ballingt•3y ago

Using Convex from Node.js

@Tristan We're going to write up some docs on this soon. If you don't mind the React dependency, the ConvexReactClient is fine! Here's an example of doing this outside of React (this is in the browser, but it works similarly)

19 Replies

ballingtOP•3y ago

https://observablehq.com/d/39c0e0503def16ab

Observable

Hello Convex Beta

How do you get shared, reactive server-side state for an Observable notebook? There are many ways: AWS, firebase, a slick solution from Tom Larkworthy — and now there's one more, Convex. You can sign up for the Convex Beta at convex.dev. To use Convex in a webapp the standard npm install convex etc. is recommended, but in Observable importing ...

ballingtOP•3y ago

But it's more direct to use the InternalConvexClient, which doesn't have a dependency on React: https://docs.convex.dev/api/classes/browser.InternalConvexClient

Class: InternalConvexClient | Convex Developer Hub

browser.InternalConvexClient

ballingtOP•3y ago

The API is a little different, you set a single function onTransition() callback that runs every time new query results are received, and client.localQueryResult(query, args) returns the current query result for that query

Tristan•3y ago

I see. And client.subscribe(query, args) as well to get things flowing?

casado•3y ago

@Tristan right. Here is a super simple global counter implementation which uses straight JS and InternalConvexClient https://github.com/61cygni/flyconv-pixi

GitHub

GitHub - 61cygni/flyconv-pixi: Bare bones convex / pixi / fly configs

Bare bones convex / pixi / fly configs. Contribute to 61cygni/flyconv-pixi development by creating an account on GitHub.

casado•3y ago

check out src/counter.js

Tristan•3y ago

Oh nice. I wasn't aware of generated clientConfig either.

casado•3y ago

Yeah, if you want pure JS boilerplate you can use the project. I don't use react with anything. Also, afaict the trigger for InternalConvexClient calls back for any change to the table and not just the given query. So you need to check the query tokens to see if your particular query fired. something like function reactive_update (updatedQueries) { if(updatedQueries == ""){ return; } list_table_from_db(); }

ballingtOP•3y ago

@Tristan note that that generated clientConfig was just removed in this most recent release, now InternalConvexClient accepts a string as its first argument, the deployment URL you want to talk to generally we expect folks will use environment variables for this, but you can hardcode a string too if you want

ballingtOP•3y ago

as of the new release today, when you create a Convex project it will offer to store the URL in a .env file — see https://blog.convex.dev/announcing-convex-0-7-0/#breaking-environment-variable-configuration for more

The Convex Blog

Announcing Convex 0.7.0

Meet Convex 0.7.0! This release adds file storage, deployment history, and more!

Tristan•3y ago

Ok, thanks, will try that too. When I created my new project I got some message about autodetecting a Vite project, which wasn't accurate, so I just skipped the .env stuff.

ballingtOP•3y ago

ah that's great feedback, will look into falsely assuming Vite issue this logic is because different environment variables need to be used for different bundlers, but sounds like we're not getting it right yet Just fixed that, thanks again for the report @Tristan Skipping the .env stuff should be fine, it's useful when you want separate dev and prod deployments but you can start with just a string like

const url = "https://keen-llama-684.convex.cloud"
const client = new InternalConvexClient(url);

const url = "https://keen-llama-684.convex.cloud"
const client = new InternalConvexClient(url);

Tristan•3y ago

Success! Ok, so I got basic client face sort of working. A few (very newbie) notes in case they're helpful: - It was hard to find docs on ContextHttpClient and ContextInternalClient, these probably should have proper sections in the docs. - The docs about mutations were surprisingly hard to find. Maybe there should be a Mutation section under Using Convex. - ConvexInternalClient threw error about lacking global Websocket. Ideally there would be some isomorphic client. - The “ws” Websocket package types don’t match the types expected by the websocketConstructor option, so I had to cast to any. The client also required me to set window warning to false. - Query tokens seem to be returned as JSON encoded strings, but then some functions take the udfPath and args. It was a little awkward to figure out how to use them, and I may still be doing it incorrectly. - Subscribe takes name and locateQueryResult and mutate take udfPath. I wasn’t sure if these are the same thing or not. Both short and long names/udfPaths seem to work. My draft wrapper is here: https://gist.github.com/tristanz/f6311becb43937a685547e1950f4327e. I may be misusing things wildly and I haven't taken the time to properly grok the types yet.

ballingtOP•3y ago

This looks great @Tristan, thanks so much for writing these up. Sending mutations over HTTP while listening over the websocket protocol has some drawbacks around local optimistic updates and read-your-own-writes, so the version we'd want to provide would use the internal client for this instead of http. So helpful to hear these stumbling blocks.

Tristan•3y ago

@ballingt Is there an API on the InternalConvexClient that lets me issue a query without subscribing to future updates? That's what led me down the dual client route. I basically wanted the API to separate between subscriptions and (one-off) queries.

ballingtOP•3y ago

there is not, helpful to hear this would be useful. Here's one way to build it (missing some error handling)

nonReactiveQuery = async (client, query, ...args) => {
  const watch = client.watchQuery(query, ...args);
  const cur = watch.localQueryResult();
  if (cur !== undefined) return Promise.resolve(cur);
  return new Promise((r) => {
    const cleanup = watch.onUpdate(() => {
      cleanup();
      r(watch.localQueryResult());
    });
  });
}

nonReactiveQuery = async (client, query, ...args) => {
  const watch = client.watchQuery(query, ...args);
  const cur = watch.localQueryResult();
  if (cur !== undefined) return Promise.resolve(cur);
  return new Promise((r) => {
    const cleanup = watch.onUpdate(() => {
      cleanup();
      r(watch.localQueryResult());
    });
  });
}

casado•3y ago

Oh, very interesting @ballingt, thank you. I went the dual client route for the same reason as Tristan. Over time, it would be great to see the optimizations that are in the react client make their way to InternalClient, such as message deduping in queue convoys.

Tristan•3y ago

Thanks. I'll try that. In terms on consistency issues, you're just saying that the dual client approach doesn't allow me to do optimistic updates, but I'd need to write those myself, right? FWIW, so far I've been thinking this is not super important on the server side. I might be nice to have a long-poll HTTP subscribe method on ConvexHttpClient so I could get rid of websocket dependency altogether. @ballingt Is there any API for all the CRUD operations you have in the UI? For instance, truncate a table. I was going to write some tests and wanted to clean things up before and after.

ballingtOP•3y ago

That's correct, you'd need to write these yourself. They can be intelligently applied to queries based on logical timestamp if you do (applying all updates not reflected in the current timestamp) but if you don't need them this seems fine. Message received about the long-poll, I think that'd be a nice API too, we've talked about this and it's planned! Short of data import and export or streaming ingress and egress, there is not. Since functions are the place auth is implemented, autogenerated CRUD would bypass any validation and auth checks. This isn't out of the question, but we want to think hard before exposing this. That said, anything we let you do in the dashboard seems reasonable to automate. Right now you need to write these mutations yourself and expose them in some safe way (e.g. they check that a submitted secret matches an deployment environment variable or check the request is authenticated as an admin user) but we could make these easier to write, generate them etc. Right now requiring you do write these mutations yourself is our way of making sure you're opting into modifying your data in this way, since all RPC is public beyond the auth that you implement it in it. We're thinking about internal mutations etc. that could only be run from the dashboard, only be run from the other UDFs, etc. — so would love to hear more about what your use cases @Tristan

Using Convex from Node.js

Did you find this page helpful?