too_easy•4d ago

possible support for top level await from npm packages

Is it possible to allow for top level await from certain npm packages? Either through a convex.json parameter or some other custom build property?

14 Replies

Convex Bot•4d ago

Thanks for posting in <#1088161997662724167>. Reminder: If you have a Convex Pro account, use the Convex Dashboard to file support tickets. - Provide context: What are you trying to achieve, what is the end-user interaction, what are you seeing? (full error message, command output, etc.) - Use search.convex.dev to search Docs, Stack, and Discord all at once. - Additionally, you can post your questions in the Convex Community's <#1228095053885476985> channel to receive a response from AI. - Avoid tagging staff unless specifically instructed. Thank you!

ballingt•4d ago

What npm packages are you seeing use this, is there another way we could support these? Top-level await is not currently allowed, but we'd love to hear why it would be useful, it's something we could implement but it'd be great to know what it's useful for because it would make the runtime more complicated.

too_easyOP•4d ago

loro-crdt - https://www.loro.dev/docs/tutorial/get_started

Getting Started – Loro

How to use Loro to build real-time or asynchronous collaboration software.

too_easyOP•4d ago

It's just its implementation I guess, probably something to do with how they load their wasm module I'd really like to have it work on the convex V8 runtime not the node runtime so that I can make use of acid transaction I.e. have it work in mutations/queries rather than actions Yjs would work fine inside mutations/queries but its causes other issues with how large the update history gets over time..

ian•4d ago

Do you need it in the convex backend, or just in the Vite config? Can you avoid importing code that needs it? Sorry not a Loro expert I think the underlying issue is more to do with wasm: https://discord.com/channels/1019350475847499849/1197172506084265984/1214297354794963014

too_easyOP•4d ago

Need it in convex backend tbh - the server is a central authority that can merge updates neatly via transactions. If it was left to client (or even actions) theres a chance updates get overwritten rather than merged

ballingt•4d ago

@too_easy is it async imports or wasm support you need here?

too_easyOP•4d ago

top level await is what I need, pretty sure I read somewhere that wasm is supported on the convex v8 runtime already right? If so, its just the top level await that is breaking things

ian•4d ago

Did you check out the thread I linked? It's all about getting it working in the backend. it's from a while ago but there's some nuances to using wasm in various serverless environments, including convex v8. Often it works the same way as cloudflare workers iirc

too_easyOP•4d ago

Pretty sure the wasm part is fine and the error comes from how the wasm library is instantiated (using a top level await) which is happening in js land, not wasm land. Loro runs fine in the convex "use node" environment - so I don't see why it couldn't run in the convex native v8 environment - perhaps it is because one is using commonJS and the other is using esm? The convex compile/upload error I get literally says "top level await is not supported"

ian•3d ago

yeah you'll want to instantiate it lazily when you need it. the "use node" environment runs in aws lambda. I ran into some of this for using Automerge wasm. note all their different docs based on platform: https://automerge.org/docs/library_initialization/#cloudflare-workers I ended up using a node environment for it, but in another project I got it working in the convex v8 environment lazily loading the wasm: https://github.com/get-convex/automerge-sync/blob/component/src/component/automerge.ts#L9

Library Initialization | Automerge CRDT

Automerge is implemented in Rust and compiled to WebAssembly for use in javascript environments. Unfortunately the way that WebAssembly modules are loaded varies across environments. In some situations this can be handled by your build tool, but in others you may need to manually load the module. This page describes how to load automerge in vari...

GitHub

automerge-sync/src/component/automerge.ts at component · get-convex...

Sync Automerge documents with Convex. Contribute to get-convex/automerge-sync development by creating an account on GitHub.

too_easyOP•3d ago

Looking at this from the other way, because I can't seem to find a nice wasm loading function for loro like automerge - Is there anyway to get acid transactions while using node environment actions? The problem with putting the crdt lib in node actions is that its possible for overwrites to happen

ballingt•2d ago

From Node.js you can run transactions in Convex with await ctx.runMutation()

too_easyOP•2d ago

Correct me if I'm wrong but doing something like in nodejs environment:

const snapshot = await ctx.runQuery(api.getSnapshot);

// do some thing with Loro

await ctx.runMutation(api.updateSnapshot)

const snapshot = await ctx.runQuery(api.getSnapshot);

// do some thing with Loro

await ctx.runMutation(api.updateSnapshot)

Assuming getSnapshot reads a point in time, Loro changes that snapshot as a binary / bytes / buffer, then calls an update snapshot - isn't there a chance another call was made simultaneously in between and that data was overwritten? I guess if the crdt is doing its job then its probably fine for it to be overwritten as the client could just request the updates be made again?

possible support for top level await from npm packages

Did you find this page helpful?