mikeysee•12mo ago

Error while trying to use Loro CRDTs (Uncaught ReferenceError: FinalizationRegistry is not defined)

Hey guys im trying some experiments with Loro and Convext together (https://www.loro.dev/docs/tutorial/get_started) but am running into an issue. Here is a basic query I am trying to write:

export const myQuery = query({
  args: {},
  handler: async (ctx, args) => {
    const doc = new Loro();
    doc.getList("mylist").insert(0, "A");
    return doc.exportSnapshot();
  },
});

export const myQuery = query({
  args: {},
  handler: async (ctx, args) => {
    const doc = new Loro();
    doc.getList("mylist").insert(0, "A");
    return doc.exportSnapshot();
  },
});

The error I get is:

✖ Error: Unable to push deployment config to https://kindly-elephant-254.convex.cloud
400 Bad Request: InvalidModules: Hit an error while pushing:
Loading the pushed modules encountered the following
    error:
Failed to analyze auth.js: Uncaught ReferenceError: FinalizationRegistry is not defined
    at <anonymous> (../node_modules/loro-wasm/bundler/loro_wasm_bg.js:209:9)

✖ Error: Unable to push deployment config to https://kindly-elephant-254.convex.cloud
400 Bad Request: InvalidModules: Hit an error while pushing:
Loading the pushed modules encountered the following
    error:
Failed to analyze auth.js: Uncaught ReferenceError: FinalizationRegistry is not defined
    at <anonymous> (../node_modules/loro-wasm/bundler/loro_wasm_bg.js:209:9)

I suspect the issue is because loro uses wasm. I have read however that WASM is supported in both runtimes. Is this not correct?

Getting Started – Loro

32 Replies

lee•12mo ago

Wasm should work in general. The error seems to be about FinalizationRegistry https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/FinalizationRegistry which Convex runtime does not support because it is nondeterminstic (it relies on knowing when garbage collection runs) so it messes with query subscriptions and mutation conflict detection which rely on determinism. I don't know why Loro uses FinalizationRegistry (it shouldn't be required to use wasm) so it may be possible to avoid the usage or hack around it if not.

MDN Web Docs

FinalizationRegistry - JavaScript | MDN

A FinalizationRegistry object lets you request a callback when a value is garbage-collected.

mikeyseeOP•12mo ago

ah okay, thats a shame but it makes sense, thanks Lee

lee•12mo ago

It's also possible we can work around this on our side. I started an internal discussion. Thanks for reporting!

mikeyseeOP•12mo ago

BTW for anyone that stumbles across this in the futurte, I raised this issue with Loro here: https://github.com/loro-dev/loro/issues/248

GitHub

Loro and FinalizationRegistry · Issue #248 · loro-dev/loro

Hi, I am experimenting around with Loro and Convex bun unfortunately I ran into an issue: ✖ Error: Unable to push deployment config to https://kindly-elephant-254.convex.cloud 400 Bad Request: Inva...

Cruz•12mo ago

Hey @mikeysee, I was interested in convex + local first but didn't see anyone using them by now, since Loro is kind of for local first apps I was going to ask if you had a chance to combine this stacks?

mikeyseeOP•12mo ago

thats exactly what I was investigating until I ran into this issue I would like to combine convex and local first, this seemed like a way of doing it

Cruz•12mo ago

would love to hear about your experience if you find a way!

Indy•12mo ago

@mikeysee I'm curious if you thought about giving https://replicache.dev/ a try?

Replicache

Replicache: Framework for local-first web apps.

Replicache is a client-side datastore and sync framework for building local-first web apps. It works with most backend stacks.

Cruz•12mo ago

For some reason everyone is suggesting replicache but there is no single working example or implementation of replicache + convex @Indy why do you think that replicache can work w/ convex?

Indy•12mo ago

I think conceptually it should work. But I have not seen anyone try it. Thus I was curious if @mikeysee had given it a try.

mikeyseeOP•12mo ago

no I havent tried it, I think the fact that they charge for something I think I could implement myself put me off. Ill give it another look 🙂

ian•12mo ago

@nb - @sujayakar made a demo recently: https://github.com/sujayakar/replicache-convex He hasn't put it on convex.dev/templates but is working on it - let us know what you think

GitHub

GitHub - sujayakar/replicache-convex

Contribute to sujayakar/replicache-convex development by creating an account on GitHub.

Cruz•12mo ago

@sujayakar do we still get type safety on the front end?

mikeyseeOP•12mo ago

nice project @sujayakar ! the amount of extra work you have to do is kind of annoying:

  replicacheClient: defineTable({
    clientId: v.string(),
    clientGroupId: v.string(),
    lastMutationId: v.number(),
    version: v.number(),
  })
    .index("by_client_id", ["clientId"])
    .index("by_client_group_id", ["clientGroupId", "version"]),

  replicacheServer: defineTable({
    serverId: v.string(),
    version: v.number(),
  }).index("by_server_id", ["serverId"]),

  replicacheClient: defineTable({
    clientId: v.string(),
    clientGroupId: v.string(),
    lastMutationId: v.number(),
    version: v.number(),
  })
    .index("by_client_id", ["clientId"])
    .index("by_client_group_id", ["clientGroupId", "version"]),

  replicacheServer: defineTable({
    serverId: v.string(),
    version: v.number(),
  }).index("by_server_id", ["serverId"]),

But I guess this is the price you have to pay if you want to ensure conflict free and offline-first. I suspect I was going to have to do something simmilar with Loro. Its a shame that I cant work around that FinalizationRegistry issue

sujayakar•12mo ago

@Cruz, it needs some work to cleanup the types and API, but I think typesafety is mostly there. the replicache tutorial has me pass in the Message type directly to tx.scan on the client side, and then we use the MessageWithID type in the mutator that then would connect to the mutation type on the server. and yeah @mikeysee, the hope would be that most of this could be abstracted away in a convex-replicache library (or... component 😃 ) related: I just submitted a PR to add no-op implementations of FinalizationRegistry and WeakRef to our isolate environment, so hopefully loro can work after that!

mikeyseeOP•12mo ago

oooo! ❤️ that would be awesome! ye sound like a good candidate for the upcoming convex modules (or whatever its going to be called) project

sujayakar•12mo ago

i’ll update the thread when it’s in prod!

Cruz•12mo ago

@mikeysee why you chose Loro over other lo-fi techs? I quickly take a look at their website and saw that this tech is mostly suitable for text editing apps where LWW(Last Write Wins) and how does it work in the case of authoritative serves where some logic is included, for example for the systems like POS?

mikeyseeOP•12mo ago

I dont know, I just wanted to tinker with Loro, I like the idea of CRDTs and it looked like a really nice and simple implementation. Im not convinced convex is neccessarily a good fit for it TBH. Its probably a lot lower cost if you just run your own node server or use a cheaper FaaS platform like Cloudlfare workers or something. The reasoning is that we dont really need the transactional gaurentees of convex as the CRDTs take care of that for us. But im a Convex fanboy so everything looks like a nail when you have a Convex hammer 😉 ye its always going to be a tradeoff based on usecase Its not always text applications that CRDTs work for tho. Think Figma, Canva, Google Sheets, etc etc

Cruz•12mo ago

definitely, but all these apps you listed work w/ the same principle LWW right?

mikeyseeOP•12mo ago

(well google sheets is OT based but you know what I mean) im not sure about LWW to be honest, CRDTs just gaurentee that there wont be any conflicts when the two timelines are merged together I was going to experiment around with it a bit more but my understanding is that user A can make changes to a list of items while user B makes changes to the same list. The list's final state will be a combination of those changes NOT just whoever made the last change

mikeyseeOP•12mo ago

Loro site has a bunch of examples you can play with (https://www.loro.dev/docs/tutorial/time_travel)

How to Use Time Travel in Loro – Loro

sujayakar•12mo ago

stub implementations of FinalizationRegistry and WeakRef are in production now! let me know if you run into any issues.

mikeyseeOP•11mo ago

wooo! Ill take a look when I can ❤️

mikeyseeOP•11mo ago

sadly I now get this error 😦

sujayakar•11mo ago

huh very interesting. do you have a small repro @mikeysee? or is just trying to use loro sufficient?

mikeyseeOP•11mo ago

yep simply trying to use loro in any mutation or query or action does it I can give you an example but it would be like 1 line new Loro()

sujayakar•11mo ago

heh fair enough. will give it a shot 🙂

mikeyseeOP•10mo ago

@sujayakar did you ever get to the bottom of this?

sujayakar•10mo ago

I did a bit of digging but am currently a bit stuck. I think the core issue is that we set up WASM imports like Cloudflare Workers (see the WASM snippet in https://developers.cloudflare.com/workers/wrangler/bundling/#files-which-will-not-be-bundled), but loro uses another convention where the ESM import is already instantiated and linked (see screenshot, taken from https://www.npmjs.com/package/loro-wasm?activeTab=code). cc @presley (we were just talking about this in person) and @ballingt

mikeyseeOP•10mo ago

@sujayakar Is there anything I can do to hack around it for now? Maybe fork and change something in Loro? thanks for looking into this BTW I very much appreciate it! This isnt mission critical for me BTW, just as a side-interest

sujayakar•10mo ago

of course! @ballingt’s looking into some bundler changes we can make to get this working.