Handling conflicts on specific tables
Convex handles conflict management for you. Can I handle it myself for specific tables? I have a writing app that uses Operational Transformation and need to handle those conflicts myself. (reposted)
7 Replies
The conflict management that Convex does is very low-level, and exists to provide you transactions https://docs.convex.dev/database/advanced/occ. You can't opt out of this behavior.
Adding Operation Transform (OT) logic is relatively simple: in the mutation where you modify the table you can run run your OT logic.
@RJ has written up a version of this before, it's not up to date but it shows the pattern: https://github.com/rjdellecese/prosemirror-collab-convex/blob/main/src/convex/index.ts
Pulling @Jacob Wright who asked
Anytime you write a Convex mutation you're writing a transaction that might be replayed if (but will only be committed once) so generally reads the current state, applies a modification, and then writes that modification. You can use this same pattern to apply an operational transform.
@Jacob Wright does this pattern make sense? I thought we had another example using ProseMIrror Collab https://prosemirror.net/examples/collab/#edit-Example but I'm not finding it — but a key thing to understand about Convex is that your mutations can be arbitrarily complex JavaScript functions, so Convex mutations are how we provide custom conflict management on tables: these are server-side functions that can read and write arbitrary data.
Where "arbitrarily complex" includes most npm modules, with the exception of some Node.js specific APIs like
fs.readFileSync("/home/me/file.txt")
Backing up a bit, to the high level question of handling conflict management:
Yes! You can always throw an error, which will roll back the transaction.
Then you could retry your edit submission from the client. This is the approach I've take before when writing collaborative editing tools: rebase the change on the client, where you have as much information as possible about user intent.
Additionally with Convex you can run arbitrary logic in your server endpoints, so you may not need to retry from the client.
@Jacob Wright poking around https://github.com/dabblewriter, very cool stuff!That sounds perfect. Coming from Firestore, the way to ensure this is to have security rules that essentially create a unique index on the revision number field. It’s been a while since I worked with transactions like this. Sounds perfect.
Thank you! It’s been in production for almost six years now with over 2 billion words written on it (live count on the home page https://www.dabblewriter.com)
Most of the logic is on the client and uses retries with rebasing there. But to get better collaborative performance, pushing the logic to the server to transform and apply the changes is necessary. M
Also looking to bring it to the edge for UK and Australian customers benefit.
I should just open-source the little project of mine where the Convex-ProseMirror OT was working
I’ll do that later today
The demo is at https://scroll.ink
Here it is! https://github.com/rjdellecese/scroll/
Of primary interest are:
- The
convex directory: https://github.com/rjdellecese/scroll/tree/fe3ac33935ef03b7ddf3b14d1f991cf632ffc515/src/convex
- The note.tsx module: https://github.com/rjdellecese/scroll/blob/fe3ac33935ef03b7ddf3b14d1f991cf632ffc515/src/elm-ts/note.tsx
The Convex functions are written in "normal" TS, but the frontend code (e.g. note.tsx) heavily employs a functional programming library and so is probably going to be harder for most people to grok. I'd just recommend starting with the Convex directory, ProseMirror OT is basically the only thing going on in there. note.tsx is where the client-side stuff happens--look there for uses of collab (https://github.com/rjdellecese/scroll/blob/fe3ac33935ef03b7ddf3b14d1f991cf632ffc515/src/elm-ts/note.tsx#L23C14-L23C14) and useQuery/useMutation.
Also happy to answer any questions I canLooks like a great candidate for #show-and-tell ◡̈
Gladly! I’ll spruce it up a bit (add a README or something) and then post it over there