Trigger/onUpdate Functions
I'd love to have Trigger functions (in addition to scheduled) : functions that are executed when a
query returns.
Similar to having a service that runs new Convexclient().onUpdate(query) but that actually runs "inside convex.
I'm aware of the schedule.runAt(0...) pattern which is also nice, but I think the trigger pattern is more expressive, allowing muliple paralel behaviors to occur independently, and allows better modelling of fail scenarios and keeps the mutations cleaner.
Also, I cant implement onUpdate patterns in for example vercel functions, having this would allow me to stay reactive withouth the burden of a stateful server.
Usecase:
I have datapoints being created in a table, with multiple mutations, from multiple sources and I want to model multiple parallel background job/queue that are triggered as soon as possible bc they are user facing. Cron not ideal because "asap" not "later".13 Replies
Are you imagining a trigger that runs in the same transaction as the mutation that changed the data? (what trigger usually means in SQL) Or something that runs afterward, once it's too late to fail the transaction? (similar to
.runAt(0, ...))
It's helpful to hear you want this. For now it sounds like a data access layer that enforces .runAt(0, ...) is called whenever certain kinds of changes are committed would be good, which can be implemented in user space (not that that's the end of the conversation, someone needs to implement it).
If you want server-side query listeners instead of triggers, the semantics are different: can these updates be debounced? what happens when they fail? Which there are reasonable answers to but want to check which behaviors would match what you need.
As long as debouncing (a server query listener running only once even though several mutations occurred) is ok and you're ok with errors not propagating back to the mutation that caused the change, then the query listeners thing makes sense and it's something we've talked about.
You mention crons being related but not quite right because of the delay — that's a good model of how a server query listener might work, basically a cron that runs more frequently (say 5 times a second) but only if a query result has updated.You are right, its not really like trigger in the sense that I dont want to abort transactions, its really more like a pub/sub but where the
pub does not care who are the subs. I think the Reactive behavior of onUpdate is fine for me though I'd love if it was a bit more like pubsub ensuring "message consumption" but I'm manmaging this with mutations "start processing" / "end processing" and using this flag to drive the query.
Usecase is this:
supose I have tasks being created on a table. Everytime I have a task or a set of tasks created I want to initiate enrichment workers in paralel, they have their own time, some are now, some are later some work in batches. The tasks table should not really care about who does this, the logic is separate.
Flow is something like this:
onUpdate( select * from notes where state = unprocessed)
worker mutates state to "in progress" (user visibile)
worker does job (e.g. some AI stuff)
worker mutates state to "done"This sounds more like a trigger, you really want this to run once per mutation
Also, the reason why I dont like the cron model is bc this behavor is user facing, Imagine your screen and a bunch of dashboards popping up one after the other but you knowing that they are doing something behind the scenes.
I've simulated this very nicelly with a node job with onUpdate the issue is that it requires me to manage a long living server in Vercel (which is not supported) so I need to give money to fly.io or something which I'd rather pay you 😉
It does, except that I'm happy with grouping things up / handling multiple messages at the time as some of my downstream services can enrich in paralel and I expect many users to need enrichment at the same time
so theoretically trigger is fine, architecturaly, batching is better.
In this particular case, would it be ok if you missed a note that was momentarily unprocessed, then disappeared?
yes, but because of queries plus the transactionality of the processor (mutate (started) <do work> mutate(done )) I can allways find things that are started but not finished and clean up later.
or decide to simply ignore them
cool, yeah as long as you model this
Thanks for going into detail, there's been some discussion about whether folks can cope with this batching
Yes, I think with some AI applications the user can forgive some misses.
and how to make error reporting clear, unfortunately we can't propagate a failure in this query up the mutation(s) that caused it
you see what I like is the elegance of modeling a subscription with SQL llike syntax which I can do with onUpdate + query very well...
anyhow yeah I think you get my point on the "long living server" vs "inside the box" point up there. 🙂 lmk if you want to discuss further
Totally, we also think your money will be happier being spent with us 😆 and we're in an intermediate state where we could run long-running actions for you that did this but want to be sure to design our primitives carefully
Agree. I think theres also some beauty in bringing the elegance of useQuery in the client to the backend... its such a nice mental model knowing that your code will run when a condition is met...
its quite unique as oposed to the declarative model of scheduler run 0.
(but I understand your point on ... it might not run you're on your own keeping track of it)
I'm working on an abstraction that wraps database access, and can call specified code when conditions are met. Your code can do things in-transaction or
scheduler.runAfter(0. If your functions are defined with a custom query / mutation where you configure this, all those writes will work without having to find the exact spots. Does this sound like what you're looking for?
an explicit onUpdate could also be made like
Does that sound more like the API you're dreaming of?
I also like the pattern of writing to the DB saying it's in-progress, scheduling the action After(0, then the action calling a mutation when it's done, with the client reading the query, as you are doing it sounds like.