Best-practice for (mutation+action) function?
Hi! 👋
Brief question:
When I need, in response to a client's clickety-clicking, to:
a) write some changes to the DB; and
b) call some external APIs (using custom libraries) Is it better practice: * to call an Action from the client, and runQuery within the action as needed; or
* to call a Mutation, and then... I guess schedule an action? I strongly assume the former, now that I've typed it out—but would appreciate feedback anyway!
What's the best way to achieve only-once'ness/atomicity? A semaphore table maybe? Thanks!
a) write some changes to the DB; and
b) call some external APIs (using custom libraries) Is it better practice: * to call an Action from the client, and runQuery within the action as needed; or
* to call a Mutation, and then... I guess schedule an action? I strongly assume the former, now that I've typed it out—but would appreciate feedback anyway!
What's the best way to achieve only-once'ness/atomicity? A semaphore table maybe? Thanks!
38 Replies
It's a little more work, but the most reliable way to sequence actions is (Mutation -> schedule action) steps
these are the fundamental ingredients to reliable workflow. we'll have higher level libraries soon for this, but you can build it yourself for now ( ahem because of https://stack.convex.dev/the-software-defined-database )
Convex: The Software-Defined Database
Which to choose, the expressive power of code, or the robustness of built-in database features? With Convex, you can have both. By eliminating the bou...
the reasons why this is the right sequence:
In convex scheduling jobs is atomic with respect to any other updates you have. so ideally, you would persist some information about the job you intend to do and then schedule it to be done in the background in one atomic step. that way it can be retried if it fails without needing the browser to remain online to detect failure and retry it (which is what the "action first") step requires. action first is much more fragile.
re: idempotency, at-least-once vs. at-most-once. by default convex actions are intentionally at-most-once (fallible) because (a) it is the most fundamental ingredient and (b) b/c actions can have effects, convex cannot assume it is safe to re-run a partially failed action. we don't know it's safely idempotent
however, you can easily use convex's primitive to make retriable at-least-once actions. and we wrote up a little library to do it, which is being used by a lot of teams right now: https://github.com/JamesCowling/convex-action-retrier
GitHub
GitHub - JamesCowling/convex-action-retrier: Helper function to ret...
Helper function to retry a Convex action until it succeeds. - JamesCowling/convex-action-retrier
a little writeup about it here: https://stack.convex.dev/retry-actions
Automatically Retry Actions
Learn how to automatically retry actions in Convex while also learning a little about scheduling, system tables, and function references.
so in a nutshell, if you want reliable workflows of background activity, think (mutation -> action) sequences, and use something like the library above for each step to ensure reliable completion along the way
and we hope to have higher level abstractions that wrap up all these practices in a nice API within a few months
So if I understand this right, these are the possible flows
(I suck at diagrams.
Well, I made one with fountain pen that's nice, but not a very portable format...)
I if I do mutation -> schedule action...
I probably have to decouple my workloads a bit more than anticipated 🤔
queries don't directly interact with the scheduler
My current steps are:
* client submits order
* patch order document in the DB with verious fields (maxPrice, comments, delivery address, status="SUBMITTING")
* send off API call to get redirect URL from payment provider
* synchronously wait for reply from payment API
* create a document in the
payments table with references to the order, and IDs from the payment provider
* return the URL back to the client, so the front-end can redirect them to the payment provider hosted payment page
(in-between most of these I also want to write a lint to a makeshift audit table, because we're still free tier for our MVP :P)Templates
The backend application platform with everything you need to build your product.
I didn't check it out, assumed it was stripe-specific
we're going to sue worldline
but I guess the high-order flow is the same... will check it out!
aye, just looked it up again
my bad! 👍
no problem
otherwise, it's a great visual. we need more and more things like this for sure to make the learning process smoother, so I appreciate you explaining your thinking so clearly! 😃
Thanks a lot, Jamie! I'll check out the Stripe repo, and shout if I'm more confused after (not unlikely!)
❤️
@Michal Srb also wrote up notes on the design here as well, but probably works for any payment provider: https://stack.convex.dev/stripe-with-convex
Wake up, you need to make money! (Add Stripe to your product)
If you’re building a full-stack app, chances are you’ll want some of your users to pay you for the service you provide. How to use Stripe with Convex ...
Me thinking out loud about how I think is how convex found me in the first place 😛
https://softwareengineeringdaily.com/2023/08/29/building-a-full-cloud-backend/
Software Engineering Daily
Building a Full Cloud Backend with James Cowling - Software Enginee...
Serverless backend platforms are cloud services that simplify the process of building a backend. These platforms are growing rapidly in popularity because they can greatly accelerate application development, and improve the developer experience. Convex is a real-time backend platform that uses 100% TypeScript and is designed with reactive UI fra...
nice! lol, I just put two and two together.
Looking into this rn (who needs a sane sleep schedule?)
Just a QQ aside:
I assume you can schedule a Query... but it doesn't make any sense... ever.
Correct? 🤔
You can only schedule mutations and actions
Cool. Because it literally makes no sense to schedule a query?^^
(I.e. not because of any inherent technical limitations?)
correct!
it makes no sense to do so
it's like a tree falling in the forest etc etc
Perfect, then I understand!
Thanks, guys!
and a query can't schedule anything b/c it isn't allowed to mutate anything. that's why those two lines in your earlier flow chart were the problematic ones
yep, got it now!
I've read the Stripe example now (also useful for general React idiomatic-ness)
I understand it, but sadly it uses Action->Mutation 😅
yeah, and in this case, that's okay. I do feel like we're learning this topic needs a stack post
so, if the action is only on behalf of the in-app user
so now digging into
retrier to understand how to build my own guarantees 😛
elaborate... 🤔then perhaps there is no need for a server intention to "finish" it if the app breaks off
the mutation-first approach in a way is delegating the "agent" to ensure the effect finishes to be the server instead of the app
but if your particular app doesn't need that, it's okay to directly have the action called by the app. and it's simpler to not involve an extra hop
Yeah... I need to think about the "failure cases" I think
yep
the reason why I usually recommend mutation -> action
is unless someone has put a bunch of thinking into this...
they might find themselves rebuilding convex-action-retrier in their app!
the only thing I don't want is for the payment to have gone through, but some information having been lost about it
which... if I record the external payment system's reference chronologically before the payment link is ever sent back ... is not possible
(we still do the normal time-space stuff, right? let me know when an update messes with that ;P )
yeah, the rabbit hole is deep, but in my experience most flows that don't record the intention to conduct some effects before beginning are inherently fragile at the limit
but for some apps, it's not Important Enough to be correct, so the simple way is good enough.
yeah
so if I record stuff, even in my Action, and then it breaks
I can retry, safely, if the retry checks existing recorded intentions, and their status (internally and/or externally as applicable)
yep, the most durable way would be something like (app -> mutation (record transactional intention, then use convex-action-retrier to drive to completion) -> on completion mutation -> app will reflect success due to reactivity)
✅
(I can see this relatively clearly now. God d*mn why can I never think this straight during the normal human day, with the sunlight and all?! 😑 )
lol
Thanks for your help @jamwt ! ❤️
no problem! thanks for the patience
The stripe example probably needs updating, we wrote it a while back 😇