Workflow Scalability
Hey Convex folks đź‘‹,
I’m building a product where our customers create little “AI programs” (basically chains of llm calls + tool execution). Depending on the complexity of the program, runs last anywhere up to about 20 minutes (maybe longer), and users can fire them manually or schedule them (e.g., “every day at 9 AM”).
I’ve been prototyping with workflows as a way to manage the execution of a program run, but I’d love a sanity-check before we go too far down that path. A few things I’m chewing on:
1. Is Workflow the right choice?
There are two main ways a user runs a program...
- On a schedule — basically a cron job of sorts. For this scenario we need durability, retries, and background processing — workflows seems perfect.
- In the app — a user can run a program directly from the UI. Right now we are planning on still using workflows here, but i could see an argument to reach for something else.
2. Parallelism & caps
The docs suggest keeping the sum of maxParallelism ≲ 50. I'm trying to understand if this a hard cap or just best practices? My concern is that many of our programs might be long running, meaning that we might hit this cap quite easily. What are my options here? Any way to scale beyond this easily?
3. General scalability
If a few thousand customers all schedule 9 AM runs, we’ll spike concurrency for an hour and really hammer the workflow system. Obviously there is built in queuing to spread out the load a bit, but we also need to trigger the runs approximately around the scheduled time, so too long of a queue starts to effect customer experience. Is Convex + workflows designed for this challenge? Or should we be reaching for something else?
I've looked into other solutions (like Temporal), but would love to keep everything on Convex. The DX of the platform has been great and it' been a joy so far.
Let me know if more details are helpful, thanks!
1 Reply
Thanks for posting in <#1088161997662724167>.
Reminder: If you have a Convex Pro account, use the Convex Dashboard to file support tickets.
- Provide context: What are you trying to achieve, what is the end-user interaction, what are you seeing? (full error message, command output, etc.)
- Use search.convex.dev to search Docs, Stack, and Discord all at once.
- Additionally, you can post your questions in the Convex Community's <#1228095053885476985> channel to receive a response from AI.
- Avoid tagging staff unless specifically instructed.
Thank you!