Long-running Convex jobs (16MB limit + 100 item batch constraint
I'm trying to build a background job in convex where we:
- loop through a table (~20k projects)
- for each, run logic and maybe call an external API (e.g. Fly.io)
i found that convex’s migration tooling forces batches of 100 by default to avoid transaction limits
any suggestions on how to structure this whole process (db reads, marking, external calls) given that constraint and the 16mb limit?
1 Reply
Thanks for posting in <#1088161997662724167>.
Reminder: If you have a Convex Pro account, use the Convex Dashboard to file support tickets.
- Provide context: What are you trying to achieve, what is the end-user interaction, what are you seeing? (full error message, command output, etc.)
- Use search.convex.dev to search Docs, Stack, and Discord all at once.
- Additionally, you can post your questions in the Convex Community's <#1228095053885476985> channel to receive a response from AI.
- Avoid tagging staff unless specifically instructed.
Thank you!