What's the best way to manage long running functions in convex actions?
I have 3 async functions that depend one on each other so I can not do Promise.all
I'm running these in internal action ("use node")
each function can take ~5 seconds so total action execution time could take ~15 seconds
Is this is healthy for convex? is there a better way todo this?
16 Replies
Am I understanding this right that they need to run sequentially, and you don't want to just run the code from one place inline? Having a top level action is the simple approach, but it contributes to action execution time for something that is running idle. You could also have each one schedule the next one when it's done.
An older post that still has some good ideas: https://stack.convex.dev/background-job-management
Background Job Management
Implement asynchronous job patterns using a table to track progress. Fire-and-forget, cancelation, timeouts, and more.
I might have misunderstood - if it's not calling 3 convex functions but just doing inline slow operations, then you're doing it right. Actions can run for up to 10m so it's "healthy" for Convex.
Yeah I want to run 3 long running api calls (no reading writing convex data).
Great. Then yes just calling them from an action that sits and waits is what I would do.
how does that impact my billing though? Execution time in lamdas cost money, right? I wouldn't expect that function which runs 15-20s be running very often though but anyway
For actions you pay for GBh - a combo of how much memory it's using and how long it runs. If you're using 5MB and running for 15s, you can do ~1 million of them each month on the free plan (20GBh) or 12 million on the pro plan, after which every 1 million calls would be ~$6 if my napkin math is right
I'm not sure how much memory you'd be using though. You can check in the log stream and attribute it to each function call if you want though
Log Streams | Convex Developer Hub
Configure logging integrations for your Convex deployment
other solutions would look like calling something that calls back to a webhook, or kicking something off and polling with the scheduler, but personally I wouldn't worry about 15s - the cost of whatever service you're hitting is likely much larger than some action compute in Convex
thanks for reply!
Hi, things may have changed since this answer was given but I am a little confused about this. For the test, I wrote a dummy action with setTimeout that waits 600000ms (10m). This is reflected in the usage as 0.01 GB-hours. If my calculation is correct, the minimum RAM usage seems to be around ~64MB. So a usage like 5MB is not possible in practice. Is this correct?
yep, that's correct
Thank you for clarifying 🙂
Hi, I need to be subscribed to a blockchain’s node and stream some info to Convex.
Is it a bad idea to use a long-running stateful convex action to achieve this? it will be just one process running 24/7 calling internal mutations every 5~10 seconds.
I don’t want to manage another backend provider like Fly.io. I love Convex.
Yes, it's bad. Do cron jobs work for you?
Actions time out after 10 minutes
Yeah a cron that runs once a minute and picks up where the last left off seems like the best bet - or recursive scheduling with some delay between each call.
If you want to run 24/7, you could run for 5min then schedule itself to start up again, passing through whatever cursor is necessary. AI Town runs nonstop and schedules itself
thanks, will checkout AI Town.