Handling Rate Limits with Convex Scheduler
If I were to mirror a queue that runs exactly like a setInterval, how do I do that?
Scenario. I want to call a third-party service that has a 1req per minute rate limit. With most queue workers, using a simple configuration, you can specify that you want the worker to run jobs every 1 minute to beat the rate-limiting problem. If 10,000 users make a request at once, no problem since all this will be in the queue and the worker will process it every minute.
Hw do I configure a scheduler to work this way? Imagine 10, 000 users would hit the function to be scheduled. I want to process this using FIFO at set intervals
5 Replies
Have you seen the docs on scheduling in Convex? https://docs.convex.dev/scheduling
With that plus a table to store the requests in you can build this behavior, and there are a lot of Stack posts about this kind of thing. https://stack.convex.dev/implementing-work-stealing is related and recent.
https://stack.convex.dev/background-job-management looks great
I recently wrote a post on this @Mordsith - I would suggest using my rate limiting helper with the "reserve" capability - I'd suggest a token bucket with capacity of 1, rate of 1, period of MINUTE (60_000). That will let it run immediately if there's capacity, and if not it tells you when to schedule it to run (where you don't have to re-check rate limit then - you've already been allocated capacity)
https://stack.convex.dev/rate-limiting
Implementing Rate Limiting with only two numbers
Implementing application rate limiting when you have fast access to a database with strong ACID guarantees. Token bucket and fixed window, with fairne...
Specifically the "Making LLM requests with reserved capacity" section:
https://stack.convex.dev/rate-limiting#making-llm-requests-with-reserved-capacity
Implementing Rate Limiting with only two numbers
Implementing application rate limiting when you have fast access to a database with strong ACID guarantees. Token bucket and fixed window, with fairne...
Thank you @ian and @ballingt
I'll work with this 🙏🏾
@ian The
ctxargument in rateLimitdoesn't work with action ctxThat's right - you need to call a mutation to rate limit
It's transactional and read/writes to DB, so needs to be in a mutation. I'd batch it with other DB operations you do transactionally when you decide to go ahead with the operation, if any