Schedulers with priority
It would be nice if the schedulers can have a priority to control how functions are executed.
In my scenario, I'm consumin a Webhook from Microsoft Azure that calls my convex route which triggers a scheduled function (rate Limited / 1minute).
There were 10 failed webhooks, the 3rd party provider keeps retrying the failed webhook when a new request comes in.
For example, a new user makes a request, expected to receive a result in 1minute (rateLimit), however, the provider automatically retries the previous webhooks that failed in random order. Each of this run waits for 1minute thereby making the new user wait 10 minute before his request is processed.
If there was a way to prioritize the scheduler + rateLimit,
HIGH, LOW, MEDIUM or something close so that even when we have 100 items scheduled to run, it starts with functions with HIGH priorities first, assuming the failed request was set with LOW priority.
With this in place, failed webhooks won't stall results for new users as they'll be processed later.4 Replies
@Mordsith we should have support for this coming soon!
namely, there will be things like convex components that implement a WorkPool that provider higher-level abstractions on top of the "raw" scheduler that can help with traffic control and things like this
Is there a reason you don't accept all webhooks and save them for processing so the webhook doesn't have to retry them over and over?
hmm, I might be misunderstanding. I thought this was a scheduler backup, not our webhook handlers (which are not scheduled)
@ampp No particular reason, I never considered it since it's small volume of requests at the moment. I'll add this to my setup
@jamwt This is for schedulers...
The flow is like this
Azure Task Complete -> Trigger Convex Webhook URL -> Schedule a task to run