Rate limiting / limiting paginated queries
Hi Convex community 🙂 I'm looking for a way to rate limit or limit paginated queries, or related workarounds.
I've already read the docs - https://docs.convex.dev/database/pagination
however, I could not find a good way to limit number of calls, or number of paginated contents of paginated queries from the server side.
Paginated Queries | Convex Developer Hub
Load paginated queries.
6 Replies
@daun take a look at this https://discord.com/channels/1019350475847499849/1203007687344914503/1203114033570644019
Depending on what sort of rate limiting use case you're thinking of this may also be useful: https://stack.convex.dev/help-my-app-is-overreacting
Help, my app is overreacting!
Reactive backends like Convex make building live-updating apps a cinch, but default behavior might be too reactive for some use cases. Not to worry! L...
Thanks, I've read the thread, I'm asking this to prevent too much resource being somewhat automatically scraped or therefore too much network bandwidth is used. However I wonder if it's possible to limit the pagination cursor from the server-side query function. For example, I'm trying to implement 'Free tier users can only retrieve 500 posts.' I can easily check if the user is authenticated or not already but I'm not sure if limiting pagination cursor is available in the API.
Here's a basic example of rate limiting pagination: https://github.com/get-convex/convex-nextjs-app-router-demo/blob/bcc7749acb93ad29f4a2ff62ad750822cb87ad28/convex/posts.ts#L88-L101
GitHub
convex-nextjs-app-router-demo/convex/posts.ts at bcc7749acb93ad29f4...
Demo showing a Next.js App Router app powered by Convex backend - get-convex/convex-nextjs-app-router-demo
for a free tier you could also do more hard-coded offset pagination. e.g. the first request fetches the first 100. The next fetches 200 and returns the last 200, and you limit it to 500 max. Then for pro you could have infinite scroll. just a thought
I just added a helper for rate limiting in
convex-helpers@0.1.38. More info in this new Stack post: https://stack.convex.dev/rate-limitingImplementing Rate Limiting with only two numbers
Implementing application rate limiting when you have fast access to a database with strong ACID guarantees. Token bucket and fixed window, with fairne...