Web Dev Cody•11mo ago

Understanding usePaginatedQuery

I'm trying to wrap my head around how to use a page based usePaginatedQuery, so basically if a user hits /posts?page=1 or /posts?page=2, I want to fetch 5 items and change the offset.
the usePaginatedQuery seems to be more like a lazy scroll type of thing. Maybe I'm missing something?

28 Replies

Convex Bot•11mo ago

Thanks for posting in <#1088161997662724167>. Reminder: If you have a Convex Pro account, use the Convex Dashboard to file support tickets. - Provide context: What are you trying to achieve, what is the end-user interaction, what are you seeing? (full error message, command output, etc.) - Use search.convex.dev to search Docs, Stack, and Discord all at once. - Additionally, you can post your questions in the Convex Community's <#1228095053885476985> channel to receive a response from AI. - Avoid tagging staff unless specifically instructed. Thank you!

lee•11mo ago

yes usePaginatedQuery isn't really conducive to offset-based pagination. the problem is you have to know where in the table to start, and we can't just tell the database "skip over 500 documents, then give a page of size 50" if you want the pages to be defined by a field, like each page is a day's worth of documents, you can query and filter against the by_creation_time index if you actually want offset-based pagination, you can use the Aggregate component because one you have aggregates, it's feasilbe to say "skip over 500 documents"

Hmza•11mo ago

i faced same thing a little while ago so i came up with this https://github.com/hamzasaleem2/convex-tanstack-table/blob/main/src/useSimplePaginatedQuery.ts its just a wrap over useQuery but did the job for me for offsets it gives you this hook to use:

const {
  status,
  currentResults,
  loadNext,
  loadPrev,
  currentPageNum,
  setPageSize,
  pageSize
} = useSimplePaginatedQuery(myQuery, initialArgs, { initialNumItems: 10 });

const {
  status,
  currentResults,
  loadNext,
  loadPrev,
  currentPageNum,
  setPageSize,
  pageSize
} = useSimplePaginatedQuery(myQuery, initialArgs, { initialNumItems: 10 });

tanstack table demo: https://convex-tanstack-table.vercel.app/ i didn't worked out alot of things in this which can be better. but you are welcome to try and happy to take a feedback on this too.

GitHub

convex-tanstack-table/src/useSimplePaginatedQuery.ts at main · hamz...

Contribute to hamzasaleem2/convex-tanstack-table development by creating an account on GitHub.

Web Dev CodyOP•11mo ago

@Hmza would the query on the convex query just need to return a pagination object?

lee•11mo ago

^ that helper looks great and is the pattern i would recommend if you're okay with calling loadNext until you get to the page you want

Web Dev CodyOP•11mo ago

easiest approach I can think is just add an auto incrementing id to the records and then calculate the page / offset using that with a .take max page length?

lee•11mo ago

if auto incrementing id is feasible (no deletes, ordering by creation time) that ^ is equivalent to and better than going all-in on aggregates

Web Dev CodyOP•11mo ago

yeah I just want a list of blog posts by date order desc going to add this export const addAutoIncrementingIdToPosts = internalMutation({ args: {}, async handler(ctx) { const posts = await ctx.db.query("blogPosts").order("desc").collect(); await Promise.all( posts.map((post, index) => ctx.db.patch(post._id, { incrementingId: index + 1 }) ) ); }, }); run it, then just do a take(PAGE_SIZE)

lee•11mo ago

how will you calculate the index for new blog posts? (i'm asking because this looks like it won't work, unless the set of blog posts is fixed or you recalculate the incrementingId field whenever a new one is added) i.e. looks like this will generate incrementingId: 1 for the most recent blog post, incrementingId: 2 for the next, etc. so if you want to add a new blog post and keep the ordering, it would need to have incrementingId: 0

Web Dev CodyOP•11mo ago

sorry I've modified since posting

const PAGE_SIZE = 5;

export const getPosts = query({
  args: {
    page: v.number(),
  },
  handler: async (ctx, args) => {
    const startId = (args.page - 1) * PAGE_SIZE + 1;

    const page = await ctx.db
      .query("blogPosts")
      .withIndex("incrementingId_index", (q) =>
        q.gte("incrementingId", startId)
      )
      .take(PAGE_SIZE);

    return page;
  },
});

const PAGE_SIZE = 5;

export const getPosts = query({
  args: {
    page: v.number(),
  },
  handler: async (ctx, args) => {
    const startId = (args.page - 1) * PAGE_SIZE + 1;

    const page = await ctx.db
      .query("blogPosts")
      .withIndex("incrementingId_index", (q) =>
        q.gte("incrementingId", startId)
      )
      .take(PAGE_SIZE);

    return page;
  },
});

I could also probably just do order('desc') so that id 1 is the oldest I think or just don't use ids and instead use proper dates

lee•11mo ago

you're right; looks like it does work. carry on 🙂

ampp•4mo ago

Do you know of any other examples of this being used? I have been using it for the last month with shadcn tables on next js. I've had lots of issues and had to hack around things, like with a debounced search. I'm not even doing the page 1,2,3 thing, just been trying to get to a elegant table design using tanstack table. And i cant seem to get a happy path.

Hmza•4mo ago

Hey, can you elaborate on what kind of issues you are facing? If you are not doing pagination in numbers(1,2,3) and just using LoadMore function call to load page by page there shouldn't be any issues atleast with loading results.

ampp•4mo ago

It would take me a bit of time to explain all the issues but the most frustrating one right now is scroll to top anytime i press previous or next and the page is scrolled down. I've checked the usually suspects

erquhart•4mo ago

It sounds like either your table is remounting, an intermediate container within your table housing the rows is remounting, or an intermediate empty state is occurring, which wouldn't be from the query itself. How to troubleshoot depends on your implementation, but I would first look at how you're handling the loading state, if at all.

ampp•4mo ago

Yeah i've been in crunch mode for days, i had ai try to tackle it for a bit trying random things. That's why i was hoping for some other examples of this in use so i could just cross reference the differences between the forms. I'm always looking for something more complex than one i got but that seems rare 😅 @erquhart hmm the demo also has this remounting issue. https://convex-tanstack-table.vercel.app/

erquhart•4mo ago

Just looked at some reference implementations - this is normally how page based pagination works. Eg., google results, github issues - the whole page reloads and scrolls to top. I'm not saying what you're trying to do isn't possible, just pointing this out in case it makes this issue less of a blocker for you. But as for solving it, this is the kind of thing I meant by check the loading state: https://github.com/hamzasaleem2/convex-tanstack-table/blob/0b40b85dfaf2390dc5d7185fdf7db867212e3885/src/App.tsx#L72 Rendering is branched on loading state, it'll scroll up every time there's a loading event because the rendered table disappears. You'd have to continuously show the old data until the new data is available to do what you're trying to do.

ampp•4mo ago

Thanks for pointing this out, since i jumped to another project I've been trying to iterate on my tables so i need to go back and see. Between the frameworks, the tanstack query code, the tanstack table w/shadcn, and all the native paginated helpers in convex my brain is fried on tables, i thought this was generally expected. I do think this is something everyone suffers through or just lives with in various ways. It's probably low on the list of priorities but high on the list of friction if you attempted to analyze what was the perfect table in the convex ecosystem is.

erquhart•4mo ago

That's fair. Also tables are just hard in general. TanStack's tooling is the best I've seen, but... they're still tables, and they're still kinda hard.

ampp•3mo ago

I'm very lost, so i made a repoduction, i could hack this all the way to working but i only know the dirty hacky way, and i'd be left wondering the right way. I left it with many state issues To me these are minimal table features, Debounced Search using convex search Index, A filter field, a results per page, previous and next, and change sort order. https://github.com/amp127/real-convex-tanstack-table I'd be happy to tip anyone for taking this across the finish line. Interested @Hmza ?

GitHub

GitHub - amp127/real-convex-tanstack-table

Contribute to amp127/real-convex-tanstack-table development by creating an account on GitHub.

ampp•3mo ago

I asked chef for basically the same thing, it did it all with useQuery() https://chef.show/dee178

Chef

Calendar App with Google Sync

Cooked with Chef by amp127

erquhart•3mo ago

Like the implementation doesn’t have the issues you’ve been encountering? Can’t tell if “just useQuery” is good or bad lol

ampp•3mo ago

Exactly, its probably not useful for a table that is refreshing as that was why the team made useStablePaginatedQuery for that temporary undefined state. But i don't see that happening.. (i added a buttons for quick adding) so im a bit more confused. i think ill upload my various iterations of these things to that github

ampp•3mo ago

Generated a full comparisons system https://github.com/amp127/convex-pagination-demo

GitHub

GitHub - amp127/convex-pagination-demo

Contribute to amp127/convex-pagination-demo development by creating an account on GitHub.

erquhart•3mo ago

Nice!! Even the anti flicker one still flickers though so your status is none of these actually fix your problem, is that right? yeah it's explicitly not rendering the table on every page load: https://github.com/amp127/convex-pagination-demo/blob/c7919a8cdc947d3268d14010bc7c1b940c5895ac/src/app/companies/stable-native/page.tsx#L444

ampp•3mo ago

My status is: I don't know if any of them are working as expected, i'd like help. The flicker is the least of my concerns right now, but my computer is so fast that it makes it appear as if its not (almost). Reset to top of the page is the same? but is that just life? Is tanstack table part of the issue. Simple Paginated Query: to make it work is hacky as shit, with useEffects or setting other variables to reset state and i would like help I'll probably switch to RJ's Next/Prev as ai can at least not screw it up. Am i missing any other ways. It would be nice to add infinate scroll demos and use some of the newer features for optiomistic updates.

erquhart•3mo ago

Resetting to top is normal for previous page / next page. For the infinite scroll examples, you're explicitly rendering a loading screen every time they press load more, so it unmounts and remounts scrolled to the top. I use regular usePaginatedQuery in production for infinite scroll and it works great, any of these approaches should be fine. Just have to look at the code to troubleshoot specific issues that you're seeing. If you're using AI to write it all the agents may have difficulty getting it right. If you want more help with specific issues, let's start with one problem, and one implementation, and solve it from there. Feels like a bit of a moving target currently.

ampp•3mo ago

Yeah, it all started with Simple paginated query. I'd be super grateful if if you could look at it's debounced search and clear/reset, which should help resolve other issues on that page. I've tried so many things but something is always broken. I was surprised that ai one-shot working solutions the other 4 forms. My workflow is basically get one table working right then have ai duplicate it 10-15 times with small variations. The problem is garbage in garbage out. Especially when it tries to fix "working code" as it sees a linter message. Makes me want to abandon the simple hook.

Understanding usePaginatedQuery

Did you find this page helpful?