Nishil Faldu•2w ago

Combination of .paginate and getPage question

    const following = await ctx.db
      .query('follows')
      .withIndex('uniqueRelationship', (q) => q.eq('followerType', 'user')
      .eq('followerId', ctx.user._id))
      .paginate(paginationOpts);

    if (following.page.length === 0) {
      return {
        page: [],
        isDone: true,
        continueCursor: undefined
      };
    }

    const discussions = [];
    for (const follow of following.page) {
      const { page: discussionsPage } = await getPage(ctx, 
        {
          table: 'discussions', 
          index: 'byUserOrgUniversity', 
          startIndexKey: [follow.followeeId], 
          endIndexKey: [follow.followeeId], 
          absoluteMaxRows: 10
        });

      discussions.push(...discussionsPage);
    }

    const following = await ctx.db
      .query('follows')
      .withIndex('uniqueRelationship', (q) => q.eq('followerType', 'user')
      .eq('followerId', ctx.user._id))
      .paginate(paginationOpts);

    if (following.page.length === 0) {
      return {
        page: [],
        isDone: true,
        continueCursor: undefined
      };
    }

    const discussions = [];
    for (const follow of following.page) {
      const { page: discussionsPage } = await getPage(ctx, 
        {
          table: 'discussions', 
          index: 'byUserOrgUniversity', 
          startIndexKey: [follow.followeeId], 
          endIndexKey: [follow.followeeId], 
          absoluteMaxRows: 10
        });

      discussions.push(...discussionsPage);
    }

I have this piece of code where in I want to return discussion posts from a users' following - but I would want to return all pages of posts from a particular user and then move on to the next user i am following...how can that be done? I tried to read the blog and docs too, but I could not wrap my head around to actually doing it

5 Replies

Convex Bot•2w ago

Thanks for posting in <#1088161997662724167>. Reminder: If you have a Convex Pro account, use the Convex Dashboard to file support tickets. - Provide context: What are you trying to achieve, what is the end-user interaction, what are you seeing? (full error message, command output, etc.) - Use search.convex.dev to search Docs, Stack, and Discord all at once. - Additionally, you can post your questions in the Convex Community's <#1228095053885476985> channel to receive a response from AI. - Avoid tagging staff unless specifically instructed. Thank you!

lee•2w ago

you're right, this is a tricky pattern. getPage doesn't make it easy to do pagination across multiple tables. it may help by designing the cursor for the pagination, and then you can figure out how to fetch the next page starting at the cursor. it sounds like your desired cursor might be a tuple of an index key of uniqueRelationship followed by an index key of byUserOrgUniversity. that is, (guessing about your schema based on names) your results are ordered by followeeId and then org and then university and then discussions._creationTime, so a tuple of those defines a cursor in your pagination. then to get the next page you could do something like this

const page = [];

const following = ctx.db.query("follows").withIndex("uniqueRelationship", q=>q.eq("followerType", "user").eq("followerId", ctx.user._id).gte("followeeId", cursor[0]));
let newCursor = cursor;

for await (const follow of following) {
  const discussions = await getPage(ctx,
    {
      table: "discussions",
      index: "byUserOrgUniversity",
      schema,
      startIndexKey: follow.followeeId === cursor[0] ?
        [follow.followeeId, ...cursor[1]] 
        : [follow.followeeId],
      endIndexKey: [follow.followeeId],
      absoluteMaxRows: 100 - page.length,
    });
  page.push(...discussions.page);
  newCursor = [follow.followeeId, discussions.indexKeys[discussion.indexKeys.length-1]];
  if (discussion.hasMore || page.length >= 100) {
    break;
  }
}

const page = [];

const following = ctx.db.query("follows").withIndex("uniqueRelationship", q=>q.eq("followerType", "user").eq("followerId", ctx.user._id).gte("followeeId", cursor[0]));
let newCursor = cursor;

for await (const follow of following) {
  const discussions = await getPage(ctx,
    {
      table: "discussions",
      index: "byUserOrgUniversity",
      schema,
      startIndexKey: follow.followeeId === cursor[0] ?
        [follow.followeeId, ...cursor[1]] 
        : [follow.followeeId],
      endIndexKey: [follow.followeeId],
      absoluteMaxRows: 100 - page.length,
    });
  page.push(...discussions.page);
  newCursor = [follow.followeeId, discussions.indexKeys[discussion.indexKeys.length-1]];
  if (discussion.hasMore || page.length >= 100) {
    break;
  }
}

Nishil FalduOP•2w ago

thank you so much for answering. i am still trying to wrap my head around it. will come back to ask more questions, if any

Slewt•2w ago

Convex team should make a short youtube tutorial for this topic in general. The way supabase does short helpful youtube tutorials is something I would love for Convex. Our team is always struggling with pagination on queries that have to manually join between multiple tables.

Lucas Couto•2w ago

Yeah, we also always struggle with that and opened discord today exactly because of it. Wanted to see if I could find a clearer path for it, but couldn't find. I usually go with the batch loading approach, but I always get uncomfortable with it since it loads way more data than needed.

Combination of .paginate and getPage question

Did you find this page helpful?