Billzabob•2d ago

Paginating Stream Not Working with "Join"

I'm paginating through a convex-helpers stream which I'm using to do a "join" and getting strange behavior where the stream ends early. I'm trying to join both the userWants and userHaves table on their userId and paginate through the results. Here is my query:

export const matchingTrades = query({
  args: {
    paginationOpts: paginationOptsValidator,
  },
  handler: async (ctx, args) => {
    const me = await getCurrentUser(ctx);

    const myWants = stream(ctx.db, schema)
      .query('userWants')
      .withIndex('by_user', (q) => q.eq('userId', me._id));

    const myTrades = myWants.flatMap(
      async (myWant) => {
        const myHaves = stream(ctx.db, schema)
          .query('userHaves')
          .withIndex('by_user', (q) => q.eq('userId', me._id));

        return myHaves.map(async (myHave) => {
          return {
            userId: me.username,
            want: myWant.cardName,
            have: myHave.cardName,
          };
        });
      },
      ['userId', 'cardId']
    );

    return myTrades.paginate(args.paginationOpts);
  },
});

export const matchingTrades = query({
  args: {
    paginationOpts: paginationOptsValidator,
  },
  handler: async (ctx, args) => {
    const me = await getCurrentUser(ctx);

    const myWants = stream(ctx.db, schema)
      .query('userWants')
      .withIndex('by_user', (q) => q.eq('userId', me._id));

    const myTrades = myWants.flatMap(
      async (myWant) => {
        const myHaves = stream(ctx.db, schema)
          .query('userHaves')
          .withIndex('by_user', (q) => q.eq('userId', me._id));

        return myHaves.map(async (myHave) => {
          return {
            userId: me.username,
            want: myWant.cardName,
            have: myHave.cardName,
          };
        });
      },
      ['userId', 'cardId']
    );

    return myTrades.paginate(args.paginationOpts);
  },
});

And here is the relavant part of the schema:

  userWants: defineTable({
    userId: v.id('users'),
    cardId: v.id('cards'),
    cardName: v.optional(v.string()),
    rarityCode: v.string(),
  })
    .index('by_user', ['userId', 'cardId']),

  userHaves: defineTable({
    userId: v.id('users'),
    cardId: v.id('cards'),
    cardName: v.optional(v.string()),
    rarityCode: v.string(),
  })
    .index('by_user', ['userId', 'cardId']),

  userWants: defineTable({
    userId: v.id('users'),
    cardId: v.id('cards'),
    cardName: v.optional(v.string()),
    rarityCode: v.string(),
  })
    .index('by_user', ['userId', 'cardId']),

  userHaves: defineTable({
    userId: v.id('users'),
    cardId: v.id('cards'),
    cardName: v.optional(v.string()),
    rarityCode: v.string(),
  })
    .index('by_user', ['userId', 'cardId']),

I'm testing with 3 entries in each table all with the same userId. If I use collect() I get the expected count of 3 x 3 = 9. If I use paginate() with an initialNumItems of 9 or more, it works and loads them all. If initialNumItems is 1 and I keep calling loadMore(1) it incorrectly terminates the stream after 3 items. If initialNumItems is 4 and I keep calling loadMore(1) it incorrectly terminates the stream after 6 items. So something with the cursor gets confused at multiples of the flatMap size.

1 Reply

Convex Bot•2d ago

Thanks for posting in <#1088161997662724167>. Reminder: If you have a Convex Pro account, use the Convex Dashboard to file support tickets. - Provide context: What are you trying to achieve, what is the end-user interaction, what are you seeing? (full error message, command output, etc.) - Use search.convex.dev to search Docs, Stack, and Discord all at once. - Additionally, you can post your questions in the Convex Community's <#1228095053885476985> channel to receive a response from AI. - Avoid tagging staff unless specifically instructed. Thank you!

Paginating Stream Not Working with "Join"

Did you find this page helpful?