Billzabob Posts - Convex Community

Billzabob

•Created by Billzabob on 5/16/2025 in #support-community

Paginating Stream Not Working with "Join"

I'm paginating through a convex-helpers stream which I'm using to do a "join" and getting strange behavior where the stream ends early. I'm trying to join both the userWants and userHaves table on their userId and paginate through the results. Here is my query:

export const matchingTrades = query({
  args: {
    paginationOpts: paginationOptsValidator,
  },
  handler: async (ctx, args) => {
    const me = await getCurrentUser(ctx);

    const myWants = stream(ctx.db, schema)
      .query('userWants')
      .withIndex('by_user', (q) => q.eq('userId', me._id));

    const myTrades = myWants.flatMap(
      async (myWant) => {
        const myHaves = stream(ctx.db, schema)
          .query('userHaves')
          .withIndex('by_user', (q) => q.eq('userId', me._id));

        return myHaves.map(async (myHave) => {
          return {
            userId: me.username,
            want: myWant.cardName,
            have: myHave.cardName,
          };
        });
      },
      ['userId', 'cardId']
    );

    return myTrades.paginate(args.paginationOpts);
  },
});

export const matchingTrades = query({
  args: {
    paginationOpts: paginationOptsValidator,
  },
  handler: async (ctx, args) => {
    const me = await getCurrentUser(ctx);

    const myWants = stream(ctx.db, schema)
      .query('userWants')
      .withIndex('by_user', (q) => q.eq('userId', me._id));

    const myTrades = myWants.flatMap(
      async (myWant) => {
        const myHaves = stream(ctx.db, schema)
          .query('userHaves')
          .withIndex('by_user', (q) => q.eq('userId', me._id));

        return myHaves.map(async (myHave) => {
          return {
            userId: me.username,
            want: myWant.cardName,
            have: myHave.cardName,
          };
        });
      },
      ['userId', 'cardId']
    );

    return myTrades.paginate(args.paginationOpts);
  },
});

And here is the relavant part of the schema:

  userWants: defineTable({
    userId: v.id('users'),
    cardId: v.id('cards'),
    cardName: v.optional(v.string()),
    rarityCode: v.string(),
  })
    .index('by_user', ['userId', 'cardId']),

  userHaves: defineTable({
    userId: v.id('users'),
    cardId: v.id('cards'),
    cardName: v.optional(v.string()),
    rarityCode: v.string(),
  })
    .index('by_user', ['userId', 'cardId']),

  userWants: defineTable({
    userId: v.id('users'),
    cardId: v.id('cards'),
    cardName: v.optional(v.string()),
    rarityCode: v.string(),
  })
    .index('by_user', ['userId', 'cardId']),

  userHaves: defineTable({
    userId: v.id('users'),
    cardId: v.id('cards'),
    cardName: v.optional(v.string()),
    rarityCode: v.string(),
  })
    .index('by_user', ['userId', 'cardId']),

I'm testing with 3 entries in each table all with the same userId. If I use collect() I get the expected count of 3 x 3 = 9. If I use paginate() with an initialNumItems of 9 or more, it works and loads them all. If initialNumItems is 1 and I keep calling loadMore(1) it incorrectly terminates the stream after 3 items. If initialNumItems is 4 and I keep calling loadMore(1) it incorrectly terminates the stream after 6 items. So something with the cursor gets confused at multiples of the flatMap size.

2 replies