Riki•7d ago

.paginate combined with .withIndex is causing an error

let say I have two tables in my scheme that represents a user that lives in a city and posts that are made in a city. All the code below could be better arranged, but this is for the simplicity of the example:

const userTable = defineTable({
  name: v.string()
  cityId: v.string()
})

const postTable = defineTable({
message: v.string()
cityId: v.string()
})

const userTable = defineTable({
  name: v.string()
  cityId: v.string()
})

const postTable = defineTable({
message: v.string()
cityId: v.string()
})

Let's say I want to display all the posts from the city of the user, so this is how I achieve it:

export const getPostForUser = query({
args: {
  userId: v.id("user")
  paginationOpts: paginationOptsValidator
},
handler: async (ctx, args) => {
  const user = await ctx.db.get(args.userId)
  if (user == null) { throw new ConvexError("Error") }
  const storiesByUsers = await ctx.db
    .query('post')
    .withIndex('by_cityId', q => q.eq('cityId', user.cityId))
    .paginate(args.paginationOpts);
}
})

export const getPostForUser = query({
args: {
  userId: v.id("user")
  paginationOpts: paginationOptsValidator
},
handler: async (ctx, args) => {
  const user = await ctx.db.get(args.userId)
  if (user == null) { throw new ConvexError("Error") }
  const storiesByUsers = await ctx.db
    .query('post')
    .withIndex('by_cityId', q => q.eq('cityId', user.cityId))
    .paginate(args.paginationOpts);
}
})

The problem is that when a user change his city by patching his data, the getPostForUser function returns an error:

Uncaught Error: InvalidCursor: Tried to run a query starting from a cursor, but it looks like this cursor is from a different query.

I think this happens because the args haven't changed but the result has, given it has been retriggered by the inner data change from the user table. A way to circumvent this would be to instead pass the city of the user as an argument of the function, so the query result changes when the args change. However, I would like to improve the ergonomics here: I feel that the code above looks good and it is unfortunate not being able to rely on a table for a paginated query. I don't think it makes sense this function returns an error conceptually.

19 Replies

Convex Bot•7d ago

Thanks for posting in <#1088161997662724167>. Reminder: If you have a Convex Pro account, use the Convex Dashboard to file support tickets. - Provide context: What are you trying to achieve, what is the end-user interaction, what are you seeing? (full error message, command output, etc.) - Use search.convex.dev to search Docs, Stack, and Discord all at once. - Additionally, you can post your questions in the Convex Community's <#1228095053885476985> channel to receive a response from AI. - Avoid tagging staff unless specifically instructed. Thank you!

erquhart•7d ago

I see this message a lot but the client handles it - is this crashing for you? I thought the convex client was catching and handling but not actually sure

RikiOP•7d ago

Oh yes definitely throwing an error, and so, caught by our error boundary screen

mikeysee•7d ago

Hi Pierre I personally havent encountered this but I saw something similar mentioned recently by @lee on our Slack:

this would happen if you call .paginate in two different branches. it doesn't know how to continue the pagination like if you do if (Math.random() < 0.5) { return await ctx.db.query("messages").paginate(opts); } else { return await ctx.db.query("users").paginate(opts); } then the cursors would get confused, because a cursor into the messages table can't be used in the users table

Im a little confused however as it seems like you arent branching in your query here so it should work 🤔

lee•7d ago

Are you using usePaginatedQuery? That hook should catch InvalidCursor errors so they wouldn't hit the error boundary. Also can you share the full query? The code you posted doesn't have a return and i'm curious if there's a try/catch in there that might be affecting how the error is passed through

RikiOP•7d ago

@mikeysee Thanks for your message! I confirm I am using only one paginated query in the code. @lee yes indeed I am using usePaginatedQuery. Here is the code that I have simplified a bit but that's still triggering the error, sorry if it is still a bit long.

export const getAllV2 = query({
  args: {
    paginationOpts: paginationOptsValidator,
  },
  handler: async (ctx, args) => {
    const {authId} = await ensureAuthenticatedV2(ctx);

    const me = await ctx.db
      .query('user')
      .withIndex('authId', q => q.eq('authId', authId))
      .unique();
    if (me == null) {
      throw new ConvexError(ApiError.RESOURCE_NOT_FOUND);
    }

    const storiesByUsers = await ctx.db
      .query('vHomeStory')
      .withIndex('by_communityId', q => q.eq('communityId', me.communityId))
      .paginate(args.paginationOpts);

    const storiesWithImage = await asyncMapWitoutNull(
      storiesByUsers.page,
      async storiesByUser => {
        const user = await ctx.db.get(storiesByUser.userId);

        if (user == null) {
          return;
        }

        const lastStories = await asyncMapWitoutNull(
          storiesByUser.lastStories,
          async lastStory => {
            return {
              creationTime: 12340123,
              storyId: lastStory,
              userId: storiesByUser.userId,
              kind: 'IMAGE',
              image: '',
              duration: 100,
              groupTitle: 'some_title',
              groupId: 'some_id',
              isLikedByMe: false,
            };
          },
        );

        return {
          userId: storiesByUser.userId,
          lastStoryTimestamp: storiesByUser.lastStoryTimestamp,
          lastStories: lastStories,
          imageUrl: user.imageUrls[0],
          name: user.name,
          authId: user.authId,
          canMessage: true,
          canLike: true,
          canShareToSocial: true,
        };
      },
    );

    return {
      ...storiesByUsers,
      page: storiesWithImage,
    };
  },
});

export const getAllV2 = query({
  args: {
    paginationOpts: paginationOptsValidator,
  },
  handler: async (ctx, args) => {
    const {authId} = await ensureAuthenticatedV2(ctx);

    const me = await ctx.db
      .query('user')
      .withIndex('authId', q => q.eq('authId', authId))
      .unique();
    if (me == null) {
      throw new ConvexError(ApiError.RESOURCE_NOT_FOUND);
    }

    const storiesByUsers = await ctx.db
      .query('vHomeStory')
      .withIndex('by_communityId', q => q.eq('communityId', me.communityId))
      .paginate(args.paginationOpts);

    const storiesWithImage = await asyncMapWitoutNull(
      storiesByUsers.page,
      async storiesByUser => {
        const user = await ctx.db.get(storiesByUser.userId);

        if (user == null) {
          return;
        }

        const lastStories = await asyncMapWitoutNull(
          storiesByUser.lastStories,
          async lastStory => {
            return {
              creationTime: 12340123,
              storyId: lastStory,
              userId: storiesByUser.userId,
              kind: 'IMAGE',
              image: '',
              duration: 100,
              groupTitle: 'some_title',
              groupId: 'some_id',
              isLikedByMe: false,
            };
          },
        );

        return {
          userId: storiesByUser.userId,
          lastStoryTimestamp: storiesByUser.lastStoryTimestamp,
          lastStories: lastStories,
          imageUrl: user.imageUrls[0],
          name: user.name,
          authId: user.authId,
          canMessage: true,
          canLike: true,
          canShareToSocial: true,
        };
      },
    );

    return {
      ...storiesByUsers,
      page: storiesWithImage,
    };
  },
});

RikiOP•7d ago

And when I change my community id (communityId attribute set up in a user entity) , this is what I receive:

RikiOP•7d ago

@lee At least in React-Native an error thrown by the usePaginatedQuery hook needs an error boundary when connected to a prod convex environment (on dev we just have a warning, and it looks like it is handled by the hook itself which "restarts" a paginated query)

erquhart•7d ago

Really seems like something else is at play here. I have the same approach, paginated queries with user adjustable args, in a React Native app, usePaginatedQuery handles this well. Hmm that error about a different query is not the one I'd expect though Nah I guess InvalidCursor is the one: https://github.com/get-convex/convex-auth/issues/53#issuecomment-2536298464 Anything interesting going on in the client code that calls this?

RikiOP•7d ago

@erquhart I think you shared a wrong link? it seems this talk about convex website urls

RikiOP•7d ago

@erquhart that is just very basic client code: A paginated query function and a flatlist

RikiOP•7d ago

So you have in your codebase some paginated queries that rely on some internal query data and not "outside" query arguments?

erquhart•7d ago

oh yeah that's the wrong link

erquhart•7d ago

Here's where InvalidCursor is handled, it resets pagination state when encountered: https://github.com/get-convex/convex-js/blob/14f9d318f594e5e63e5a7513b6be28643560b62c/src/react/use_paginated_query.ts#L249-L267

GitHub

convex-js/src/react/use_paginated_query.ts at 14f9d318f594e5e63e5a7...

TypeScript/JavaScript client library for Convex. Contribute to get-convex/convex-js development by creating an account on GitHub.

erquhart•7d ago

Oh interesting it sounds like this is working as expected in dev but throwing in prod missed that part

RikiOP•7d ago

Oh lovely, thanks for the link I guess that's an unwanted side effect? I imagine they wanted here to bubble up the error in case it wasn't the InvalidCursor one given that should be the only acceptable error for paginated queries that the convex package can handle gracefully? But given if I'm not mistaken, the convex prod doesn't receive error messages, it is not able to find the "InvalidCursor" string and so falls back to bubbling up the error.

erquhart•6d ago

oohh yeah that makes sense But I've never had to do that in my production usage... I wonder how they're working around error obfuscation

lee•6d ago

agreed i think this may be a problem in prod because of the error obfuscation yep i found some internal discussion about this bug. turns out it's a known bug. i'll re-raise it internally

RikiOP•6d ago

Awesome, thanks @lee !!