Jordan Comments - Convex Community

Jordan

•Created by Jordan on 1/23/2025 in #support-community

filter index based on multiple possible values?

As long as you're aware that the function walks the requests for the user, and only stops when it finds enough that fit the filter criteria.

yes, i think i follow.

 return await ctx.db
   .query("requests")
   .withIndex("by_userId", (q) => q.eq("userId", user._id))
   .filter(createStatusCallback(args.statuses))
   .paginate(args.paginationOpts);

 return await ctx.db
   .query("requests")
   .withIndex("by_userId", (q) => q.eq("userId", user._id))
   .filter(createStatusCallback(args.statuses))
   .paginate(args.paginationOpts);

for this above query i expect it do the following: 1. lets assume i am querying for 10 initial items with our paginate 2. using the by_userid index on requests table, we get all requests by a given user (this is performant due to index) 3. then we iterate or "walk" through all the requests of that given user only to match the filter until we get the correct number of results to return for pagination 4. once we have the correct number of results for the pagination, then we STOP walking the remainder of the user requests I wanted to confirm that (4) is accurate and how the cursor pagination is working. if so, the above is good enough performance IMO

15 replies

CCConvex Community

•Created by Jordan on 1/23/2025 in #support-community

filter index based on multiple possible values?

FWIW, this is the approach i will likely take for now:

 return await ctx.db
   .query("requests")
   .withIndex("by_userId", (q) => q.eq("userId", user._id))
   .filter(createStatusCallback(args.statuses))
   .paginate(args.paginationOpts);

 return await ctx.db
   .query("requests")
   .withIndex("by_userId", (q) => q.eq("userId", user._id))
   .filter(createStatusCallback(args.statuses))
   .paginate(args.paginationOpts);

function createStatusCallback(statuses: string[]) {
  return (q: any) =>
    q.or(...statuses.map((status) => q.eq(q.field("status"), status)));
}

function createStatusCallback(statuses: string[]) {
  return (q: any) =>
    q.or(...statuses.map((status) => q.eq(q.field("status"), status)));
}

I know convex docs recommend to avoid using filter however, the above is the only way i know to combine indexes/filter/pagination. the by_userId index at least helps with performance some, but then .filter allows us to filter out based on args.statuses. and then we still get pagination after all of that.

15 replies

CCConvex Community

•Created by Jordan on 1/23/2025 in #support-community

filter index based on multiple possible values?

oh i see now after trying to chain multiple .eq() in the callback of .withIndex() that thats not allowed since you can only have one equality check per field convex has been a helpful technology, but combining dynamic filters + indexing for performance + pagination is the biggest pain point I'm struggling to solve. there are solutions to solve each individually, but no good solution when wanting to use a combination of them together. i feel like its common logic on an application to fetch given a data-set index with user defined filters and paginate the results. curious if others have examples of how they implement this

15 replies

CCConvex Community

•Created by Jordan on 1/23/2025 in #support-community

filter index based on multiple possible values?

moreover, this pattern wouldn't pair well with pagination either, which is sort of a deal breaker for my given implementation

15 replies

CCConvex Community

•Created by Jordan on 1/23/2025 in #support-community

filter index based on multiple possible values?

lets say i knew the status array, how might these two methods differ in performance? I feel like if the given index is still large, then re-querying it n-times over would still be concerning option 1:

const userPosts = await ctx.db
      .query("posts")
      .withIndex("by_userId_status", (q) => q.eq("userId", user?._id).eq('status', 'pending').eq('status', 'in_progress))
      .collect();

const userPosts = await ctx.db
      .query("posts")
      .withIndex("by_userId_status", (q) => q.eq("userId", user?._id).eq('status', 'pending').eq('status', 'in_progress))
      .collect();

option 2:

const userPosts = ['status', 'in_progress'];
for (const status of args.status) {
  const userPostsWithStatus = await ctx.db
      .query("posts")
      .withIndex("by_userId_status", (q) => q.eq("userId", user?._id).eq('status', status))
      .collect();
  userPosts.push(...userPostsWithStatus);
}

const userPosts = ['status', 'in_progress'];
for (const status of args.status) {
  const userPostsWithStatus = await ctx.db
      .query("posts")
      .withIndex("by_userId_status", (q) => q.eq("userId", user?._id).eq('status', status))
      .collect();
  userPosts.push(...userPostsWithStatus);
}

15 replies

CCConvex Community

•Created by Jordan on 12/7/2024 in #support-community

Any idea why `NEXT_PUBLIC_CONVEX_URL` is required in a vercel deployment of using convex auth?

no it is not referenced in my codebase. i reference it as NEXT_PUBLIC_CONVEX_CLOUD_URL instead.

10 replies