KinKon•11mo ago

Querying Multiple Documents by IDs in Convex

I'm currently working on an application where I have a list of document IDs and I need to fetch their corresponding data from Convex. I am considering a few approaches and would appreciate some help on what is the best approach. 1. Is there a way to use the filter method to directly query for a set of IDs in one go, something akin to an IN clause in SQL? For example, can I pass an array of IDs to filter or use a combination of filter and or to achieve this? Or maybe with some other method? 2. If such an array-based filtering is not supported, would it be more efficient to: (a)Retrieve all documents and filter them on the client-side using JavaScript? () (b)Make separate queries for each ID? (c) Some other pattern

5 Replies

ian•11mo ago

If you have an array it sounds like this: https://stack.convex.dev/relationship-structures-let-s-talk-about-schemas#direct-references-simple And a helper to make this easy (in convex-helpers npm): https://stack.convex.dev/functional-relationships-helpers#direct-references-the-getall-helper

Database Relationship Helpers

Traverse database relationships in a readable, predictable, and debuggable way. Support for one-to-one, one-to-many, and many-to-many via utility func...

Relationship Structures: Let's Talk About Schemas

In this post we’ll look at some patterns for structuring relationships in the Convex database.

ian•11mo ago

You'd do this in a query on the server-side, so it fetches all documents in parallel, and is much more efficient than a filter. So this is (b) from your list, but in parallel

KinKonOP•11mo ago

Thanks for the info. This should work my use case. I'm going to give it a proper review and try to implement it I got it implemented. Actually I think I spoke to soon. I realize now that even though I'm providing different IDs, I'm actually getting back multiple results of the same Doc My code looks something like this

import { v } from "convex/values";
import { getAll } from "convex-helpers/server/relationships";

import { mutation, query } from "./_generated/server";

export const getPersonsByIds = query({
  args: { personIds: v.array(v.id("persons")) },
  handler: async (ctx, args) => {
    console.log(args.personIds);
    const details = await getAll(ctx.db, args.personIds); 
    console.log(details, "details");
    return details;
  },
});

import { v } from "convex/values";
import { getAll } from "convex-helpers/server/relationships";

import { mutation, query } from "./_generated/server";

export const getPersonsByIds = query({
  args: { personIds: v.array(v.id("persons")) },
  handler: async (ctx, args) => {
    console.log(args.personIds);
    const details = await getAll(ctx.db, args.personIds); 
    console.log(details, "details");
    return details;
  },
});

Meanwhile the logs show that we are getting the same document back, multiple times

[ 'kd79kcvw4qyp9maw6yet8qv7yd6jw42b', 'kd79kcvw4qyp9maw6yet8qv7yd6jw42be' ] 'args.personIds'
log
[
  {
    _creationTime: 1707091577934.881,
    _id: 'kd79kcvw4qyp9maw6yet8qv7yd6jw42b',
    address: 'Jeas, Former 1 way, California',
    dob: '2024-02-09',
    email: 'sample@gmail.com',
    name: 'Michael Reevesdd',
    phoneNumber: '1234567890',
    updatedAt: '2024-02-05T00:06:17.939Z',
    userId: 'user_2YNF6WtiGxiSlhmCd9b0cha5Ffw'
  },
  {
    _creationTime: 1707091577934.881,
    _id: 'kd79kcvw4qyp9maw6yet8qv7yd6jw42b',
    address: 'Jeas, Former 1 way, California',
    dob: '2024-02-09',
    email: 'sample@gmail.com',
    name: 'Michael Reevesdd',
    phoneNumber: '1234567890',
    updatedAt: '2024-02-05T00:06:17.939Z',
    userId: 'user_2YNF6WtiGxiSlhmCd9b0cha5Ffw'
  }
] 'details'

[ 'kd79kcvw4qyp9maw6yet8qv7yd6jw42b', 'kd79kcvw4qyp9maw6yet8qv7yd6jw42be' ] 'args.personIds'
log
[
  {
    _creationTime: 1707091577934.881,
    _id: 'kd79kcvw4qyp9maw6yet8qv7yd6jw42b',
    address: 'Jeas, Former 1 way, California',
    dob: '2024-02-09',
    email: 'sample@gmail.com',
    name: 'Michael Reevesdd',
    phoneNumber: '1234567890',
    updatedAt: '2024-02-05T00:06:17.939Z',
    userId: 'user_2YNF6WtiGxiSlhmCd9b0cha5Ffw'
  },
  {
    _creationTime: 1707091577934.881,
    _id: 'kd79kcvw4qyp9maw6yet8qv7yd6jw42b',
    address: 'Jeas, Former 1 way, California',
    dob: '2024-02-09',
    email: 'sample@gmail.com',
    name: 'Michael Reevesdd',
    phoneNumber: '1234567890',
    updatedAt: '2024-02-05T00:06:17.939Z',
    userId: 'user_2YNF6WtiGxiSlhmCd9b0cha5Ffw'
  }
] 'details'

ian•11mo ago

Those look like the same ID but the second one has an extra 'e' - could it be that you accidentally added that e? Maybe we're accepting the ID and trimming the extra if you call ctx.normalizeId("persons", args.personIds[1]) does it give back kd79kcvw4qyp9maw6yet8qv7yd6jw42b out of curiosity? I'd be very curious where the extra 'e' came from

KinKonOP•11mo ago

Thanks for helping me debug Ian. It turns out I was relying on data that became corrupt days ago. My best guess is that extra 'e' was manually added from the dashboard, probably at a time where the field accepted strings as opposed to Ids. I didn't catch it because once I noticed the last character was different I assumed they were unique. When I call normalizeId with kd79kcvw4qyp9maw6yet8qv7yd6jw42be I do indeed get back kd79kcvw4qyp9maw6yet8qv7yd6jw42b. Same if I switch the trailing 'e' to 'a'. The moment I add more than one extra character I get back null

Querying Multiple Documents by IDs in Convex

Did you find this page helpful?