Can I skip some filter field in withIndex

Guys I have question about withIndex schema:

profile: defineTable(profile).index("by_filter", ["gender", "age"])

profile: defineTable(profile).index("by_filter", ["gender", "age"])

query:

const gender = "male"
const minAge = 18
const profile = await ctx.db.query("profiles")
     .withIndex("by_filter", q => 
       q.eq("gender", gender)
       q.lte("age", minAge)
     )

const gender = "male"
const minAge = 18
const profile = await ctx.db.query("profiles")
     .withIndex("by_filter", q => 
       q.eq("gender", gender)
       q.lte("age", minAge)
     )

but if gender is "everyone" seem withIndex from "by_filter" is force 1st condition should check "gender" before "age" how can I skip that this example have only 2 fields but, my filter have about 8 fields, can I skip some field via filter with withIndex thank you

5 Replies

Convex Bot•6d ago

Thanks for posting in <#1088161997662724167>. Reminder: If you have a Convex Pro account, use the Convex Dashboard to file support tickets. - Provide context: What are you trying to achieve, what is the end-user interaction, what are you seeing? (full error message, command output, etc.) - Use search.convex.dev to search Docs, Stack, and Discord all at once. - Additionally, you can post your questions in the Convex Community's <#1228095053885476985> channel to receive a response from AI. - Avoid tagging staff unless specifically instructed. Thank you!

Clever Tagline•5d ago

Just echoing the link to the Stack article about the streams helper that I shared in #general yesterday. If you feel that this will address your use case, please mark this thread as resolved. https://stack.convex.dev/merging-streams-of-convex-data

Merging Streams of Convex data

New convex-helpers are available now for fetching streams of documents, merging them together, filtering them them out, and paginating the results. Wi...

Sikarin (Afu)OP•5d ago

Thank you. I already try it. but I just found another way It's work too, I'm not sure it's best way or not but seem 100k records filter very fast (I'm happy now) for .withIndex("by_filter") take around 100 - 200 ms => item per page is 50 for .withIndex("by_filter") take around 20 - 40 ms => item per page is 10 for without withIndex take 1-2 sec. it's is complex query because dating app need to filter geo too, I found convex components but seem can not query with complex. here how I work schema

export default defineSchema({
  profiles: defineTable(profiles)
    .index("by_clerkId", ["clerkId"])
    .index("by_filter", [
      "isReady",
      "gender",
      "birthDateTimestamp",
      "sexualOrientation",
      "tall",
      "hereToWhat",
      "drink",
      "smoke",
      "pet",
    ]),
})

export default defineSchema({
  profiles: defineTable(profiles)
    .index("by_clerkId", ["clerkId"])
    .index("by_filter", [
      "isReady",
      "gender",
      "birthDateTimestamp",
      "sexualOrientation",
      "tall",
      "hereToWhat",
      "drink",
      "smoke",
      "pet",
    ]),
})

query code

Sikarin (Afu)OP•5d ago

Clever Tagline•5d ago

Glad that you found a solution, though frankly I'm not sure how efficient it is. If you use withIndex without a range expression (the second argument), it performs a full table scan, which is exactly the thing that the index is designed to avoid. (Check the yellow warning box just below this header in the docs ) Full table scans are expensive because they increase your read amount dramatically, especially with large tables. Check your stats in the dashboard to see how that approach is affecting your usage.

Indexes | Convex Developer Hub

Speed up queries with database indexes

Can I skip some filter field in withIndex

Did you find this page helpful?