fakesilent•2d ago

Ordered listings created within the last 6 days gives too many bytes read error

export const getTopRatedListings = internalQuery({
  handler: async (ctx) => {
    const sixDaysAgo = Date.now() - 6 * 24 * 60 * 60 * 1000;

    return ctx.db
      .query("listings")
      .withIndex("by_cashflow")
      .order("desc")
      .filter((q) => q.gte(q.field("_creationTime"), sixDaysAgo))
      .take(1);
  },
});

export const getTopRatedListings = internalQuery({
  handler: async (ctx) => {
    const sixDaysAgo = Date.now() - 6 * 24 * 60 * 60 * 1000;

    return ctx.db
      .query("listings")
      .withIndex("by_cashflow")
      .order("desc")
      .filter((q) => q.gte(q.field("_creationTime"), sixDaysAgo))
      .take(1);
  },
});

schema

    ...,
    monthlyCashflow: v.optional(v.number()),
  })
    .index("by_cashflow", ["monthlyCashflow"])

    ...,
    monthlyCashflow: v.optional(v.number()),
  })
    .index("by_cashflow", ["monthlyCashflow"])

This is giving me the following too many bytes read error:

Error: [CONVEX Q(fetchListings:getTopRatedListings)] [Request ID: 49905af9f0732044] Server Error
Uncaught Error: Too many bytes read in a single function execution (limit: 16777216 bytes). Consider using smaller limits in your queries, paginating your queries, or using indexed queries with a selective index range expressions.

Error: [CONVEX Q(fetchListings:getTopRatedListings)] [Request ID: 49905af9f0732044] Server Error
Uncaught Error: Too many bytes read in a single function execution (limit: 16777216 bytes). Consider using smaller limits in your queries, paginating your queries, or using indexed queries with a selective index range expressions.

I am genuinely puzzled on how to do this without running into this error. ~6k documents

6 Replies

Convex Bot•2d ago

Thanks for posting in <#1088161997662724167>. Reminder: If you have a Convex Pro account, use the Convex Dashboard to file support tickets. - Provide context: What are you trying to achieve, what is the end-user interaction, what are you seeing? (full error message, command output, etc.) - Use search.convex.dev to search Docs, Stack, and Discord all at once. - Additionally, you can post your questions in the Convex Community's <#1228095053885476985> channel to receive a response from AI. - Avoid tagging staff unless specifically instructed. Thank you!

Clever Tagline•this hour

Your withIndex syntax is incomplete. You're not using it to make any comparison using the cashflow field (guessing the field name from the index name), so it's effectively pushing the full table read through to the order step. Review the docs if needed: https://docs.convex.dev/database/reading-data/indexes/#querying-documents-using-indexes

Indexes | Convex Developer Hub

Speed up queries with database indexes

fakesilentOP•this hour

I wouldn't like to make a comparison on the cashflow field, I am merely using the index for sorting. I just need to get highest cashflow listings from the last 6 days.

Clever Tagline•12h ago

Gotcha. I also overlooked that you'd included the table index in your original post, so I didn't need to guess the field name. 😂 While the index does help with sorting, by not applying any criteria it's going to return the entire table. Instead of doing the filter outside the index, do it in the index and remove the .filter() section. Per the docs, you'll need to first make at least one .eq() comparison, and then you can do the date comparison with the _creationTime field (that field is included in every index by default). I feel that this might yield better results than filtering outside the index, though I could be mistaken.

fakesilentOP•8h ago

Well that is kind of the issue haha, i need to do one eq but it has to be on the monthlyCashflow field which is just not possible because that is what I am trying to sort. It seems so weird to not be able to do this normally. Granted of course I have wide documents but I don't mind if it takes longer, I just want to be able to do it lol. The docs mention if you run into this limit, to use a index, but that is precisely what I am using... very frustrating.

Clever Tagline•8h ago

Yeah, I hear you. I'm still not familiar enough with the nuances of complex queries to wrap my head around how else to accomplish this. And I just dug deeper into the index part of the docs and saw some examples following the pattern you used (.withIndex but no range expression), which I somehow thought was less efficient, but there are some use cases mentioned there (though they aren't clicking as readily as I'd hoped).

Ordered listings created within the last 6 days gives too many bytes read error

Did you find this page helpful?