Select specific db columns on server (bytes execution limit)

The following code in convex/lead.ts:

const leads = await ctx.db.query("lead").collect();
const mapLeads = leads.map((lead) => {
  return {_id: lead._id, latitude: lead.latitude,longitude: lead.longitude,};
});

const leads = await ctx.db.query("lead").collect();
const mapLeads = leads.map((lead) => {
  return {_id: lead._id, latitude: lead.latitude,longitude: lead.longitude,};
});

Uncaught Error: Too many bytes read in a single function execution (limit: 8388608 bytes). Consider using smaller limits in your queries, paginating your queries, or using indexed queries with a selective index range expressions.

Uncaught Error: Too many bytes read in a single function execution (limit: 8388608 bytes). Consider using smaller limits in your queries, paginating your queries, or using indexed queries with a selective index range expressions.

Is it possible to select only the columns (fields) I want in the db query? An SQL analogy would be:

SELECT _id, latitude, longitude FROM lead

SELECT _id, latitude, longitude FROM lead

7 Replies

Michal Srb•2y ago

It's not possible. Given your other question, I wonder what the data looks like in your documents, and whether it would make sense to model it (normalize it) better before storing in the database?

zoomie | TuloOP•2y ago

Yes, I could do that. I would prefer to not specify too much of the data's structure (i.e. normalise), as we haven't solidified on a schema/solution. With the validation off, I could pull apart the locations from the rest of the data (not sure if that counts as normalisation as it's 1 to 1) as that seems like the only way.

ian•2y ago

Does pagination work for your usecase? That'd be the most scalable approach

zoomie | TuloOP•2y ago

Unfortunately pagination won't work, I need all the rows, but only two columns.

Michal Srb•2y ago

(You can always accumulate the data from pagination on the client - but given you're running into this error, it will be a lot of data)

ian•2y ago

If it's run in a one-off context, you could paginate from an action and accumulate the results there before returning it all at once. I'm curious what usecase can't use pagination

zoomie | TuloOP•2y ago

Oh of course; as we're using a map, I assumed you meant paginate as the user moves around. But you're 100% right, I could accumulate the results with multiple calls on page load.

Select specific db columns on server (bytes execution limit)

Did you find this page helpful?