zoomie•10mo ago

Select specific db columns on server (bytes execution limit)

The following code in convex/lead.ts:

const leads = await ctx.db.query("lead").collect();
const mapLeads = leads.map((lead) => {
  return {_id: lead._id, latitude: lead.latitude,longitude: lead.longitude,};
});

const leads = await ctx.db.query("lead").collect();
const mapLeads = leads.map((lead) => {
  return {_id: lead._id, latitude: lead.latitude,longitude: lead.longitude,};
});

Uncaught Error: Too many bytes read in a single function execution (limit: 8388608 bytes). Consider using smaller limits in your queries, paginating your queries, or using indexed queries with a selective index range expressions.

Uncaught Error: Too many bytes read in a single function execution (limit: 8388608 bytes). Consider using smaller limits in your queries, paginating your queries, or using indexed queries with a selective index range expressions.

Is it possible to select only the columns (fields) I want in the db query? An SQL analogy would be:

SELECT _id, latitude, longitude FROM lead

SELECT _id, latitude, longitude FROM lead

7 Replies

Michal Srb•10mo ago

It's not possible. Given your other question, I wonder what the data looks like in your documents, and whether it would make sense to model it (normalize it) better before storing in the database?

zoomieOP•10mo ago

Yes, I could do that. I would prefer to not specify too much of the data's structure (i.e. normalise), as we haven't solidified on a schema/solution. With the validation off, I could pull apart the locations from the rest of the data (not sure if that counts as normalisation as it's 1 to 1) as that seems like the only way.

ian•10mo ago

Does pagination work for your usecase? That'd be the most scalable approach

zoomieOP•10mo ago

Unfortunately pagination won't work, I need all the rows, but only two columns.

Michal Srb•10mo ago

(You can always accumulate the data from pagination on the client - but given you're running into this error, it will be a lot of data)

ian•10mo ago

If it's run in a one-off context, you could paginate from an action and accumulate the results there before returning it all at once. I'm curious what usecase can't use pagination

zoomieOP•10mo ago

Oh of course; as we're using a map, I assumed you meant paginate as the user moves around. But you're 100% right, I could accumulate the results with multiple calls on page load.

Select specific db columns on server (bytes execution limit)

Did you find this page helpful?