Too many bytes read in a single function execution in migration
Hi, I am trying to run a migration on a table. When docs come in in the migrateOne function, i get the above error. How can I possibly deal with my docs if I can't load a single doc in a query?
2 Replies
Thanks for posting in <#1088161997662724167>.
Reminder: If you have a Convex Pro account, use the Convex Dashboard to file support tickets.
- Provide context: What are you trying to achieve, what is the end-user interaction, what are you seeing? (full error message, command output, etc.)
- Use search.convex.dev to search Docs, Stack, and Discord all at once.
- Additionally, you can post your questions in the Convex Community's <#1228095053885476985> channel to receive a response from AI.
- Avoid tagging staff unless specifically instructed.
Thank you!
You define the migration based on a single document, but the migration actually runs on many docs in batches. You can reduce the batch size like
new Migrations(components.migrations, { defaultBatchSize: 10 }) or migrations.define(table, migrateOne, 10)
In the worst case you'll always be able to migrate documents with batch size 1, since if you were able to write a document to convex then you're able to read and modify it (read limits are higher than write limits)