punn
punn•16mo ago

Efficient backfills

Is there a more efficient backfill than this approach? (backfilling info directly throws the function read limit error)
backfillForAllUsers:
users = all users
for user in users:
schedule backfillForOneUser

backfillForOneUser:
infos = this user's info[]
for info in infos:
patch info
backfillForAllUsers:
users = all users
for user in users:
schedule backfillForOneUser

backfillForOneUser:
infos = this user's info[]
for info in infos:
patch info
3 Replies
ballingt
ballingt•16mo ago
Does efficient mean faster here, or are you trying to cut down on use of some other resource? This looks pretty reasonable, if you wanted to use fewer function calls or try to push its performance you could do some batching but this looks like exactly how I'd write it, and how I hope developers write their code on Convex: it expresses what needs to be done and lets Convex handle the scaling
ian
ian•16mo ago
In my migration helpers, I run each mutation in batches of 100 but expose a "change one document" function to make the API easy. https://stack.convex.dev/migrating-data-with-mutations
Migrating Data With Mutations
Using mutations to migrate data in Convex.
punn
punnOP•16mo ago
awesome thank you. getting used to letting convex take the wheel and it feels great 🙂

Did you find this page helpful?