50 replies

Batch insertions

Hey guys, quick question - whats the best way to bulk insert documents? Im scheduling a function to run every morning that hits an api that grabs pricing on about 19k documents & have a table to store the price for that day (looking to make a historical pricing for my app)

Wondering tho what is the best way to go about it.. i have internal mutations that add one doc and currently looping over all the data batched at 250 each, and using the convex-helpers

asyncMap

asyncMap

like so:

        await asyncMap(cardVariants, async (card) => {
          await ctx.runMutation(
            internal.cardPricing.mutations.internalInsertCardPriceDoc.default,
            card
          );
          totalProcessed++;
        });

        await asyncMap(cardVariants, async (card) => {
          await ctx.runMutation(
            internal.cardPricing.mutations.internalInsertCardPriceDoc.default,
            card
          );
          totalProcessed++;
        });

Wondering how this all works when it comes to pricing if i need to call functions 19k times a day just for this function & if this is a good idea

Batch insertions

Similar Threads