winsoroaks•8mo ago

query and update inside for-loop

(not urgent) hi team! im running into a weird situation. hopefully it's just that im blind to the logic implementation. i have the following code that "updates" the usage table with the image size the user uploaded

await Promise.all(
  args.storageIds.map(async (s) => {
    const metadata = await ctx.db.system.get(s.storageId as Id<"_storage">)
    console.log(
      "🚀 ~ file: images.ts:43 ~ args.storageIds.map ~ metadata:",
      metadata.size
    ) // 93558, 68153, 68948

    const usage = await ctx.db
      .query("usage")
      .withIndex("by_userId", (q) => q.eq("userId", ctx.userId))
      .unique()
    console.log(
      "🚀 ~ file: images.ts:53 ~ args.storageIds.map ~ usage:",
      usage.used
    ) // 0, 0, 0, should be 0, 93558, 93558 + 68153
    await ctx.db.patch(usage._id, { used: usage.used + metadata.size })

    await ctx.db.insert("images", {
      userId: ctx.userId,
      uploadedImageId: s.storageId,
    })
  })
)

await Promise.all(
  args.storageIds.map(async (s) => {
    const metadata = await ctx.db.system.get(s.storageId as Id<"_storage">)
    console.log(
      "🚀 ~ file: images.ts:43 ~ args.storageIds.map ~ metadata:",
      metadata.size
    ) // 93558, 68153, 68948

    const usage = await ctx.db
      .query("usage")
      .withIndex("by_userId", (q) => q.eq("userId", ctx.userId))
      .unique()
    console.log(
      "🚀 ~ file: images.ts:53 ~ args.storageIds.map ~ usage:",
      usage.used
    ) // 0, 0, 0, should be 0, 93558, 93558 + 68153
    await ctx.db.patch(usage._id, { used: usage.used + metadata.size })

    await ctx.db.insert("images", {
      userId: ctx.userId,
      uploadedImageId: s.storageId,
    })
  })
)

im not sure why the usage returned is 0, 0, 0 where i'd expect it to be 0, 93558, 93558 + 68153. i managed to change the implementation to the following:

let size = 0
await Promise.all(
  args.storageIds.map(async (s) => {
    const metadata = await ctx.db.system.get(s.storageId as Id<"_storage">)
    size += metadata.size
    await ctx.db.insert("images", {
      userId: ctx.userId,
      uploadedImageId: s.storageId,
    })
  })
)

const usage = await ctx.db
  .query("usage")
  .withIndex("by_userId", (q) => q.eq("userId", ctx.userId))
  .unique()
await ctx.db.patch(usage._id, { used: usage.used + size })

let size = 0
await Promise.all(
  args.storageIds.map(async (s) => {
    const metadata = await ctx.db.system.get(s.storageId as Id<"_storage">)
    size += metadata.size
    await ctx.db.insert("images", {
      userId: ctx.userId,
      uploadedImageId: s.storageId,
    })
  })
)

const usage = await ctx.db
  .query("usage")
  .withIndex("by_userId", (q) => q.eq("userId", ctx.userId))
  .unique()
await ctx.db.patch(usage._id, { used: usage.used + size })

which is def more efficient and worked. im just curious if there's some special case with my first implementation. thank you.

6 Replies

winsoroaksOP•8mo ago

@kapa.ai do you know?

kapa.ai•8mo ago

Let me check the knowledge base for you. One moment! :books: ---------------------- I'm sorry, but your question is too vague. Could you please provide more details or context? I'm here to provide information about Convex, a tool designed to simplify the development of web applications. ---------------------- Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: ----------------------

kapa.ai•8mo ago

<@1112389092118892677> kapa.ai is still learning and improving, please let us know how it did by reacting below

erquhart•8mo ago

I think you're expecting both examples to work like a for loop, accessing array items in sequence. Because your mapping function is async, you can expect the mapping function to run for all three items in parallel. Your second approach worked because you moved the usage table read/write outside of the loop. In the first code sample those read/writes were overwriting one another due to parallelism. For getting a sum of async values, you generally would either use an actual loop, out if you prefer functional you could use an async reducer and have the reducer always await the preceding result.

Michal Srb•8mo ago

@erquhart is correct, the first version runs concurrently, so first all the metadata documents are loaded, then all the usage documents are loaded (all with 0), and only then the patches run (overwriting each other). If you replace Promise.all with a for loop things will work as you'd expect, but your second version is faster since you parallelize the reads.

winsoroaksOP•8mo ago

thank you all!

query and update inside for-loop

Did you find this page helpful?