conradkoh•9mo ago

Race condition when reading and writing data

I encountered a race condition somewhere in my code and I can't figure out where my understanding is lacking in how convex handles concurrent write requests. This is the query for my code, where I am attempting to use convex as a cache, while I am migrating my system over. The way it works is that my system will call convex to update the cache when the data in my other backend changes. The frontend is subscribed to convex as the source of truth.

/**
 * This should be called by the server
 */
export const update = mutation({
  args: {
    accessTokenHash: v.string(),
    userId: v.string(),
    accounts: v.array(AccountValue()),
  },
  handler: async (ctx, args) => {
    const tokenHash = args.accessTokenHash;

    //invalidate all other user accounts caches
    const caches = await ctx.db
      .query('cache_userAccounts')
      .withIndex('by_userId', (q) => q.eq('userId', args.userId))
      .collect();

    //grant access to updater
    const existsForToken =
      caches.find(
        (v) => v.tokenHash === tokenHash && v.userId == args.userId
      ) != undefined;
    if (!existsForToken) {
      await ctx.db.insert('cache_userAccounts', {
        tokenHash: tokenHash,
        userId: args.userId,
        accounts: args.accounts,
      });
    }

    //update all caches
    await Promise.allSettled(
      caches.map(async (cache) => {
        await ctx.db.patch(cache._id, {
          tokenHash: tokenHash,
          userId: args.userId,
          accounts: args.accounts,
        });
      })
    );
  },
});

/**
 * This should be called by the server
 */
export const update = mutation({
  args: {
    accessTokenHash: v.string(),
    userId: v.string(),
    accounts: v.array(AccountValue()),
  },
  handler: async (ctx, args) => {
    const tokenHash = args.accessTokenHash;

    //invalidate all other user accounts caches
    const caches = await ctx.db
      .query('cache_userAccounts')
      .withIndex('by_userId', (q) => q.eq('userId', args.userId))
      .collect();

    //grant access to updater
    const existsForToken =
      caches.find(
        (v) => v.tokenHash === tokenHash && v.userId == args.userId
      ) != undefined;
    if (!existsForToken) {
      await ctx.db.insert('cache_userAccounts', {
        tokenHash: tokenHash,
        userId: args.userId,
        accounts: args.accounts,
      });
    }

    //update all caches
    await Promise.allSettled(
      caches.map(async (cache) => {
        await ctx.db.patch(cache._id, {
          tokenHash: tokenHash,
          userId: args.userId,
          accounts: args.accounts,
        });
      })
    );
  },
});

The issue I am facing is that existsForToken returns false and multiple repeated rows are inserted into the table, with the same tokenHash and userId. my intent is for the tokenHash + userId to for a unique pair. I've seen some other threads where .withIndex(<index name>).unique() was suggested, but I don't really see how .unique() is helpful, because this errors out on read (aka the data is already in an inconsistent state).

4 Replies

lee•9mo ago

can you explain the //update all caches part? It looks to me like every cache with userId=args.userId is patched to have the same tokenHash, which would explain why multiple caches end up with the same userId and tokenHash

ian•9mo ago

+1 to what Lee said. In general Convex protects you from race conditions. If you read and something isn't there, then you write it, you don't need to worry about two racing requests that both read and both write - one of them will be retried on the conflict. read more here: https://docs.convex.dev/database/advanced/occ

OCC and Atomicity | Convex Developer Hub

In Queries, we mentioned that determinism

conradkohOP•9mo ago

oh, you guys are completely right, it’s a bug in what i wrote. thanks for the help and sorry for wasting your time! I’ve seen the occ stuff which is why it was confusing to me that it didn’t behave in the way I expected (at least so I thought). appreciate the help!

ian•9mo ago

no sweat - glad it's working. and reminder you can use the bulk edit in the dashboard to edit your data if you need to clear values / etc: https://stack.convex.dev/lightweight-zero-downtime-migrations

Lightweight Zero-Downtime Migrations

Patch all of your data in your database table with the bulk edit feature on the Convex dashboard, without writing migration code.

Race condition when reading and writing data

Did you find this page helpful?