Mutations race condition
I have a table called
messageBuffer that does the following:
- Creates a new row for some conversation's first message
- If another message in the conversation arrives, we append a buffer variable within that conversation's row
- After 30 seconds, each message in buffer is processed, and the row is deleted (note: this deletion happens in an action that takes a long time to execute).
I'm currently running into the issue of losing message_2 in the following scenario:
for one conversation,
message_1 arrives
30 seconds pass and we begin to process and flush the buffer
message_2 arrives
buffer is deleted
message_3 arrives
.....
Am I able to fix this by replacing my deletion with an updateOrDelete function instead?5 Replies
so "process and flush" takes some time in an action, correct?
correct
cool
so, the way to do this is: (1) create a sequence number in the table...1, 2, 3. (2) add an index on it so you can get the maximum value quickly on every mutation; (3) on the mutation, grab this maximum and insert the next record with the next higher value, which will give you your ordered count; (4) when you grab the batch to process (flush) you'll have the "cursor" which represents how much of the queue you grabbed; (5) only delete records
<= end of processed buffer when you're done
at the end of the day, if you have some long-running, potentially failing, non-transactional thing that needs to run while the transactional store changes underneath, you're going to need a cursoring strategy like this to do it safely. or just delete all the ids you pulled manually instead of wiping the whole buffergot it, that makes sense. ordering within the buffer doesn't need to be conserved here so I'm taking the approach of just deleting specific ids from the buffer rather than wiping it completely.
will definitely remember the first approach if we ever need the ordering though. Thank you
sounds great!