Database row version history?
It seems Convex internally keeps some version history of DB rows. Is it possible to view the previous version / snapshot of a given row, at a point in time?
I would like to do a diff between versions to see what data has changed. Thanks
8 Replies
we don't expose this yet, but we have some vague plans in the future to provide a high-performance CDC (change data capture) API using our transaction log (https://stack.convex.dev/how-convex-works#the-transaction-log)
How Convex Works
The full, unabridged story on how the Convex internals work.
That would be really useful for debugging. Given that Convex can now replace a lot of the client side state management, it would open up the possibility of a Redux-like time travel experience (but server side), where I would be able to restore the exact state of the app at any point in time.
Correlating this with errors raised would allow the app to be restored to its state when the error occurred.
How far off in the future is this?
we do have this API https://docs.convex.dev/http-api/#streaming-export-api which can show you a snapshot of a table, and changes that were made to it, if the timestamp is within the past month or so.
alternatively, you can keep the history yourself with a component https://github.com/ldanilek/table-history
GitHub
GitHub - ldanilek/table-history: Convex component for storing and a...
Convex component for storing and accessing an edit history or audit log of a Convex table - ldanilek/table-history
Convex HTTP API | Convex Developer Hub
Connecting to Convex directly with HTTP
but nothing like cdc with is a stream ?
you can set up CDC with airbyte or fivetran https://docs.convex.dev/production/integrations/streaming-import-export
but with the plain APIs the closest we have is
/api/document_deltas http://docs.convex.dev/http-api/#get-apidocument_deltasStreaming Data in and out of Convex | Convex Developer Hub
Streaming Data in and out of Convex
Convex HTTP API | Convex Developer Hub
Connecting to Convex directly with HTTP
If I'm not wanting to use airbyte or fivetran as I'm self hosting. Is there a way to achieve this? Happy to write something myself and fork the code. 😄
The streaming export API is what those providers consume. You can write your own consumer for it.
That was the first link that Lee provided.
I’m not 100% sure if it’s up and running in the docker container version right now…