mysql to convex
Is there a way to import data from a mysql database to convex? I saw the cli has a snapshot import, but will that also import primary keys, foriegn keys, etc?
5 Replies
hey there! the airbyte destination connector will do this ( https://docs.airbyte.com/integrations/destinations/convex )
Convex | Airbyte Documentation
This page contains the setup guide and reference information for the Convex destination connector.
but the keys won't magically be linked document ids. maybe @lee (who knows a lot about this streaming import system) has some ideas about how to preserve references on this import path
streaming import doesn't support foreign references to
_id columns, sorry.
snapshot import from a zip file does support foreign references, but all of the ids need to be valid convex ids from tables of the same name. so you can construct fake data in convex, do a snapshot export, swap the real data into the zip file, and do a snapshot import. but that's pretty error prone.
i would suggest using streaming import (or snapshot import) and put the primary keys in fields not called _id. then you don't get the nice syntax for db.get -- have to do indexed queries against the primary key field -- and you can't use v.id() validators -- have to use v.string(). but it should work.Or if you're ok with downtime, you could import it all with the non-
_id foreign references, then do a snapshot export, then do some surgery on the documents in the export to change the references from strings to the _id values of the other documents. snapshot import maintains IDs / references, so you could import the documents pointing at the _id value instead of the older primary key
Or write an online migration that does the rewritingThank you all for the feedback! I will look into what you said and determine the best option. Thanks again!