import & export improvements
Suggestion about Database Import & Export. If I understood correctly from the documentation, at the moment you can only export the entire schema. It could be nice to have a utility like Data Pump from Oracle. It gives you the ability to export certain tables, with data or just metadata (not sure how useful this part might be). Also, you can specify certain queries: for example, if I only want to export some of the data and not all I can do that. Another cool feature is that it allows you to import in a different table. (ex. you export from a environment from a Table_1 , and you can import the data in Table_2).
5 Replies
great ideas!
Table selection/deselection is on the list, the tricky part is getting a good UI so any feedback is helpful.
can you describe a metadata-only export? what it contains and how you would use it.
for importing into a different table name, you can do that by modifying the zip file (both _tables/documents.jsonl and the directory) or by using the
--table flagin the context of Oracle Database, the meta-data only export was referring to the structure of the table; like exporting an empty table.
ex. You wanted to create a new env and have all the tables from a different one. So you would do an export schema meta-data and then import in the new env. You'll have the schema prepared.
Was that specified in the docs and I somehow missed it?
ex of data pump oracle.
expdp system/password@db10g full=Y directory=TEST_DIR dumpfile=full.dmp logfile=expdp_full.log query='SCOTT.EMP:"WHERE deptno=0",SCOTT.DEPT:"WHERE deptno=0"'
To get the schema in Convex, you just copy the schema.ts file - you don't need to import a schema via data.
A few options to consider to get specific data out:
1. Write a query (paginated if there's thousands of rows or more) that you run from a node script, writing out JSON objects to a
.jsonl file in whatever form you like.
2. Setting up streaming export to continuously get data out of convex into another tool (for analytics, e.g.)
3. Use snapshot export, but then inspect the data with DuckDB locallyStreaming Data in and out of Convex | Convex Developer Hub
Streaming Data in and out of Convex
Observing your app in production
By setting up dedicated tools, you can get actionable data to help understanding errors, performance, user behavior and allow you respond quickly when...
Fair enough.
I'll try it out.
Thank you