What’s the best way to import a large table — around 16 million records?
Hi folks, I see that the current way to import a table is using npx convex import, but it takes around 2 hours and 40 minutes each time.
Is there a faster way to import large tables — or even larger datasets? I noticed that after uploading the data, the import processes documents in batches of 8k, which adds up and takes quite a while.
3 Replies
Is this from an external database or importing from a backup?
from a csv
So the source is external though, right? Was trying to find out if streaming import with airbyte straight from the external source is faster, but I don’t know that it is