Importing a snapshot during deploy stored on s3
Hi, my snapshot is now above 100mb. This means I can no longer commit it to github. Is there a way to tell the deploy command
npx convex import --preview-name "$VERCEL_GIT_COMMIT_REF" ./data/mock-data/snapshot.zip during a deploy to tell it to grab the file from a remote resource, like an s3 bucket? What are workarounds for large snapshots that can't be commited to a repo?19 Replies
Thanks for posting in <#1088161997662724167>.
Reminder: If you have a Convex Pro account, use the Convex Dashboard to file support tickets.
- Provide context: What are you trying to achieve, what is the end-user interaction, what are you seeing? (full error message, command output, etc.)
- Use search.convex.dev to search Docs, Stack, and Discord all at once.
- Additionally, you can post your questions in the Convex Community's <#1228095053885476985> channel to receive a response from AI.
- Avoid tagging staff unless specifically instructed.
Thank you!
currently this isn't possible; npx convex import requires the file to exist on the local filesystem. But @nipunn is working on a new feature that should help
Thanks
If I wanted to import manually, can I run
npx convex import --replace --preview-name "development" ./data/mock-data/snapshot.zip locally to upload it to a preview deployment whose branch name is "development"? i.e. is the VERCEL_GIT_COMMIT_REF used in the deploy command npx convex deploy --cmd 'npm run build'&&[ "$VERCEL_ENV" = preview ]&&npx convex import --preview-name "$VERCEL_GIT_COMMIT_REF" ./data/mock-data/snapshot.zip;npx convex run --preview-name "$VERCEL_GIT_COMMIT_REF" migrations:runMigrations equal to the branch name? If not, how do I get it?I suppose you could split the file into many files for github and then merge them in the build command.
Wow yea actually what I did above seemed to work!
I believe
$VERCEL_GIT_COMMIT_REF is the branch name, yes. Are you trying to import into your dev deployment?no, was trying to import into preview environment
Just happens to be that my preview environment is off the pushed branch development
Nice
Now that I'm doing it manually, how would I remove it properly from the deploy command but still keep the runmigrations part?
to
?
Do you want to runMigrations after importing the seed data? Then that won't work
Oh I guess there's no point in running the migration then and there
yea
As another option, you could store the file in s3 and then curl it to download it before importing
So just
npx convex deploy --cmd 'npm run build' and then locally run npx convex import --replace --preview-name "development" ./data/mock-data/snapshot.zip;npx convex run --preview-name "development" migrations:runMigrations ?
As another option, you could store the file in s3 and then curl it to download it before importingI can do all that in a build command? That'd be cool
yep build command should be able to curl
wow ok will try that
Or maybe ill do it in github actions since i can store the snapshot with lfs
Ooh good idea
In the end I was able to put it into an s3 bucket and write a custom build command file that downloads it and then runs and it works!
Is there a place where the community can write custom posts to help others? I'd be happy to write this one as an example and share it with the community
nice work! you could post in #general or #show-and-tell , since that build command sounds useful 🙂
👍