So on a project I have, I'm trying to
So on a project I have, I'm trying to self host and creat a terraform configuration that can launch convex into a VPC on Google Cloud. I'm having trouble with the generate_admin_key step. both calling that command and getting the key. I'd much rather prefer to generate my own key and pass that in to the container. But I didn't see any options discussed for that. Has anyone successfully packaged this up into something that can be run that way? or is everyone just setting this up on Cloud accounts where they have root access and logging into the box and generating keys?
17 Replies
It seems reasonable to require some kind of root access to create a key?
The admin key is generated from here
https://github.com/get-convex/convex-backend/blob/main/self-hosted/docker-build/generate_admin_key.sh
And requires the instance secret which is generated by default within the container here
https://github.com/get-convex/convex-backend/blob/main/self-hosted/docker-build/read_credentials.sh
If you want - you could modify and build your own container that passes in the instance secret from the outside rather than generating it. Would even consider PRs that create that option.
https://github.com/get-convex/convex-backend/blob/main/self-hosted/docker-build/Dockerfile.backend
But I suspect you'll have an easier time sticking to being able to run the
generate_admin_key for your container. It's gotta be possible to run scripts inside your container on Google Cloud.It's not that it's impossible, it's that it's disabled on most cloud accounts from an organizational point of view.
like, they'd want to run this inside of like.. cloud run or GKE
Are you using the docker-compose.yml ?
You can set INSTANCE_NAME and INSTANCE_SECRET here
https://github.com/get-convex/convex-backend/blob/main/self-hosted/docker/docker-compose.yml#L13
You can generate a valid instance secret the same way as read_credentials does
https://github.com/get-convex/convex-backend/blob/main/self-hosted/docker-build/read_credentials.sh
with
openssl rand -hex 32
Then you can generate an admin key manually outside of the container with those two pieces of information with the same technique as here
https://github.com/get-convex/convex-backend/blob/main/self-hosted/docker-build/generate_admin_key.sh
generate_key being a script you'd have to build from sourceI spun it up locally that way, but on the VPC I was using GCE containers and using the images and arranging a VPC in the same way the docker compose describes
I think the environment variables I described can still work even if you're hooking things up yourself.
okay, so I can pass in instance name and instance secret, then I can tell the box to run the key generation as part of its initialization, and then I can calculate the admin key outside the box just based on knowledge of the instance name and secret I passed in using a local version of the generate key script
so if that's.. accurate, then I'm looking for the source for that generate key script
GitHub
convex-backend/crates/keybroker/src/bin/generate_key.rs at main · g...
The open-source reactive database for app developers - get-convex/convex-backend
this one?
https://github.com/get-convex/convex-backend/blob/main/self-hosted/docker-build/Dockerfile.backend#L85 - this is how it's built.
I suspect you could also do a
docker compose up locally with the same instance name and instance secret and it would work with the normal instructionsoh... yeah, so I could set the instance name and secret locally.. generate .. see what that is.. and then do the same on my remote instance
I see
I just built it locally it was quick
oh cool. So I can just do that. excellent. I'm pretty new to rust and playing with cargo
yeah - either approach works - whatever you want to do.
Docker is nice because you don't have to set up all the build toolchains yourself - but ... yeah you have to set up docker which can be its own pain sometimes
If you get a setup that you like working, I would be open to accepting a PR with an additional guide for generating your admin keys outside of the container (another md file in here https://github.com/get-convex/convex-backend/tree/main/self-hosted) - if you want to contribute.
you're sorta breaking ground in this approach
Awesome. Yeah, I'll mess with it and get something up that I like
we've done something like this internally for our cloud product (which we run in our infra) so it's definitely possible, but it's not in the simple docker setup guide
what would be cool, is if the instance secret and name are deliberately set, that the generate key script runs automatically -- because it would be the same everytime, so no harm in have regenerating it as part of startup.. just thinking out loud
and then the user could just what their admin secret is via the CLI
have there been any conversations around basic scaling methods for the self hosting crowd?
I'm going to need to have at least a back of the envelope understanding of what size servers I'm going to need for a particular quantity of users. So I'm preparing to do that math. we're likely to have a 10k institutional user base and that probably won't change a ton, so we're not too worried about scale, but still need to size the architecture correctly
Just kinda live-attempted this setup and got it working in under 2 minutes.... took a couple of tries, but got it.
Seems doable!