Chad Maycumber•2y ago

Shared environment variables

It would be really helpful if environment variables could be shared across developers inside an organization. So that when new variables are added the backend environment is always up to date.

12 Replies

ian•2y ago

Great suggestion. I had some ideas around this recently- one idea was to have an ENV config in the repo that prompts users for missing ENV variables as part of convex dev An extension of that could be to offer to copy from prod? Currently I’m guessing each dev working with you can access the prod env and copy from there. Is the friction: 1. Devs get surprised by new variables they need and have to go copy things. 2. The secrets for not-prod are different and annoying to share between teammates 3. Updating secrets doesn’t propagate to other deployments 4. Something else?

Chad MaycumberOP•2y ago

I'm currently thinking about setting up multiple stages (prod, staging, and maybe even dev) as separate projects. All of the "deployed" versions of the site would be the corresponding production branch in convex. I think the biggest issue rn is if a dev adds a new service for example and merges that code it'll break the other devs environments (this is fine but then it requires a dev to go and get the corresponding env variable from whichever team member added the change). But the secrets for not prod are also different. Kinda all of the above.

ian•2y ago

You'll likely want each developer to have their own "dev" but I agree with a separate project for prod & staging (and maybe "dev" is only on staging except for testing hot fixes). This mirrors the GCP approach I've been recommended of having duplicate GCP projects for staging. I understand the pain - thanks for clarifying. I hope we get a better workflow here. For now, here's a couple patterns I've used on projects that require env variables: 1. Have a query that returns a list of the env variables that aren't set, and a UI element (only shown in development) that warns about which env variables need to be set, potentially hiding the rest of the app until they are, with a description of where to get them. e.g. https://github.com/ianmacartney/embeddings-in-convex/blob/main/convex/lib/pinecone.ts#L22C18-L22C18 which is consumed by https://github.com/ianmacartney/embeddings-in-convex/blob/main/src/EnvCheck.tsx 2. Have a function that is run before running npx convex dev / npm run dev, that adds initialization data and throws if an environment variable is missing. The package.json scripts section can look like:

  "scripts": {
    "dev": "npm-run-all dev:init --parallel dev:server dev:client",
    "build": "tsc && vite build",
    "dev:server": "convex dev",
    "dev:client": "vite --open",
    "dev:init": "convex run init"
  },

  "scripts": {
    "dev": "npm-run-all dev:init --parallel dev:server dev:client",
    "build": "tsc && vite build",
    "dev:server": "convex dev",
    "dev:client": "vite --open",
    "dev:init": "convex run init"
  },

And you can have a convex/init.ts file with:

export default internalMutation(async () => {
  if (!process.env.OPENAI_API_KEY) {
    throw new Error("OPENAI_API_KEY not set: get it from ...");
  }
  // add seed data
});

export default internalMutation(async () => {
  if (!process.env.OPENAI_API_KEY) {
    throw new Error("OPENAI_API_KEY not set: get it from ...");
  }
  // add seed data
});

Chad MaycumberOP•2y ago

Ahh cool! I currently am using a zod version of an env check that does seem to catch it in dev. The init script before dev starts is a cool touch though

presley•2y ago

If you are trying to have a list of env vars and fail quickly and loudly if some are not set, there is a simpler way. We now use the latest environment variables to evaluate modules during deployment. So you can just assert the env vars are defined at global scope. If you don’t have then set for the respective deployment, you will get an error during ‘npx convex dev’ or ‘npx convex deploy’. The caveat is if you delete the env vars later, your code will start failing at runtime.

ian•2y ago

Thanks for getting that out Presley! That’s great news, and makes a lot of this way simpler.

Chad MaycumberOP•15mo ago

I know there was talk about this on the way! Just dropping this here as I thought it could be interesting. It would be nice if the convex CLI supported a command that mimicked dotenv -- so that environment variables could be totally managed inside of convex

ballingt•15mo ago

@Chad Maycumber what does dotenv -- do?

Chad MaycumberOP•15mo ago

@ballingt Oh sorry. That would be helpful context. It executes the command with the environments variables in the current working directory from your .env, .env.local, etc. The additional -- just acts as a pass through so that you can still pass additional commands to whatever follows. Here's how I'm currently using it with my dev command: "dev": "dotenv -- turbo run dev",. Going back to earlier I think it'd be cool if I could do something like: "dev": "convex env "turbo run dev"",. Then I don't even need to worry about an env file

ian•15mo ago

In this case would the env variables be going from .env to convex, or from convex into a locally running service? The browser env variables I wouldn't want locally, but maybe there are some shared with next.js api routes or something that run locally, is that it?

Chad MaycumberOP•15mo ago

I guess in my case I have multiple backends that share .env variables. I was thinking from convex to the locally running service so that convex could be used as the .env management solution to the other backends

ian•15mo ago

@Chad Maycumber Env variables can be shared now in 1.5 https://news.convex.dev/announcing-convex-1-5/ However, note that it doesn't sync them, it just configures the defaults for new deployments

Convex News

Announcing Convex 1.5

Convex 1.5 adds support for preview deployments, improvements to the Convex runtime, and more.

Shared environment variables

Did you find this page helpful?