RJ•2y ago

Env var availability in Convex functions

I'm having some trouble understanding how/when env vars are made accessible in the context of Convex functions. Are they inlined as part of a compilation step or something? If I run console.log(process.env) in a Convex function (an HTTP action), I see in the Convex dashboard logs log [process.env], and if I run Object.keys(process.env).forEach((envKey) => console.log(envKey)) I see no output in the Convex dashboard logs at all, which I find rather surprising.

13 Replies

lee•2y ago

they're implemented with a Proxy where handler has a get method. this way we can detect which env variables you're using, so they are added to the "read set", which means your query subscriptions will trigger when the specific env variable changes, and mutation conflicts will only happen (with OCC retries) if the same env variable was concurrently changed. all of this is implementation detail, and it's not necessarily the same between actions and queries/mutations, but that's why it's implemented like it is now

RJOP•2y ago

Hmm ok, so it's expected given this implementation that Object.keys(process.env) will always produce []? I'm not very familiar with how Proxy works, but I guess I'd like to know what other behavioral differences exist between the usual process.env object and Convex's process.env implemented with a Proxy?

Michal Srb•2y ago

We need to document this better. Basically process.env is not iterable, but process.env.FOO will happen at runtime (or push time if it happens in the module scope). Do you have a need for iterating through process.env?

RJOP•2y ago

I'm using, or would prefer to use, a library which is incompatible with this approach of mocking process.env. Some references: - https://github.com/Effect-TS/io/blob/main/src/internal/configProvider.ts#L99 - https://www.effect.website/docs/configuration Check out the getEnv and load functions in the first link to see whence the problem arises

Michal Srb•2y ago

I see, makes sense. I'd suggest to not use the Configuration part of the library for now. process.env.FOO is always a string, which you can convert to whatever type you expect.

RJOP•2y ago

Yeah for sure, it's just more convenient to have the library do that for me 🙂 I also have the option of writing a custom ConfigProvider which reads from process.env but can handle Convex's method of mocking process.env. Do you imagine this library's current approach to reading process.env could ever be supported by Convex's approach to mocking process.env, or probably not (again, not familiar at all with the features/limitations of Proxy)? ChatGPT thinks it's possible 😛

Is it possible to make a JS Proxy iterable?

Yes, it is possible to make a JavaScript Proxy object iterable by defining the Symbol.iterator method on the proxy's handler. Here's an example:

const originalArray = [1, 2, 3];

const iterableProxy = new Proxy(originalArray, {
  get(target, property) {
    if (property === Symbol.iterator) {
      return function* () {
        for (let i = 0; i < target.length; i++) {
          yield target[i];
        }
      };
    }

    return target[property];
  },
});

for (const item of iterableProxy) {
  console.log(item);
}

const originalArray = [1, 2, 3];

const iterableProxy = new Proxy(originalArray, {
  get(target, property) {
    if (property === Symbol.iterator) {
      return function* () {
        for (let i = 0; i < target.length; i++) {
          yield target[i];
        }
      };
    }

    return target[property];
  },
});

for (const item of iterableProxy) {
  console.log(item);
}

In this example, we create a Proxy object around an array (originalArray). The proxy's get trap is used to intercept the access to the Symbol.iterator property. When Symbol.iterator is accessed, we return a generator function that yields the elements of the target array. By implementing the Symbol.iterator method in the proxy's handler, we make the proxy object iterable, allowing us to use it in for...of loops and other constructs that work with iterables.

magicseth•2y ago

Thanks for explaining that, I was confused also about why i couldn't access env variables outside of a function! Bettter error messaging could help there. too @RJ I think the issue is that it would make it impossible to do accurate dependency tracking with that code, thereby silently hurting performance now or in the future

RJOP•2y ago

Ah right, that would make sense. Agreed that better error messaging would be nice, in that case! Revisiting this one last time—I imagine that supporting process.env as an iterable could still allow for env var dependency tracking, but would have to assume that if process.env is accessed as an iterable that all env vars are dependencies of that function, right? The cost of that support is obviously performance, although I imagine env vars aren't changed very often by users, so I wonder how signficant that would be. But the upside is that process.env would work as most users would expect it to, by default. I'm not trying to dictate an outcome one way or the other, but would still like to know whether you all view this is as something that would ever be worth supporting, or not?

sshader•2y ago

Yep this sounds like a thing we could do -- take a dependency on all env vars when process.env is accessed as an iterable (or when it's console.log-ed). I believe the design so far has been optimized for keeping the dependencies pretty narrow, so it's helpful to hear these concrete examples where perhaps it would be preferred to just let you grab all your env variables (behaving more like the usual process.env).

RJOP•2y ago

Makes sense for sure, thanks for the response!

magicseth•2y ago

@sshader it would be nice to be able to access env vars from outside of functions. anything that makes it easier to port in existing code is great. Maybe any env variables you use outside of a function become dependencies for all of those functions?

Michal Srb•2y ago

@magicseth you should be able to access environment variables outside functions (if you're on latest version of Convex). Let us know if you run into any issues with it.

presley•2y ago

It does work on older versions too. The support got added couple months back and was server only fix

Env var availability in Convex functions

Did you find this page helpful?