Richard•2d ago

R2 Environment Variables Issue / Project Structure?

Hello all, First of all thank you for taking the time to read this, and I am not 100% sure of the cause of this problem, and have spent more hours than I would like to admit hunting down the cause, and I am honestly not too sure.

✖ Error fetching POST  http://127.0.0.1:3210/api/deploy2/start_push 400 Bad Request: InvalidModules: Hit an error while pushing:
Loading the pushed modules encountered the following
    error:
Failed to analyze adapter.js: Uncaught Error: R2 configuration is missing required fields:
Missing: R2_BUCKET, R2_ENDPOINT, R2_ACCESS_KEY_ID, R2_SECRET_ACCESS_KEY
    at parseConfig (../../../../node_modules/@convex-dev/r2/src/client/index.ts:49:20)
    at new R2 (../../../../node_modules/@convex-dev/r2/src/client/index.ts:141:2)
    at <anonymous> (../../services/R2/index.ts:16:1)

✖ Error fetching POST  http://127.0.0.1:3210/api/deploy2/start_push 400 Bad Request: InvalidModules: Hit an error while pushing:
Loading the pushed modules encountered the following
    error:
Failed to analyze adapter.js: Uncaught Error: R2 configuration is missing required fields:
Missing: R2_BUCKET, R2_ENDPOINT, R2_ACCESS_KEY_ID, R2_SECRET_ACCESS_KEY
    at parseConfig (../../../../node_modules/@convex-dev/r2/src/client/index.ts:49:20)
    at new R2 (../../../../node_modules/@convex-dev/r2/src/client/index.ts:141:2)
    at <anonymous> (../../services/R2/index.ts:16:1)

Running convex dashboard opens up a dashboard, where the state of the env vars is shown by the attached picture (note all variables are defined) So for context, my convex project is within a monorepo, and has the following structure (this may / may not be the problem).

├── convex (re-export functions here to maintain a clean generated api structure)
│   ├── _generated
│   ├── convex.config.ts
│   ├── http.ts
│   ├── schema.ts (reexport from resources/schemas, etc)
│   └── tsconfig.json
├── package.json
├── resources (my defined tables, split into schema, queries, mutations files)
├── services (interacting with 3rd parties)
│   └── R2
│       └── index.ts
├── shared
│   ├── errors.ts (frontends use this file)
│   ├── types.ts
│   └── utils.ts (convex-helpers custom queries)
└── tsconfig.json

├── convex (re-export functions here to maintain a clean generated api structure)
│   ├── _generated
│   ├── convex.config.ts
│   ├── http.ts
│   ├── schema.ts (reexport from resources/schemas, etc)
│   └── tsconfig.json
├── package.json
├── resources (my defined tables, split into schema, queries, mutations files)
├── services (interacting with 3rd parties)
│   └── R2
│       └── index.ts
├── shared
│   ├── errors.ts (frontends use this file)
│   ├── types.ts
│   └── utils.ts (convex-helpers custom queries)
└── tsconfig.json

... (1/2)

6 Replies

RichardOP•2d ago

I am using the local deployment feature, but I have tested this on live deployments, and received the same issue. Basically, in my shared/utils.ts file, I define custom queries (e.g. authQuery, where I ensure the user is authenticated, and then inject helpful things into the context, such as R2, auth, etc (see code snippet below).

typescript 
        import { r2 } from "../services/R2";
        ...
        // Helper: generate authenticated context
    const buildAuthContext = (sessionData: { session: Session; user: User }) => ({
        auth: {
            ...ctx.auth, // (maintain convex auth functions)
            session: sessionData.session,
            user: sessionData.user,
        },
                // various services, etc
        ...{ r2 },
    });

typescript 
        import { r2 } from "../services/R2";
        ...
        // Helper: generate authenticated context
    const buildAuthContext = (sessionData: { session: Session; user: User }) => ({
        auth: {
            ...ctx.auth, // (maintain convex auth functions)
            session: sessionData.session,
            user: sessionData.user,
        },
                // various services, etc
        ...{ r2 },
    });

I do not have all code in the convex folder to prevent reexporting the same thing multiple times for the generated api (e.g. if i export a query in a resource and then rexport to cleanup the api). I think that this might be related to the file structure that I have, but am not 100% sure. I feel like that maybe do the file structure convex is bundling the external files, and not including the environment variables here, or only includes environment variables at inference time, and my use of the r2 within this "injected" context is illegal. Note that I have begun migrating to this new file structure, to cleanup the api paths, and this implementation (including r2 in context) worked in the past. Notably, this is not the highest priority issue, but some feedback/thoughts from others would be appreciated. I have since removed the injection of the r2 object from context and made all files import r2 manually, so I do have a fix, but would like to further understand the crux of this problem so I do not make the same mistake in the future. (2/2)

erquhart•this hour

Can you try console logging the environment variables right before wherever the error is happening to confirm whether environment variables are being picked up

RichardOP•22h ago

Yes, so the only way I can actually get any console.log output is by hooking into the constructor of the R2 class. Editing the dist code in node_modules:

    constructor(component, options = {}) {
        console.log(JSON.stringify(process.env)); // <-- added this
        this.component = component;
        this.options = options;
        this.config = {
            bucket: options?.R2_BUCKET ?? process.env.R2_BUCKET,
            endpoint: options?.R2_ENDPOINT ?? process.env.R2_ENDPOINT,
            accessKeyId: options?.R2_ACCESS_KEY_ID ?? process.env.R2_ACCESS_KEY_ID,
            secretAccessKey: options?.R2_SECRET_ACCESS_KEY ?? process.env.R2_SECRET_ACCESS_KEY,
        };
        this.r2 = createR2Client(parseConfig(this.config));
    }

    constructor(component, options = {}) {
        console.log(JSON.stringify(process.env)); // <-- added this
        this.component = component;
        this.options = options;
        this.config = {
            bucket: options?.R2_BUCKET ?? process.env.R2_BUCKET,
            endpoint: options?.R2_ENDPOINT ?? process.env.R2_ENDPOINT,
            accessKeyId: options?.R2_ACCESS_KEY_ID ?? process.env.R2_ACCESS_KEY_ID,
            secretAccessKey: options?.R2_SECRET_ACCESS_KEY ?? process.env.R2_SECRET_ACCESS_KEY,
        };
        this.r2 = createR2Client(parseConfig(this.config));
    }

Yields an empty object at runtime: '{}' I feel like there is something in the wrong order, as it seems like the code is being executed prior to convex loading in the environment vars from the dashboard? Not 100% sure with the internals. BTW: Thank you for getting back to me

erquhart•21h ago

I mean before the constructor of the r2 class is called, log env vars and make sure the things you expect to be defined are defined Don't log process.env as a whole, that's often disabled at the runtime level Log specific variables

RichardOP•21h ago

Okay, I actually resolved this (the reason I was process.env as a whole is no outputs were being recorded). One of my local convex components imported one of the helper functions from utils.ts. Since R2 was imported in this file as well, it was trying to resolve variables that did not exist yet. I moved the function that was being imported by a local component into its own file and all is good. Thank you for your help/insight, probably wouldn't have figured it out

erquhart•21h ago

Glad you're up and running!

R2 Environment Variables Issue / Project Structure?

Did you find this page helpful?