erquhart•8mo ago

convex-helpers migrations code failing tsc

I cannot get rid of tsc failures coming out of the migration helper. I never have node modules show up in type checking, and I do have compilerOptions.skipLibCheck set to true on both of my tsconfigs. Convex's own typecheck doesn't fail, my root project tsc does. I've confirmed that I'm only import from convex-helpers within the convex/ directory. Any thoughts? (cc/ @ian)

node_modules/convex-helpers/server/migrations.ts:365:56 - error TS2345: Argument of type 'string | undefined' is not assignable to parameter of type 'string'.
  Type 'undefined' is not assignable to type 'string'.

365                 .withIndex("name", (q) => q.eq("name", next[i]))
                                                           ~~~~~~~

node_modules/convex-helpers/server/migrations.ts:476:35 - error TS2345: Argument of type 'FunctionReference<"mutation", "internal", { cursor?: string | null | undefined; batchSize?: number | undefined; next?: string[] | undefined; dryRun?: boolean | undefined; fn: string; }> | undefined' is not assignable to parameter of type 'SchedulableFunctionReference'.
  Type 'undefined' is not assignable to type 'SchedulableFunctionReference'.

476   await ctx.scheduler.runAfter(0, fnRef, {
                                      ~~~~~

node_modules/convex-helpers/server/migrations.ts:477:25 - error TS2345: Argument of type 'FunctionReference<"mutation", "internal", { cursor?: string | null | undefined; batchSize?: number | undefined; next?: string[] | undefined; dryRun?: boolean | undefined; fn: string; }> | undefined' is not assignable to parameter of type 'AnyFunctionReference'.
  Type 'undefined' is not assignable to type 'AnyFunctionReference'.

477     fn: getFunctionName(fnRef),
                            ~~~~~


Found 3 errors in the same file, starting at: node_modules/convex-helpers/server/migrations.ts:365

node_modules/convex-helpers/server/migrations.ts:365:56 - error TS2345: Argument of type 'string | undefined' is not assignable to parameter of type 'string'.
  Type 'undefined' is not assignable to type 'string'.

365                 .withIndex("name", (q) => q.eq("name", next[i]))
                                                           ~~~~~~~

node_modules/convex-helpers/server/migrations.ts:476:35 - error TS2345: Argument of type 'FunctionReference<"mutation", "internal", { cursor?: string | null | undefined; batchSize?: number | undefined; next?: string[] | undefined; dryRun?: boolean | undefined; fn: string; }> | undefined' is not assignable to parameter of type 'SchedulableFunctionReference'.
  Type 'undefined' is not assignable to type 'SchedulableFunctionReference'.

476   await ctx.scheduler.runAfter(0, fnRef, {
                                      ~~~~~

node_modules/convex-helpers/server/migrations.ts:477:25 - error TS2345: Argument of type 'FunctionReference<"mutation", "internal", { cursor?: string | null | undefined; batchSize?: number | undefined; next?: string[] | undefined; dryRun?: boolean | undefined; fn: string; }> | undefined' is not assignable to parameter of type 'AnyFunctionReference'.
  Type 'undefined' is not assignable to type 'AnyFunctionReference'.

477     fn: getFunctionName(fnRef),
                            ~~~~~


Found 3 errors in the same file, starting at: node_modules/convex-helpers/server/migrations.ts:365

43 Replies

ian•8mo ago

Thanks I’ll take a look. This is on latest version of convex & helpers ya?

erquhartOP•8mo ago

Yep, convex@1.13.0, convex-helpers@0.1.45

ian•8mo ago

I have a few other things I need to look into first - does changing your tsconfig to skip lib checks fix it for now?

erquhartOP•8mo ago

It doesn't, that's the issue I'll try nuking node_modules and see if that helps for some reason Didn't work Given that you're exporting TypeScript files, I guess I would expect those to get checked if I'm importing, even with skipLibCheck on. Root tsconfig:

{
  "extends": "expo/tsconfig.base",
  "compilerOptions": {
    "skipLibCheck": true,
    "paths": {
      "@/*": ["./src/*"],
      "~/*": ["./*"],
      "@cvx/*": ["./convex/*"]
    },
    "lib": ["ES2023"],
    "noImplicitAny": true,
    "useUnknownInCatchVariables": false,
    "noUnusedLocals": false,
    "noUncheckedIndexedAccess": true,
    "strict": true,
    "noErrorTruncation": true
  }
}

{
  "extends": "expo/tsconfig.base",
  "compilerOptions": {
    "skipLibCheck": true,
    "paths": {
      "@/*": ["./src/*"],
      "~/*": ["./*"],
      "@cvx/*": ["./convex/*"]
    },
    "lib": ["ES2023"],
    "noImplicitAny": true,
    "useUnknownInCatchVariables": false,
    "noUnusedLocals": false,
    "noUncheckedIndexedAccess": true,
    "strict": true,
    "noErrorTruncation": true
  }
}

Convex tsconfig:

{
  /* This TypeScript project config describes the environment that
   * Convex functions run in and is used to typecheck them.
   * You can modify it, but some settings required to use Convex.
   */
  "compilerOptions": {
    /* Added these myself, don't know if they're allowed */
    "noErrorTruncation": true,
    "noImplicitAny": true,
    "skipLibCheck": true,
    "strictNullChecks": true,
    "noUnusedLocals": false,
    "noUncheckedIndexedAccess": true,
    "useUnknownInCatchVariables": false,
    "paths": {
      "~/*": ["../*"],
      "@cvx/*": ["./*"]
    },

    /* These settings are not required by Convex and can be modified. */
    "strict": true,

    /* These compiler options are required by Convex */
    "target": "ESNext",
    "lib": ["ES2023", "dom"],
    "forceConsistentCasingInFileNames": true,
    "allowSyntheticDefaultImports": true,
    "module": "ESNext",
    "moduleResolution": "Bundler",
    "isolatedModules": true,
    "noEmit": true
  },
  "include": ["./**/*", "../shared-util/**/*"],
  "exclude": ["./_generated"]
}

{
  /* This TypeScript project config describes the environment that
   * Convex functions run in and is used to typecheck them.
   * You can modify it, but some settings required to use Convex.
   */
  "compilerOptions": {
    /* Added these myself, don't know if they're allowed */
    "noErrorTruncation": true,
    "noImplicitAny": true,
    "skipLibCheck": true,
    "strictNullChecks": true,
    "noUnusedLocals": false,
    "noUncheckedIndexedAccess": true,
    "useUnknownInCatchVariables": false,
    "paths": {
      "~/*": ["../*"],
      "@cvx/*": ["./*"]
    },

    /* These settings are not required by Convex and can be modified. */
    "strict": true,

    /* These compiler options are required by Convex */
    "target": "ESNext",
    "lib": ["ES2023", "dom"],
    "forceConsistentCasingInFileNames": true,
    "allowSyntheticDefaultImports": true,
    "module": "ESNext",
    "moduleResolution": "Bundler",
    "isolatedModules": true,
    "noEmit": true
  },
  "include": ["./**/*", "../shared-util/**/*"],
  "exclude": ["./_generated"]
}

Adding moduleResolution: "Bundler" to my root config seems to have fixed it I don't know how Expo/react native feels about that, but so far so good Calling this solved for now

ian•8mo ago

interesting. I bet it's noUncheckedIndexedAccess that's causing at least one of those. I can add some stricter checks to helpers to try and catch more of these. Thanks for flagging

ian•8mo ago

I just cut https://github.com/get-convex/convex-helpers/pull/166 for this

GitHub

Stricter tsconfig by ianmacartney · Pull Request #166 · get-convex/...

Improving the tsconfig strictness so we don't cause type issues for developers importing this code By submitting this pull request, I confirm that you can use, modify, copy, and redistribute ...

erquhartOP•8mo ago

Quick!

adam•8mo ago

Not sure if it's related but I experienced type errors caused by .js file references in convex-helpers. I'm using import { zid } from 'convex-helpers/server/zod'; in my @packages/shared I use zid() Convex helper in my Zod schema definitions in my shared package, but when the @apps/frontend tried to pull in and use my Zod schema defined in @packages/shared is didn't work because of the .js references in Example occurrence of the .js reference:

// convex-helpers/server/zod.ts
...
import {
  Mod,
  NoOp,
  Registration,
  UnvalidatedBuilder,
  splitArgs,
} from "./customFunctions.js";
...

// convex-helpers/server/zod.ts
...
import {
  Mod,
  NoOp,
  Registration,
  UnvalidatedBuilder,
  splitArgs,
} from "./customFunctions.js";
...

I'm using ESM, "type": "module" in my @apps/frontend. To solve this, I needed to copy the zid helper from convex-helpers over to my @packages/shared package and rework the imports to remove these .js references and it works fine. Is there a better way to solve this issue?

ian•8mo ago

What is your moduleResolution setting in your tsconfig? bundler doesn't require the file extensions, nodenext I believe does which is why I put the extensions in there. https://www.typescriptlang.org/docs/handbook/modules/reference.html#relative-file-path-resolution implies that every moduleResolution setting should support the relative import with .js, but maybe something else is messing it up..

adam•8mo ago

I'm using bundler as my moduleResolution in both @packages/shared and @apps/frontend

ian•8mo ago

Would you mind sharing the rest of your tsconfig so I can try to repro? Or do you have a repro you can share?

adam•8mo ago

I'm just trying to reproduce the error again to see the specific issue. Ah yes, now I recall the issue. It was with my react native app (expo v51). This is the specific error:

iOS Bundling failed 19108ms index.js (4764 modules)
Unable to resolve "./customFunctions.js" from "../../node_modules/convex-helpers/server/zod.ts"

iOS Bundling failed 19108ms index.js (4764 modules)
Unable to resolve "./customFunctions.js" from "../../node_modules/convex-helpers/server/zod.ts"

Here is my tsconfig.json for the expo app:

{
  "compilerOptions": {
    "allowJs": false,
    "allowSyntheticDefaultImports": true,
    "experimentalDecorators": true,
    "jsx": "react-native",
    "module": "es2015",
    "moduleResolution": "node",
    "strict": true,
    "noImplicitAny": true,
    "noImplicitReturns": true,
    "noImplicitThis": true,
    "noUnusedLocals": false,
    "sourceMap": true,
    "target": "esnext",
    "lib": ["esnext", "dom"],
    "skipLibCheck": true,
    "resolveJsonModule": true,
    "baseUrl": ".",
    "paths": {
      "app/*": ["./app/*"],
      "assets/*": ["./assets/*"]
    }
  },
  "extends": "expo/tsconfig.base",
  "ts-node": {
    "compilerOptions": {
      // compilerOptions specified here will override those declared above,
      // but *only* in ts-node. Useful if you want ts-node and tsc to use
      // different options with a single tsconfig.json.
      "module": "commonjs"
    }
  },
  "include": ["index.js", "App.tsx", "app", "types", "plugins", "app.config.ts"],
  "exclude": ["node_modules", "test/**/*"]
}

{
  "compilerOptions": {
    "allowJs": false,
    "allowSyntheticDefaultImports": true,
    "experimentalDecorators": true,
    "jsx": "react-native",
    "module": "es2015",
    "moduleResolution": "node",
    "strict": true,
    "noImplicitAny": true,
    "noImplicitReturns": true,
    "noImplicitThis": true,
    "noUnusedLocals": false,
    "sourceMap": true,
    "target": "esnext",
    "lib": ["esnext", "dom"],
    "skipLibCheck": true,
    "resolveJsonModule": true,
    "baseUrl": ".",
    "paths": {
      "app/*": ["./app/*"],
      "assets/*": ["./assets/*"]
    }
  },
  "extends": "expo/tsconfig.base",
  "ts-node": {
    "compilerOptions": {
      // compilerOptions specified here will override those declared above,
      // but *only* in ts-node. Useful if you want ts-node and tsc to use
      // different options with a single tsconfig.json.
      "module": "commonjs"
    }
  },
  "include": ["index.js", "App.tsx", "app", "types", "plugins", "app.config.ts"],
  "exclude": ["node_modules", "test/**/*"]
}

So it's "moduleResolution": "node", not bundler sorry. Btw, switching "allowJs" to true doesn't seem to help..

ian•8mo ago

Does changing moduleResolution here to bundler or nodenext help?

adam•8mo ago

I tried both bundler and nodenext, but neither fix the issue.

adam•8mo ago

ian•8mo ago

When I try to remove the .js I get an error for my own moduleResolution, so there must be some way..

adam•8mo ago

It it possible for convex-helpers to use bundler as the moduleResolution given this is what the main convex package now uses? I'm using bundler in my @packages/shared with these helper functions copied over and everything works ok (with the .js removed from the imports).

ian•8mo ago

I've reproduced it and it seems to be the expo bundler. I've removed all .js imports in the package in favor of absolute ones in an alpha build: 0.1.46-alpha.0 you can use. I'm having some trouble getting tests to like the approach, so I may have to go a different direction with the non-alpha release, but hopefully that works for you @adam convex-helpers@0.1.46 goes with the approach you suggested: changing them all to node-style imports and using bundler module resolution. I might go back to nodenext but I'll make sure expo is happy with it. I validated it against our monorepo template (which I updated to newest expo / turbo while I was there) @erquhart ^^ also since you I believe have been looking at convex-helpers with Expo recently?

adam•8mo ago

I've just tested 0.1.46-alpha.1 with my bun monorepo setup and it now looks to be working well. Thanks very much - nice work. Note that I needed to update my Expo tsconfig.json to use "moduleResolution": "bundler" and not "moduleResolution": "node" which I believe is the default. convex-helpers@0.1.46 is also working well.

ian•8mo ago

good to know, and glad to hear it - I wonder if the alpha.0 version didn't need the resolution change 🤔. It's pretty unfortunate that Expo's bundler seems incompatible with nodenext typescript packages - unless I can find another way to tell it to start looking in dist. it ignores the exports directive unless you toggle an unstable option (which then broke another package's bundling for my app). Maybe it's worth a patch that strips the .js before looking for other extensions.

erquhartOP•8mo ago

I'll try the alpha 0 in a bit @ian alpha 0 does fix my original issue without requiring a change to moduleResolution actually... now it's not having the issue at all anymore, even on latest 🤦‍♂️ What happened so far: - I posted this issue, and fixed by changing moduleResolution of root tsconfig to bundler - at some point after that, for unrelated reasons, I deleted node_modules entirely and reinstalled everything - today, I went to check if alpha 0 addressed my original issue without a module resolution change - before doing anything, I removed the module resolution change to make sure tsc still failed, which it did - updated to alpha.0 - no failure - updated to alphal.1 - no failure - reverted to 0.1.46 (latest) - no failure - uninstalled and reinstalled convex-helpers - no failure 🤷‍♂️

ian•8mo ago

Latest includes the alpha changes. I released it last night. I would expect it not to work on 1. 45. 0.1.46 has a bad behavior where vitest expects the nodenext-style relative imports - so any project using vitest and convex-helpers would fail. I've cut 0.1.47-alpha.2 that should support everything - relative imports, node resolution, nodenext, bundler, Expo, etc. I'm going to cut 0.1.47 soon unless I hear it's causing either of you issues @erquhart @adam

erquhartOP•8mo ago

trying the alpha now looks to be working for me, expo only, no vite

adam•8mo ago

0.1.47 is working well for me at runtime. I've tested using expo, vite and vitest (v2). I am however getting tsc errors which seem to be related to the validators change. I'm still on convex@1.12.2 due to outstanding optional types bug in convex@1.13.0. I'll revert back to 0.1.43 for now.

ian•8mo ago

which types bugs are you blocked on in convex@1.13.0?

adam•8mo ago

This one here https://discord.com/channels/1019350475847499849/1259106419521622056

ian•8mo ago

I believe that was also fixed in 0.1.46 - can you try with 0.1.47 and convex 1.13? Sorry I didn't follow up in that thread - Sarah fixed that along with two other issues in https://github.com/get-convex/convex-helpers/pull/167

GitHub

Fix type issues from npm 1.13 by sshader · Pull Request #167 · get-...

Fixes three issues: myCustomFunciton({ handler: (ctx, args: { foo: string } => ... }) now compiles zodToConvex(z.object({ foo: z.number().optional() })) now yields { foo?: number | undefined } ...

adam•8mo ago

Ok great. When I try to update: convex 1.12.2 => 1.13.0 convex-helpers 0.1.43 => 0.1.47 convex seems to get stuck in an endless loop when running npx convex dev

ballingt•8mo ago

Do you have another npx convex dev process running somewhere by chance?

adam•8mo ago

I suspected that too but it doesn’t seem so. Also, when I undo the change then start up convex again everything works fine (no looping).

ballingt•8mo ago

Huh! Could you try npx convex dev --verbose @adam

ian•8mo ago

the mac file watcher also isn't the best piece of technology - I can't remember what's fixed it before when I saw this, whether it was a machine restart, finder restart, VSCode restart, etc.

adam•8mo ago

I successfully got it upgraded to 1.13.0/0.1.47 after a restart of my macbook. Thanks @ballingt @ian

erquhartOP•8mo ago

Just a note: I'm now seeing something like the original issue here after migrating to Convex auth, and it's specifically with files that import from the auth lib. I've changed "moduleResolution" to "Bundler" in the root of my project to fix, but as before, ideally this wouldn't be necessary, as I'm not certain it's a welcome override for Expo.

ian•8mo ago

Something like the original issue being type errors about the unchecked index access? Can you paste the updated error?

erquhartOP•8mo ago

Good point, these weren't errors from node_modules, so it's different. It's almost like npx tsc at root isn't respecting the convex directory tsconfig, but the vscode editor is. So the cli has errors but the editor doesn't. But this hasn't been an issue before, and it's all to do with code from the auth library (says it can't find the module). I can't dig into it anymore though, changing module resolution is working 🤷‍♂️ will update if that changes. Thanks for checking in 👍

ian•8mo ago

you're using RN / Expo, so metro bundler? I believe that only works with node module resolution, and the convex auth package currently doesn't seem to have non-react entrypoints for node - something we can improve.

erquhartOP•8mo ago

Metro hasn't complained yet. But I also haven't attempted a production build since migrating. Local dev bundling is working though. But yeah that might fix it. I still don't understand why vscode editor had no type issues, but cli did. Using terminal in vscode and both using the same Typescript version. Might just be a setup issue.

ballingt•8mo ago

re vscode respecting convex/tsconfig.json but not tsc, that's always been the behavior; the convex tsconfig.json is only used in editors and when Convex bundles, not when the app compiles

erquhartOP•8mo ago

Interesting, maybe I should run two separate checks then

ballingt•8mo ago

You can run the convex check with npx convex typecheck, it runs on dev automatically and the main app one with something like tsc --noEmit depending on your TypeScript setup

erquhartOP•8mo ago

Oh nice didn't know about the convex command. I'll probably have to exclude the convex directory from my root config for the app check. Actually that doesn't work because I import from convex all over the place. I don't think there's a current workaround that doesn't involve setting moduleResolution to bundler. I've always run with noEmit set to true via config.

ballingt•8mo ago

makes sense, we'll add these somehow; Ian has written some scripts to generate these a few different ways @erquhart if it's easy, I'd love to hear if

npm install github:get-convex/convex-auth#node10-entry-points

npm install github:get-convex/convex-auth#node10-entry-points

fixes for you

convex-helpers migrations code failing tsc

Did you find this page helpful?