milk enjoyer
milk enjoyer13mo ago

Cannot resolve node packages even with "use node"

I have "use node" indicated in my actions file, and that is the only place in my entire code where I am using the Node APIs. However, I am getting these errors:
✘ [ERROR] Could not resolve "buffer"

node_modules/safer-buffer/safer.js:5:21:
5 │ var buffer = require('buffer')
╵ ~~~~~~~~

The package "buffer" wasn't found on the file system but is built into node. Are
you trying to bundle for node? You can use "platform: 'node'" to do that, which
will remove this error.

✘ [ERROR] Could not resolve "string_decoder"

node_modules/iconv-lite/encodings/internal.js:49:28:
49 │ var StringDecoder = require('string_decoder').StringDecoder;
╵ ~~~~~~~~~~~~~~~~

The package "string_decoder" wasn't found on the file system but is built into
node. Are you trying to bundle for node? You can use "platform: 'node'" to do that,
which will remove this error.
✘ [ERROR] Could not resolve "buffer"

node_modules/safer-buffer/safer.js:5:21:
5 │ var buffer = require('buffer')
╵ ~~~~~~~~

The package "buffer" wasn't found on the file system but is built into node. Are
you trying to bundle for node? You can use "platform: 'node'" to do that, which
will remove this error.

✘ [ERROR] Could not resolve "string_decoder"

node_modules/iconv-lite/encodings/internal.js:49:28:
49 │ var StringDecoder = require('string_decoder').StringDecoder;
╵ ~~~~~~~~~~~~~~~~

The package "string_decoder" wasn't found on the file system but is built into
node. Are you trying to bundle for node? You can use "platform: 'node'" to do that,
which will remove this error.
51 Replies
milk enjoyer
milk enjoyerOP13mo ago
Upon more testing, it seems like the actions are indeed using the node APIs buffer and string_decoder correctly (as part of iconv-lite). But it still throws these errors every single time I save any file which is distracting me from other genuine errors Follow up: very bad news, this fails on convex deploy so I cannot push this to prod
ballingt
ballingt13mo ago
Are you importing this "use node" file from any non-use node files? a file with "use node" can use node modules for any actions it exports
milk enjoyer
milk enjoyerOP13mo ago
If I call the action from a non use node mutation does that count?
milk enjoyer
milk enjoyerOP13mo ago
No description
milk enjoyer
milk enjoyerOP13mo ago
generateVocabCardsAction is from the use node file
ballingt
ballingt13mo ago
That's fine, it's only importing that is a problem.
milk enjoyer
milk enjoyerOP13mo ago
hmm
ballingt
ballingt13mo ago
and are this iconv-lite imported in any other files?
milk enjoyer
milk enjoyerOP13mo ago
only this file it's not imported directly by this file but it is imported by another file that this file imports
ballingt
ballingt13mo ago
"use node" causes that file to be bundled differently such that it can no longer declare mutations and queries and it can use node APIs like "string_decoder"
milk enjoyer
milk enjoyerOP13mo ago
yes but that file has nothing except 2 actions both of which use node APIs
ballingt
ballingt13mo ago
so any file that does not have "use node" will cause thus error if it imports that library hm that sounds fine then
milk enjoyer
milk enjoyerOP13mo ago
yes it's very strange but i am stuck now because i cant deploy
ballingt
ballingt13mo ago
hm, I'd try removing files from the convex directly to see which one is causing this issue (assuming there's no indication of which file was being bundled when this error occurs)
milk enjoyer
milk enjoyerOP13mo ago
i know exactly which file which i remove the one that uses iconv-lite the errors stop
ballingt
ballingt13mo ago
One of the convex files that doesn't use "use node" is causing it
milk enjoyer
milk enjoyerOP13mo ago
nope it's this file specifically
ballingt
ballingt13mo ago
or "use node" isn't working
milk enjoyer
milk enjoyerOP13mo ago
when the imported file stops importing iconv-lite the error immediately goes away
ballingt
ballingt13mo ago
what does the top of that file look like? the "use node" part
milk enjoyer
milk enjoyerOP13mo ago
No description
milk enjoyer
milk enjoyerOP13mo ago
"@/lib/functions/openai/vocab" uses iconv-lite (because openai has a bug now where they return non utf8 strings as utf8 so we need to manually decode)
ballingt
ballingt13mo ago
huh could you double check that "vocab" isn't being imported from somewhere else?
milk enjoyer
milk enjoyerOP13mo ago
confirm
milk enjoyer
milk enjoyerOP13mo ago
No description
ballingt
ballingt13mo ago
Not openai/vocab, this file generation/vocab
milk enjoyer
milk enjoyerOP13mo ago
No description
ballingt
ballingt13mo ago
it might be "./vocab", but looks like you always use the top level imports, nice Would you be able to share this code? I'm tom@convex.dev, you could zip and email or share a GitHub repo with thomasballinger on GitHub
milk enjoyer
milk enjoyerOP13mo ago
Sure Let me send it to you now @ballingt added you on GitHub
ballingt
ballingt13mo ago
cloning to take a look
milk enjoyer
milk enjoyerOP13mo ago
thank you
ballingt
ballingt13mo ago
I think I see convex/server/meaning.ts --imports--> handle-duplicate-meanings.ts --imports--> "decode-openai" --imports--> iconv-lite
milk enjoyer
milk enjoyerOP13mo ago
ah i see i only imported the zod schemas but seems like it might have traced the other imports too
ballingt
ballingt13mo ago
hm yeah it doesn't seem to be able to tree-shake this
milk enjoyer
milk enjoyerOP13mo ago
ok let me move the schemas to it's own file and try again
ballingt
ballingt13mo ago
("it" == esbuild here the way it is invoked by convex)
milk enjoyer
milk enjoyerOP13mo ago
also one more problem zid() resolves to a string for some reason not a big problem because i can override it but slightly annoying
ballingt
ballingt13mo ago
Huh interesting, could you file that in https://github.com/get-convex/convex-helpers/issues
milk enjoyer
milk enjoyerOP13mo ago
sure
milk enjoyer
milk enjoyerOP13mo ago
No description
milk enjoyer
milk enjoyerOP13mo ago
No description
ballingt
ballingt13mo ago
thanks ah I see, hm
milk enjoyer
milk enjoyerOP13mo ago
my zid is the same one i copied from the convex helpers file
ballingt
ballingt13mo ago
oh so it's the same code, but not actually the one imported from the library, got it — curious if it repros in the code. Ian recently has been packaging some of these up to make them importable
milk enjoyer
milk enjoyerOP13mo ago
from convex/helpers right? that package gives me super weird typescript errors when i install it
milk enjoyer
milk enjoyerOP13mo ago
yup this one, i tried installing it and it started throwing weird typescript errors it has some magic where it is able to throw typescript errors from node_modules even though it is ignored according to tsconfig
ballingt
ballingt13mo ago
yeah that's it, cool it's under very active development so would be good to get a repro of that sometime
milk enjoyer
milk enjoyerOP13mo ago
ok cool will check it out again
ballingt
ballingt13mo ago
I'm headed to bed, good luck + thanks for reporting these
milk enjoyer
milk enjoyerOP13mo ago
thank you very much for saving my repo have a good rest! just an update: you are right, the latest implementation in convex-helpers does indeed fix the zid problem

Did you find this page helpful?