I have a large JSON file with objects, what is the best way to get that into an internal action?
Specifically I have a JSON file where I maintain a list of Google's supported canonical locations. The file has more objects in the array than can be committed in one mutation.
I want some way to get this large JSON array into an internalAction I can call to have it insert or patch documents in my locations table.
I've attempted multiple things so far:
- Tried using the args of a function by the CLI; the argument length was too large at least when I ran through npx
- Tried reading the file from a "use node" action; the source file isn't present on the bundled version
- Tried statically importing the JSON into a "use node" action; the bundle size became too large to upload
- Tried uploading the JSON to Convex and passing the storageId to the action; Upload went fine but
I want some way to get this large JSON array into an internalAction I can call to have it insert or patch documents in my locations table.
I've attempted multiple things so far:
- Tried using the args of a function by the CLI; the argument length was too large at least when I ran through npx
- Tried reading the file from a "use node" action; the source file isn't present on the bundled version
- Tried statically importing the JSON into a "use node" action; the bundle size became too large to upload
- Tried uploading the JSON to Convex and passing the storageId to the action; Upload went fine but
await file.text(); on the blob from const file = await ctx.storage.get(storageId); fails with an "Array buffer allocation failed" error and the documentation says nothing about supporting reading the file from an action.
