milk enjoyer•12mo ago

Uploading files from Node/backend

ctx.storage.store takes in a Blob, but Blobs are not really well supported in Node and backend environments. What's the preferred way to upload a file from a server?

11 Replies

lee•12mo ago

The convex action environments from which you can run ctx.storage.store should support Blob (https://nodejs.org/dist/latest-v16.x/docs/api/buffer.html#class-blob ). What data structure do you have for your file? Are you running an action in the default Convex environment or with "use node" ?

milk enjoyerOP•12mo ago

@lee After playing around with actions and uploading, i found that this works:

const response = await fetch(testSource);
    console.log(response);
    const buffer = await response.arrayBuffer();
    const contentType =
      response.headers.get("Content-Type") || "application/octet-stream";

    const blob = new Blob([buffer], { type: contentType });

    const storageId = await ctx.storage.store(blob);

const response = await fetch(testSource);
    console.log(response);
    const buffer = await response.arrayBuffer();
    const contentType =
      response.headers.get("Content-Type") || "application/octet-stream";

    const blob = new Blob([buffer], { type: contentType });

    const storageId = await ctx.storage.store(blob);

But there are some problems with the "use node" fetch which causes it to fail, so I have to use another provider to run my fetches for this use case which runs in a more standard node environment. What would be the best way to upload from there? By the way regarding the fetch problems with "use node" actions in convex, I have been trying to figure this out for a long time but I really have no idea why. I've ran the same code to download a media file on my computer and different node environments and the browser and they work perfectly fine, but whenever that code runs on convex I end up getting a 403 from the server. I have checked the cookies and everything and there is really no difference, everything is exactly the same including the code. Is it something to do with the convex runtime?

lee•12mo ago

the code you pasted works for me, although const blob = await response.blob() is a big simpler. it sounds to me like your issue is the fetch is returning 403s. what URL are you trying to fetch? And can you describe the problem with storage/blobs (it sounds like the code works for you too)?

milk enjoyerOP•12mo ago

blob = await response.blob()

blob = await response.blob()

does not work for me with "use node" @lee can i message you directly?

lee•12mo ago

what error are you getting?

milk enjoyerOP•12mo ago

failure Uncaught Error: store() expects a Blob. If you are trying to store a Request, await request.blob() will give you the correct input. at handler (../../../convex/ofApi/media/actions.ts:28:25)

lee•12mo ago

is it possible you're importing fetch from somewhere instead of using the global fetch?

milk enjoyerOP•12mo ago

nope, let me DM you an example link and you can try it out

lee•12mo ago

sure, can DM, although it would be great to do as much as we can here so others can benefit for troubleshooting 🙂

milk enjoyerOP•12mo ago

it's super strange, all my fetches work except when in convex can you accept my friend request? @lee the link might be slightly sensitive which is why i prefer not to share it here

lee•12mo ago

yep sorry

Uploading files from Node/backend

Did you find this page helpful?