Smurfens
Smurfens3mo ago

OpenAi assistant vision - stored image url

Hi! I am trying to use an image, that is stored in Convex, with OpenAi Assistant vision. Using the image url, i am getting an error from openai "Uncaught Error: 400 Invalid 'content[1].image_url.url'. Expected url to end in a supported format: jpeg, jpg, png, gif, webp.". Url format used: "https://xxxx-xxxx-xxxx.convex.cloud/api/storage/:id" As it says, the url for the stored image is missing the filetype ending, like a "normal" image url have. Is there a way to serve the image with the image format at the end? Am I missing something in the docs...? Would hate to have to store seperate images in third-party just for this.... Openai vison docs: https://platform.openai.com/docs/guides/vision Exampel from openai: { "type": "image_url", "image_url": { "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg", }, }, Thanks! All the best, Hampus
5 Replies
Convex Bot
Convex Bot3mo ago
Thanks for posting in <#1088161997662724167>. Reminder: If you have a Convex Pro account, use the Convex Dashboard to file support tickets. - Provide context: What are you trying to achieve, what is the end-user interaction, what are you seeing? (full error message, command output, etc.) - Use search.convex.dev to search Docs, Stack, and Discord all at once. - Additionally, you can post your questions in the Convex Community's <#1228095053885476985> channel to receive a response from AI. - Avoid tagging staff unless specifically instructed. Thank you!
lee
lee3mo ago
can you put a filename in a query param that's not used, like https://***.convex.cloud/api/storage/<id>?name=image.jpg ? If that doesn't work you could make an http action https://discord.com/channels/1019350475847499849/1019350478817079338/1296092275822166051
Smurfens
SmurfensOP3mo ago
I tried both, both the query param directily on the image url and setting up the http action, as the linked suggested, it works fine in the browser, with the param. Seems like openai sanitizes the params, so I am left with the same error const editedImageUrl = https://xxx.convex.site/getImage?storageId=<id>&filename=hi.jpeg; if (imageUrl) { content.push({ type: "image_url", image_url: { url: editedImageUrl }, // image_url: { url: ${imageUrl}?name=image.jpg }, }); }
lee
lee3mo ago
Interesting. What if you do the http action but as part of the path instead of as a query param like *.convex.site/getImage/<id>/hi.jpeg
Smurfens
SmurfensOP3mo ago
Thanks! That worked like a charm. For future reference I will paste some of my code OpenAi assistant vision, input object: { type: "image_url", image_url: { url: https://xxx.convex.site/getImage/${imageStorageId}/img.jpg}, (check metadata for image type maybe?) } And my http action (there is probably a better way to get the path param): http.route({ pathPrefix: "/getImage/", method: "GET", handler: httpAction(async (ctx, request) => { const splitUrlArray = request.url.split("/"); const blob = await ctx.storage.get(splitUrlArray[4] as Id<"_storage">); if (blob === null) { return new Response("Image not found", { status: 404, }); } return new Response(blob); }), });

Did you find this page helpful?