OpenAi assistant vision - stored image url
Hi!
I am trying to use an image, that is stored in Convex, with OpenAi Assistant vision. Using the image url, i am getting an error from openai
"Uncaught Error: 400 Invalid 'content[1].image_url.url'. Expected url to end in a supported format: jpeg, jpg, png, gif, webp.".
Url format used:
"https://xxxx-xxxx-xxxx.convex.cloud/api/storage/:id"
As it says, the url for the stored image is missing the filetype ending, like a "normal" image url have. Is there a way to serve the image with the image format at the end? Am I missing something in the docs...?
Would hate to have to store seperate images in third-party just for this....
Openai vison docs:
https://platform.openai.com/docs/guides/vision
Exampel from openai:
{
"type": "image_url",
"image_url": {
"url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg",
},
},
Thanks!
All the best,
Hampus
5 Replies
Thanks for posting in <#1088161997662724167>.
Reminder: If you have a Convex Pro account, use the Convex Dashboard to file support tickets.
- Provide context: What are you trying to achieve, what is the end-user interaction, what are you seeing? (full error message, command output, etc.)
- Use search.convex.dev to search Docs, Stack, and Discord all at once.
- Additionally, you can post your questions in the Convex Community's <#1228095053885476985> channel to receive a response from AI.
- Avoid tagging staff unless specifically instructed.
Thank you!
can you put a filename in a query param that's not used, like
https://***.convex.cloud/api/storage/<id>?name=image.jpg
? If that doesn't work you could make an http action https://discord.com/channels/1019350475847499849/1019350478817079338/1296092275822166051I tried both, both the query param directily on the image url and setting up the http action, as the linked suggested, it works fine in the browser, with the param. Seems like openai sanitizes the params, so I am left with the same error
const editedImageUrl =
https://xxx.convex.site/getImage?storageId=<id>&filename=hi.jpeg
;
if (imageUrl) {
content.push({
type: "image_url",
image_url: { url: editedImageUrl },
// image_url: { url: ${imageUrl}?name=image.jpg
},
});
}Interesting. What if you do the http action but as part of the path instead of as a query param
like *.convex.site/getImage/<id>/hi.jpeg
Thanks! That worked like a charm. For future reference I will paste some of my code
OpenAi assistant vision, input object:
{
type: "image_url",
image_url: { url:
https://xxx.convex.site/getImage/${imageStorageId}/img.jpg}
, (check metadata for image type maybe?)
}
And my http action (there is probably a better way to get the path param):
http.route({
pathPrefix: "/getImage/",
method: "GET",
handler: httpAction(async (ctx, request) => {
const splitUrlArray = request.url.split("/");
const blob = await ctx.storage.get(splitUrlArray[4] as Id<"_storage">);
if (blob === null) {
return new Response("Image not found", {
status: 404,
});
}
return new Response(blob);
}),
});