Sean Aguinaga
Sean Aguinaga3mo ago

@convex-dev/agent Vector Embedding Size Limitations

export const VectorDimensions = [
128, 256, 512, 768, 1024, 1536, 2048, 3072, 4096,
] as const;
export function validateVectorDimension(
dimension: number
): asserts dimension is VectorDimension {
if (!VectorDimensions.includes(dimension as VectorDimension)) {
throw new Error(
`Unsupported vector dimension${dimension}. Supported: ${VectorDimensions.join(", ")}`
);
}
}
export const VectorDimensions = [
128, 256, 512, 768, 1024, 1536, 2048, 3072, 4096,
] as const;
export function validateVectorDimension(
dimension: number
): asserts dimension is VectorDimension {
if (!VectorDimensions.includes(dimension as VectorDimension)) {
throw new Error(
`Unsupported vector dimension${dimension}. Supported: ${VectorDimensions.join(", ")}`
);
}
}
If I use google's multimodalEmbedding001, which outputs 1408 dimensions, won't it break with existing code above?
7 Replies
Convex Bot
Convex Bot3mo ago
Thanks for posting in <#1088161997662724167>. Reminder: If you have a Convex Pro account, use the Convex Dashboard to file support tickets. - Provide context: What are you trying to achieve, what is the end-user interaction, what are you seeing? (full error message, command output, etc.) - Use search.convex.dev to search Docs, Stack, and Discord all at once. - Additionally, you can post your questions in the Convex Community's <#1228095053885476985> channel to receive a response from AI. - Avoid tagging staff unless specifically instructed. Thank you!
Sean Aguinaga
Sean AguinagaOP3mo ago
Does this mean we can only use openai?
ian
ian3mo ago
I tried to include all the dimensions for all the providers I could find, but I guess I missed that one. I'll add it to the list! I added it in version 0.1.1 but just FYI the embeddings are currently only on the text part of the message, which this model seems to not be well optimized for. Adding a future image/video embedder may make sense, but right now I'd suggest using a different embedding model.
Sean Aguinaga
Sean AguinagaOP3mo ago
Yeah 32 tokens is hilarious Thank you I also ask because the agent component has some methods in the source code like
addFile()
addFile()
Which currently don't seem to be utilizied? And I was wondering if there was some multimodal work going on there to make file embeddings easier
ian
ian3mo ago
Oh yeah that's for adding images/ files to messages, but the context RAG isn't currently doing vector search on anything but text fields. For files outside of the chat history I was thinking a RAG component would be better suited. But for things in the chat itself doing embeddings on more things could be beneficial - if the surrounding conversation doesn't pull it in. would be interesting to do some evals
Sean Aguinaga
Sean AguinagaOP3mo ago
Do you send the text message and then add a file to it after the fact? So it can be displayed inline?
ian
ian3mo ago
You can add a file, put it in a message (as a URL to convex storage), and send it to the LLM in the messages argument. I put an example in the repo that doesn't (yet) use the Agent file tracking: https://github.com/get-convex/agent/blob/ian/text-streaming/example/convex/example.ts#L344-L370
GitHub
agent/example/convex/example.ts at ian/text-streaming · get-convex...
Build AI agents on Convex with persistent chat history - get-convex/agent

Did you find this page helpful?