Convex Agent component + AI SDK + Vercel AI elements integration
Hey everyone! I'm working on a side project to test out some patterns and would love to get your thoughts on the best approach.
The Application
I'm building a basic AI app that lets users query a database with natural language. The database consists of a single table with around 20,000 documents. Think asking questions like "how much electrical capacity is available at this substation" and getting direct database responses. It's just a testing project to explore the limitations of Convex and different AI integration patterns.
I'm considering two different architectural patterns:
Pattern 1: AI SDK + Convex Tools
Using Next.js route handlers where I define AI SDK to structure outputs (convex query filter) and call Convex queries as tools. The flow would be: user input → AI SDK processes it → sends structured output → calls Convex query as a tool → sends response to Vercel AI elements . The nice thing here is AI SDK has solid integration with Vercel AI elements.
Pattern 2: Convex Agents + AI Elements
Instead of using AI SDK, I'd use Convex Agents for the backend logic and still use Vercel AI elements for the frontend. But I'm not sure how well AI elements integrate with Convex's Agent component responses. This is where I'm getting stuck.
I also have some concerns about authentication when you have different backend sources like Next.js route handlers versus Convex. How do you handle auth consistently across both?
Another issue is that when the database sends back large amounts of data, I don't want this to be part of the LLM output because it'll consume tons of output tokens and get expensive fast. Instead, I want the large data to bypass the LLM entirely and go straight to table or CSV rendering elements on the frontend as soon as the response is received. Is this even possible with either pattern?
Between these two patterns, which one do you think is better suited for this kind of setup? Has anyone tried mixing Convex Agents with Vercel AI elements before?
The Application
I'm building a basic AI app that lets users query a database with natural language. The database consists of a single table with around 20,000 documents. Think asking questions like "how much electrical capacity is available at this substation" and getting direct database responses. It's just a testing project to explore the limitations of Convex and different AI integration patterns.
I'm considering two different architectural patterns:
Pattern 1: AI SDK + Convex Tools
Using Next.js route handlers where I define AI SDK to structure outputs (convex query filter) and call Convex queries as tools. The flow would be: user input → AI SDK processes it → sends structured output → calls Convex query as a tool → sends response to Vercel AI elements . The nice thing here is AI SDK has solid integration with Vercel AI elements.
Pattern 2: Convex Agents + AI Elements
Instead of using AI SDK, I'd use Convex Agents for the backend logic and still use Vercel AI elements for the frontend. But I'm not sure how well AI elements integrate with Convex's Agent component responses. This is where I'm getting stuck.
I also have some concerns about authentication when you have different backend sources like Next.js route handlers versus Convex. How do you handle auth consistently across both?
Another issue is that when the database sends back large amounts of data, I don't want this to be part of the LLM output because it'll consume tons of output tokens and get expensive fast. Instead, I want the large data to bypass the LLM entirely and go straight to table or CSV rendering elements on the frontend as soon as the response is received. Is this even possible with either pattern?
Between these two patterns, which one do you think is better suited for this kind of setup? Has anyone tried mixing Convex Agents with Vercel AI elements before?
