Issue with AI agent
Hi I'm working on a study platform that allows people to generate flashcards from documents. It's already generating them but it fails on the return validation of .generateObject. I'm lost since it seems to be behind agent code
3 Replies
Thanks for posting in <#1088161997662724167>.
Reminder: If you have a Convex Pro account, use the Convex Dashboard to file support tickets.
- Provide context: What are you trying to achieve, what is the end-user interaction, what are you seeing? (full error message, command output, etc.)
- Use search.convex.dev to search Docs, Stack, and Discord all at once.
- Additionally, you can post your questions in the Convex Community's <#1228095053885476985> channel to receive a response from AI.
- Avoid tagging staff unless specifically instructed.
Thank you!
Hi I'm working on a study platform that allows people to generate flashcards from documents. It's already generating them but it fails on the return validation of .generateObject. I'm lost since it seems to be behind agent code
import { action, mutation, query } from "../_generated/server";
import { v } from "convex/values";
import * as enums from "../../src/enums";
import { getAuthUserId } from "@convex-dev/auth/server";
import { Agent, createTool } from "@convex-dev/agent";
import { components } from "../_generated/api";
import { tool } from "ai";
import { openai } from "@ai-sdk/openai";
import z from "zod";
export const generateCardsFromDocuments = action({
args: v.object({
type: v.union(...Object.values(enums.CardType).map(v.literal)),
sourceContent: v.optional(v.string()),
}),
handler: async (ctx, args) => {
const { type, sourceContent } = args;
const amountOfCards = 5;
const supportAgent = new Agent(components.agent, {
chat: openai.chat("gpt-4o-mini"),
instructions: `You are a learning buddy and only a backend tool. All you do is generate ${type} cards based on the provided file. Use tools to see the structure of each type of card return.`,
tools: {
getCardStructure: getCardStructure,
},
});
const userId = await getAuthUserId(ctx);
if (!userId) throw new Error("User not authenticated");
const { threadId } = await supportAgent.createThread(ctx, { userId });
const result = await supportAgent.generateObject(
ctx,
{ threadId, userId },
{
prompt: `Generate cards ${amountOfCards} based on ${sourceContent}`,
schema: z.object({
cards: z.array(
z.object({
question: z.string(),
correctAnswer: z.string(),
options: z.array(z.string()).optional(),
})
),
}),
}
);
console.log("Generated cards:", result);
if (typeof result === "string") {
return JSON.parse(result);
}
return result;
},
});
//TOOLS
const getCardStructure = createTool({
description: "Get the structure of a specific type of card",
args: z.object({
type: z.enum(Object.values(enums.CardType) as [string, ...string[]]),
}),
handler: async (ctx, args) => {
const { type } = args;
switch (type) {
case enums.CardType.Flashcard:
return {
question: "What is the capital of France?",
correctAnswer: "Paris",
options: ["Paris", "London", "Berlin"],
};
case enums.CardType.Explain:
return {
question: "Explain the theory of relativity.",
correctAnswer:
"The theory of relativity explains how space and time are linked for objects that are moving at a consistent speed in a straight line.",
};
case enums.CardType.MultipleChoice:
return {
question: "What is the largest planet in our solar system?",
options: ["Earth", "Jupiter", "Saturn"],
correctAnswer: "Jupiter",
};
default:
throw new Error("Unknown card type");
}
},
});
import { action, mutation, query } from "../_generated/server";
import { v } from "convex/values";
import * as enums from "../../src/enums";
import { getAuthUserId } from "@convex-dev/auth/server";
import { Agent, createTool } from "@convex-dev/agent";
import { components } from "../_generated/api";
import { tool } from "ai";
import { openai } from "@ai-sdk/openai";
import z from "zod";
export const generateCardsFromDocuments = action({
args: v.object({
type: v.union(...Object.values(enums.CardType).map(v.literal)),
sourceContent: v.optional(v.string()),
}),
handler: async (ctx, args) => {
const { type, sourceContent } = args;
const amountOfCards = 5;
const supportAgent = new Agent(components.agent, {
chat: openai.chat("gpt-4o-mini"),
instructions: `You are a learning buddy and only a backend tool. All you do is generate ${type} cards based on the provided file. Use tools to see the structure of each type of card return.`,
tools: {
getCardStructure: getCardStructure,
},
});
const userId = await getAuthUserId(ctx);
if (!userId) throw new Error("User not authenticated");
const { threadId } = await supportAgent.createThread(ctx, { userId });
const result = await supportAgent.generateObject(
ctx,
{ threadId, userId },
{
prompt: `Generate cards ${amountOfCards} based on ${sourceContent}`,
schema: z.object({
cards: z.array(
z.object({
question: z.string(),
correctAnswer: z.string(),
options: z.array(z.string()).optional(),
})
),
}),
}
);
console.log("Generated cards:", result);
if (typeof result === "string") {
return JSON.parse(result);
}
return result;
},
});
//TOOLS
const getCardStructure = createTool({
description: "Get the structure of a specific type of card",
args: z.object({
type: z.enum(Object.values(enums.CardType) as [string, ...string[]]),
}),
handler: async (ctx, args) => {
const { type } = args;
switch (type) {
case enums.CardType.Flashcard:
return {
question: "What is the capital of France?",
correctAnswer: "Paris",
options: ["Paris", "London", "Berlin"],
};
case enums.CardType.Explain:
return {
question: "Explain the theory of relativity.",
correctAnswer:
"The theory of relativity explains how space and time are linked for objects that are moving at a consistent speed in a straight line.",
};
case enums.CardType.MultipleChoice:
return {
question: "What is the largest planet in our solar system?",
options: ["Earth", "Jupiter", "Saturn"],
correctAnswer: "Jupiter",
};
default:
throw new Error("Unknown card type");
}
},
});
6/21/2025, 10:48:15 PM [CONVEX M(messages:addMessages)] ArgumentValidationError: Value does not match validator.
Path: .messages[0].message
Value: {content: "{\"cards\":[{\"question\":\"How many letters are in the word 'Letter'?\",\"correctAnswer\":\"6\"},{\"question\":\"What is the first letter of the word 'Letter'?\",\"correctAnswer\":\"L\"},{\"question\":\"What is the last letter of the word 'Letter'?\",\"correctAnswer\":\"r\"},{\"question\":\"Is the word 'Letter' a noun?\",\"correctAnswer\":\"Yes\"},{\"question\":\"Does the word 'Letter' contain any vowels?\",\"correctAnswer\":\"Yes\"}]}", id: "chatcmpl-BkzQEkjOrHl0IlWAd8CldqwH3ZXSm", role: "assistant"}
Validator: v.union(v.object({content: v.union(v.string(), v.array(v.union(v.object({providerOptions: v.optional(v.record(v.string(), v.record(v.string(), v.any()))), text: v.string(), type: v.literal("text")}), v.object({image: v.union(v.string(), v.bytes()), mimeType: v.optional(v.string()), providerOptions: v.optional(v.record(v.string(), v.record(v.string(), v.any()))), type: v.literal("image")}), v.object({data: v.union(v.string(), v.bytes()), filename: v.optional(v.string()), mimeType: v.string(), providerOptions: v.optional(v.record(v.string(), v.record(v.string(), v.any()))), type: v.literal("file")})))), providerOptions: v.optional(v.record(v.string(), v.record(v.string(), v.any()))), role: v.literal("user")}), v.object({content: v.union(v.string(), v.array(v.union(v.object({providerOptions: v.optional(v.record(v.string(), v.record(v.string(), v.any()))), text: v.string(), type: v.literal("text")}), v.object({data: v.union(v.string(), v.bytes()), filename: v.optional(v.string()), mimeType: v.string(), providerOptions: v.optional(v.record(v.string(), v.record(v.string(), v.any()))), type: v.literal("file")}), v.object({providerOptions: v.optional(v.record(v.string(), v.record(v.string(), v.any()))), signature: v.optional(v.string()), text: v.string(), type: v.literal("reasoning")}), v.object({data: v.string(), providerOptions: v.optional(v.record(v.string(), v.record(v.string(), v.any()))), type: v.literal("redacted-reasoning")}), v.object({args: v.any(), providerOptions: v.optional(v.record(v.string(), v.record(v.string(), v.any()))), toolCallId: v.string(), toolName: v.string(), type: v.literal("tool-call")})))), providerOptions: v.optional(v.record(v.string(), v.record(v.string(), v.any()))), role: v.literal("assistant")}), v.object({content: v.array(v.object({args: v.optional(v.any()), experimental_content: v.optional(v.array(v.union(v.object({text: v.string(), type: v.literal("text")}), v.object({data: v.string(), mimeType: v.optional(v.string()), type: v.literal("image")})))), isError: v.optional(v.boolean()), providerOptions: v.optional(v.record(v.string(), v.record(v.string(), v.any()))), result: v.any(), toolCallId: v.string(), toolName: v.string(), type: v.literal("tool-result")})), providerOptions: v.optional(v.record(v.string(), v.record(v.string(), v.any()))), role: v.literal("tool")}), v.object({content: v.string(), providerOptions: v.optional(v.record(v.string(), v.record(v.string(), v.any()))), role: v.literal("system")}))
6/21/2025, 10:48:15 PM [CONVEX M(messages:addMessages)] ArgumentValidationError: Value does not match validator.
Path: .messages[0].message
Value: {content: "{\"cards\":[{\"question\":\"How many letters are in the word 'Letter'?\",\"correctAnswer\":\"6\"},{\"question\":\"What is the first letter of the word 'Letter'?\",\"correctAnswer\":\"L\"},{\"question\":\"What is the last letter of the word 'Letter'?\",\"correctAnswer\":\"r\"},{\"question\":\"Is the word 'Letter' a noun?\",\"correctAnswer\":\"Yes\"},{\"question\":\"Does the word 'Letter' contain any vowels?\",\"correctAnswer\":\"Yes\"}]}", id: "chatcmpl-BkzQEkjOrHl0IlWAd8CldqwH3ZXSm", role: "assistant"}
Validator: v.union(v.object({content: v.union(v.string(), v.array(v.union(v.object({providerOptions: v.optional(v.record(v.string(), v.record(v.string(), v.any()))), text: v.string(), type: v.literal("text")}), v.object({image: v.union(v.string(), v.bytes()), mimeType: v.optional(v.string()), providerOptions: v.optional(v.record(v.string(), v.record(v.string(), v.any()))), type: v.literal("image")}), v.object({data: v.union(v.string(), v.bytes()), filename: v.optional(v.string()), mimeType: v.string(), providerOptions: v.optional(v.record(v.string(), v.record(v.string(), v.any()))), type: v.literal("file")})))), providerOptions: v.optional(v.record(v.string(), v.record(v.string(), v.any()))), role: v.literal("user")}), v.object({content: v.union(v.string(), v.array(v.union(v.object({providerOptions: v.optional(v.record(v.string(), v.record(v.string(), v.any()))), text: v.string(), type: v.literal("text")}), v.object({data: v.union(v.string(), v.bytes()), filename: v.optional(v.string()), mimeType: v.string(), providerOptions: v.optional(v.record(v.string(), v.record(v.string(), v.any()))), type: v.literal("file")}), v.object({providerOptions: v.optional(v.record(v.string(), v.record(v.string(), v.any()))), signature: v.optional(v.string()), text: v.string(), type: v.literal("reasoning")}), v.object({data: v.string(), providerOptions: v.optional(v.record(v.string(), v.record(v.string(), v.any()))), type: v.literal("redacted-reasoning")}), v.object({args: v.any(), providerOptions: v.optional(v.record(v.string(), v.record(v.string(), v.any()))), toolCallId: v.string(), toolName: v.string(), type: v.literal("tool-call")})))), providerOptions: v.optional(v.record(v.string(), v.record(v.string(), v.any()))), role: v.literal("assistant")}), v.object({content: v.array(v.object({args: v.optional(v.any()), experimental_content: v.optional(v.array(v.union(v.object({text: v.string(), type: v.literal("text")}), v.object({data: v.string(), mimeType: v.optional(v.string()), type: v.literal("image")})))), isError: v.optional(v.boolean()), providerOptions: v.optional(v.record(v.string(), v.record(v.string(), v.any()))), result: v.any(), toolCallId: v.string(), toolName: v.string(), type: v.literal("tool-result")})), providerOptions: v.optional(v.record(v.string(), v.record(v.string(), v.any()))), role: v.literal("tool")}), v.object({content: v.string(), providerOptions: v.optional(v.record(v.string(), v.record(v.string(), v.any()))), role: v.literal("system")}))
The validation error seems to be with messages:addMessage so its very likely that the agent is trying to use addMessage and passing invalid data. Maybe you can show how addMessage gets used?