sbkl•2mo ago

Using gemini with openai library working in action but not http action

I followed this guide to integrate ai streaming in my convex app. But instead of using openai, I want to use gemini because of their larger context window (1M tokens for gemini-1.5-flash). I get a bad request answer just using the basic example from google docs here. However, the exact same code works in actions which I assume it is due to the fact that http action cannot run node packages? What I don't get is why it is supposed to work with openAI (cannot try as I am not located in a country where opeanai is available) but it doesn't work for gemini. But still works in actions. Any idea if there is a way to make it work? Here is the code as an example (gemini api key created via google ai studio):

  const openai = new OpenAI({
    apiKey: env.GEMINI_API_KEY,
    baseURL: "https://generativelanguage.googleapis.com/v1beta/openai/",
  });
  const response = await openai.chat.completions.create({
    model: "gemini-1.5-flash",
    messages: [
        { role: "system", content: "You are a helpful assistant." },
        {
            role: "user",
            content: "Explain to me how AI works",
        },
    ],
  });

  const openai = new OpenAI({
    apiKey: env.GEMINI_API_KEY,
    baseURL: "https://generativelanguage.googleapis.com/v1beta/openai/",
  });
  const response = await openai.chat.completions.create({
    model: "gemini-1.5-flash",
    messages: [
        { role: "system", content: "You are a helpful assistant." },
        {
            role: "user",
            content: "Explain to me how AI works",
        },
    ],
  });

AI Chat with HTTP Streaming

By leveraging HTTP actions with streaming, this chat app balances real-time responsiveness with efficient bandwidth usage. Users receive character-by-...

6 Replies

Convex Bot•2mo ago

Thanks for posting in <#1088161997662724167>. Reminder: If you have a Convex Pro account, use the Convex Dashboard to file support tickets. - Provide context: What are you trying to achieve, what is the end-user interaction, what are you seeing? (full error message, command output, etc.) - Use search.convex.dev to search Docs, Stack, and Discord all at once. - Additionally, you can post your questions in the Convex Community's <#1228095053885476985> channel to receive a response from AI. - Avoid tagging staff unless specifically instructed. Thank you!

FleetAdmiralJakob 🗕 🗗 🗙•2mo ago

Use of the Node.js environment is restricted to action functions only. If you want to use a library designed for Node.js and interact with the Convex database, you need to call the Node.js library from an action, and use runQuery or runMutation helper to call a query or mutation.

https://docs.convex.dev/functions/runtimes#nodejs-runtime

Runtimes | Convex Developer Hub

Convex functions can run in two runtimes:

FleetAdmiralJakob 🗕 🗗 🗙•2mo ago

In your case the http action has to call an action to do node specific stuff

sbkl.•2mo ago

Yes that was my first assumption but that is contradictory with the example from convex I linked to stream data from a llm and I think it is more linked to a bug on the integration of the OpenAI library with Gemini. I ended up building a simple class reproducing the OpenAI library api to integrate with Gemini and it works great.

lee•2mo ago

To state it explicitly, http actions can use npm packages as long as the package doesn't depend on node. Last i checked, the openai library doesn't depend on node, but I wouldn't be surprised if that changed with a recent update

sbklOP•2mo ago

Thanks. Could be a reason. Ended up building a simple class mocking the openai library api for gemini api. Works great.

Using gemini with openai library working in action but not http action

Did you find this page helpful?