Khalil•10mo ago

Using Hono for HTTP in Convex issues with SSE

I am trying to make use of Hono's first class support for SSE https://hono.dev/helpers/streaming but Convex returns 502 (bad gateway). Is this not support by the platform?

8 Replies

KhalilOP•10mo ago

for reference: I am using the hono-convex template https://stack.convex.dev/hono-with-convex

Abhishek•10mo ago

Does for non streaming you are getting the response ?

KhalilOP•10mo ago

yes

 async (c) => {
    const chatbotId = c.req.param("chatbotId");

    const data = c.req.valid("json");
    const message = data.message;
    const chatHistory = data.messages ?? [];

    return streamSSE(c, async (stream) => {
      const writeSSE = async (message: SSEMessage) =>
        await stream.writeSSE(message);

      try {
        await chatHandler(chatbotId, message, chatHistory, writeSSE);
      } finally {
        await stream.close();
      }
    });
  }

 async (c) => {
    const chatbotId = c.req.param("chatbotId");

    const data = c.req.valid("json");
    const message = data.message;
    const chatHistory = data.messages ?? [];

    return streamSSE(c, async (stream) => {
      const writeSSE = async (message: SSEMessage) =>
        await stream.writeSSE(message);

      try {
        await chatHandler(chatbotId, message, chatHistory, writeSSE);
      } finally {
        await stream.close();
      }
    });
  }

sshader•10mo ago

At the moment we don't have support for response streaming (and the 502 is a poor error message here). We're investigating adding response streaming to support use cases like this. Also I'd be happy to learn more about your use case / try and brainstorm workarounds here

KhalilOP•10mo ago

thanks @sshader. I want to stream AI generated messages, and I don't want to use the DB to work as a "middleman", since i don't want to persist conversations (this is important) if convex supports SSE it would be a killer feature for me, currently migrating from a TRPC + Hono setup to Convex, let me know if you need any more context 🙂

ian•8mo ago

We just released http streaming! https://discord.com/channels/1019350475847499849/1019372556693815418/1240087869914218586

ian•8mo ago

Example code: https://github.com/sshader/streaming-chat-gpt/blob/sshader-streaming/convex/http.ts

GitHub

streaming-chat-gpt/convex/http.ts at sshader-streaming · sshader/st...

An example of streaming ChatGPT via the OpenAI v4.0 node SDK. - sshader/streaming-chat-gpt

ian•8mo ago

@Khalil ^

Using Hono for HTTP in Convex issues with SSE

Did you find this page helpful?