John•11mo ago

{"code":"BadJsonBody","message":"Failed to buffer the request body: length limit exceeded"}

I'm currently trying to use the stable diffusion image-to-image api to generate images and save them to Convex storage. I initially tried using a Nextjs API route to call the api, and then call a Convex action to store each generated image. The issue is that the API returns images as base64 encoded strings. When I try to pass the string to the Convex action, some images will throw the following error and not get stored (seems like the image strings are too long):

{"code":"BadJsonBody","message":"Failed to buffer the request body: length limit exceeded"}

{"code":"BadJsonBody","message":"Failed to buffer the request body: length limit exceeded"}

I then tried creating a Convex action to make the API request directly in an action and store the images. However, if I try to generate and store more than ~3 images in a single request, calling the action from the client throws the following error:

Error: [CONVEX A(stablediffusion:generateImage)] Connection lost while action was in flight

Error: [CONVEX A(stablediffusion:generateImage)] Connection lost while action was in flight

The images get stored successfully, but because of the timeout, I'm not getting the return data to display. Any suggestions for a better way to set this up?

4 Replies

JohnOP•11mo ago

This is the Convex action that times out when called directly.

"use node";
import { v } from "convex/values";
import { ActionCtx, action } from "./_generated/server";
import { internal } from "./_generated/api";
import { Id } from "./_generated/dataModel";

interface GenerationResponse {
  artifacts: Array<{
    base64: string;
    seed: number;
    finishReason: string;
  }>;
}

export const generateImage = action({
  args: { base64Image: v.string() },
  handler: async (ctx, args) => {
    const identity = await ctx.auth.getUserIdentity();
    if (!identity) {
      throw new Error("you must be logged in to upload images");
    }
    const user = await ctx.runQuery(internal.users.getUserById, {
      userId: identity.subject,
    });

    if (!user) {
      throw new Error("user not found");
    }

    const sourceImage = args.base64Image;
    const imageBlob = base64ToBlob(sourceImage);

    const formData = new FormData();
    formData.append("init_image", imageBlob);
    formData.append("samples", "10"); // number of images to generate
     //... more params, out of text space for discord

    const response = await fetch(
      "https://api.stability.ai/v1/generation/stable-diffusion-xl-1024-v1-0/image-to-image",
      {
        headers: {
          Authorization: `Bearer ${process.env.STABLE_DIFFUSION_API_KEY}`,
        },
        body: formData,
        method: "POST",
      },
    );

    const responseJSON = (await response.json()) as GenerationResponse;

    const images = responseJSON.artifacts.map(
      (artifact) => `data:image/jpeg;base64,${artifact.base64}`,
    );

    const storeImagePromises = images.map((image) => {
      console.log("storing", image);
      return store(ctx, image, user._id);
    });

    await Promise.all(storeImagePromises);

    return images;
  },
});

"use node";
import { v } from "convex/values";
import { ActionCtx, action } from "./_generated/server";
import { internal } from "./_generated/api";
import { Id } from "./_generated/dataModel";

interface GenerationResponse {
  artifacts: Array<{
    base64: string;
    seed: number;
    finishReason: string;
  }>;
}

export const generateImage = action({
  args: { base64Image: v.string() },
  handler: async (ctx, args) => {
    const identity = await ctx.auth.getUserIdentity();
    if (!identity) {
      throw new Error("you must be logged in to upload images");
    }
    const user = await ctx.runQuery(internal.users.getUserById, {
      userId: identity.subject,
    });

    if (!user) {
      throw new Error("user not found");
    }

    const sourceImage = args.base64Image;
    const imageBlob = base64ToBlob(sourceImage);

    const formData = new FormData();
    formData.append("init_image", imageBlob);
    formData.append("samples", "10"); // number of images to generate
     //... more params, out of text space for discord

    const response = await fetch(
      "https://api.stability.ai/v1/generation/stable-diffusion-xl-1024-v1-0/image-to-image",
      {
        headers: {
          Authorization: `Bearer ${process.env.STABLE_DIFFUSION_API_KEY}`,
        },
        body: formData,
        method: "POST",
      },
    );

    const responseJSON = (await response.json()) as GenerationResponse;

    const images = responseJSON.artifacts.map(
      (artifact) => `data:image/jpeg;base64,${artifact.base64}`,
    );

    const storeImagePromises = images.map((image) => {
      console.log("storing", image);
      return store(ctx, image, user._id);
    });

    await Promise.all(storeImagePromises);

    return images;
  },
});

These are the helper functions used in the action:

async function store(ctx: ActionCtx, base64Image: string, userId: Id<"users">) {
  const blob = base64ToBlob(base64Image);
  const storageId: Id<"_storage"> = await ctx.storage.store(blob as Blob);
  await ctx.runMutation(internal.images.storeResult, {
    storageId: storageId,
    user: userId,
  });
}

function base64ToBlob(base64: string) {
  const byteCharacters = atob(base64.split(",")[1]);
  const byteNumbers = new Array(byteCharacters.length);
  for (let i = 0; i < byteCharacters.length; i++) {
    byteNumbers[i] = byteCharacters.charCodeAt(i);
  }
  const byteArray = new Uint8Array(byteNumbers);
  return new Blob([byteArray], { type: "image/png" });
}

async function store(ctx: ActionCtx, base64Image: string, userId: Id<"users">) {
  const blob = base64ToBlob(base64Image);
  const storageId: Id<"_storage"> = await ctx.storage.store(blob as Blob);
  await ctx.runMutation(internal.images.storeResult, {
    storageId: storageId,
    user: userId,
  });
}

function base64ToBlob(base64: string) {
  const byteCharacters = atob(base64.split(",")[1]);
  const byteNumbers = new Array(byteCharacters.length);
  for (let i = 0; i < byteCharacters.length; i++) {
    byteNumbers[i] = byteCharacters.charCodeAt(i);
  }
  const byteArray = new Uint8Array(byteNumbers);
  return new Blob([byteArray], { type: "image/png" });
}

Sam J•11mo ago

Hi John, do you have the instance name where you're seeing these errors in the Action? You can find in the url on your dashboard or as part of your deployment URL. If you post that here or DM me I can look into the connection lost issue. In the meantime, you may be able to use your original approach, but then upload the blobs directly from the client. You can then pass the storage IDs into an action or mutation to store the user <-> id mapping. See https://docs.convex.dev/file-storage/upload-files for an example of how to do this.

Uploading and Storing Files | Convex Developer Hub

Files can be uploaded by your users and stored in Convex.

JohnOP•11mo ago

Instance name: precious-squid-669

Sam J•11mo ago

Thanks. We need to improve the errror message, but I believe you're seeing that error because the size of the return value from your Action, images, exceeds an internal limit. Can you modify your Action to return the Storage IDs, rather than the image blobs?

{"code":"BadJsonBody","message":"Failed to buffer the request body: length limit exceeded"}

Did you find this page helpful?