Help with Openai Whipser

so in convex, I'm trying to generate some text to speech using this sdk

async function generateAudio(text: string) {
  const mp3 = await openai.audio.speech.create({
    model: "tts-1",
    voice: "onyx",
    input: text,
  });
  const buffer = Buffer.from(await mp3.arrayBuffer());
  return buffer;
}

async function generateAudio(text: string) {
  const mp3 = await openai.audio.speech.create({
    model: "tts-1",
    voice: "onyx",
    input: text,
  });
  const buffer = Buffer.from(await mp3.arrayBuffer());
  return buffer;
}

and then I'm trying to pass that to this type of function

export async function getTranscripts(audio: Buffer) {
  try {
    const transcription = await openai.audio.transcriptions.create({
      file: audio,
      model: "whisper-1",
      timestamp_granularities: ["word"],
      response_format: "verbose_json",
    });
    return transcription;
  } finally {
    // No cleanup needed as we didn't create any files
  }
}

export async function getTranscripts(audio: Buffer) {
  try {
    const transcription = await openai.audio.transcriptions.create({
      file: audio,
      model: "whisper-1",
      timestamp_granularities: ["word"],
      response_format: "verbose_json",
    });
    return transcription;
  } finally {
    // No cleanup needed as we didn't create any files
  }
}

3 Replies

Web Dev CodyOP•4mo ago

but file needs to be an actual file, but File doesn't exist on node nvm got it

DeepakDahiya•4mo ago

Would you mind sharing your solution?

Web Dev CodyOP•4mo ago

@DeepakDahiya yeah

      file: await toFile(audio, "audio.mp3", { type: "audio/mp3" }),

      file: await toFile(audio, "audio.mp3", { type: "audio/mp3" }),

toFile is an import from openai import { toFile } from "openai/uploads"; audio is a Buffer

Help with Openai Whipser

Did you find this page helpful?