ibrahimyaacob•7mo ago

how to create a temporary file

i need to create a file on backend, in Nextjs, i have the ability to create a file using fs module on /tmp dir. Is there a way to do the same on convex? how do i run fs.createWriteStream ?

4 Replies

Michal Srb•7mo ago

You can create a Blob. What are you trying to achieve, why do you need a file?

ibrahimyaacobOP•7mo ago

i want to create a .jsonl file for open ai batch request. i previously used this snippet to achieve this on next backend

async function uploadJsonlFile(dataArray: any[], name: string) {
  // Create a temporary file path
  const tempFilePath = path.join('/tmp', `${name}.jsonl`);

  // Write array of objects to JSONL file
  const fileStream = fs.createWriteStream(tempFilePath);
  dataArray.forEach((obj) => {
    fileStream.write(JSON.stringify(obj) + '\n');
  });
  fileStream.end();

  // Wait for the file to be fully written
  await new Promise((resolve, reject) => {
    fileStream.on('finish', resolve);
    fileStream.on('error', reject);
  });

  const file = fs.createReadStream(`/tmp/${name}.jsonl`);
  const createdFile = await openai.files.create({
    file, // also can fetch file remotely
    purpose: 'batch',
  });

  const batch = await openai.batches.create({
    input_file_id: createdFile.id,
    endpoint: '/v1/chat/completions',
    completion_window: '24h',
  });

  return { fileId: createdFile.id, batchId: batch.id };
}

async function uploadJsonlFile(dataArray: any[], name: string) {
  // Create a temporary file path
  const tempFilePath = path.join('/tmp', `${name}.jsonl`);

  // Write array of objects to JSONL file
  const fileStream = fs.createWriteStream(tempFilePath);
  dataArray.forEach((obj) => {
    fileStream.write(JSON.stringify(obj) + '\n');
  });
  fileStream.end();

  // Wait for the file to be fully written
  await new Promise((resolve, reject) => {
    fileStream.on('finish', resolve);
    fileStream.on('error', reject);
  });

  const file = fs.createReadStream(`/tmp/${name}.jsonl`);
  const createdFile = await openai.files.create({
    file, // also can fetch file remotely
    purpose: 'batch',
  });

  const batch = await openai.batches.create({
    input_file_id: createdFile.id,
    endpoint: '/v1/chat/completions',
    completion_window: '24h',
  });

  return { fileId: createdFile.id, batchId: batch.id };
}

im not so sure how to achieve the same with convex

Michal Srb•7mo ago

Can you try

const fileContent: string = "Hello, this is the content of the file.";

// Convert the string into a Blob
const blob: Blob = new Blob([fileContent], { type: 'text/plain' });

// Create a File from the Blob
const file: File = new File([blob], "example.txt", { type: 'text/plain' });

const fileContent: string = "Hello, this is the content of the file.";

// Convert the string into a Blob
const blob: Blob = new Blob([fileContent], { type: 'text/plain' });

// Create a File from the Blob
const file: File = new File([blob], "example.txt", { type: 'text/plain' });

ibrahimyaacobOP•7mo ago

hey @ibrahimyaacob sorry taking a while to response to this, i can only work on my side project on weekends, your solution works like charm many thanks!

how to create a temporary file

Did you find this page helpful?