Zeroday•3w ago

Migrations calling HTTP

Hi, I wanted some advice on how I would have a migration call an HTTP function. For example: Run migration on all documents that will Call an API, wait for the API to return Update the document with some info returned from API call I'm not a typescript pro, so some help would be amazing.

5 Replies

Convex Bot•3w ago

Thanks for posting in <#1088161997662724167>. Reminder: If you have a Convex Pro account, use the Convex Dashboard to file support tickets. - Provide context: What are you trying to achieve, what is the end-user interaction, what are you seeing? (full error message, command output, etc.) - Use search.convex.dev to search Docs, Stack, and Discord all at once. - Additionally, you can post your questions in the Convex Community's <#1228095053885476985> channel to receive a response from AI. - Avoid tagging staff unless specifically instructed. Thank you!

erquhart•3w ago

It doesn’t sound like you need http actions, just have your migration function be an action that fetches data and uses ctx.runMutation to update the document.

ZerodayOP•3w ago

Hey! Thanks for the reply. Can you give me an example?

erquhart•3w ago

I can, I’ll be back at my machine in a couple of hours if no one beats me to it (on mobile) Ran your original post through kapa in the docs (you can also chat with kapa in #ask-ai), the recommendation here looks good to me: The proper approach for this migration scenario would be to use the scheduler to run an action after the mutation completes:

export const callApiAndUpdateDocs = migrations.define({
  table: "yourTableName",
  migrateOne: async (ctx, document) => {
    // Schedule an action to run after the mutation completes
    await ctx.scheduler.runAfter(0, internal.yourModule.fetchAndUpdateFromExternalApi, {
      documentId: document._id,
      someField: document.someField
    });
    
    // Mark that we've scheduled the action
    await ctx.db.patch(document._id, { 
      migrationScheduled: true
    });
  },
});

export const callApiAndUpdateDocs = migrations.define({
  table: "yourTableName",
  migrateOne: async (ctx, document) => {
    // Schedule an action to run after the mutation completes
    await ctx.scheduler.runAfter(0, internal.yourModule.fetchAndUpdateFromExternalApi, {
      documentId: document._id,
      someField: document.someField
    });
    
    // Mark that we've scheduled the action
    await ctx.db.patch(document._id, { 
      migrationScheduled: true
    });
  },
});

Then create an action that both makes the API call and updates the document:

export const fetchAndUpdateFromExternalApi = internalAction({
  args: { 
    documentId: v.id("yourTableName"),
    someField: v.string() 
  },
  handler: async (ctx, args) => {
    // Make your API call
    const response = await fetch("https://your-api-endpoint.com", {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        field: args.someField
      }),
    });

    if (!response.ok) {
      throw new Error(`API call failed: ${response.statusText}`);
    }

    // Parse the API response
    const data = await response.json();
    
    // Update the document with the API result
    await ctx.runMutation(internal.yourModule.updateDocWithApiResult, {
      documentId: args.documentId,
      apiData: data
    });
    
    return data;
  },
});

export const updateDocWithApiResult = internalMutation({
  args: { 
    documentId: v.id("yourTableName"),
    apiData: v.any()
  },
  handler: async (ctx, args) => {
    await ctx.db.patch(args.documentId, { 
      apiData: args.apiData,
      migrationComplete: true
    });
  }
});

export const fetchAndUpdateFromExternalApi = internalAction({
  args: { 
    documentId: v.id("yourTableName"),
    someField: v.string() 
  },
  handler: async (ctx, args) => {
    // Make your API call
    const response = await fetch("https://your-api-endpoint.com", {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        field: args.someField
      }),
    });

    if (!response.ok) {
      throw new Error(`API call failed: ${response.statusText}`);
    }

    // Parse the API response
    const data = await response.json();
    
    // Update the document with the API result
    await ctx.runMutation(internal.yourModule.updateDocWithApiResult, {
      documentId: args.documentId,
      apiData: data
    });
    
    return data;
  },
});

export const updateDocWithApiResult = internalMutation({
  args: { 
    documentId: v.id("yourTableName"),
    apiData: v.any()
  },
  handler: async (ctx, args) => {
    await ctx.db.patch(args.documentId, { 
      apiData: args.apiData,
      migrationComplete: true
    });
  }
});

ZerodayOP•3w ago

Interesting, I was trying something similar and it was basically scheduling the API calls and continuing to the next batch resulting in ratelimits because the API was being bombarded with requests, but this worked for me thanks!

Migrations calling HTTP

Did you find this page helpful?