Passing `ctx` to AI agent tools

I'm currently doing the following inside a convex action:

const result = await createBlockFromPromptAls.run(
      {
        ctx: ctx,
        clerkOrgId: args.clerkOrgId,
        blockId: args.blockId,
        workspaceCollectionInfo: args.workspaceCollectionInfo,
      },
      () =>
        generateText({
          model: anthropic("claude-3-5-sonnet-20240620"),
          prompt,
          maxToolRoundtrips: 5, // allow up to 5 tool roundtrips
          tools: {
            getFields: FireviewAITools.getFields,
            createTableBlockOneShot: FireviewAITools.createTableBlockOneShot,
          },
        })
    );

const result = await createBlockFromPromptAls.run(
      {
        ctx: ctx,
        clerkOrgId: args.clerkOrgId,
        blockId: args.blockId,
        workspaceCollectionInfo: args.workspaceCollectionInfo,
      },
      () =>
        generateText({
          model: anthropic("claude-3-5-sonnet-20240620"),
          prompt,
          maxToolRoundtrips: 5, // allow up to 5 tool roundtrips
          tools: {
            getFields: FireviewAITools.getFields,
            createTableBlockOneShot: FireviewAITools.createTableBlockOneShot,
          },
        })
    );

where I'm using an ALS like so:

export const createBlockFromPromptAls = new AsyncLocalStorage<{
  ctx: AuthActionCtx;
  clerkOrgId: string;
  blockId: Id<"blocks">;
  workspaceCollectionInfo: z.infer<typeof zWorkspaceCollectionsInfo>;
}>();

export const createBlockFromPromptAls = new AsyncLocalStorage<{
  ctx: AuthActionCtx;
  clerkOrgId: string;
  blockId: Id<"blocks">;
  workspaceCollectionInfo: z.infer<typeof zWorkspaceCollectionsInfo>;
}>();

so that tools can access the ctx to query or write to the Convex db, e.g.

export const getFields = tool({
    parameters: ...
    execute: async ({ collectionId }) => {
      const store = createBlockFromPromptAls.getStore();
      const { ctx, clerkOrgId } = store;
      ctx.runQuery...

export const getFields = tool({
    parameters: ...
    execute: async ({ collectionId }) => {
      const store = createBlockFromPromptAls.getStore();
      const { ctx, clerkOrgId } = store;
      ctx.runQuery...

HOWEVER, I'm running into:

[ERROR] Could not resolve "node:async_hooks"
[CONVEX]     lib/ai/als.ts:1:34:
[CONVEX]       1 │ import { AsyncLocalStorage } from "node:async_hooks";
[CONVEX]         ╵                                   ~~~~~~~~~~~~~~~~~~
[CONVEX] 
[CONVEX]   The package "node:async_hooks" wasn't found on the file system but is built into node. Are you trying to bundle for node? You can use "platform: 'node'" to do that, which will remove this error.

[ERROR] Could not resolve "node:async_hooks"
[CONVEX]     lib/ai/als.ts:1:34:
[CONVEX]       1 │ import { AsyncLocalStorage } from "node:async_hooks";
[CONVEX]         ╵                                   ~~~~~~~~~~~~~~~~~~
[CONVEX] 
[CONVEX]   The package "node:async_hooks" wasn't found on the file system but is built into node. Are you trying to bundle for node? You can use "platform: 'node'" to do that, which will remove this error.

which i could fix by using the use node directive, but I'm trying to stay away from it for performance reasons. Any ideas on how do do this with edge runtime packages?

3 Replies

ballingt•5mo ago

The Convex JS runtime doesn't currently support AsyncLocalStorage, so you won't be able to do this there. However only one mutation/query/action is running in any given isolate, so you can just set a global variable with this; you don't need to use AsyncLocalStorage to try to keep the right kind of context to each of the different async threads of execution. If you need it for a library you're using we want to hear about it, but instead of using it manually for this kind of thing you can just set a global variable. This global state won't be shared between mutation / query / action runs.

David AlonsoOP•5mo ago

Thanks for the suggestion @ballingt ! I'm trying the following: Action file

import "../../../../lib/ai/globalStore";
....
  handler: async (ctx, args) => {
    // Set global state
    globalStore = {
      ctx,
      clerkOrgId: args.clerkOrgId,
      blockId: args.blockId,
      workspaceCollectionInfo: args.workspaceCollectionInfo,
    };

import "../../../../lib/ai/globalStore";
....
  handler: async (ctx, args) => {
    // Set global state
    globalStore = {
      ctx,
      clerkOrgId: args.clerkOrgId,
      blockId: args.blockId,
      workspaceCollectionInfo: args.workspaceCollectionInfo,
    };

Global store file:

export type GlobalStore = {
  ctx: AuthActionCtx;
  clerkOrgId: string;
  blockId: Id<"blocks">;
  workspaceCollectionInfo: z.infer<typeof zWorkspaceCollectionsInfo>;
} | null;

declare global {
  var globalStore: GlobalStore;
}

export type GlobalStore = {
  ctx: AuthActionCtx;
  clerkOrgId: string;
  blockId: Id<"blocks">;
  workspaceCollectionInfo: z.infer<typeof zWorkspaceCollectionsInfo>;
} | null;

declare global {
  var globalStore: GlobalStore;
}

When I call the action I get:

Server Error
Uncaught ReferenceError: globalStore is not defined

not very familiar with the best way to manage global stores..

ballingt•5mo ago

Ah are you trying to use a truly global variable? You can do that by setting properties on globalThis, globalThis.globalStore. When you write globalStore = { ... } ~~that just creates a module-level variable~~ that's invalid syntax in strict mode.

Passing `ctx` to AI agent tools

Did you find this page helpful?