Zephyrrr•3mo ago

How to Get File Size in onUpload with Convex R2 When getMetadata Returns Null?

Hi everyone, I'm working with Convex and the R2 component for file uploads. My goal is to get the size of the uploaded file immediately after the upload completes, specifically within the onUpload callback, so I can track storage usage per user. The issue I'm encountering is that when I call r2.getMetadata(ctx, key) inside the onUpload function, it returns null. I suspect this is because onUpload might be running as part of the syncMetadata process, and the metadata isn't fully available or settled at that exact moment through r2.getMetadata. Here's the relevant part of my r2.clientApi configuration:

export const { generateUploadUrl, syncMetadata } = r2.clientApi({
  checkUpload: async (ctx, bucket) => {
    const user = await ctx.runQuery(api.users.current);
    if (!user) {
      throw new ConvexError("unauthorized");
    }
    // TODO: check if the storage is full
  },
  onUpload: async (ctx, key) => {
    // Problem: metadata is null here
    const metadata = await r2.getMetadata(ctx, key);
    console.log("Metadata in onUpload:", metadata);

    const user = await ctx.runQuery(api.users.current);
    if (!user || !metadata) {
      return;
    }

    await totalStorageAggregate.insert(ctx, {
      key: user._id,
      id: key,
      sumValue: metadata.size, // metadata.size would be undefined if metadata is null
    });
  },
  onDelete: async (ctx, key) => {
    const user = await ctx.runQuery(api.users.current);
    if (!user) {
      throw new ConvexError("unauthorized");
    }

    await totalStorageAggregate.delete(ctx, {
      key: user._id,
      id: key,
    });
  },
});

export const { generateUploadUrl, syncMetadata } = r2.clientApi({
  checkUpload: async (ctx, bucket) => {
    const user = await ctx.runQuery(api.users.current);
    if (!user) {
      throw new ConvexError("unauthorized");
    }
    // TODO: check if the storage is full
  },
  onUpload: async (ctx, key) => {
    // Problem: metadata is null here
    const metadata = await r2.getMetadata(ctx, key);
    console.log("Metadata in onUpload:", metadata);

    const user = await ctx.runQuery(api.users.current);
    if (!user || !metadata) {
      return;
    }

    await totalStorageAggregate.insert(ctx, {
      key: user._id,
      id: key,
      sumValue: metadata.size, // metadata.size would be undefined if metadata is null
    });
  },
  onDelete: async (ctx, key) => {
    const user = await ctx.runQuery(api.users.current);
    if (!user) {
      throw new ConvexError("unauthorized");
    }

    await totalStorageAggregate.delete(ctx, {
      key: user._id,
      id: key,
    });
  },
});

How can I reliably get the file size (and other metadata) for a newly uploaded file within the Convex backend flow, ideally associated with the onUpload event or shortly thereafter? Any insights, examples, or alternative approaches would be greatly appreciated! Thanks!

15 Replies

Convex Bot•3mo ago

Thanks for posting in <#1088161997662724167>. Reminder: If you have a Convex Pro account, use the Convex Dashboard to file support tickets. - Provide context: What are you trying to achieve, what is the end-user interaction, what are you seeing? (full error message, command output, etc.) - Use search.convex.dev to search Docs, Stack, and Discord all at once. - Additionally, you can post your questions in the Convex Community's <#1228095053885476985> channel to receive a response from AI. - Avoid tagging staff unless specifically instructed. Thank you!

erquhart•3mo ago

Makes sense, onUpload() runs before the actual upload, so you can throw an error to stop it if you choose. An afterUpload() hook is what you're looking for

ZephyrrrOP•3mo ago

yes, but there is no afterUpload() hook available is there any workaround?🥲

erquhart•3mo ago

I can put out a quick update that adds size to the onUpload args It would be straight from the file object before upload. Would that work? Or do you need more info

ZephyrrrOP•3mo ago

that'll be great👍 I think size is all I need

erquhart•3mo ago

to confirm, you're using useUploadFile right Because onUpload is technically onSyncMetadata

ZephyrrrOP•3mo ago

yes

erquhart•3mo ago

has to be afterUpload, anything else is just trusting file info passed in from the client should still be doable tonight

ZephyrrrOP•3mo ago

makes sense, never trust client. Look forward to seeing it🫡

erquhart•3mo ago

Released in 0.6.0: https://github.com/get-convex/r2?tab=readme-ov-file#accessing-metadata-after-upload

ZephyrrrOP•3mo ago

cool, will check it out once have time! It seems that I can't get auth info from the context

export const r2 = new R2(components.r2);
const callbacks: R2Callbacks = internal.r2;

export const { generateUploadUrl, syncMetadata, onSyncMetadata } = r2.clientApi(
  {
    callbacks,
    checkUpload: async (ctx, bucket) => {
      const user = await ctx.runQuery(api.users.current);
      if (!user) {
        throw new ConvexError("unauthorized");
      }
      const totalStorageStats = await ctx.runQuery(api.r2.getTotalStorageStats);

      if (
        totalStorageStats.totalSizeMB >
        (totalStorageStats.storageLimit ?? 0) * 1024
      ) {
        throw new ConvexError("storage full");
      }
    },
    onSyncMetadata: async (ctx, args) => {
      const user = await ctx.runQuery(api.users.current);
      const identity = await ctx.auth.getUserIdentity() 
      
      const metadata = await r2.getMetadata(ctx, args.key);
      
      console.log({ metadata, user, identity });
      if (!user) {
        throw new ConvexError("unauthorized");
      }

      if (!metadata) {
        throw new ConvexError("file not found");
      }

      await totalStorageAggregate.insert(ctx, {
        key: user._id,
        id: args.key,
        sumValue: metadata.size,
      });
    },
    onDelete: async (ctx, key) => {
      const user = await ctx.runQuery(api.users.current);
      if (!user) {
        throw new ConvexError("unauthorized");
      }

      await totalStorageAggregate.delete(ctx, {
        key: user._id,
        id: key,
      });
    },
  }
);

export const r2 = new R2(components.r2);
const callbacks: R2Callbacks = internal.r2;

export const { generateUploadUrl, syncMetadata, onSyncMetadata } = r2.clientApi(
  {
    callbacks,
    checkUpload: async (ctx, bucket) => {
      const user = await ctx.runQuery(api.users.current);
      if (!user) {
        throw new ConvexError("unauthorized");
      }
      const totalStorageStats = await ctx.runQuery(api.r2.getTotalStorageStats);

      if (
        totalStorageStats.totalSizeMB >
        (totalStorageStats.storageLimit ?? 0) * 1024
      ) {
        throw new ConvexError("storage full");
      }
    },
    onSyncMetadata: async (ctx, args) => {
      const user = await ctx.runQuery(api.users.current);
      const identity = await ctx.auth.getUserIdentity() 
      
      const metadata = await r2.getMetadata(ctx, args.key);
      
      console.log({ metadata, user, identity });
      if (!user) {
        throw new ConvexError("unauthorized");
      }

      if (!metadata) {
        throw new ConvexError("file not found");
      }

      await totalStorageAggregate.insert(ctx, {
        key: user._id,
        id: args.key,
        sumValue: metadata.size,
      });
    },
    onDelete: async (ctx, key) => {
      const user = await ctx.runQuery(api.users.current);
      if (!user) {
        throw new ConvexError("unauthorized");
      }

      await totalStorageAggregate.delete(ctx, {
        key: user._id,
        id: key,
      });
    },
  }
);

Logs:

5/27/2025, 10:10:45 PM [CONVEX M(r2:onSyncMetadata)] [LOG] {
  metadata: {
    bucket: 'yxxxxx',
    bucketLink: 'https://dash.cloudflare.com/fe8f2b07ab2xxxxxcef93995b405e00/r2/default/buckets/xxxxx',
    contentType: 'image/webp',
    key: '7bdc4f1c-cxxxx6fd-2957e8c07642',
    lastModified: '2025-05-28T05:10:45.000Z',
    link: 'https://dash.cloudflare.com/fe8f2b07ab2xxxxef93995b405e00/r2/default/buckets/yixxxst/objects/7bdc4f1c-xxxxx-86fd-2957e8c07642/details',
    size: 11522,
    url: 'https://xyxx-xxx.fe8f2b07ab27d9xx3995b405e00.r2.cloudflarestorage.com/7bdc4f1c-c6ac-4385-8xx8c07642?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=d60ccad9b8695062d7d010ed21cbcc3a%2F20250528%2Fauto%2Fs3%2Faws4_request&X-Amz-Date=20250528T051045Z&X-Amz-Expires=900&X-Amz-Signature=6b860bdb18ef3c583ad9a3c8a1add793b490bb963be7e3a18275117d91f59fb5&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject'
  },
  user: null,
  identity: null
}

5/27/2025, 10:10:45 PM [CONVEX M(r2:onSyncMetadata)] [LOG] {
  metadata: {
    bucket: 'yxxxxx',
    bucketLink: 'https://dash.cloudflare.com/fe8f2b07ab2xxxxxcef93995b405e00/r2/default/buckets/xxxxx',
    contentType: 'image/webp',
    key: '7bdc4f1c-cxxxx6fd-2957e8c07642',
    lastModified: '2025-05-28T05:10:45.000Z',
    link: 'https://dash.cloudflare.com/fe8f2b07ab2xxxxef93995b405e00/r2/default/buckets/yixxxst/objects/7bdc4f1c-xxxxx-86fd-2957e8c07642/details',
    size: 11522,
    url: 'https://xyxx-xxx.fe8f2b07ab27d9xx3995b405e00.r2.cloudflarestorage.com/7bdc4f1c-c6ac-4385-8xx8c07642?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=d60ccad9b8695062d7d010ed21cbcc3a%2F20250528%2Fauto%2Fs3%2Faws4_request&X-Amz-Date=20250528T051045Z&X-Amz-Expires=900&X-Amz-Signature=6b860bdb18ef3c583ad9a3c8a1add793b490bb963be7e3a18275117d91f59fb5&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject'
  },
  user: null,
  identity: null
}

@erquhart am I missing something? I am using clerk btw, and I can get identify from ctx in other functions

erquhart•3mo ago

Ah yeah, that runs in a scheduled function. Have to think about how to get auth to it

ZephyrrrOP•3mo ago

I see, so it is expected behavior.

erquhart•3mo ago

Yeah, but it needs to be documented at least. Best bet is to use onUpload to persist whatever part of the operation you’re trying to do that makes sense - you have the key and an authenticated user there, just no metadata yet. Then after metadata sync, look up by key to get the user associated with the file.

ZephyrrrOP•3mo ago

good point, I'll try it It works like a charm! Thank you so much

How to Get File Size in onUpload with Convex R2 When getMetadata Returns Null?

Did you find this page helpful?