whoami•2y ago

Is there a way to get the progress of file uploading in convex storage?

Something like this, provided by upload.io

const uploadResult = await uploader.uploadFile(file, {
        onProgress: (p) => {
          setUploadProgress(p.progress)
        },
      })

const uploadResult = await uploader.uploadFile(file, {
        onProgress: (p) => {
          setUploadProgress(p.progress)
        },
      })

9 Replies

ian•2y ago

There isn't anything provided right now. If it's important, you could have an httpAction that you upload the file to, and that one could use the generateUploadUrl functionality, and read from the client stream and write to the url stream, capturing progress along the way. Admittedly this is a bit hand-wavy - I haven't ever done this. You could also be doing this from the client, if you're uploading to a getUploadUrl-provided URL, as I believe those support streaming out of the box. Everything over the websocket gets fully serialized before sending up as a message, so you wouldn't have much insight if sent that way. In general if you're uploading a tiny thing (<1MB), an action is OK. If it's a medium-sized thing (~5MB), sending to an httpAction is OK. If it's on the order of GB, then getting an upload URL and sending it directly is your best bet. Also, feel free to use upload.io alongside convex. You could probably use their SDK inside an action, though I'm not sure if you'll have to add the "use node"; directive (though it seems you could pass the built-in fetch rather than node-fetch to their sdk, so you might not need any node-specific dependencies

aheimlich•2y ago

Another option, if you want Convex to store your files, is to use Uppy (https://uppy.io) and configure its XHR uploader (https://uppy.io/docs/xhr-upload/) with either the URL to a HTTP Action, or the URL returned by generateUploadUrl. Uppy itself will handle displaying upload progress, among many other things. You might even be able to write a custom Uppy uploader plugin that uses upload.io's SDK, if you still want to upload files to upload.io.

Uppy

Sleek, modular open source JavaScript file uploader

Uppy

Sleek, modular open source JavaScript file uploader

jamwt•2y ago

cool idea!

aheimlich•2y ago

Thanks! Uppy is a wonderful library, and I have been using it on various projects for years.

whoamiOP•2y ago

wow thanks for the suggestion! @aheimlich hey there, I’m really new to uppy, is it easy to customize the UI for uppy with convex upload url without having to use their dashboard to show the progress bar? I found their examples are little bit tricky. Thanks in advance.

aheimlich•2y ago

I normally use Uppy's dashboard and customize the CSS. You can also use the status bar completely by itself. However, there was one instance where I tied Uppy to an instance of Material UI's Circular Progress Bar component, because the area I wanted to display upload progress in was simply too small for any of Uppy's built-in UI components. I can throw together a quick code sample sometime tommorrow

whoamiOP•2y ago

Oh great thank you ❤️

aheimlich•2y ago

import * as React from 'react';
import PropTypes from 'prop-types';
import CircularProgress from '@mui/material/CircularProgress';
import Typography from '@mui/material/Typography';
import Box from '@mui/material/Box';
import Uppy from "@uppy/core";

/**
 * See https://mui.com/material-ui/react-progress/#circular-with-label
 */
function CircularProgressWithLabel(props) {
  return (
    <Box sx={{ position: 'relative', display: 'inline-flex' }}>
      <CircularProgress variant="determinate" {...props} />
      <Box
        sx={{
          top: 0,
          left: 0,
          bottom: 0,
          right: 0,
          position: 'absolute',
          display: 'flex',
          alignItems: 'center',
          justifyContent: 'center',
        }}
      >
        <Typography variant="caption" component="div" color="text.secondary">
          {`${Math.round(props.value)}%`}
        </Typography>
      </Box>
    </Box>
  );
}

CircularProgressWithLabel.propTypes = {
  /**
   * The value of the progress indicator for the determinate variant.
   * Value between 0 and 100.
   * @default 0
   */
  value: PropTypes.number.isRequired,
};

export default function UppyProgressBar(props) {
  const { uppy } = props;
  const [progress, setProgress] = React.useState(0);

  React.useEffect(() => {
    /**
     * See https://uppy.io/docs/uppy/#progress
     */
    const onProgress = (progress) => {
      setProgress(progress);
    };

    /**
     * See https://uppy.io/docs/uppy/#reset-progress
     */
    const onResetProgress = () => {
      setProgress(0);
    };

    uppy
      .on("progress", onProgress)
      .on("reset-progress", onResetProgress);

    return () => {
      uppy
        .off("progress", onProgress)
        .off("reset-progress", onResetProgress);
    };
  }, [uppy]);

  return <CircularProgressWithLabel value={progress} />;
}

UppyProgressBar.propTypes = {
  uppy: PropTypes.instanceOf(Uppy).isRequired
}

import * as React from 'react';
import PropTypes from 'prop-types';
import CircularProgress from '@mui/material/CircularProgress';
import Typography from '@mui/material/Typography';
import Box from '@mui/material/Box';
import Uppy from "@uppy/core";

/**
 * See https://mui.com/material-ui/react-progress/#circular-with-label
 */
function CircularProgressWithLabel(props) {
  return (
    <Box sx={{ position: 'relative', display: 'inline-flex' }}>
      <CircularProgress variant="determinate" {...props} />
      <Box
        sx={{
          top: 0,
          left: 0,
          bottom: 0,
          right: 0,
          position: 'absolute',
          display: 'flex',
          alignItems: 'center',
          justifyContent: 'center',
        }}
      >
        <Typography variant="caption" component="div" color="text.secondary">
          {`${Math.round(props.value)}%`}
        </Typography>
      </Box>
    </Box>
  );
}

CircularProgressWithLabel.propTypes = {
  /**
   * The value of the progress indicator for the determinate variant.
   * Value between 0 and 100.
   * @default 0
   */
  value: PropTypes.number.isRequired,
};

export default function UppyProgressBar(props) {
  const { uppy } = props;
  const [progress, setProgress] = React.useState(0);

  React.useEffect(() => {
    /**
     * See https://uppy.io/docs/uppy/#progress
     */
    const onProgress = (progress) => {
      setProgress(progress);
    };

    /**
     * See https://uppy.io/docs/uppy/#reset-progress
     */
    const onResetProgress = () => {
      setProgress(0);
    };

    uppy
      .on("progress", onProgress)
      .on("reset-progress", onResetProgress);

    return () => {
      uppy
        .off("progress", onProgress)
        .off("reset-progress", onResetProgress);
    };
  }, [uppy]);

  return <CircularProgressWithLabel value={progress} />;
}

UppyProgressBar.propTypes = {
  uppy: PropTypes.instanceOf(Uppy).isRequired
}

Sorry for the delay, here (above) is a very basic code sample tying Uppy to Material UI's Circular Progress Bar component

whoamiOP•2y ago

wow beautiful, thank you @aheimlich

Is there a way to get the progress of file uploading in convex storage?

Did you find this page helpful?