CLI Export issue on VM with older Node version
❔Advice
Howdy, we are fairly new users of Convex, I'm working with our BI team to do OLAP reporting on what's happening in our app.
Currently we are attempting to build an automated pipeline that exports, parses and uploads our Convex data to Big Query for analysis. This is done with a series of Python scripts and a Convex CLI command (export, specifically).
The scripts all work fine except for one issue, we are using Matillion ETL as our orchestration platform, and when it runs the "npx convex export --[path]" command, I get an error that it using an older version of Node.js than the Convex CLI needs.
I have been able, on the same machine in a different account, to upgrade the Node.js version and successfully run the CLI command, but I can't seem to force Matillion to use that version when calling Matillion components (Python or Bash scripts). (Note: I have raised this with their support team and am working on that side as well, but thought I might try here too).
Because we have to (?) use the prod deploy key to run the export for prod data, I am very reluctant to just sling the key around a lot of places.
If anyone has any ideas on how I might implement this, either with this configuration or a different option (maybe it's own VM with a cron job), I am open to suggestions.
Also, I don't know if the "export" functionality is avialable via the Python or Node convex libraries, my understanding is that it's only avialable via the CLI.
Any insight or information would be much appreciated.
Currently we are attempting to build an automated pipeline that exports, parses and uploads our Convex data to Big Query for analysis. This is done with a series of Python scripts and a Convex CLI command (export, specifically).
The scripts all work fine except for one issue, we are using Matillion ETL as our orchestration platform, and when it runs the "npx convex export --[path]" command, I get an error that it using an older version of Node.js than the Convex CLI needs.
I have been able, on the same machine in a different account, to upgrade the Node.js version and successfully run the CLI command, but I can't seem to force Matillion to use that version when calling Matillion components (Python or Bash scripts). (Note: I have raised this with their support team and am working on that side as well, but thought I might try here too).
Because we have to (?) use the prod deploy key to run the export for prod data, I am very reluctant to just sling the key around a lot of places.
If anyone has any ideas on how I might implement this, either with this configuration or a different option (maybe it's own VM with a cron job), I am open to suggestions.
Also, I don't know if the "export" functionality is avialable via the Python or Node convex libraries, my understanding is that it's only avialable via the CLI.
Any insight or information would be much appreciated.
