k.yr•2y ago

Invalid Modules Error with metaplex and web3js packages

Hi all 👋 When utilizing the packages @metaplex-foundation/js and @solana/web3.js I am encountering the following error

400 Bad Request: UnableToPush: Hit an error while pushing:
InvalidModules: Loading the pushed modules encountered the following
    error:
Failed to analyze metaplex.js: Assignment to constant variable.

400 Bad Request: UnableToPush: Hit an error while pushing:
InvalidModules: Loading the pushed modules encountered the following
    error:
Failed to analyze metaplex.js: Assignment to constant variable.

Here is my action that is utilising the libraries

"use node";
import { action } from "./_generated/server";
import { v } from "convex/values";
import { Connection, PublicKey } from "@solana/web3.js";
import { Metaplex } from "@metaplex-foundation/js";

const solanaRpc = process.env.CHAINSTACK_URL ?? "null";

export const getMetaplexNfts = action({
  args: { wallet: v.string() },
  handler: async (ctx, args) => {
    const walletKey = new PublicKey(args.wallet);

    const connection = new Connection(solanaRpc);
    const metaplex = new Metaplex(connection);

    const nfts = await metaplex.nfts().findAllByOwner({ owner: walletKey });

    return nfts;
  },
});

"use node";
import { action } from "./_generated/server";
import { v } from "convex/values";
import { Connection, PublicKey } from "@solana/web3.js";
import { Metaplex } from "@metaplex-foundation/js";

const solanaRpc = process.env.CHAINSTACK_URL ?? "null";

export const getMetaplexNfts = action({
  args: { wallet: v.string() },
  handler: async (ctx, args) => {
    const walletKey = new PublicKey(args.wallet);

    const connection = new Connection(solanaRpc);
    const metaplex = new Metaplex(connection);

    const nfts = await metaplex.nfts().findAllByOwner({ owner: walletKey });

    return nfts;
  },
});

Has anyone had success using web3 and metaplex in their convex projects, is there something that I am overlooking here? Cheers!

9 Replies

mikeysee•2y ago

I work with @k.yr and just wanted to add a couple of other things I tried if it helps. I tried to "dynamically import" the dependencies within the function body:

"use node";
import { action } from "./_generated/server";
import { v } from "convex/values";

const solanaRpc = process.env.CHAINSTACK_URL ?? "null";

export const getMetaplexNfts = action({
  args: { wallet: v.string() },
  handler: async (ctx, args) => {
    const { Connection, PublicKey } = await import("@solana/web3.js");
    const { Metaplex } = await import("@metaplex-foundation/js");

    const walletKey = new PublicKey(args.wallet);

    const connection = new Connection(solanaRpc);
    const metaplex = new Metaplex(connection);

    const nfts = await metaplex.nfts().findAllByOwner({ owner: walletKey });

    return nfts;
  },
});

"use node";
import { action } from "./_generated/server";
import { v } from "convex/values";

const solanaRpc = process.env.CHAINSTACK_URL ?? "null";

export const getMetaplexNfts = action({
  args: { wallet: v.string() },
  handler: async (ctx, args) => {
    const { Connection, PublicKey } = await import("@solana/web3.js");
    const { Metaplex } = await import("@metaplex-foundation/js");

    const walletKey = new PublicKey(args.wallet);

    const connection = new Connection(solanaRpc);
    const metaplex = new Metaplex(connection);

    const nfts = await metaplex.nfts().findAllByOwner({ owner: walletKey });

    return nfts;
  },
});

I get the following error:

mikeysee•2y ago

Also if you remove the "use node"; at the top you get a bunch of errors you would expect for a module that depends upon other node modules:

mikeysee•2y ago

Anyways, its super simply to replicate, just open any convex project, add @metaplex-foundation/js to your project. Then add the function that @k.yr listed and try to build

ballingt•2y ago

I've reproduced this too and am poking around now

ballingt•2y ago

No solution here, roughly I think the UMD build of the relatively old library typescript-collections used by metaplex is difficult for esbuild to bundle for Node.js. If this is blocking you you might set up a Node.js server somewhere (or use a Vercel API endpoint if you're hosting a Next.js app or similar there). --- When I look at the bundled code I indeed see the variable name __require both being imported and being assigned to so this looks like a bundling issue. The UMD artifact of typescript-collections does this reassignment of require https://unpkg.com/browse/typescript-collections@1.3.3/dist/lib/umd.js#L21. Bundling for Node.js is less common than bundling for the browser so even some popular libraries have packaging issues like this, but if I import typescript-collections directly it's not a problem, so whatever's going on is more complicated. Convex uses esbuild to bundle modules for Node.js using the settings here: https://github.com/get-convex/convex-js/blob/541ca6507a5d62b38f46de967db220b0edabebaf/src/bundler/index.ts#L47, the next step is making a simple reproduction with esbuild and these libraries. typescript-collections hasn't been updated in 4 years so the simplest fix may be forking it to fix its build process to use tsc --module CommonJS or esbuild instead of umd and then using package.json overrides https://docs.npmjs.com/cli/v8/configuring-npm/package-json#overrides to make metaplex use that fork. We're also investigating fixing this whole class of errors by changing the way we bundle code for Node.js, but there's no timeline on this yet.

UNPKG - typescript-collections

The CDN for typescript-collections

package.json | npm Docs

Specifics of npm's package.json handling

GitHub

convex-js/src/bundler/index.ts at 541ca6507a5d62b38f46de967db220b0e...

TypeScript/JavaScript client library for Convex. Contribute to get-convex/convex-js development by creating an account on GitHub.

k.yrOP•2y ago

Thank you for the in depth answer @ballingt ! I think the separate node server will be the go, @mikeysee

ballingt•2y ago

Sounds good, we'll keep you updated on getting it working

mikeysee•2y ago

thanks again @ballingt

presley•17mo ago

UPDATE: We have released a new beta feature in convex 1.4 (https://news.convex.dev/announcing-convex-1-4/) that will allow us to skip bundling and install the packages on the server. We believe this will likely get @metaplex-foundation/js to work. More details on how to use the feature https://docs.convex.dev/functions/bundling?ref=news.convex.dev#external-packages.

Convex News

Announcing Convex 1.4

Convex 1.4 introduces a new option to install packages used in your Node action environment on the server, a variety of logging improvements, a new option to pause deployments, a new npm create convex command and more! External packages in Node.js Before 1.4, several NPM dependencies were

Bundling | Convex Developer Hub

Bundling is the process of gathering, optimizing and transpiling the JS/TS

Invalid Modules Error with metaplex and web3js packages

Did you find this page helpful?