Chad Maycumber•2y ago

Trying to run Langchain inside convex action

Hi, I'm trying to run langchain inside of a convex action and I'm running into the following error: 400 Bad Request: InvalidModules: Loading the pushed modules encountered the following error: Failed to analyze _deps/node/Y425MVOI.js: ENOENT: no such file or directory, open '/var/task/tiktoken_bg.wasm' I wonder if it's something to do with the bundling? Or maybe the runtime env. Any help with this would be great!

24 Replies

ian•2y ago

I’m not an expert, but just to confirm- are you on 0.13.0 and do you have “use node”; at the top of the file with the action in it?

Chad MaycumberOP•2y ago

Yeah I am. I was doing some weird imports so I'll try changing those out maybe

ballingt•2y ago

Having trouble with importing ./tiktoken_bg.wasm' makes sense, the tiktoken library (dependency of langchain) uses WebAssembly to run code written in Rust and bundling this .wasm file works differently (in this case, doesn't work) in Convex actions than other runtimes. It's not just you! We'll let you know when this changes, we definitely want folks using things like LangChain on Convex.

Chad MaycumberOP•2y ago

Thanks I have a tRPC API so leveraging that is working in the meantime!

ballingt•2y ago

Cool, and your tRPC runs on another server, not Convex? Curious to hear which pieces of your app are on the server vs Convex, glad you have a way to do this for now.

Chad MaycumberOP•2y ago

They run on NextJS in a serverless function

Ross W•2y ago

Following, as I was about to create a langchain action 😅

ian•2y ago

Our JS environment now supports wasm imports, such as langchain: https://blog.convex.dev/announcing-convex-0-14-0/

Convex News

Announcing Convex 0.14.0

Meet the brand new Convex Rust client! We’ve also open sourced the JS client, added schema validation, and more. 0.14.0 is the best version of Convex yet!

presley•2y ago

One clarifications. Langchain will work in Convex's Javascript environment, but since we use a bundler won't work if run in node (by adding "use node";) directive. If you need to run some code in Node, you can call it from the langchain action using runAction or vice verca.

Chad MaycumberOP•2y ago

Gotcha thanks for the heads up!

whoami•2y ago

Sorry I am confused, do I have to install langchain's dependency in convex's Javascript environment? I thought I can only do fetch in convex's javascript environment? How am I suppose to call langchain in the environment then

Michal Srb•2y ago

Hey @whoami you can use node_modules dependencies without "use node" via import (so just like you're used to). A lot of Node packages work in our edge runtime, but some require specific things that only our Node runtime provides.

whoami•2y ago

Oh that's really cool, is there an official node libraries that are supported by convex's javascript runtime (I guess that list include langchain)? I browsed the documentation and found little information about the javascript runtime and do I need to add dependency in the convex folder? I know I probably don't have to but I guess my ide would complain...

Michal Srb•2y ago

The node_modules/ folder should be a sibling to the convex/ folder and your package.json with the dependency. We're working on the runtime docs, but JS runtime compatibility is tricky, so it won't be possible to fully list all node packages that work with either runtime. That said we're working towards broad compatibility for both runtimes.

whoami•2y ago

Thanks this makes sense, very good to know this feature and super excited to try out! What are the differences between js runtime and node runtime from a tech perspective? Just curious

Michal Srb•2y ago

Our JS runtime is similar to other Edge runtimes, plus unlike other runtime it is deterministic (for the reasons described here: https://docs.convex.dev/functions/query-functions#caching--reactivity ). I'll let other folks on the team add more details.

Queries | Convex Developer Hub

Queries are the bread and butter of your backend API. They fetch data from the

whoami•2y ago

Is pdf-parse supported in the js runtime? I know langchain has a pdf loader but it also has pdf-parse as an additional dep hmmm I got the error message suggesting me to try node runtime after switching to node runtime, got a even weirder error associated with storage, const blob = await storage.get(storageId) is this supported by node runtime now?

Presley•2y ago

It should be. What is the error?

whoami•2y ago

ok I figured the storageId part

✘ [ERROR] Could not resolve "langchain/document_loaders"

    convex/action.ts:2:26:
      2 │ ...Loader } from "langchain/document_loaders"
        ╵                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  The path "./document_loaders" is not currently
  exported by package "langchain":

    node_modules/langchain/package.json:598:13:
      598 │   "exports": {
          ╵              ^

  None of the conditions provided ("node") match any of
  the currently active conditions ("browser",
  "default", "import"):

    node_modules/langchain/package.json:845:26:
      845 │     "./document_loaders": {
          ╵                           ^

  Consider enabling the "node" condition if this
  package expects it to be enabled. You can use
  "conditions: ['node']" to do that:

    node_modules/langchain/package.json:846:6:
      846 │       "node": {
          ╵       ~~~~~~

  You can mark the path "langchain/document_loaders" as
  external to exclude it from the bundle, which will
  remove this error.

✘ [ERROR] Could not resolve "langchain/document_loaders"

    convex/action.ts:2:26:
      2 │ ...Loader } from "langchain/document_loaders"
        ╵                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~

  The path "./document_loaders" is not currently
  exported by package "langchain":

    node_modules/langchain/package.json:598:13:
      598 │   "exports": {
          ╵              ^

  None of the conditions provided ("node") match any of
  the currently active conditions ("browser",
  "default", "import"):

    node_modules/langchain/package.json:845:26:
      845 │     "./document_loaders": {
          ╵                           ^

  Consider enabling the "node" condition if this
  package expects it to be enabled. You can use
  "conditions: ['node']" to do that:

    node_modules/langchain/package.json:846:6:
      846 │       "node": {
          ╵       ~~~~~~

  You can mark the path "langchain/document_loaders" as
  external to exclude it from the bundle, which will
  remove this error.

This is the error I got when using pdf-loader from langchain, seems like it is not supported by js runtime yet

Presley•2y ago

Yes, such imports won’t work. Does this work in If you add “use node”? In general, all npm libraries should work in Node.js, with only unfortunate example of ones that depend on wasm files being present on the local filesystem. We plan to fix this in the future.

whoami•2y ago

yea it works with node env! While it works for pdf-loader with use node, the following code doesn't work in both js and node env

import { v } from "convex/values"
import { ChatOpenAI } from "langchain/chat_models/openai"
import { HumanChatMessage } from "langchain/dist/schema"

import { action } from "./_generated/server"

export const extract = action({
  args: {
    report: v.id("reports"),
  },
  handler: async ({ runMutation, runQuery }, { report }) => {
    const docs = "..."
    const chat = new ChatOpenAI({ temperature: 0 })

    const response = await chat.call([
      new HumanChatMessage(`Translate the doc: ${docs}`),
    ])

    runMutation("reports:patch", { id: report, insights: [response.text] })
  },
})

import { v } from "convex/values"
import { ChatOpenAI } from "langchain/chat_models/openai"
import { HumanChatMessage } from "langchain/dist/schema"

import { action } from "./_generated/server"

export const extract = action({
  args: {
    report: v.id("reports"),
  },
  handler: async ({ runMutation, runQuery }, { report }) => {
    const docs = "..."
    const chat = new ChatOpenAI({ temperature: 0 })

    const response = await chat.call([
      new HumanChatMessage(`Translate the doc: ${docs}`),
    ])

    runMutation("reports:patch", { id: report, insights: [response.text] })
  },
})

and the error is

✖ Error: Unable to bundle Convex modules
esbuild failed: Error: Build failed with 1 error:
convex/insights.ts:3:33: ERROR: Could not resolve "langchain/dist/schema"
✘ [ERROR] Could not resolve "langchain/dist/schema"

    convex/insights.ts:3:33:
      3 │ ... from "langchain/dist/schema"
        ╵          ~~~~~~~~~~~~~~~~~~~~~~~

  The path "./dist/schema" is not exported
  by package "langchain":

    node_modules/langchain/package.json:598:13:
      598 │   "exports": {
          ╵              ^

  You can mark the path
  "langchain/dist/schema" as external to
  exclude it from the bundle, which will
  remove this error.

✖ Error: Unable to bundle Convex modules
esbuild failed: Error: Build failed with 1 error:
convex/insights.ts:3:33: ERROR: Could not resolve "langchain/dist/schema"

✖ Error: Unable to bundle Convex modules
esbuild failed: Error: Build failed with 1 error:
convex/insights.ts:3:33: ERROR: Could not resolve "langchain/dist/schema"
✘ [ERROR] Could not resolve "langchain/dist/schema"

    convex/insights.ts:3:33:
      3 │ ... from "langchain/dist/schema"
        ╵          ~~~~~~~~~~~~~~~~~~~~~~~

  The path "./dist/schema" is not exported
  by package "langchain":

    node_modules/langchain/package.json:598:13:
      598 │   "exports": {
          ╵              ^

  You can mark the path
  "langchain/dist/schema" as external to
  exclude it from the bundle, which will
  remove this error.

✖ Error: Unable to bundle Convex modules
esbuild failed: Error: Build failed with 1 error:
convex/insights.ts:3:33: ERROR: Could not resolve "langchain/dist/schema"

any insights?

ballingt•2y ago

Is this your code trying to import from "langchain/dist/schema"? That looks like a bad autocompleted import looks like it should be from "langchain/schema" https://github.com/hwchase17/langchainjs/blob/af842d7813861bd810d43fbfee2ab8c2482df506/langchain/package.json#L990

whoami•2y ago

thanks!

presley•15mo ago

UPDATE: tiktoken and .wasm files in general support in our Node.js environment. Previously, we only supported in the Convex environment, but in we have released a new beta feature in convex 1.4 (https://news.convex.dev/announcing-convex-1-4/) that will allow us to skip bundling and install the packages on the server. This is needed to make tiktoken work in Node.js. More details on the feature here https://docs.convex.dev/functions/bundling?ref=news.convex.dev#external-packages.

Convex News

Announcing Convex 1.4

Convex 1.4 introduces a new option to install packages used in your Node action environment on the server, a variety of logging improvements, a new option to pause deployments, a new npm create convex command and more! External packages in Node.js Before 1.4, several NPM dependencies were

Bundling | Convex Developer Hub

Bundling is the process of gathering, optimizing and transpiling the JS/TS

Trying to run Langchain inside convex action

Did you find this page helpful?