leventov•2mo ago

Inline queries from actions

I would like to call some very simple queries and mutations (that do nothing except a single db.get() and db.insert() call, respectively) from an action. These specific query and mutation are not going to be used anywhere except this action. Creating a separate internalQuery() and internalMutation() for each with full signature specification feels like too much ceremony and hair splitting. Is there a way to do db.get() and db.insert() directly from an action, or another way of defining "inline query and mutation" right within the action block without much boilerplate? There is an old thread: https://discord.com/channels/1019350475847499849/1019350478817079338/1213551918052544593 but I’m not sure it is still relevant.

11 Replies

erquhart•2mo ago

Every Convex function is a separate isolate, and queries, mutations, and actions are isolates with different capabilities. An action can't touch the database, so it needs to call a query or mutation to do so, no way around that.

ari-cake•5w ago

I wish this was relaxed a bit, to be honest - even if there just was the capability for e.g. an inlineQuery as an iife or sth. I do get the concerns around transactions etc, and closures are complicated and even impossible to network, so it's understandable - but a single index get would solve a lot of things I have to wrap in a fairly verbose internalQuery

erquhart•5w ago

Can you give an example scenario you've run into?

leventovOP•5w ago

In my backend, "get something -- do a network call -- insert something" is a very common pattern. In fact, the majority of "conceptual mutations" end up this triple of an action, query, and mutation, because of the need to do a network call. Maybe the best way to describe this generically is that the backend is orchestrating/integrating external systems -- so for most of its mutations, it also does a "mutation to the external system" via an API call in the middle. And in this case, a good fraction of queries and mutations end up being "one-off", used in exactly one action. Hence the feel of boilerplate and ceremony of defining each internalQuery and internalMutation separately.

ari-cake•5w ago

https://github.com/moritzuehling/dms/blob/main/convex/features/paperless.ts#L111 Here's an example. Context i'm building an open-source document management system. Think of a cool piece of software where you can scan in the letters you get in the mail, email, and other ways, and then it makes them searchable. "I need the insurance certificate for my home insurance that I made 4 years ago" is a thing that should be solved by this. The most popular open-source solution for this right now is a paperless-ngx, and I want to be able to import documents from there. The issue A user makes a call to the importPaperlessDocs mutation, which should - connect to paperless instance of the user - fetch new documents to download - and queue an action for each document to download it (metadata + backing pdf) - and store it now, the user tells me an paperlessInstance ID in the request, and I need to get the HTTP url for this. Which means, I have to m,ake an internal query to get that. Then I connect to paperless and get the docs. Then, I have to make a second query later to figure out which documents exist already in my convex database, so I need a 2nd internal mutation. So, two internalMutations already :/ Then, here I have to again get the instance information in the scheduled action. I need to store the file (and why can't I do this atomically with the mutation? How do I GC files where the mutation fails?!) And then I need to run another mutation: https://github.com/moritzuehling/dms/blob/main/convex/features/paperless.ts#L201 What this means is that the code is constantly broken up, because I need to jump out of the function to do things that imo should be in the same place.

ari-cake•5w ago

Like, this function here is kinda outragous in terms of "lines that do" vs. "lines that are boilderplate"

ari-cake•5w ago

(Ignore the remoteDocId, that one is unused now, oops)

ari-cake•5w ago

same with this one here, too

ari-cake•5w ago

like, the getMyInstance doesn't do much, and so I've writen 23 lines of code for basically

ari-cake•5w ago

Like, the perfect (but probably impossible to implement) API would be:

ctx.inlineQuery((qCtx) => {
  const instance = qCtx.get(instanceId);
  if (instance?.owner != await getUserId(qCtx))
    throw new ConvexError("Not your instance");

  return instance;
}

ctx.inlineQuery((qCtx) => {
  const instance = qCtx.get(instanceId);
  if (instance?.owner != await getUserId(qCtx))
    throw new ConvexError("Not your instance");

  return instance;
}

If I did it inline, it would be super nice. Heck, then I could pass in the existing getMyInstance function, instead of making an internalQuery and then doing ctx.runQuery --- I'm aware this is nitpicky, but it's something that I've had a couple times I think just a simple get(documentId) that reads a document in an action would solve 80% of my issues, which maybe is something that isn't too difficult to mock hm

erquhart•5w ago

If you can stomach get(documentId) looking like get(ctx, documentId), you can do this today

// convex/utils.ts
export const getById = internalQuery({
  args: {
    id: v.string(),
  },
  handler: async (ctx, args) => {
    return ctx.db.get(args.id as any)
  },
})

const get = <T extends TableNames, Id extends GenericId<T>>(
  ctx: ActionCtx,
  id: Id,
): Promise<Doc<T> | null> => {
  return ctx.runQuery(internal.utils.getById, {
    id,
  }) as Promise<Doc<T> | null>
}

// convex/someFile.ts
import { get } from './utils'

export const myAction = action({
  args: {
    someId: v.id('someTable'),
  },
  handler: async (ctx, args) => {
    // get a doc
    const doc = await get(ctx, args.someId)
    // do stuff and return
    return
  },
})

// convex/utils.ts
export const getById = internalQuery({
  args: {
    id: v.string(),
  },
  handler: async (ctx, args) => {
    return ctx.db.get(args.id as any)
  },
})

const get = <T extends TableNames, Id extends GenericId<T>>(
  ctx: ActionCtx,
  id: Id,
): Promise<Doc<T> | null> => {
  return ctx.runQuery(internal.utils.getById, {
    id,
  }) as Promise<Doc<T> | null>
}

// convex/someFile.ts
import { get } from './utils'

export const myAction = action({
  args: {
    someId: v.id('someTable'),
  },
  handler: async (ctx, args) => {
    // get a doc
    const doc = await get(ctx, args.someId)
    // do stuff and return
    return
  },
})

You can usually get pretty close to the ergonomic you want via utils. I have a lot of utils lol. But baked in would be nice for sure, agree.

Inline queries from actions

Did you find this page helpful?