casen•12mo ago

Code organization patterns in Convex

Hello everyone! After playing around and creating a slack app that's wrapping an OpenAI assitant with a knowledge base, I've gotten to kick the tires a bit on Convex. As the various back and forth interactions have grown in complexity, I've noticed some rather strange architectural patterns emerge. I'd like to discuss some best practices around overall app structure, which I have not noticed in any examples or docs, since they are all relatively trivial. 1. Convex seems to force all code into two buckets: internal, and api (public and private). 2. Convex furthur devides all code into actions, and then all the rest (mutations, queries). 3. All files in the root of the convex folder seem to represent a resource, or a domain, and thus become the tip of the spear for the entire application. Given the above, I've found it rather challenging to organize the codebase. Convex seems to want a CQRS style, which makes sense given the separation of queries and mutations. That much is nice, when everything lives in convex land with pure mutations and queries, the code really looks good. Where it gets challenging is with actions and the nodejs runtime. Has anyone else wrestled with this? Any example codebases or tips for better organizing actions? Often times, things are foced into actions merely because they need the nodejs runtime, or they are calling an external service. Because of this they need to be in a separate file, even if the code is highly coupled or related to mutations and queries. Overall it creates a bit of spaghetti where code bounces around between files, calling mutations, actions, mutations, queries, actions, etc. Perhaps I am just inexperienced with this database-first reactive style of programming. I welcome any tips!

7 Replies

erquhart•12mo ago

Definitely wrestled with this. I landed on a pretty basic solution: when a function would normally have an obvious home in my convex runtime files, but needs to be run in node, I create a Node suffixed variant of the file it would belong to. It's clunky, but it cleared it up for me. So I end up with eg., user.ts and userNode.ts. I also wonder if CQRS adds more complexity than maybe necessary for Convex. I've found, oddly, that MVC actually sort of works across the entire stack with Convex. The entire frontend being the view (which can be broken down into it's own pattern), and then splitting Convex code between controller and model, has worked really well for me. Within that, I haven't found a need to draw hard lines between read and write operations.

jamwt•12mo ago

/me Googles CQRS

erquhart•12mo ago

really appreciate you saying that, I most certainly did as well lol

jamwt•12mo ago

@casen so, I think two different dimension to what you're feeling we're aware of 1. Q/M/A, code having to be organized that way this we view as a Very Good Thing and is essential to convex being better. it does have tradeoffs but in general we think they are 10x worth it 2. What this ^ structure though imposes on your code and we're not thrilled with some parts about that, agreed two big warts we know about in particular a. 'use node' being a module-level directive sucks to have to break out entire modules for this b. lots of internal mutations or queries just to service actions in an ideal world you'd be able to somehow define inline Q + M that service actions just to keep things more linear, organized, and not "pollute" a top-level namespace for a single-use function we hope to make progress on all these, but most of what we run into right now is we have less control over things like the bundler and the semantics of the language than we'd like we're in sort of a similar boat to vercel with 'use client'; we have some interesting internal prototypes for things like inline queries, but doing it in a way without accidental, subtle bugs involving bundling and closures is tough we'll keep working on it though! I'm sure the team will keep coming up with small improvements to make this more ergonomic in the mean time, ideas like the ones @erquhart suggested are good -- the folks with bigger codebases have found "coping mechanisms" they can live with we (the convex team) have learned from them too because at this point lots of convex customers have much bigger app code bases than we've ever managed on convex. we'll invest more into writing up some of these learnings soon for better discoverability by everyone reading up on CQRS, yeah, I think convex agrees with some of those principles, but in some ways, it's formalizing things which were already true? like for example, SELECT is not terribly congruent with INSERT and never has been reading data normally involves creating aggregates and derivations and computations and writes are record-level rather than asking the database to somehow infer underling table structure from some complex aggregate 😛 so convex is "owning this" more than previous systems have but we're not unique here or breaking ground; graphql also does this, as do other sort of declarative read systems like react itself the read path in react (dom reflow) is completely different than the write path (setState) and that’s probably a more useful way to model reading and writing, so convex agrees and aligns. Thanks for the feedback!

Michael Rea•12mo ago

I personally make sub folders per table in the schema and break files down to actions, mutations, queries and helper functions (per table)

erquhart•12mo ago

Same. I did find down the road that my application logic didn't actually group well into a per-table setup, so I ended up repurposing per table files just for schema and direct data access (lists, paging, relations, cascading deletes, authz, constraints, validation), which made them pretty uniform. My application logic is separate and leans on those table-based files for data interactions. I keep the per-table files in convex/db, exported as a single object similar to how convex exports api. The db layer has no queries, mutations, or actions, except for some that are required to support cascading deletes. A query then looks something like this:

// convex/schedule.ts

import { v } from 'convex/values'
import { query } from '@cvx/_generated/server'
import db from '@cvx/db'

export const listEvents = query({
  args: { scheduleId: v.id('schedule') },
  handler: async (ctx, args) => {
    // generate an authorized context
    const authzCtx = await db.schedule.authz(ctx, args.scheduleId)
    // get the list of events
    const events = await db.event.listByScheduleId(authzCtx, args.scheduleId)
    return { data: events }
  },
})

// convex/schedule.ts

import { v } from 'convex/values'
import { query } from '@cvx/_generated/server'
import db from '@cvx/db'

export const listEvents = query({
  args: { scheduleId: v.id('schedule') },
  handler: async (ctx, args) => {
    // generate an authorized context
    const authzCtx = await db.schedule.authz(ctx, args.scheduleId)
    // get the list of events
    const events = await db.event.listByScheduleId(authzCtx, args.scheduleId)
    return { data: events }
  },
})

casenOP•12mo ago

I really love all the sharing about this, thanks! Indeed. You send commands, and receive the query over a web hook. In typical MVC we tend to write, and return the result in the same request Sometimes people make a second request. It’s a messy space. I prefer the segregation between commands and queries personally

Code organization patterns in Convex

Did you find this page helpful?