mikeysee•13mo ago

folder structure in a project

hey this seems like an obvious question but what is the reccomended way of structuring a projects files and folders?? For now I have been putting client-side files in /src and convex related files (queries, mutations, actions etc) in /convex but what about any code that is shared between client and server? What about various other "model" or util files that shouldnt be included in the automated API generation?

15 Replies

mikeyseeOP•13mo ago

I was looking at how AI Town was structured and noticed the same issue https://github.com/a16z-infra/ai-town

GitHub

GitHub - a16z-infra/ai-town: A MIT-licensed, deployable starter kit...

A MIT-licensed, deployable starter kit for building and customizing your own version of AI town - a virtual town where AI characters live, chat and socialize. - a16z-infra/ai-town

mikeyseeOP•13mo ago

even tho there are numerious "util" files that dont contain queries, mutations, actions or anything explicity "api" they are included in the generated API because of the folder structure: https://github.com/a16z-infra/ai-town/blob/main/convex/_generated/api.d.ts

GitHub

ai-town/convex/_generated/api.d.ts at main · a16z-infra/ai-town

A MIT-licensed, deployable starter kit for building and customizing your own version of AI town - a virtual town where AI characters live, chat and socialize. - a16z-infra/ai-town

mikeyseeOP•13mo ago

the problem is that it is possible to easily get into circular reference issues like this, so would it be better to include a sepparate /shared/ or /utils/ directory outside of the /convex one for non-api functions?

jamwt•13mo ago

@ian probably has some good thoughts about this

erquhart•13mo ago

Yeah I use a shared-util folder in root for this.

mikeyseeOP•13mo ago

Hmm, it seems like this should be codified in the docs or a stack post at least. I guess it depends on the library you use to some degree but circular references in typescript are a footgun that could be avoided with a convention

ian•13mo ago

tl;dr my recommendation is a convex/shared folder to make it explicit, but it's not a strong preference. The issue is real: beyond what's possible in TS, it's useful to know from the client-side what code ought to be imported to avoid leaking server logic to clients, and to know where the minimal imports are. Especially for types, having the schema and functions and client all import from one place is much nicer than having schema depend on a /convex/users.ts file, and that file using the schema. I don't have a one-size-fits-all strategy here, though a clear winner will likely evolve. Having src/ shared/ convex/ is a good pattern, and relative imports will help you out there. For monorepos, the story is more complex since many apps may want access to shared/. Importing from a convex/shared/ directory on the client isn't a problem - if it isn't exposing functions, it won't leak any function implementations to clients. The imports will be tree shaken.

mikeyseeOP•13mo ago

tl;dr my recommendation is a convex/shared folder to make it explicit, but it's not a strong preference.

The problem is that is still exported into the api (I just tested it, see image below). I think what im going to do going forward is 1. /convex 2. /client or /src 3. /shared - if there is stuff shared between server and client and a server 4. /server or /server-shared to hold any logic that /convex needs but we dont want included in the api. I might write some linting rules to enorce these folder rules. We do something like this in our big monorepo at Gangbusters and it works well.

mikeyseeOP•13mo ago

Another issue with lots of files unnecessarily in the /convex directory is that this starts to really slow down typescript as it has to do a bunch of typechecking and inference unnecessarily to build the API this becomes very noticable if you start to use xstate which really slows down the compiler.

ian•13mo ago

makes sense! we could probably avoid adding things to the api if there aren't any exported functions to help out here. also Tim is going to look into faster bundling / incremental building, since this is a known pain point. The type inference is related but types outside of convex won't benefit directly from that work.

mikeyseeOP•13mo ago

we could probably avoid adding things to the api if there aren't any exported functions to help out here

Seems like a good idea tbh

ampp•3mo ago

Did you ever try having a packages/shared folder in your mono-repo? I've been trying to find if anyone has done this. I cant import anything from packages/shared without it attempting to build the convex folder, so i have to use a generated API in that folder. But the apps/web1 apps/etc all can import from convex. So ive been moving a lot of the objects to the shared folder then extending them within the convex folder to add the database functionality. I can do do anything like what is done in ai-town's serverGame.ts file. 😅

mikeyseeOP•2mo ago

Yep I have mono-repo'ed a few projects now with convex and have a packages/shared folder. As long as the tsconfigs are setup correctly it should be okay

ampp•2mo ago

If you could share the tsconfig files i'd be forever grateful, like for /app/web /packages/shared /packages/backend and / . It's such a needle in a haystack type problem as everything else is working. Our /packages/backend accesses a lot of stuff in shared, so i hope to keep it that way.

mikeyseeOP•2mo ago

Sure, this is from my "stashit" project: in /tsconfig.json:

{
  "extends": "./tsconfig.base.json",
  "references": [
    { "path": "./packages/ui" },
    { "path": "./packages/shared" },
    // { "path": "./packages/convex" },
    { "path": "./apps/client" },
    { "path": "./apps/extension" }
  ]
}

{
  "extends": "./tsconfig.base.json",
  "references": [
    { "path": "./packages/ui" },
    { "path": "./packages/shared" },
    // { "path": "./packages/convex" },
    { "path": "./apps/client" },
    { "path": "./apps/extension" }
  ]
}

in /tsconfig.base.json

{
  "compilerOptions": {
    "strict": true,
    "declaration": true,
    "declarationMap": true,
    "esModuleInterop": true,
    "incremental": true,
    "preserveWatchOutput": true,
    "emitDeclarationOnly": true,
    "tsBuildInfoFile": ".typescript/.tsbuildinfo",
    "isolatedModules": true,
    "lib": ["es2022", "DOM", "DOM.Iterable"],
    "module": "NodeNext",
    "moduleDetection": "force",
    "moduleResolution": "NodeNext",
    "noUncheckedIndexedAccess": true,
    "resolveJsonModule": true,
    "skipLibCheck": true,
    "jsx": "react-jsx",
    "target": "ES2022"
  },
  "include": [],
  "exclude": [
    "**/node_modules",
    "**/dist",
    "*.d.ts",
    "**/tmp",
    "**/blueprint-templates",
    "**/.github"
  ]
}

{
  "compilerOptions": {
    "strict": true,
    "declaration": true,
    "declarationMap": true,
    "esModuleInterop": true,
    "incremental": true,
    "preserveWatchOutput": true,
    "emitDeclarationOnly": true,
    "tsBuildInfoFile": ".typescript/.tsbuildinfo",
    "isolatedModules": true,
    "lib": ["es2022", "DOM", "DOM.Iterable"],
    "module": "NodeNext",
    "moduleDetection": "force",
    "moduleResolution": "NodeNext",
    "noUncheckedIndexedAccess": true,
    "resolveJsonModule": true,
    "skipLibCheck": true,
    "jsx": "react-jsx",
    "target": "ES2022"
  },
  "include": [],
  "exclude": [
    "**/node_modules",
    "**/dist",
    "*.d.ts",
    "**/tmp",
    "**/blueprint-templates",
    "**/.github"
  ]
}

in /packages/shared/tsconfig.json

{
  "extends": "../../tsconfig.base.json",
  "compilerOptions": {
    "composite": true,
    "outDir": "dist",
    "rootDir": "./src",
    "tsBuildInfoFile": ".typescript/.tsbuildinfo"
  },
  "include": ["src"]
}

{
  "extends": "../../tsconfig.base.json",
  "compilerOptions": {
    "composite": true,
    "outDir": "dist",
    "rootDir": "./src",
    "tsBuildInfoFile": ".typescript/.tsbuildinfo"
  },
  "include": ["src"]
}

in /packages/convex/tsconfig.json

{
  "extends": "../../tsconfig.base.json",
  "compilerOptions": {
    "composite": true,
    "outDir": "dist",
    "rootDir": "./convex",
    "module": "ESNext",
    "moduleResolution": "Bundler",
    "tsBuildInfoFile": ".typescript/.tsbuildinfo"
  },
  "include": ["convex/**/*.ts", "convex/**/*.tsx"]
}

{
  "extends": "../../tsconfig.base.json",
  "compilerOptions": {
    "composite": true,
    "outDir": "dist",
    "rootDir": "./convex",
    "module": "ESNext",
    "moduleResolution": "Bundler",
    "tsBuildInfoFile": ".typescript/.tsbuildinfo"
  },
  "include": ["convex/**/*.ts", "convex/**/*.tsx"]
}

in /packages/convex/convex/tsconfig.json is standard convex tsconfig

{
  /* This TypeScript project config describes the environment that
   * Convex functions run in and is used to typecheck them.
   * You can modify it, but some settings required to use Convex.
   */

  "compilerOptions": {
    /* These settings are not required by Convex and can be modified. */
    "allowJs": true,
    "strict": true,

    /* These compiler options are required by Convex */
    "target": "ESNext",
    "lib": ["ES2021", "dom"],
    "forceConsistentCasingInFileNames": true,
    "allowSyntheticDefaultImports": true,
    "preserveWatchOutput": true,
    "module": "ESNext",
    "moduleResolution": "Bundler",
    "isolatedModules": true,
    "skipLibCheck": true,
    "noEmit": true,

    "paths": {
      "@repo/ui/*": ["../../../packages/ui/src/*"],
      "@repo/shared/*": ["../../../packages/shared/src/*"],
    }
  },
  "include": ["./**/*"],
  "exclude": ["./_generated"]
}

{
  /* This TypeScript project config describes the environment that
   * Convex functions run in and is used to typecheck them.
   * You can modify it, but some settings required to use Convex.
   */

  "compilerOptions": {
    /* These settings are not required by Convex and can be modified. */
    "allowJs": true,
    "strict": true,

    /* These compiler options are required by Convex */
    "target": "ESNext",
    "lib": ["ES2021", "dom"],
    "forceConsistentCasingInFileNames": true,
    "allowSyntheticDefaultImports": true,
    "preserveWatchOutput": true,
    "module": "ESNext",
    "moduleResolution": "Bundler",
    "isolatedModules": true,
    "skipLibCheck": true,
    "noEmit": true,

    "paths": {
      "@repo/ui/*": ["../../../packages/ui/src/*"],
      "@repo/shared/*": ["../../../packages/shared/src/*"],
    }
  },
  "include": ["./**/*"],
  "exclude": ["./_generated"]
}

in /apps/client/tsconfig.json

{
  "extends": "../../tsconfig.base.json",
  "compilerOptions": {
    "composite": true,
    "emitDeclarationOnly": true,
    "outDir": "dist",
    "rootDir": "./src",
    "allowImportingTsExtensions": true,
    "module": "ESNext",
    "moduleResolution": "Bundler",
    "tsBuildInfoFile": ".typescript/.tsbuildinfo",
    "paths": {
      "@repo/ui/*": ["../../packages/ui/src/*"],
      "@repo/shared/*": ["../../packages/shared/src/*"],
    }
  },
  "references": [
    { "path": "../../packages/shared" },
    { "path": "../../packages/ui" }
  ],
  "include": ["src"]
}

{
  "extends": "../../tsconfig.base.json",
  "compilerOptions": {
    "composite": true,
    "emitDeclarationOnly": true,
    "outDir": "dist",
    "rootDir": "./src",
    "allowImportingTsExtensions": true,
    "module": "ESNext",
    "moduleResolution": "Bundler",
    "tsBuildInfoFile": ".typescript/.tsbuildinfo",
    "paths": {
      "@repo/ui/*": ["../../packages/ui/src/*"],
      "@repo/shared/*": ["../../packages/shared/src/*"],
    }
  },
  "references": [
    { "path": "../../packages/shared" },
    { "path": "../../packages/ui" }
  ],
  "include": ["src"]
}

I just realised that client doesnt actually reference convex in its tsconfig but it is able to use it:

import { api } from "@repo/convex/convex/_generated/api";
import { Id } from "@repo/convex/convex/_generated/dataModel";
import { useEffect } from "react";
import { toast } from "sonner";
import { useMutation, useQuery } from "convex/react";

export const useGetOrCreateItemChats = (itemId: Id<"items"> | null) => {
  const chat = useQuery(api.itemChats.findMyChatForItem, itemId ? { itemId } : "skip");
  const createChat = useMutation(api.itemChats.createMyChatForItem);

  useEffect(() => {
    if (!itemId) return;
    if (chat === undefined) return;
    if (chat) return;
    console.log(`Creating chat for item ${itemId}`);
    createChat({ itemId }).catch(toast.error);
  }, [itemId, chat]);

  return {
    isLoading: chat === undefined,
    chat,
  };
};

import { api } from "@repo/convex/convex/_generated/api";
import { Id } from "@repo/convex/convex/_generated/dataModel";
import { useEffect } from "react";
import { toast } from "sonner";
import { useMutation, useQuery } from "convex/react";

export const useGetOrCreateItemChats = (itemId: Id<"items"> | null) => {
  const chat = useQuery(api.itemChats.findMyChatForItem, itemId ? { itemId } : "skip");
  const createChat = useMutation(api.itemChats.createMyChatForItem);

  useEffect(() => {
    if (!itemId) return;
    if (chat === undefined) return;
    if (chat) return;
    console.log(`Creating chat for item ${itemId}`);
    createChat({ itemId }).catch(toast.error);
  }, [itemId, chat]);

  return {
    isLoading: chat === undefined,
    chat,
  };
};

folder structure in a project

Did you find this page helpful?