David Alonso•10mo ago

Type of schema field depend on value of other field

Say we have a schema that on a field called props stored different kinds of objects depending on the value of a field called type. How would we define this schema in convex?

13 Replies

David AlonsoOP•10mo ago

@Michal Srb @sshader 🙏

sshader•10mo ago

https://docs.convex.dev/typescript + https://stack.convex.dev/types-cookbook are generally good resources, but to answer the question directly, you probably want something like

v.union(v.object({ kind: v.literal("bird"), props: v.object({ wingspan: v.number() }) }), v.object({ kind: v.literal("dog"), props: ... }))

(a union of objects, each with a field with a different v.literal value -- I used kind vs. type out of personal preference) This corresponds to this feature in TypeScript https://www.typescriptlang.org/docs/handbook/unions-and-intersections.html#discriminating-unions

TypeScript | Convex Developer Hub

Move faster with end-to-end type safety.

Types and Validators in TypeScript: A Convex Cookbook

It can be tough to wrangle types to behave how you want them to. Thankfully, Convex was designed to make the experience with types perfect. Learn why ...

Handbook - Unions and Intersection Types

How to use unions and intersection types in TypeScript

David AlonsoOP•9mo ago

tagging @Sirian I've got a bit of a more complex case in my hands and was wondering if I can improve something here:

filters: v.optional(v.array(v.object({
    index: v.number(),
    fieldId: v.id("firestoreFields"),
    // TODO: this union should be based on the type of the fieldId ideally
    operator: v.union(v.literal("=="), v.literal("<"), v.literal(">"), v.literal("<="), v.literal(">=")),
    // TODO: this value type should be equal to the type of the fieldId ideally
    value: v.any(),
  }))),

filters: v.optional(v.array(v.object({
    index: v.number(),
    fieldId: v.id("firestoreFields"),
    // TODO: this union should be based on the type of the fieldId ideally
    operator: v.union(v.literal("=="), v.literal("<"), v.literal(">"), v.literal("<="), v.literal(">=")),
    // TODO: this value type should be equal to the type of the fieldId ideally
    value: v.any(),
  }))),

I want to complete the todos in the code snippet. Any thoughts on whether this is possible? @sshader I guess I could do a union of multiple filter objects and have different operator types and value types for each, but the goal ofc would be to also constraint the type of the firestoreField referenced in the id

sshader•9mo ago

What does the schema for firestoreFields look like here?

David AlonsoOP•9mo ago

Something like this:

const firestoreFieldsTable = defineTable({
  name: v.string(),
  collectionId: v.id("firestoreCollections"),
  type: v.union(
    v.literal("string"),
    v.literal("int"),
    v.literal("double"),
    v.literal("boolean"),
    v.literal("timestamp"),
    v.literal("ref"),
    v.literal("geopoint"),
    v.literal("enum"),
    v.literal("map"),
    v.literal("list"),
    v.literal("array"),
    v.literal("image"),
    v.literal("url")
  ),

const firestoreFieldsTable = defineTable({
  name: v.string(),
  collectionId: v.id("firestoreCollections"),
  type: v.union(
    v.literal("string"),
    v.literal("int"),
    v.literal("double"),
    v.literal("boolean"),
    v.literal("timestamp"),
    v.literal("ref"),
    v.literal("geopoint"),
    v.literal("enum"),
    v.literal("map"),
    v.literal("list"),
    v.literal("array"),
    v.literal("image"),
    v.literal("url")
  ),

the type field is of essence in this scenario Not to distract you from the previous question, but is this good practice?

const baseBlockTableProps = {
  workspaceId: v.id("workspaces"),
 ...
  updatedAt: v.number(),
};

const blocksTable = defineTable(v.union(
    v.object({
      ...baseBlockTableProps,
      type: v.literal("documentField"),
      properties: documentFieldBlockProperties,
    }),
    v.object({
      ...baseBlockTableProps,
      type: v.literal("table"),
      properties: tableBlockProperties,
    }),
    v.object({
      ...baseBlockTableProps,
      type: v.literal("page"),
      properties: pageBlockProperties,
    })
    // Add other block types and their corresponding properties here
  )
);

const baseBlockTableProps = {
  workspaceId: v.id("workspaces"),
 ...
  updatedAt: v.number(),
};

const blocksTable = defineTable(v.union(
    v.object({
      ...baseBlockTableProps,
      type: v.literal("documentField"),
      properties: documentFieldBlockProperties,
    }),
    v.object({
      ...baseBlockTableProps,
      type: v.literal("table"),
      properties: tableBlockProperties,
    }),
    v.object({
      ...baseBlockTableProps,
      type: v.literal("page"),
      properties: pageBlockProperties,
    })
    // Add other block types and their corresponding properties here
  )
);

*BlockProperties are standard v.object(), but I can't seem to spread an object validator as done with baseBlockTableProps

sshader•9mo ago

re: second question -- yeah that seems fine. We some day want to make it a little easier to grab the fields from a validator. re: first question -- ok yeah this seems fundamentally hard to do via schema validation since schema validation operates over a single document, and what you want is to validate something in filters based off of the document you get if you load fieldId. One option would be to have a different table for each data type (e.g. firestoreStringFieldsTable ), but I think you might be better off keeping your schema generic but having a TS wrapper + some helpers. For instance, if you always use the same function to write to filters, you can load the fieldId and assert that the types match up instead of relying on schema validation to do this.

David AlonsoOP•9mo ago

Sounds good!! Okay, back with another question: I have the following table:

const blocksTable = defineTable(
  v.union(
    v.object({
      ...commonBlockProps,
      type: v.literal("documentField"),
      properties: documentFieldBlockProperties,
    }),
    v.object({
      ...commonBlockProps,
      type: v.literal("table"),
      properties: tableBlockProperties,
    }),
    v.object({
      ...commonBlockProps,
      type: v.literal("page"),
      properties: pageBlockProperties,
    })
    // Add other block types and their corresponding properties here
  )
)
  .index("by_parent", ["parentBlockId"])
  .index("by_parent_type", ["parentBlockId", "type"]);

const blocksTable = defineTable(
  v.union(
    v.object({
      ...commonBlockProps,
      type: v.literal("documentField"),
      properties: documentFieldBlockProperties,
    }),
    v.object({
      ...commonBlockProps,
      type: v.literal("table"),
      properties: tableBlockProperties,
    }),
    v.object({
      ...commonBlockProps,
      type: v.literal("page"),
      properties: pageBlockProperties,
    })
    // Add other block types and their corresponding properties here
  )
)
  .index("by_parent", ["parentBlockId"])
  .index("by_parent_type", ["parentBlockId", "type"]);

when I run this query:

 const pages = await context.db
      .query("blocks")
      .withIndex("by_parent_type", (q) =>
        q.eq("parentBlockId", args.parentPageBlockId).eq("type", "page")
      )
      .collect();

 const pages = await context.db
      .query("blocks")
      .withIndex("by_parent_type", (q) =>
        q.eq("parentBlockId", args.parentPageBlockId).eq("type", "page")
      )
      .collect();

I'd expect the inferred type of pages to be inferred as

v.object({
      ...commonBlockProps,
      type: v.literal("page"),
      properties: pageBlockProperties,
    })

v.object({
      ...commonBlockProps,
      type: v.literal("page"),
      properties: pageBlockProperties,
    })

but instead when I hover over pages I get:

const pages: ({
    _id: Id<"blocks">;
    _creationTime: number;
    parentBlockId?: Id<"blocks"> | undefined;
    content?: Id<"blocks">[] | undefined;
    type: "documentField";
    updatedAt: number;
    workspaceId: Id<...>;
    properties: {
        ...;
    };
} | {
    ...;
} | {
    ...;
})[]

const pages: ({
    _id: Id<"blocks">;
    _creationTime: number;
    parentBlockId?: Id<"blocks"> | undefined;
    content?: Id<"blocks">[] | undefined;
    type: "documentField";
    updatedAt: number;
    workspaceId: Id<...>;
    properties: {
        ...;
    };
} | {
    ...;
} | {
    ...;
})[]

Am I doing something wrong or is this a limitation of Convex? How else should I best assert the type of pages?

Michal Srb•9mo ago

This is a limitation of Convex. You can Infer the TS type from the validator and then use it to assert the result:

const vPageWithPageBlocks = v.object({
  ...commonBlockProps,
  type: v.literal("page"),
  properties: pageBlockProperties,
});

type PageWithPageBlocks = Infer<typeof vPageWithPageBlocks>

const pages: PageWithPageBlocks[] = ....

const vPageWithPageBlocks = v.object({
  ...commonBlockProps,
  type: v.literal("page"),
  properties: pageBlockProperties,
});

type PageWithPageBlocks = Infer<typeof vPageWithPageBlocks>

const pages: PageWithPageBlocks[] = ....

David AlonsoOP•9mo ago

Thanks @Michal Srb 🙂 your joint support has been great! unless I'm missing something, I'd need something like as PageWithPageBlocks[] at the end of the query. Otherwise TS complains due to a type conflict

Michal Srb•9mo ago

Sometimes TS lets you downcast via variable declaration, but when exactly I'm not sure. as is the way to go.

David AlonsoOP•9mo ago

I just realized that when I go down this path I lose the luxury of accessing system fields, so something like pages[0]._id... any ideas? Specific example where this is an issue:

const recursiveRestore = async (blockId: Id<"blocks">) => {
      const children = await context.db
        .query("blocks")
        .withIndex("by_parent_type", (q) => q.eq("parentBlockId", blockId).eq("type", "page"))
        .collect() as PageBlock[];

      for (const child of children) {
        await context.db.patch(child._id, {
          properties: {
            ...child.properties,
            isArchived: false,
          },
        });

        await recursiveRestore(child._id);
      }
    };

const recursiveRestore = async (blockId: Id<"blocks">) => {
      const children = await context.db
        .query("blocks")
        .withIndex("by_parent_type", (q) => q.eq("parentBlockId", blockId).eq("type", "page"))
        .collect() as PageBlock[];

      for (const child of children) {
        await context.db.patch(child._id, {
          properties: {
            ...child.properties,
            isArchived: false,
          },
        });

        await recursiveRestore(child._id);
      }
    };

wondering if perhaps there's a way to hydrate types with system fields... This feels pretty hacky:

export type WithSystemFields<T, TableName extends TableNames> = T & {
  _id: Id<TableName>;
  _creationTime: number;
};

const children = await context.db
        .query("blocks")
        .withIndex("by_parent_type", (q) => q.eq("parentBlockId", blockId).eq("type", "page"))
        .collect() as WithSystemFields<PageBlock, "blocks">[];

export type WithSystemFields<T, TableName extends TableNames> = T & {
  _id: Id<TableName>;
  _creationTime: number;
};

const children = await context.db
        .query("blocks")
        .withIndex("by_parent_type", (q) => q.eq("parentBlockId", blockId).eq("type", "page"))
        .collect() as WithSystemFields<PageBlock, "blocks">[];

RJ•9mo ago

How about casting as this instead?

Extract<Doc<"blocks">, { type: "page" }>[]

Extract<Doc<"blocks">, { type: "page" }>[]

David AlonsoOP•9mo ago

nifty! didn't know this trick, thanks @RJ 🔥

Type of schema field depend on value of other field

Did you find this page helpful?