zoomie•10mo ago

Partial schema

Is it possible to have a partial schema? I have a table called lead with data in it. The schema is lead: defineTable(v.any()). I want to add an index, so I need to define the fields.

  lead: defineTable({
    latitude: v.optional(v.number()),
    longitude: v.optional(v.number()),
  })

  lead: defineTable({
    latitude: v.optional(v.number()),
    longitude: v.optional(v.number()),
  })

As there is existing data in the table, I'm getting this error:

Document with ID ... in table "lead" does not match the schema: Object contains extra field that is not in the validator.

Document with ID ... in table "lead" does not match the schema: Object contains extra field that is not in the validator.

9 Replies

erquhart•10mo ago

No, you can't have some fields defined and some fields not, you'll either want to add them to the schema or turn off schema validation (perhaps temporarily). You can have some tables defined and some not, but not fields on a table. You could always just add the fields you're not solid on as optional, and then make things required when you're ready to firm things up. You can also use v.any() for said fields

Michal Srb•10mo ago

Or you could do

lead: defineTable({
    latitude: v.optional(v.number()),
    longitude: v.optional(v.number()),
    metadata: v.any(),
  })

lead: defineTable({
    latitude: v.optional(v.number()),
    longitude: v.optional(v.number()),
    metadata: v.any(),
  })

and store in metadata an object with any fields you want.

zoomieOP•10mo ago

Thanks, I've gone with turning of the validation for now. I like the metadata idea, but I'm using npx convex import to get my csv data in, and would need to wrangle the schema in my data pipeline.

ian•10mo ago

to have schema validation for other tables, you could use v.any() for the table. And if you want the types w/o validation, you can cast:

    lead: defineTable(
      v.any() as ObjectValidator<{
        latitude: Validator<number>;
        longitude: Validator<number>;
      }>
    ),

    lead: defineTable(
      v.any() as ObjectValidator<{
        latitude: Validator<number>;
        longitude: Validator<number>;
      }>
    ),

sshader•10mo ago

also whenever you do import in your data and get it stable, try the "Show Schema" > "Generated" buttons in the dashboard to jumpstart writing out a full schema

zoomieOP•10mo ago

For some reason the "Generated" tab only ever showed v.any()?

zoomieOP•10mo ago

sshader•10mo ago

It unfortunately can't always infer the schema from your data -- common causes are fields that aren't valid identifiers (identifiers are alphanumeric and underscore strings), which can't be used in schema definitions or objects with a lot of keys (where we give up and say v.any())

zoomieOP•10mo ago

Yes it's definatly that, good to know why, I appreciate the help 🙂

Partial schema

Did you find this page helpful?