Feedback: schemaValidation and strictTableNameTypes documentation is difficult to comprehend
The docs for schemaValidation and strictTableNameTypes https://docs.convex.dev/database/schemas is primarily written in prose. A chart may help. I'm trying to understand the impact of my settings for these two properties
Rhetorical questions:
1) Which tables are validated during runtime?
2) Which tables get Intellisense in VS Code?
3) What are the implications for deploying table definition changes?
4) What changes if a table is in defineSchema() or not? It seems like I have two ways to control behavior: Include tables in defineSchema() or change these settings. So what do I do if I want Intellisense, but I don't want strict validation during runtime, but I also don't want to compromise my entire schema by setting schemaValidation=false?
Schemas | Convex Developer Hub
Schema validation keeps your Convex data neat and tidy. It also gives you end-to-end TypeScript type safety!
15 Replies
Question: Is it correct Convex design to only have one defineSchema() call in a Convex project? Maybe if I have multiple defineSchema(), then I can have a different defineSchema() for the four possible combinations of schemaValidation and strictTableNameTypes values
I can see one gap in the doc, it doesn't say that you can still write to tables not in your schema.
Answering your questions:
1) Which tables are validated during runtime?The tables in your schema.
Which tables get Intellisense in VS Code?The tables in your schema.
3) What are the implications for deploying table definition changes?Schema will only be deployed if it matches the existing documents in your tables.
So what do I do if I want Intellisense, but I don't want strict validation during runtime
schemaValidation: false
don't want to compromise my entire schemaWhat do you mean by this?
Is it correct Convex design to only have one defineSchema()The doc could be more precise, yes, there can only be a single schema, which is the default export from the
schema.ts file.
I do think the two options are explained pretty clearly, and they are orthogonal to each other.Haha well, I give you the benefit of the doubt Michal: it's probably my own inability to understand the docs.
Let me de-scope my point: the two options are orthogonal to each other. But the inclusion of tables in defineSchema() is not orthogonal to the two options.
By the way, I don't think the word orthogonal is that helpful. In this context, it's a metaphor. Let me reflect on this and then explain what I mean about compromising my defineSchema
By orthogonal I mean that the two options don't affect each other.
schemaValidation affects only runtime behavior, while strictTableNameTypes only affects type-checking.
I agree that the doc is missing a clear explanation of what happens to tables not present in the schema, will definitely fix that.Here's the context for what I mean about "compromising" the defineSchema() in a feature branch:
I'm the only developer on my project. I'm using the GitHub flow strategy for change management: https://githubflow.github.io/
Ideally, I'm making feature branches that only have changes specific to that feature. As of now, I don't have various environment-specific branches that have different values for schemaValidation and strictTableNameTypes.
Based on your earlier answers, it seems like a feature branch would have schemaValidation and strictTableNameTypes set to false, and so "compromise" the mature, unchanged tables by removing their schema validation in that feature branch.
When I'm creating a new table, I find myself leaving it out of defineSchema(), even though I'd like to have Intellisense for it.
For a given feature branch, I'm not making changes to most of the tables in my schema. So I'd like these "mature" tables to have the behavior of schemaValidation=true and strictTableNameTypes=true.
But for my new feature, I may have a pre-existing table I want to change, or a new table new for that feature branch. I want these new tables to have Intellisense.
So, I have two approaches: A) I can leave the new tables out of defineSchema() so that I can still keep schemaValidation=true intact for the mature tables. B) Or I can include these changed tables in defineSchema, but then set the schemaValidation=false for all tables. (And then revert schemaValidation back to true during testing, or at least before I merge this feature branch to main.)
So, I have two approaches: A) I can leave the new tables out of defineSchema() so that I can still keep schemaValidation=true intact for the mature tables. B) Or I can include these changed tables in defineSchema, but then set the schemaValidation=false for all tables. (And then revert schemaValidation back to true during testing, or at least before I merge this feature branch to main.)
GitHub Flow
GitHub Flow : The best way to use Git and GitHub
Yeah so this a schema branching problem. I agree that currently this is not easy to do, especially if you care about the dev deployment data.
I think our current answer is to have a script that generates your dev deployment data, so that you can easily wipe your whole deployment and populate it. Then the flow can be:
1. Empty dev deployment
2. Branch off
3. Populate db
3. Change schema
4. Test
5. Wipe deployment
6. Switch branches
7. (now the schema is different)
8. Populate db
9. Test
etc.
You can also use a snapshot export from the main (trunk) branch to do the wipe + populate.
Well, even if I were to do this flow, I still face the same issue in step #4, right? Fundamentally, I want some of my tables in defineSchema() to have the behavior of schemaValidation=true and some to not. This is why I had asked that question about multiple defineSchema() calls.
You haven't written why you want
schemaValidation: false for the tables modified on the feature branch (although I can think of some reasons). I generally have schemaValidation: true all the time, except when I have to make the dance for migrating data in an existing table.
I think we have discussed internally options for turning on off schema validation on individual tables, no conrete plans I'm aware off though.Ok thanks Michal. That’s helpful. How do you usually set these options? When developing, do you generally exclude new tables in defineSchema() and set strictTableNameTypes: false? If so, that’s why docs are confusing. The burden is on the new developer to figure out the implications of including tables in defineSchema()
I usually don't set either of the options, except for when I'm migrating existing data to a new changed schema, when I disable
schemaValidation. I include all tables in the schema, usually upfront, sometimes after having written some data I can copy the table definition from the dashboard.
You are correct that the whole schema experience is far from ideal and we don't give very clear advice on the workflow (still better than most traditional databases tbh).Oh Yeah, with say, drizzle, there is no type safety - you build your own interfaces.
I think I get it now: I think StrictTableNameTypes is usually False, and dev cycle is comfortable with that.
One of the very first things a new convex developer does is set StrictTableNameTypes to false despite the default value being true
Or alternatively, like you mentioned, one can set both properties to true and forego the rapid prototyping capabilities altogether
Re: prototyping on individual tables, sometimes I'll do something like
myNewTable: defineTable(v.any()) while keeping schema validation enabled. This lets me try out different things with this new table for fast prototyping, while maintaining type safety on everything else.What specifically do you gain by doing that? I noticed that if I don’t include a table in a defineSchema(),
intellisense would also say any as the type for the return from a useQuery
If you also use the default
strictTableNameTypes: true, TypeScript will catch typos like db.insert("mesages", {...}) but will treat the Doc type for the table you're prototyping with as any. It's also more self-documenting -- you can see in your schema which tables have the any type vs. strictTableNameTypes: false.Ok, all things considered, I think the docs are pretty good, it just took me this support thread to understand it. Thank you Michal and Sarah!