RJ•3mo ago

Schema validation succeeds when adding a non-optional field to a union variant

Let's say I have a table like this:

defineSchema({
  my_table: defineTable(v.union(
    v.object({
      tag: v.literal("A"),
    }),
    v.object({
      tag: v.literal("B"),
    })
  ))
})

defineSchema({
  my_table: defineTable(v.union(
    v.object({
      tag: v.literal("A"),
    }),
    v.object({
      tag: v.literal("B"),
    })
  ))
})

Which contains some documents, like these:

[{ tag: "A" }, { tag: "B" }]

[{ tag: "A" }, { tag: "B" }]

If I add a new (non-optional) field to one of the variants:

defineSchema({
  my_table: defineTable(v.union(
    v.object({
      tag: v.literal("A"),
+     a_field: v.string()
    }),
    v.object({
      tag: v.literal("B"),
    })
  ))
})

defineSchema({
  my_table: defineTable(v.union(
    v.object({
      tag: v.literal("A"),
+     a_field: v.string()
    }),
    v.object({
      tag: v.literal("B"),
    })
  ))
})

Schema validation succeeds, thus permitting invalid documents in that table. Validation seems to work correctly in other contexts (e.g. insertion, patching).

9 Replies

Convex Bot•3mo ago

Thanks for posting in <#1088161997662724167>. Reminder: If you have a Convex Pro account, use the Convex Dashboard to file support tickets. - Provide context: What are you trying to achieve, what is the end-user interaction, what are you seeing? (full error message, command output, etc.) - Use search.convex.dev to search Docs, Stack, and Discord all at once. - Additionally, you can post your questions in the Convex Community's <#1228095053885476985> channel to receive a response from AI. - Avoid tagging staff unless specifically instructed. Thank you!

ballingt•3mo ago

I can't repro this, is there anything else that might be different?

RJOP•3mo ago

Gosh Tom I'm sorry, I just went to try to reproduce this myself and I think I see what happened. I was adding a new field (a_field) to my schema, which looked like this:

defineSchema({
  my_table: defineTable(v.union(
    v.object({
      tag: v.literal("A"),
      a_field: v.optional(v.literal("something"), v.null())
    }),
    v.object({
      tag: v.literal("B"),
    })
  ))
})

defineSchema({
  my_table: defineTable(v.union(
    v.object({
      tag: v.literal("A"),
      a_field: v.optional(v.literal("something"), v.null())
    }),
    v.object({
      tag: v.literal("B"),
    })
  ))
})

Initially I had a few tag: "A" variants in my dev database, which didn't have any values for a_field. I tested the UI a bit, and in the process populated a_fields for all but one record. At some point in this process I then altered the schema such that a_field was required. Because there was still one document missing a value for a_field, this schema change must not have been deployed (but I didn't spot this in the moment).

defineSchema({
  my_table: defineTable(v.union(
    v.object({
      tag: v.literal("A"),
-     a_field: v.optional(v.union(v.literal("something"), v.null()))
+     a_field: v.union(v.literal("something"), v.null())
    }),
    v.object({
      tag: v.literal("B"),
    })
  ))
})

defineSchema({
  my_table: defineTable(v.union(
    v.object({
      tag: v.literal("A"),
-     a_field: v.optional(v.union(v.literal("something"), v.null()))
+     a_field: v.union(v.literal("something"), v.null())
    }),
    v.object({
      tag: v.literal("B"),
    })
  ))
})

Then I went to the dashboard to manually edit the one document missing a_field. I changed its value from unset to null successfully. Then, ~30 seconds later, I unset the a_field for that same document, but this time got a validation error, which is what confused me. It looks like when I populated the last unset a_field, convex dev saw the change and then successfully deployed the new schema. So when I came back 30s later, it was validating against the new schema, whereas before it was validating against the old one. When I went back to check the dev logs, I saw the successful schema deployment and assumed that had happened when I initially changed the schema, having missed that initially it had failed. In short: I don't think this is a bug. I'm sorry to have wasted your time 😶

lee•3mo ago

that's quite the saga 🤯. We're always happy to hear bug reports, it means people are exploring edge cases. 🙏

ballingt•3mo ago

RJ your record is like ten real bugs to a couple false alarms, always helpful to get these! and there's dx to think about for the future, maybe blocking dev or making it clear in the browser that a schema change has not deployed to avoid this chain of events

ampp•3mo ago

This has caught me off guard so many times. Especially when bundling was taking over 20 seconds. I got 4 monitors and sometimes i don't see or register error. Is there some way to get the last time the configuration was updated to our webapps, Then i could have a little debug component that is green for the first 30 seconds, yellow for 30-60s, then red. Although it may be nice to have a dashboard version of ^ a timer since the last configuration update. I got the dashboard log open almost all the time. I think its one of the best ideas for dx experiance. I wouldnt doubt if this little thing would save me a hour or two a month

ballingt•3mo ago

If it's connected locally, a green for up to date and red for waiting (and even a list of files that have changed?) would be slick

sshader•3mo ago

(I've been wanting to build a little Convex widget you can embed in your app with things like "Last pushed N seconds ago" and "Go to dashboard" and logs / currently loaded queries. Particularly excited about the first one since I've done the "why is my app not working the way I expect it" to realize I haven't actually pushed my updates since I have some silly type error buried in a terminal window somewhere)

RJOP•3mo ago

I love this idea!