17 Replies
so you're talking about uniqueness constraints, @caverax_
just do that by checking if it exists
that's essentially what the index constraint does in an RDBMs. right now in convex, check if you care. if you want this every time, ORM patterns work
yes but that's a sort of gimmick 😂
and are often used by team
why is it a gimmick?
re: ids and randomness. the whole deck of cards is based on the assumption that given a bitset large enough for the domain, a collision will not happen
like, the TLS you conduct the transaction over uses nonces that are assumed to be non-colliding
if 128 bits of uuid4 isn't enough for your application, use 256 bits -- two uuid4 together.
inside convex, there is a timestamp oracle that has more control over serializability
so it has the ability to generate more compressed "truly unique" ids
and it does it using algorithms similar to snowflake https://en.wikipedia.org/wiki/Snowflake_ID
Snowflake ID
Snowflake IDs, or snowflakes, are a form of unique identifier used in distributed computing. The format was created by X (formerly Twitter) and is used for the IDs of tweets. It is popularly believed that every snowflake has a unique structure, so they took the name "snowflake ID". The format has been adopted by other companies, including Discor...
so it feels like we now have three topics? 1. are convex ids guaranteed fixed size? no 2. can I use uuid4 for distributed, uncoordinated user-generated unique ids? In almost any system, yes, but perhaps not in a system you're anticipating; 3. How do I enforce a uniqueness constraint (or any other constraints) in my convex indexes? with code
1. if they were truly fixed size would be usable, but I get it's a sharding key
not only that, but we reserve the right to evolve them
so they need to be versioned
there have been 3 different versions of the id algorithm already, for example
as we scale up the backend
if you want an existing ORM that provides higher-level constraints, like a "unique index", the ents project is popular with some people, btw: https://labs.convex.dev/convex-ents/schema
Ent Schema - Convex Ents
Relations, default values, unique fields and more for Convex
section on uniqueness: https://labs.convex.dev/convex-ents/schema#unique-fields
Ent Schema - Convex Ents
Relations, default values, unique fields and more for Convex
I believe it uses a mutation right? would be nice to know how it does it. Like https://stack.convex.dev/tanstack-real-time-github-npm-stat-counter#creating-unique-fields-with-upserts
This does basically a select, which is not ideal, but usually should be done with a B-tree
Tutorial: How I added GitHub and npm stat counters to TanStack.com
The tutorial "How I added GitHub and npm stat counters to TanStack.com" by Convex Champion Shawn Erquhart details how Convex automates the integration...
yes, it does it in a mutation. It does use a btree, that’s the withIndex method on the table. The index is a btree.
I guess works in my current use case, but eventually will hit the ceiling in a real-time type of scenario
oh yeah? why is that, not sure i'm following yet
from what I understood this thing acquire a db lock (otherwise I don't understand how it should guarantee uniqueness)
Ah even worse than expected, just read the documentation seems like is a single-writer model
that's why I was confused for a second on how you handle multiple connections on a distributed db
is the db lock global? or defined within tables?
If I have a job inserting 100M rows that uses a mutation, a user cannot register?
there is no lock, it uses optimistic concurrency control
many many parallel writers
Convex Optimistic Concurrency Control
Shared via Claude, an AI assistant from Anthropic
details there ^
good job claude, that's pretty solid