Compressing Convex IDs
I'd like to store a Convex document ID in a more compact form than its default encoding, and would welcome any recommendations for how best to do so.
Are Convex IDs v4 UUIDs? It looks to me like they are Base36-encoded—is that right? If at least the latter question is true, I imagined it might be easiest to convert them to Base64. Thoughts on that approach?
4 Replies
More specifically, I have a 22-character limit that I need to meet.
hi! yes, they are base32 encoded. we have yet to document this, but they're effectively a variable length 32-bit integer identifying the table, 14 bytes of randomness, and a two byte timestamp. so, decoded, they are at most 20 bytes.
unfortunately, I think switching to base64 won't get you enough compression, since base64 encodes 6 input bits in 8 output bits => 20 bytes of input becomes 27 bytes of output.
if you want to get really fancy and all of your IDs are for a single table, you would only need to encode the 14 random bytes + 2 timestamp bytes => 16 bytes of input which would then be 22 bytes of output. let me know if you'd like to explore this and I can help with the format. (and we'll also document the ID format properly...)
otherwise, it might be easier to just assign another ID to your documents and use that. let me know!
Thanks @sujayakar, that's all very helpful! Given all that, I think just assigning another ID to each document in this table is probably the easiest path forward for me, in this case.
Small clarification question though—do you really mean Base32, or Base36? I thought the standard Base32 encoding omitted the numbers
8, 9, and 0 from the alphanumeric character set, but I see some of those in some documents in my Convex dashboard.
(and obviously +1 to documentation of the ID format!)sounds good!
our encoding is based off crockford's base32 format: https://www.crockford.com/base32.html, which omits "i", "l", "o", and "u".