Record counts for large numbers of records is non-trivial
I've handled counting needs for large numbers of records (any amount greater than I can reasonably list purely for a count, so over 500 maybe) by keeping count fields and incrementing/decrementing at write time.
I'm now noticing failures due to that number changing too much during batch mutations, so I'm having to rework to add special handling for those cases. Which is making me realize there are other such cases in my codebase that are probably just a chance away from a random failure.
I'm thinking through other novel approaches, such as updating count daily or on certain events, by paginating through the records and updating a given count field as a whole. But that's a lot of data moving out of the database just to get a count.
I know a number of questions have come around getting counts in this support forum. Are there better solutions on the way? This is one of those few problems that are actually harder with Convex than with a traditional db, would be nice to eventually check it off the list.
For now I'm moving toward avoiding the need for counts altogether, but that's not a fun way to steer product decisions. Thoughts/suggestions appreciated!
I'm now noticing failures due to that number changing too much during batch mutations, so I'm having to rework to add special handling for those cases. Which is making me realize there are other such cases in my codebase that are probably just a chance away from a random failure.
I'm thinking through other novel approaches, such as updating count daily or on certain events, by paginating through the records and updating a given count field as a whole. But that's a lot of data moving out of the database just to get a count.
I know a number of questions have come around getting counts in this support forum. Are there better solutions on the way? This is one of those few problems that are actually harder with Convex than with a traditional db, would be nice to eventually check it off the list.
For now I'm moving toward avoiding the need for counts altogether, but that's not a fun way to steer product decisions. Thoughts/suggestions appreciated!
