Database Architecture Concerns
Hello convex team. Currently im working on a new SAAS application and its main feature allows users to post data to my public api and store data for them to view later. In theory i want users to be able to store thousands of rows in my database for them to query later, however I was thinking would this not be bad for convex later on? Because each users data does not need to be synced with each other (always). Idealy, I would want a master database where the auth user data is stored, and other things. But users specific data is stored in there own database (table in this case) where each user (s) can only see there data.
Im not sure if convex can do something like this, im concerned that in the future If i have a lot of data, it will slow down other users accessing data because the data stored will still need to be parsed from top to bottom in the querys. Just looking for some advice, I love convex as a datavase but im not sure If its the best at my needs for this project. Vs something like sqlite where If i wanted more databases, I could just make another file and have user data separated from each other completely and not have other user data effecting performance (reads / writes ) of others
2 Replies
Convex aims to support such use-cases. As long as you're using indexes, data from one user should not slow down queries for another user.
If you're worried about Convex storing all of your data in one database, we do have plans to improve scaling internally (with "sharding") and queries will continue to be performant as long as they have reasonable access patterns, like always using indexes that start with user id. This is effectively the same as having one sqlite database for each user, but we handle it for you. I wouldn't worry about such scaling issues for a while -- I happen to know that LinkedIn and Databricks have single databases with no sharding, so you would need pretty massive scale before this becomes relevant.
Okay thanks this is good to hear.