High level questions about convex ETL story and cost scaling
Interested in trying out convex for a new greenfield app at work but have 2 big concerns:
- what does the ETL story look like for extracting data from the convex backend without going through the app?
- how does cost scale; specifically how will # of functions + compute-hours of actions scale for real-world apps (~10k MAU)
On the ETL side we will need to sync to our (semi-custom) datalake at least daily and preferably allow analysts to directly do cross-db queries (on read replicas) for the most up to date data. Realtime CDC not necessary. I see the Airbyte and Fivetran connectors but we don't use those vendors.
In addition to obvious DX wins we perceive a huge benefit of convex as being a full all-in-one stop for this app. The File API pricing likely won't fit our needs so we'll eject straight to S3 but I have little intuition for understanding how other resource costs will scale
4 Replies
Thanks for posting in <#1088161997662724167>.
Reminder: If you have a Convex Pro account, use the Convex Dashboard to file support tickets.
- Provide context: What are you trying to achieve, what is the end-user interaction, what are you seeing? (full error message, command output, etc.)
- Use search.convex.dev to search Docs, Stack, and Discord all at once.
- Additionally, you can post your questions in the Convex Community's <#1228095053885476985> channel to receive a response from AI.
- Avoid tagging staff unless specifically instructed.
Thank you!
I'm not sure how anyone can answer this unless you have a baseline idea of the number of function invocations per day per user you expect. Then your expected number of daily active users. And the number of records to sync daily with the percentage potential mutations.
If i recall a rough ballpark idea is on the team plan single user it was like $25 would get you a bit over 800 invocations per user per day assuming 1000 DAU. You could blast through that in minutes with poorly written code syncing updates to 1000 users who always have the app open. I generally don't see anyone complaining about pricing other than the premium price for file storage(or people trying to use the free tier to the max). Nor do i see many people getting deep into optimizations and with the couple people i've spoken with convex is saving them over other options with more features. Technically, the biggest cost, dev time is the least factored in most of the time. I'm so guilty of spending hours to save $2
If i recall a rough ballpark idea is on the team plan single user it was like $25 would get you a bit over 800 invocations per user per day assuming 1000 DAU. You could blast through that in minutes with poorly written code syncing updates to 1000 users who always have the app open. I generally don't see anyone complaining about pricing other than the premium price for file storage(or people trying to use the free tier to the max). Nor do i see many people getting deep into optimizations and with the couple people i've spoken with convex is saving them over other options with more features. Technically, the biggest cost, dev time is the least factored in most of the time. I'm so guilty of spending hours to save $2
Yea I guess I'm thinking the query/mutation/action model maps relatively nicely onto a traditional API so I could estimate invocations with # API calls. So to be more precise with my question I wanted to check if that's actually a decent hueristic for estimated function calls or do others see different scaling on query/mutation/action compared to API calls
ETL would be tough without using one of the streaming integrations, if it's at all possible to onboard Fivetran for this purpose it'd be worthwhile. For pricing, there's a calculator on the pricing page, and it provides a lot of controls under the expandable "advanced settings" panel: https://www.convex.dev/pricing