High level questions about convex ETL story and cost scaling
Interested in trying out convex for a new greenfield app at work but have 2 big concerns:
- what does the ETL story look like for extracting data from the convex backend without going through the app?
- how does cost scale; specifically how will # of functions + compute-hours of actions scale for real-world apps (~10k MAU)
On the ETL side we will need to sync to our (semi-custom) datalake at least daily and preferably allow analysts to directly do cross-db queries (on read replicas) for the most up to date data. Realtime CDC not necessary. I see the Airbyte and Fivetran connectors but we don't use those vendors.
In addition to obvious DX wins we perceive a huge benefit of convex as being a full all-in-one stop for this app. The File API pricing likely won't fit our needs so we'll eject straight to S3 but I have little intuition for understanding how other resource costs will scale
- what does the ETL story look like for extracting data from the convex backend without going through the app?
- how does cost scale; specifically how will # of functions + compute-hours of actions scale for real-world apps (~10k MAU)
On the ETL side we will need to sync to our (semi-custom) datalake at least daily and preferably allow analysts to directly do cross-db queries (on read replicas) for the most up to date data. Realtime CDC not necessary. I see the Airbyte and Fivetran connectors but we don't use those vendors.
In addition to obvious DX wins we perceive a huge benefit of convex as being a full all-in-one stop for this app. The File API pricing likely won't fit our needs so we'll eject straight to S3 but I have little intuition for understanding how other resource costs will scale
