need help setting up airbyte to export data from convex
Hellooo, I’m trying Airbyte for the first time because I desperately need OLAP. But I’m running into what looks like completely broken behavior:
- Every destination table contains the entire source database snapshot, not just its own corresponding records.
- This means every table I sync ends up being a massive duplicate of the whole source, instead of a clean representation of that single table.
- To make it worse, there’s still no normalization happening afterward — everything is kept in raw form.
- I tried using destinations like ClickHouse, Postgres, or S3 and they all the same. for example I have a table with 5 records. but airbyte streams like 200mb+ data for that table (which is probably my entire database raw json dump)
This looks like misconfiguration, but I can’t figure out where. Is there some step I’m missing in Airbyte that prevents this “all-database-in-every-table” behavior? Or is this a known bug?
What’s the right way to configure Airbyte so that:
1. Each destination table only gets the data from its matching source table.
2. Normalization actually produces clean, relational OLAP-ready tables instead of just raw dumps.
- Every destination table contains the entire source database snapshot, not just its own corresponding records.
- This means every table I sync ends up being a massive duplicate of the whole source, instead of a clean representation of that single table.
- To make it worse, there’s still no normalization happening afterward — everything is kept in raw form.
- I tried using destinations like ClickHouse, Postgres, or S3 and they all the same. for example I have a table with 5 records. but airbyte streams like 200mb+ data for that table (which is probably my entire database raw json dump)
This looks like misconfiguration, but I can’t figure out where. Is there some step I’m missing in Airbyte that prevents this “all-database-in-every-table” behavior? Or is this a known bug?
What’s the right way to configure Airbyte so that:
1. Each destination table only gets the data from its matching source table.
2. Normalization actually produces clean, relational OLAP-ready tables instead of just raw dumps.
