Is it really so that convex uses
Is it really so that convex uses basically one db table for all convex tables in Postgres? This seems a bit odd (and i'm a bit worried as our current db is few terabytes of data). I was planning to use Postgres directly for reporting but now it does not seem possible
8 Replies
Yeah - Convex uses postgres (or mysql or sqlite) as the persistence layer, but does not use its full power as a database.
Convex itself is implementing a reactive relational database on top of it.
https://docs.convex.dev/understanding/#database
What kind of reporting are you hoping for?
You could use convex functions to report on your convex data. That feels like the right layer of abstraction for getting information about the Convex tables.
Okay. I was hoping to plug and play Apache Superset to Convex postgres, but that is not going to happen. I don't know details of course but I wonder why to utilize Posgresql/Mysql at all then?
can you say whyyou're worried about this? one table or many doesn't really make much of a difference when it comes to scaling in a modern RDBMS
see https://discord.com/channels/1019350475847499849/1357260531035148360/1357260531035148360
Durable persistence to disk is a hard problem that (IMO) is one layer below the relational aspect of databases. It's something that these RDBMS' do really well (as @Jamie is mentioning).
Eg check out this article from SQLite
https://www.sqlite.org/atomiccommit.html. Postgres and Mysql are similarly battle tested and good at this. That's a big part of why we use them for durability/persistence to disk.
The actual relations, reactivity, subscriptions, stored procedures in typescript - that's Convex's magic! You can check out the source here. https://github.com/get-convex/convex-backend/tree/main/crates.
You can probably hook up to apache superset via streaming export in the cloud product (eg fivetran or airbyte).
Quite bold statement... Indexes are not free... But I'm not here to debate on this. For us this was/is showstopper. I know we could build integrations (and add additional systems) etc to get it up and running but then we would lose all the benefits of (relatively simple and integrated) system and lose the benefits that convex "creates". IMHO this just moves complexity, but does not solve it. Interesting project though and we will keep following it...
okay, gotcha
well, thanks for trying convex! sorry it didn't work out, it happens, I get it. I'm glad you all found another solution you're happy with!
Unfortunately we didn't find such a solution. Perfect for us would have been Convex utilizing PostgreSQL functions in more standard fashion (to be able to use tools like CNPG, Superset etc seamlessly with it). But that is life...
which postgres functions? you mean if, for example, convex fields mapped to postgres fields and tables to tables? sorry, I misunderstood your use case. I think I get it now
yep, we don't have a great solution for that yet. the way we intend to solve that soon is to embed a simple olap engine, like duckdb or something, or clickhouse, so you can point standard tools at it (superset) and do nice high performance analytics stuff
I hope we can get that out in the next 3 months or so