Why Convex Sucks — For Now
🗣️ 🎙️ New Databased Episode: Convex SUCKS 📻 🔊
Founders are happy to tell you their product is great, but you already know that, right? Right?? 😰
So on this week's episode, we've decided to dive into why Convex kind of sucks.
https://www.youtube.com/watch?v=FMhaM3yXYbk
We even made a website: https://convex.sucks
If you have more reasons you think Convex sucks, feel free to add them to this thread. We can take it! 🥹
Convex
YouTube
Why Convex Sucks — For Now
In this episode of Databased, Jamie Turner and James Cowling grapple with some bad news… they think each other’s product sucks…? Jamie and James candidly discuss the challenges and misconceptions surrounding Convex. For example, they explore the learning curve for new users, emphasizing the balance between conceptual understanding and practical ...
Convex SUCKS!
Let’s talk about how Convex sucks
44 Replies
Timestamps for Key Topics Discussed
3:55 - The initial learning experience with Convex, such as the time it takes for developers to grasp new concepts and methods.
4:38 - The difference between conceptual understanding and practical application, emphasizing the importance of both in mastering Convex.
8:09 - Concerns about developer lock-in, and how specialization in Convex might affect career mobility and skill transferability.
24:48 - The significance of opinionated frameworks that guide users, helping them make informed decisions while enhancing productivity.
24:48 - The balance between providing flexible tools and maintaining structured guidelines to support users as their projects evolve.
30:50 - The necessity of clear pricing structures, allowing users to estimate costs based on their specific application needs and user growth.
36:52 - The importance of robust customer support, ensuring users can navigate the platform effectively and maximize its benefits.
56:55 - The critical role of indexing in database management, emphasizing the need for efficient queries to optimize performance.
40:34 - The value of case studies to illustrate real-world applications and challenges, making technical concepts more relatable to users.
51:56 - The importance of educating users on writing efficient SQL queries, helping them avoid performance pitfalls in their applications.
1:01:51 - Implementing feedback forms to gather user insights and critiques, fostering continuous improvement of the platform based on user experiences.
1:03:09 - Incorporating positive affirmations and recognition within the development process to motivate teams and enhance productivity.
Timestamps for Key Takeaways
5:46 - Despite the initial learning curve, developers can become productive with Convex within a few days of use.
9:30 - The importance of viewing oneself as a software engineer rather than just an expert in a specific tool, even Convex.
15:50 - The significance of establishing trust with users, especially when introducing a new product in a competitive market.
1:01:51 - The value of feedback from the developer community, which helps them identify areas for improvement in Convex.
16:19 - Releasing Convex may inadvertently seed competition, highlighting the dynamic nature of the software development landscape.
24:48 - Implementing opinionated frameworks in software design to guide users toward making informed decisions and enhancing overall productivity.
24:48 - Achieving balance flexibility and structure in tools to accommodate evolving project needs while providing clear guidelines for users.
30:50 - Developing transparent pricing models that allow users to easily estimate costs based on their specific application requirements and anticipated growth.
36:52 - Enhancing customer support by creating comprehensive resources and training materials that empower users to navigate the platform effectively.
56:55 - Optimizing query performance by focusing on indexing strategies that improve the efficiency of SQL queries and overall database management.
40:34 - Creating a pricing simulator to help users visualize potential costs based on different scenarios, aiding in budget planning and decision-making.
40:34 - Utilizing case studies to demonstrate real-world applications platform, making technical concepts more relatable and understandable for users.
51:56 - Educating users on SQL query writing to help them avoid common performance pitfalls and improve their data analysis capabilities.
The most annoying thing are the typescript errors that appear when you create a let initialQuery = ctx.db.query("col") and then conditionally want to add to the query later,
if (something) {
initialQuery = initialQuery.filter...
}
In a bunch instances this will give a typescript error which is super annoying for being able to organize code nicely. Instead I typically have to start with a blank query variable to avoid the TS errors.
Also, I happened not to be the biggest fan of the chaining method to build a query. Would be way cooler if you could pass an object of filters/indexes, etc.
Biggest pain point though is the inability to use multiple indexes when chaining, AND the inability to filter before an index is used, instead of having to create many custom indexes. For example, I'd like a search index that's pre-filtered by a different field (such as an organization id) so it isn't first searching against your entire table and docs that aren't relevant. This has unique problems with doing pagination properly and not having to collect() the whole result of the search query only to filter it manually in js later.
BUT I love convex so far 🙂
thanks for the feedback!
re: multiple indexes, unless I'm misunderstanding you, you would use an index on multiple fields to accomplish this
the reason you can't filter before an index is used is because that would require walking the entire table, which gets to be a nonstarter after more than, say, a few hundred rows
and if you had that little data you wouldn't need indexes in the first place
this wouldn't really be possible in a SQL database either. the query planner will always prefer the index first, or, if it has to table scan, give up on indexes altogether. if you already visiting every row in the database, indexes buy you nothing! the only purpose they can serve is to limit the amount of data you need to fetch/evaluate for matching predicates
Databases are Spreadsheets
I want to share my mental model of databases:
- Databases are just big spreadsheets
- An index is just a view of the spreadsheet sorted by one or mor...
Thanks for the reply!
I'd add one more thing that'd be useful - in the defineTable, when creating or specifying a column, instead of having to write v.optional on a column (and for many other insert/patch use cases), it'd be create if you can optionally define a default value for a column if none is specified
A lot of highly-paid software engineers value their time at zero and proudly spend hours trying to save a few dollars (hi Hacker News!).shots. fired. 🔥
I initially attempted to learn
Postgresql, investing in Udemy courses and dedicating significant time to it. However, as a frontend enthusiast, I found backend development challenging. I then explored Drizzle, a type-safe SQL solution, and concurrently learned Hono.js and other related technologies.
During my journey, I encountered undocumented SQL issues and inadequate explanations of relationships. This led me to explore Prisma, which I found to be powerful but complex compared to Drizzle. I also experimented with TypeORM.
While watching YouTube tutorials, I noticed that many frontend developers use Prisma without fully understanding its capabilities, often relying on pre-written code. That's when I stumbled upon Convex, which resonated with me because it allows me to leverage my frontend skills for backend development without requiring extensive backend knowledge.
With Convex, I can create functions and let the platform handle the underlying complexities. i can run typescript in the conext of databaseAwesome to hear! That's the goal exactly. Thanks for sharing with us.
I recently encountered a challenge while filtering a database table using indices. The pain point was the requirement to use compound indices in the exact order they were defined in the schema. For example:
This meant I had to use the indices in the same order: isDeleted, isFeatured, isPublished. Here's an example of the query:
I hope this clarifies the issue I faced.
Is there another way to achieve this without creating additional indices? I'd love to hear suggestions or alternative approaches!
unfortunately no -- nested indexes are really just btrees, where the keys on
["age", "name"] end up ordered like [(14, "Johnny"), (14, "Taylor"), (18, "Randy"), (22, "Bill"), (22, "Callum")]
so there is no way to get to the "ordered set of names" without first starting with an age
you need a second index with the name first in order to "start" with name
the ordered set of names aren't actually continuous -- they're only continuous within a given value of ageWould it be acceptable to use a standard filter approach with a table containing approximately 5,000 rows? Or would the performance impact be too significant?
here's a good article on it: https://stack.convex.dev/databases-are-spreadsheets
Databases are Spreadsheets
I want to share my mental model of databases:
- Databases are just big spreadsheets
- An index is just a view of the spreadsheet sorted by one or mor...
any reason why not to just have another index? it will be like 100x faster even at 5,000 rows
I've created four indices to optimize my queries, but maintaining the code has become increasingly challenging.
ah. is it because you have reporting interfaces where they can sort or query by any field?
usually when you find yourself wanting to filter over relatively large amounts of records in arbitrary ordering, you have something more like an OLAP pattern. we have some plans to ship an OLAP engine to make it easy to do aribtrary queries over slightly stale data in a high performance way, but we're likely not going to ship that for 4-6 months unfortunately
you can definitely get into a whackamole index game if you try to build something like that with convex and OLTP patterns.
after all i decided to use normal filter like this
yeah, for now, if it's really 5k records, and you're happy enough with the performance, that's probably fine
our record caching will make it so you're not having to hit the database most of the time
it's not ideal, but until we have this embedded OLAP system, I'll admit convex doesn't have a great answer for these use cases
I think we talk about that briefly in "why convex sucks", but can't remember offhand
Could you please provide guidance on implementing offset-based pagination? I'd appreciate an example or a step-by-step explanation to help me understand how to paginate results effectively using offset.
well, if it's a relatively static data set, you can take the final list that comes back from your filters and slice it
this won't account for records that get inserted in the middle of pages, but traditional offset-based pagination always gets those wrong, so it's no worse than those systems
your query can just take a
start and count argument, and then you use those arguments to https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sliceMDN Web Docs
Array.prototype.slice() - JavaScript | MDN
The slice() method of Array instances returns a shallow copy of a portion of
an array into a new array object selected from start to end
(end not included) where start and end represent
the index of items in that array. The original array will not be modified.
after you finish your filter
return results.slice(start, start + count);It would be greatly beneficial if Convex provided a skip method, allowing for more flexible and efficient pagination
it won't be more efficient b/c you're using filter
if you're using filter, you have all the records already in memory
withIndex ranges are the only things that limit the amount of data loaded into memory from the databaseI've read the Convex Stack post 'Take Control of Pagination', which led me to conclude that Convex is committed to following best practices to make its users' lives easier, even beyond just pagination.
One of the things about coming from frontend to backend is you don't know what you don't know (as in any kind of learning, really), but you can trust the Convex team to give you all the primitives, abstractions you need to build
right -- it's sort of impossible to account for mutating pages in offset-based pagination
doesn't matter which database it is
so you can do the same thing in convex -- just count the records in the page
and then ask for the 51 if you got the first 50
you might actually miss one because the page sizes changed
but same deal in a SQL database -- it's just a fundamental property of offset-based pagination
And that's why I'm committed to sticking with Convex, no matter what. Their dedication to user-friendly design and best practices has won me over, and I don't want to compromise on that.
but for your app, if you want offset based pagination, no problem. the way to do it is just via simple list slices, and it will work the same as it does in other databases. and for some applications, that's completely sufficient
no diss on the right tool for the job
Does the usePaginatedQuery hook implement all best practices for pagination, or these applied at the database level?
yeah, it uses more like "infinite scroll" pagination which does the right thing so you don't skip records and it accounts for changing pages and things
but unless you build your UI in infinite scroll fashion, it can be harder to use. fixed page numbers and "next/prev" buttons don't mesh as well with that infinite-scroll style pagination
i used infinite scroll with table row virtualization
yeah, cool. that works! and more modern than page numbers for sure
but full disclosure, there are too many use cases where the built-in method of pagination doesn't work out. lee documents them all well in his stack post ( https://stack.convex.dev/pagination )
Take Control of Pagination
Convex offers robust control over pagination with a powerful function, getPage, enabling complex edge cases. In this article, we go over how to use th...
because of this, we haven't really taken pagination out of beta, and we think there might be a more flexible way in the future
lee's helper is probably closer to how we aim to handle this stuff in the future than the current api. we won't deprecate that api or anything, but there's likely a v2 solution to pagination we'll ship in the not to distant future that gets around some of the limitations, like joins etc.
the built in solution we have is a really great and complete solution as long as your query fits exactly the case it was designed for
if your query doesn't fit that case, then it suddenly doesn't work out anymore.
lee said "not enough examples"
https://discord.com/channels/1019350475847499849/1283043121730420808/1283157916458684581
I was wondering if I opt to implement cursor-based pagination manually, as described in the Convex documentation, rather than using the
usePaginatedQuery hook, can I still achieve the same benefits, such as overlap pages, as discussed in the Convex Stack post (I couldn't find the exact post, but I recall reading about it)?so I just read the original thread
I think you can keep it simple and just use the standard pagination, no? you're only using one index
just paginate after filtering
you'll occasionally get empty pages, but you can just query more when you do
with so few records it shouldn't be super noticiable
just do your one
withIndex, conditionally chain .filter() to the query, then paginate at the end
oh you're using search
hmmAfter reviewing the source code of the
usePaginatedQuery hook, I've concluded that it's best to stick with it, as it provides optimized functionality that would be challenging to replicate manually.yeah, dunno if the fact you're using full-text search will complicate things. otherwise, I think using
paginate and usePaginatedQuery is gonna do a lot of the hard work for you
just be ready for if the results are empty and status === "CanLoadMore" to just call loadMore right away
but the rest will just workThank you for the helpful discussion! I'm excited to share that I'm currently developing my portfolio, and I've decided to include Convex in my tech stack, alongside Next.js, TypeScript, Tailwind CSS, and Playwright for testing. I'm looking forward to leveraging Convex's capabilities to build efficient and scalable applications.
CONVEX team hard work and dedication are truly appreciated!
here's my suggestion: (1) take a look at https://github.com/get-convex/convex-demos/blob/main/pagination/src/App.tsx ; (2) augment it with an
useEffect that depends on [results, status, loadMore]; (3) if (results.length === 0 && status === "CanLoadMore") { loadMore() }GitHub
convex-demos/pagination/src/App.tsx at main · get-convex/convex-demos
Demo apps built on Convex. Contribute to get-convex/convex-demos development by creating an account on GitHub.
that will make sure that if your filters eliminate all the records in a give page, you keep trying until you find some to display
it's probably possible to even write a little react hook that's a wrapper around
usePaginatedQuery that keeps driving the loader until it gets some results to showI'm using the hook and just slicing the "page" I want. Its simple enough and works well.