I have a file full of helper functions
I have a file full of helper functions for every table. Mostly crud stuff, listing, pagination, etc. Boring "db layer" stuff. For every get-by-id, which occurs many times throughout each file, I use a shared utility function that throws if the id is not found.
Then I started having to deal with what happens when something - literally anything - gets deleted. (Things crash.) I've been avoiding this for months, but I'm now convinced this was absolutely the wrong approach. I've now changed that function to just return undefined, and am updating my entire application to handle it.
Throwing this out there in case anyone else is thinking through this sort of thing, and of course, in case anyone has helpful thoughts on the matter.
36 Replies
I'm curious how it happens that deletion causes gets by ID to fail. Where are you storing IDs such that this is possible? I've thought about this before (because I basically do the same thing), but it's never come up for me as an issue
It's ubiquitous. URL path, component state, anywhere the client gets a handle on an id.
Common example:
- view an entity using it's id in the url
- delete the entity from within that view
- component crashes
To address this, you would typically build a graceful exit from the view into the delete flow. But then if another user is viewing the data at the same time, they don't get that graceful exit and it crashes for them. So now I need graceful handling from outside of the delete flow.
The graceful handling is also not straightforward in some cases, like when the component is not only getting the entity by id, but also (for example) calling multiple paginated queries that are also driven by that id. Now those paginated queries need to fail gracefully as well, which is challenging due to the inability to return a proper error state from a paginated query.
On top of this, a view with an id in the url is just one case. You could have a view of a related entity, and maybe have some of that entity state represented in local component state. If that local state includes the deleted relation id, then this component could also crash if the related entity is deleted.
Trying to find all of the cases where something might fail if an entity is deleted is a whack-a-mole operation, and the types don't really help.
I know Effect is meant to solve this very problem, but it's such a huge commitment, I don't think it's one I'm going to make. There has to be an approach that doesn't require fundamentally changing the way all of my code is written.
@RJ I appreciate the question on where I'm storing the id, though, I hadn't considered that every instance of this problem requires that I'm referencing a convex id from outside of the reactor. I can't say I'm super hopeful, but I'm going to take a swing at categorically narrowing the impact sites based on these external references.
I should also state that I understand this isn't actually a Convex specific problem, it's a general application development problem that becomes more impactful the more realtime your app is. I'm not actually frustrated with Convex right now, I'm frustrated with application development lol
Actually, I'm just going to continue with my current plan, it's just going to be a bunch of work and a huge PR that will probably introduce some fun new bugs 🎉 . Having all queries return undefined, empty array, etc gives a type safe baseline to make sure I'm at least just showing a blank component rather than crashing entirely when an entity disappears. From there we can add graceful handling to the higher priority interfaces/flows. I suspect This is The Way, but still curious if there are approaches/angles I'm not considering.
If this is the recommended approach, we should consider how to help folks get that information early in their Convex use.
I am thinking we should code the UI against an explicit state machine. I mean a query is a state machine. We have to design that state machine once.
Effect is too powerful, I can do more with less for this problem space.
If you take a look at Redwoodjs, they are handling this nicely. Different stack, but nice.
Multiple good takes.
Totally forgot RedwoodJS existed. Just checked their docs to see what you meant and saw some Prisma code and immediately closed my browser. Panic reaction.
Seriously though, curious what specifically you saw in Redwood's error handling that felt like a good pattern
I see yeah, that all makes perfect sense. A couple unorganized thoughts:
- Given that this problem (should) only arise when you're passing in IDs from outside the Convex reactor context, maybe all you need to do is verify the presence of IDs that are provided as args to Convex functions and actions?
- Effect could help here a bit in the sense that it could make the error handling easier to deal with inside a Convex function, but I don't know that it would help that much with the main underlying problem.
Re: the first bullet above, this obviously wouldn't solve the UI issues, but would maybe make operating within the context of your Convex functions a bit easier to handle?
Since you are in a rush @erquhart , I think the main idea is to think that a query has a lifecycle. This is valid for all apps.
We are missing a piece here, and your are discovering it.
I think that:
- promises and async/await is NOT ENOUGH
- effect is TOO MUCH
maybe all you need to do is verify the presence of IDs that are provided as args to Convex functions and actions?@RJ I like this, but there's still the problem of paginated queries and what to return from them when a received id doesn't validate
Yeah, I don't have any great ideas there. I guess you'd need to use
ConvexError, or write an alternativeusePaginatedQuery hook. Or maybe you could smuggle the error message in the pages property, but that's grossI did think of that gross option as well lol
ConvexError would still need to be caught by an error boundary, and I really want to avoid using error boundaries for something that really isn't an error case. They'd be everywhere.
Really appreciate the ideas here y'all ❤️
For inspiration, take a look here. https://docs.redwoodjs.com/docs/cells/
Cells | RedwoodJS Docs
Declarative data fetching with Cells
They are serious with the metaphors lol
taking a look
man that stack is wild, I've already forgotten how utterly verbose and indirect things were before convex
But yeah, I do appreciate the lifecycle concept. It's sort of describing middleware, though, which I love less
Our design focus here can be the state machine, forget the rest, just the query lifecycle and how that affects the UI. And come up with a general pattern
I like that. I think you and RJ (from his first bullet point) are saying the same thing here, effectively describing the
beforeQuery pattern from Redwood's cells. Validating before proceeding.
A custom usePaginatedQuery might be all that's needed to make that work. And the custom hook would just return an empty state in an error case.Definitely Effect solves all the problems it's the God tool in js land.
But sometimes you want to isolate problems and their solutions in a limited domain language. So that it's easier (for the team) to design and think about the pb.
It's basically the idea of having multiple languages vs one language.
I saw that! You're a lisper! lol
But yeah agreed, I would use Effect if I didn't have to rewrite everything to do so
Also I'm at the point where I consider it a victory when I don't have to learn new things in application development
The idea is to have simpler languages instead of one powerful language.
I think for app dev it's better. We want to be close to business people and designers.
"The least power design principle"
Now to give credit to @RJ , my approach here would be:
1. Solve the pb with effects, cuz it's has all the necessary semantics.
2. Hide effect semantics and bake them in a simpler API that is less powerful and more accessible
3. Tree shake concepts and code, optimize, rewrite.
4. Share with the community
That would be ideal
Making a custom usePaginatedQuery is more involved than I hoped. But I'm also thinking that just returning an empty array for
page in the case of an invalid id should effectively reset the client results to empty - reactivity means it would rerun everything, so this should just work.@erquhart will this https://github.com/thelinuxlich/go-go-try used at the boundary between your queries and UI solve you problems, and offer better better DX ? you seem to have a good mileage in convex, and I am just starting, so I want to learn from your experience 😉. This way of doing error handling is common is other languages. I never used try/catch in elixir in the last year. I think this pattern works great ! The idea is that 'errors' are just data so we don't need another construct language construct.
From my experience in Pheonix Framework, I can confirm that it works very well.
I've had golang style error handling in the back of my mind for some time, and have considered using that approach with my queries in general. Really love the approach, and cool to see there's a lib implementing it.
between you and me, I think that convex is too naked. it needs a convenience layer on top. I feel also that conveniance layer doesnt have to be core technicaly but I think core to the convex business.
Agree! And that's actually the exact intent - build a solid core with only the required bits (the llvm), leave the rest for layers above. Some of which will be "first class", but still not technically core.
but I feel that we need some guidance to build that layer.
I prefer the Vue leadership to React leadership.
It's like we need to come up with a Roadmap for the community.
So far it's like assembly language for querying, and just hopping that that layer will emerge, this is not good enough for us app devs. We need a Vision, and so far that Vision is "here are the primitives" build whatever you want on top.
It's like what React did. We will endup with many contradicting patterns that don't serve the community. I feel that many problems that @jamwt is expecting us to build are know problems already. So the idea of having champions is great ! but I think we need some ambitious design docs of the future layer. Then we can come together.
But that leadership has to start from core IMO.
I'll rather not build a framework on top of convex, something like (next for react), (nuxt for vue) is a no go for me.
80% of people wants to build with sane defaults, successful unicorns can go nerdy with low level.
The components framework they're working on should be along the lines of what you're getting at here, or at least toward enablement and providing some patterns
any links ? roadmap ? please
There's a light mention here under Components: https://discord.com/channels/1019350475847499849/1019372556693815418/1220856197725294643
thanks @erquhart . I'll just focus on building my app, and re-check in a few weeks then. I am here for the platform. Something will emerge for sure. I'll try to:
- keep the querying isolated and do a rewrite of queries down the road. (The platform is very solid in terms of separating 'functions' from the UI, I don't have to do much here. We have 'functions' in the dashboard. that's the separation)
- Now it's just a matter of using go-go-try to have better signatures for the 'functions'. My UI will be more solid and shielded from the rewrite.
- The goal is to have stable signature to manage the lifecycle of the queries. To be used by all these 'functions.' To make it nicer to build UIs
@Chakib Ouhajjou to be clear, we’re not permanently eschewing the responsibility to formalize higher layer constructs; I’m just being honest and saying we’ll (a) do it slowly and (b) never “obscure” the llvm layer so you’re team can do it differently.
Slowly is because of two things: 1. We learn by watching what works out well for teams in practice and 2. Frankly, just flat out headcount. We’re a small team with a really ambitious and complex charter.
And when we say something is officially supported, we mean it.
So we have to have capacity to keep that commitment.
Until then, I think a lot of you will be “ahead of us” and that’s kind of great.
Thanks for transparency, it really helps set my expectations and better define the 'contract' between my project and convex.
And I completely understand I am one guy building an ERP.
It means we’re tapping into a broader and broader set of talented and creative people to help refine how to use convex. And many of you are using it “in anger” more that we are. We’re very far from owning the most complex convex codebase right now. So our perspective is not quite as valuable in many ways.
This definitely helps me shape my thinking about convex. I just have to accept that it is low level and high level at the same time. And get on with it.
Video Games for example are done that way.
Thanks for the clarity !! 😃
I am happy 😊
I've landed on handling this mostly in my existing authorization functions, which already start off every query and mutation anyway. I'm updating queries that take in ids to return an error key with a typed error for a not found entity (I'm already returning a
{ data } object explicitly from all queries anyway.)
For posterity, returning an empty page in the not found case when dealing with paginated queries: https://discord.com/channels/1019350475847499849/1256744780109582467/1256744780109582467
hahaha just realized I'm implementing my own crappy version of Effect 😂 😭
or the Success/Error paradigm anywayeffect is not stable enough as a spec based on 4 hours of research I did. I hope I am wrong. A queston for RJ
but I still think, just doing error as Data is the way to go. If your pain is local (for a specific problem)
I would use effect if I want to really improve the quality of my code base at all levels. But based on my research some people are not happy with memory consumption behavior, and reverting back from Effect. I would not be confident running it on convex. not my servers, I'll rather be carefull with memory consumption.
Yeah I haven't used it, and I have read about some of those issues. Really rooting for them, though, it feels like they're really onto something. And error handling in js/ts is just awful.
I think TS has all the primitives to do good error handling. I think it's just that conventions are set to favor try/catch. But I dont see the language lacking there. In Elixir ecosystem the conventions are that errors are treated as data. No try/catch. Elixir has try catch, but we dont use them in our web apps.
Yep, try/catch is the issue. And it's load bearing across the ecosystem, very tough to opt out of.
The worst part is the places where it's handy, like bailing on Convex mutations. Of course that could also be facilitated by calling some function provided through the Convex function api.
but you can limit it's propagation. dont make try/catch logic leak into your code.
You can kill the snake by wrapping external code that you call with a catch all.
it's like if you commit to never use try/catch around your code. and only around external code. You will have good error handling DX.
Sometimes a Style Guide fixes many problems. Here is an example:
https://v2.vuejs.org/v2/style-guide/?redirect=true