virtual / calculated fields
I'm not sure if Convex really has model classes, but is there a way to extend a schema/model with something like virtual fields?
canonical example is you have first, last name fields and want a fullName that just concats them.
In my data model, it more just seems pure objects and I'm not even sure if there are underlying classes that I could patch methods onto that can look into the data of the object.
12 Replies
You can go-style use functions that operate on these types, function userFullName(user: User){}` etc. Operating on the raw objects is the most common way people do this in Convex today.
wondering where i can apply that method so i don't have to recalc every time i return a value?
is there an actual underlying class for each table in the db? i havent looked at schema defs yet (typescript)
https://stack.convex.dev/functional-relationships-helpers
looking at the schema ex here, it seems about defining types, not functions
Functional Relationships: Helpers
In this post, we’ll look at some helper functions to help write code to traverse relationships in a readable, predictable, and debuggable way.
In general you will either recalculate the "getter" on every read or you could "memoize" it by writing it directly into the document in the db.
If you're worried about recalculating the getter many times inside of a single query, Convex does cache
db.get(id) calls but there is currently no way to hook into this, so this is not possible to avoid unless you roll your own cache (practically this is unlikely to be of concern perf wise though - if the calculation is really expensive it should probably be memoized as I described above).
There is no actual underlying class for each table. Currently the db always returns a Plain Old Javascript Object for each document.
Besides perf your concern might be with code duplication, if you have many places that read documents of the same type, you will need to apply the same function in many places (in the style Tom suggested). You could write middleware that does this automatically but I don't think we have any examples of this yet, although you could start with Lee's RLS middleware: https://stack.convex.dev/row-level-security I wouldn't suggest this though atm.Row Level Security: Wrappers as "Middleware"
Implementing row-level security on Convex as a library. Wrap access to the database with access checks written in plain old JS / TS.
could the POJO have a mixin mixed in for these extra methods? since my queries are running client side, we're not sending any code over the wire, just mixing some extra methods into the returned POJO. they would only run if the user invoked them.
If you want custom getters on the clientside you can add methods to the returned POJOs, or use a wrapper class:
(either would also work on the server side, but only POJOs will be transfered over the wire from server to client or client to server)
the code that goes into
/convex dir is eventually running client side? or those are pushed funcs that are run server side? still a bit confused about the basics here 😄No worries! Any code exposed via
query, mutation or action wrappers imported from convex/server and used in the convex/ directory runs on the server. Check out the tutorial https://docs.convex.dev/get-started for a short walkthrough of how Convex works.Get Started | Convex Developer Hub
Build your first backend with Convex
One benefit to not making custom classes is that when you return objects from the server, the class won’t be preserved. So by making the helper function that takes in a POJO and calling it when you need fullName, it will work anywhere. Alternatively having a helper that turns the DB object into another POJO with derived fields makes a lot of sense, and you could apply it to a whole array like
users.map(userWithDerivedFields)
For many documents, it’s much more common to read than write, so you could alternatively run that function before inserting / updating a user to always save the derived /denormalized data.
Beyond client/server, POJOs are also useful for server/server- like scheduling actions or running mutations from actions. The mental model is roughly that they get json serialized between function boundariesI too am familiar with Calculation / Formula fields from the likes of Airtable. Would love to see a tutorial on how things are done differently within convex when juggling similar requirements. I think I understand what's been discussed above, but a formal hands on tut would be awesome here - using the simple example of concatenation of first and last name, or Author and Book Title, where the string details have been captured in different v.strings and there's a requirement to join them.
For reference, this is what I've worked out of for a few years now, the Airtable formula reference - which is not unlike Excel formulas (but row limited, so no where near as powerful as excel).
https://support.airtable.com/docs/formula-field-reference
And with these formula fields, the biggest complaint with them is that base designers can't just run pure JavaScript within a field - and we have to use these limited/buggy/painful-to-write formulas to get the data to do the thing. But that said, I suspect that with the teqniques mentioned above, that convex is absolutely about writing a function to do-the-thing - for example, using JS intl... to do a tonne of heavy lifting when it comes to formatting and localising dates (and soon to be durations, etc).
Example of JS intl.DateTimeFormat (if only I could have run this in a formula / calculation field!!)
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl/DateTimeFormat
@ballingt - not sure if bumping old threads is the thing to do here?
Big picture, the main pattern for this in Convex is writing query functions that return a modified version of a database record.
You might have a table with columns "first" and "last," but return an object from the query function that just has name. You might store a timestamp in the DB but return an array of objects from a query function each with a formatted date
When a query function does a db.query("tasks").collect() that returns an array of objects, but you can transform these objects (say by adding properties to them) before returning them
which is probably the closest analogue to calculation / formula fields in Airtable
we haven't finished this whole direction, but the safe way forward is to just put them in functions for now. what we haven't done yet, but may explore soon enough, is have better patterns for like queries calling queries. and when this happens, there's a possibility to use the query cache for subqueries, which will automatically memoize these calculated values
then it could be something like step 1 -- have a better primitive, like queries calling queries utilizing the query cache to make these "properties" almost free to access. then all the field combination logic stays in pure javascript. then step 2 -- figure out some sort of orm layer (part of a broader need we discuss all the time) that makes it more ergonomic to express these calculated fields. said ORM would probably use queries-of-queries under the covers, but remove some boilerplate or something
no matter what -- convex will always support regular JavaScript for this, so functions (or query functions) are the right answer. convex doesn't need a DSL for this or anything