6 week eng roadmap
Hi all! I wanted to share some inside info on engineering projects we’re currently prioritizing at Convex. We have a long list to work on after these but the projects below are what we plan to achieve in the next ~6 weeks.
Note that these are our internal plans and don’t constitute any guarantees 🙂
We’ll try to give periodic updates as new features and priorities originate. Feel free to respond with any feedback or questions!
🍎 Core platform
- Better scheduled function management: ability to cancel all running scheduled functions, pause toggle for deployments that you want to stop quickly.
- Programmatic access to metadata so you can directly query and subscribe to storage metadata, scheduled functions and status, etc.
- Redesign pagination to split pages that grow too large, and potentially also allow paginating over multiple database queries in a single function.
- Lots of small ergonomic improvements like unifying our filter/index syntax, allowing unions in validators, relaxing constraints on what characters are allowed in a field name, etc.
- Full-text search improvements, e.g., prefix search, fuzzy search, snippets.
⚖️ Scale
- Scale out function execution to allow greater concurrency and dynamic memory allocation.
- Support larger code modules and npm modules that aren’t able to be bundled client-side.
👔 Enterprise/Pro
- Add preview deployments for GitHub branches.
- Export streams for logs and function metrics, plus exception reporting to Sentry.
- Complete our SOC-2 audit.
- Edge acceleration.
🤖 AI
- Develop an AI Kit for bootstrapping AI apps on Convex.
- Release Game Kit which encapsulates a game engine for apps on Convex.
- Scale up our vector database to support up to 1M vectors per index.
🧑💻 Developer engagement
- Better onboarding flow.
- First-class python client library including subscriptions.
- Additional client libraries for Node and Svelte.
39 Replies
Feel free to share any feedback or questions in here!
All sounds incredible guys, thanks for the transparency 🙂
At the risk of turning this into "where is my feature X".. is there any plans for more per-project metrics?
I know we have some metrics under team settings but I wonder if it would be more visibile to also have them on the project page?
Also, it has been previously discussed but a "v2" or more advanced version of the metrics that is able to estimate costs for function execuition and perhaps leverage AI to recommend ways with which you can improve function performance and thus cost?
Really appreciate the transparency, and excited to see what lands!
From a product standpoint, I think the search improvements will be (another) game changer. It’s a ubiquitous requirement with no easy solution (Algolia, elastic search, etc bring their own new problems), so providing it in conjunction with Convex’s reactive pagination feels like a top tier value add for the platform. Would definitely unlock some things for my current effort.
yep @mikeysee do you mean per-project cost metrics? i think we could have a bit more visibility there, agreed
if you mean metrics like requests per second and function execution time these will be part of the metric streams
we've been wanting to give suggestions about which indexes etc to add to improve function performance. we already have a service ready to go to send notifications but don't yet have the analysis to send the suggestions. will do this for sure sometime but not in the current batch of projects unfortunately
thanks @erquhart! we dropped the ball a little on full-text search because we put the beta out but then didn't have a chance to circle back and finish the feature set. those should be coming up sometime soon
also, @Mikael Lirbank , want to follow up here because I know you highlighted this as particularly important to your team. as you can see from james's roadmap above, preview deployments are the top focus to add next for pro accounts. so we expect to ship something soon and hopefully make your life easier here!
awesome to hear
fair enough, its not a big desire from me right now, just a nice to have 😉
@mikeysee some of the really, really fine grained stuff as james said will probably get solved in the short run by having a really detailed firehose available to put into external tools
the in-dashboard UI, might lag a bit behind if you want a lot of detail about what's happening inside your deployments/projects
So awesome! It will be a game changer for teams! Just imagine how fast we can integrate when we can run, review, and discuss all open PRs independently! Thanks for listening. I'm thrilled!
what are you all using for app hosting? vercel or netlify? or some other service
(just to understand the other pieces of your workflow)
Vercel
GitHub
Auth - currently Google OAuth, a library, so all user/auth data in Convex (but changing soon to passwordless, but still all data in Convex, no auth service)
Next.js
Is it out of the question to infer that Convex is aiming to turn into a one-stop-shop for web projects?
@hack_entertainer how broadly do you view "one stop shop?"
we imagine we'll always integrate heavily with specialized services like LLMs, authentication, payments, etc. and we probably will not own app hosting and app build/deploy workflow (e.g. the role netlify and vercel play) anytime soon. so those will be important complements
@jamwt Well, I meant it quite broadly. With hosting, authentication, llm, and all the other integrations available, it seems like Convex is a good starting point to prototype and ship any web service. Is that too ambitious?
nope, that's absolutely right
The thing I'm missing most so far is native auth integration. Using clerk works, but having to sync userdata from clerk to convex comes with some burdens and extra work. And it feels like it makes user onboarding kind of fragile. I'm using the clerk webhook approach to keep user records in convex up-to-date and am always afraid of the webhook having some outage or delay. It would just be awesome if something like clerk would be a native convex solution!
ye I mentioned that too in my post (https://mikecann.co.uk/posts/tinkering-with-convex).. Auth is quite a big topic tho so I understand why they are keen to let third parties handle it for now
Just a suggestion, I use Hookdeck to solve the webhook reliability issue, as I have to use them for a number of critical systems.
https://hookdeck.com
Webhook Infrastructure and Tooling - Hookdeck
Hookdeck helps developers release their features faster. Less time spent developing, testing, deploying, monitoring, and troubleshooting webhooks.
Yeah, we here this a lot. The good news is we know the clerk folks personally and they're great. We've discussed needing a great way to do this with them in the past and they're well aware of the need. Maybe we'll reconnect and see if there's a better first-class integration we can figure out.
I think you should just write a couple of example recipes for using the most common auth libraries with Convex - a lot of people don't want to use auth services. We got it working but it needs documentation. The main difference with other apps/backends is that Convex uses websockets instead of HTTP RPC, so it just takes a bit of getting used to. I think you could solve 95% of this issue with better docs, and maybe through in an NPM package that makes it even smoother. But it's mostly a documentation issue.
@Mikael Lirbank did you see this? https://stack.convex.dev/convex-with-lucia
Custom Authentication (with Lucia)
Learn how to build a full stack app with authentication without any third-party auth providers, using Convex and the Lucia library.
Templates
The backend application platform with everything you need to build your product.
I did! It came just after we got our auth working so I just glanced it. I will read it more carefully soon.
cool. would love to hear what more feedback you all have, like what other kinds of examples or more auth features etc would be useful
I'd really like to abstract passing the sessionToken as an arg to every db function. Eg. in the Lucia docs there is:
But where is the withAuth code? If you made those middlewares that Ian has posted about on the blog official npm packages (or part of the convex package) that would be epic!
@ian @Indy ^
Like this stuff https://stack.convex.dev/wrappers-as-middleware-authentication
Authentication: Wrappers as “Middleware”
Using wrapper functions like withUser can help you organize your code into middleware-like blocks that you can compose to keep your function logic con...
yeah, we're still working on how to make a great sort of module or plug-in system for convex beyond copy paste
we'll get there
But it doen't work too well to copy paste. The blog gets outdated and I may have other tsconfig settings so TS complains a lot when I copy paste.
yep
for this one specific template, here's what I found in the repo: https://github.com/get-convex/convex-lucia-auth-demo/blob/main/convex/withAuth.ts
GitHub
convex-lucia-auth-demo/convex/withAuth.ts at main · get-convex/conv...
Demo showing authentication powered by Convex and Lucia - get-convex/convex-lucia-auth-demo
Still a big fan of Convex, just sharing my expeiences as we work with it.
I'll take a look!
yeah. I mean, the good news is almost always your gripe list is our gripe list
😄
so the rest is just a matter of time and resources 😛
I'm in the same boat, so I feel you 🙂
(another boat, but yeah, same same)
The lucia template @Michal Srb did put into an npm library: https://www.npmjs.com/package/@convex-dev/convex-lucia-auth
Docs: https://github.com/get-convex/convex-lucia-auth/blob/master/DOCS.md
I'd love to hear how close this is to the format you'd want.
It still requires you to add some tables to your schema, set up some global types, add a cron to delete old sessions, etc. But it's pretty slick
npm
@convex-dev/convex-lucia-auth
This library enables authentication built entirely on top of Convex without any third-party platform. It uses Lucia for the authentication logic.. Latest version: 0.0.2, last published: 14 days ago. Start using @convex-dev/convex-lucia-auth in your project by running
npm i @convex-dev/convex-lucia-auth. There are no ...GitHub
convex-lucia-auth/DOCS.md at master · get-convex/convex-lucia-auth
Convex database adapter for Lucia Auth. Contribute to get-convex/convex-lucia-auth development by creating an account on GitHub.
Awesome! I'll take a look!
Very interesting, I'll also have a look (when I find time for this 😅 )