Serializing/deserializing `Request`/`Response` objects in HTTP action
I'm trying to integrate Effect's HTTP API library (it's like Hono, basically) with my Convex HTTP API, but (until someone adds a Convex platform for
@effect/platform 🙂) doing so requires me to run the HTTP endpoint handlers in a Node runtime. I'm at the point where I have a function (request: Request) => Response, but if I can't run the handler directly in the HTTP action, maybe I could serialize/deserialize the Request/Response objects? I'm curious whether this has come up before, and (either way) whether anyone has any knowledge of whether this is possible/feasible, either in the general case or in certain narrower ones.9 Replies
I'm still just trying to decipher effect having its own http library
I've concluded that this is a bad idea and have abandoned the effort 😄
But will say that it would be really nice to be able to run HTTP actions in the Node runtime
Heh yeah we've internally had to do serialization of HTTP requests + responses (stuff like https://github.com/get-convex/convex-backend/blob/e0e06186fd54ccf1b51a9ec95e38eddac7c6d4f8/npm-packages/udf-runtime/src/23_request.ts#L334) since a lot of our system is built to deal with Convex Values (we also borrowed from Deno on some of this). In the initial version we did stuff like serializing the request / response body to a base64 encoded string, which works for small requests / responses, but falls apart once you want larger requests / responses + streaming.
I think I agree with you assessment that this is probably harder than it's worth.
I think the main thing we'd need to think through for HTTP actions in Node would be how to express the routing, since our pattern of a single
http.ts that imports all the handlers wouldn't quite work since it would be importing things from both runtimes.GitHub
convex-backend/npm-packages/udf-runtime/src/23_request.ts at e0e061...
Open source single-machine version of the Convex backend - get-convex/convex-backend
That makes sense! Good to know I'm not completely crazy for considering it 😄
And also makes sense re: the challenge of supporting both with the current APIs. I guess something like just exporting the route spec and then hoovering those up (like how actions/queries/mutations work) would technically work but less well given that you couldn't put all the routing details (HTTP method and path) in the file name
Anyways I looked at Effect's platform module a bit more and I think it wouldn't be too hard to write a platform for Convex, so maybe I'll attempt that
Would be pretty neat to have
I made some progress on this and am now getting this error when I try to hit an endpoint using the Effect HTTP lib-generated handler:
This is the problem: https://developer.mozilla.org/en-US/docs/Web/API/URL/username
I'm a bit surprised this isn't an issue with Hono. Any interest in supporting this? Are the rest of the URL APIs supported?
Definitely interested in supporting! There's a lot of surface area so we made guesses about what we needed to implement, sometimes we're wrong! This probably isn't hard to implement.
Yeah of course, no worries, just asking! 🙂
That would be great
I have an active need to set something like this up, so if you have a sense of when y'all might be able to get to it or look into it that would be useful to know
I'll just Hono (for now) otherwise
Which would certainly not be the end of the world
@v was playing around with some patches to try to get this to work, and thanks to that work we can confirm that the following (and possibly only the following) need an implementation for the Effect HTTP lib to function:
-
URL.username
- URL.password
- Request.signalAre there any plans to implement these and just out of curiosity would PRS be accepted surrounding this?
We know this stuff is important so yes before too long, and yes PRs could be accepted! It's finicky stuff so it could take some back and forth. Check out https://github.com/get-convex/convex-backend/blob/main/npm-packages/udf-runtime/src/00_url.ts and https://github.com/get-convex/convex-backend/blob/main/npm-packages/udf-tests/convex/js_builtins/url.ts
We don't have a much of bandwidth to help with these contributions, see the caveats in https://github.com/get-convex/convex-backend/blob/main/CONTRIBUTING.md
here's this line! https://github.com/get-convex/convex-backend/blob/main/npm-packages/udf-runtime/src/00_url.ts#L322