devagr•2mo ago

Confect: Branded types don't work

cc @RJ I am using effect's branded types for certain ids like this

const CompanyId = Schema.String.pipe(Schema.brand("CompanyId"));

const CompanyId = Schema.String.pipe(Schema.brand("CompanyId"));

But these get converted back into a regular string in the type definitions when returned from a query or mutation. I want to reuse the type definitions I get out of the schemas, but the type definitions have branded types, and the confect functions don't

30 Replies

RJ•2mo ago

Hey @devagr—this is probably a current limitation (or what I would consider a limitation) of Confect that you're encountering, in that the values returned from invoking Convex functions built with Confect are the Encoded type of the returns validator schemas. If you want to convert them to the Type type, you need to manually Schema.decode. The way I recommend doing this right now is creating a separate file containing the args and returns schemas for your functions, and importing that both in your frontend and your backend, where you define your functions. You can see an example of how you might do this in the example/ directory on GitHub (example/convex/functions.schemas.ts) and (example/convex/functions.ts). There are actually some very simple React helpers in the library for helping with this as well: example/src/App.tsx.

devagrOP•2mo ago

i can't use the react helpers because i'm using tanstack query

RJ•2mo ago

I consider this the main limitation of Confect at the moment, and it's something I'm working to change in v1.0. It's actually a fair amount of work to set this up to work seamlessly, because I have to integrate with how Convex does its type generation/mapping of function paths with the file-system etc. I haven't looked into Tanstack Query yet because I'm waiting for Tanstack Start to become stable, but I do plan on supporting Confect for this framework as well. I'm happy to take a quick look and see if I can come up with a recommendation for how to do this using Tanstack Query, but basically you just need to Schema.encode/Schema.decode using your args/returns validators where you provide/receive values to/from your Convex functions.

devagrOP•2mo ago

okay so all i need is to import the same schemas on the client side and decode the result coming out of convex

RJ•2mo ago

Yes, exactly! And if you want to provide arguments to a Confect function (and you want to define those args in terms of their schema's Type), you need to encode those before passing them in. I was just looking at Tanstack Query a bit, it should be pretty easy to write a similar kind of helper as are already defined for React. I can look into adding it sometime soon; or if you come up with a solution and want to share it here or contribute it directly, let me know! Also: It's probably worth noting that some folks prefer to not use Effect types on the frontend (I am not one of those people), so for some the existing behavior is preferred.

devagrOP•2mo ago

yeah here's convex's helpers for tanstack query https://github.com/get-convex/convex-react-query/blob/main/src/index.ts

GitHub

convex-react-query/src/index.ts at main · get-convex/convex-react-...

Contribute to get-convex/convex-react-query development by creating an account on GitHub.

devagrOP•2mo ago

yeah i suspected another issue I have a schema of a record type like this

export const RetailerStateSchema = Schema.Struct({
  outlets: Schema.Record({ key: OutletId, value: RetailerOutletSchema }),
});
export const getRetailer = query({
  args: Schema.Struct({}),
  returns: Schema.Struct({
    state: RetailerStateSchema
  })
  ...

export const RetailerStateSchema = Schema.Struct({
  outlets: Schema.Record({ key: OutletId, value: RetailerOutletSchema }),
});
export const getRetailer = query({
  args: Schema.Struct({}),
  returns: Schema.Struct({
    state: RetailerStateSchema
  })
  ...

and i get this error when trying to deploy this

✖ Error fetching POST  https://upbeat-grasshopper-51.convex.cloud/api/push_config 400 Bad Request: InvalidModules: Hit an error while pushing:
Loading the pushed modules encountered the following
    error:
Failed to analyze domains/retailer.js: Uncaught IndexSignaturesAreNotSupportedError: Index signatures are not supported
    at onSome (../../node_modules/.pnpm/@rjdellecese+confect@0.0.26_@effect+platform@0.87.1_effect@3.16.10__convex@1.25.1_@cler_869f671a691d2bb251ee7ecc88f3221e/node_modules/@rjdellecese/confect/src/server/schema-to-validator.ts:417:5)
    at <anonymous> (../../node_modules/.pnpm/effect@3.16.10/node_modules/effect/src/Option.ts:438:45)
    at <anonymous> (../../node_modules/.pnpm/effect@3.16.10/node_modules/effect/src/Function.ts:248:8)
    at pipe (../../node_modules/.pnpm/effect@3.16.10/node_modules/effect/src/Function.ts:1147:9)
    at handleTypeLiteral (../../node_modules/.pnpm/@rjdellecese+confect@0.0.26_@effect+platform@0.87.1_effect@3.16.10__convex@1.25.1_@cler_869f671a691d2bb251ee7ecc88f3221e/node_modules/@rjdellecese/confect/src/server/schema-to-validator.ts:411:19)
    at Object.evaluate (../../node_modules/.pnpm/@rjdellecese+confect@0.0.26_@effect+platform@0.87.1_effect@3.16.10__convex@1.25.1_@cler_869f671a691d2bb251ee7ecc88f3221e/node_modules/@rjdellecese/confect/src/server/schema-to-validator.ts:337:8)
    at add [as add] (../../node_modules/.pnpm/effect@3.16.10/node_modules/effect/src/internal/matcher.ts:77:56)
    at Arguments.<anonymous> (../../node_modules/.pnpm/effect@3.16.10/node_modules/effect/src/internal/matcher.ts:337:48)
    at pipe (../../node_modules/.pnpm/effect@3.16.10/node_modules/effect/src/Function.ts:1163:6)
    at compileAst

✖ Error fetching POST  https://upbeat-grasshopper-51.convex.cloud/api/push_config 400 Bad Request: InvalidModules: Hit an error while pushing:
Loading the pushed modules encountered the following
    error:
Failed to analyze domains/retailer.js: Uncaught IndexSignaturesAreNotSupportedError: Index signatures are not supported
    at onSome (../../node_modules/.pnpm/@rjdellecese+confect@0.0.26_@effect+platform@0.87.1_effect@3.16.10__convex@1.25.1_@cler_869f671a691d2bb251ee7ecc88f3221e/node_modules/@rjdellecese/confect/src/server/schema-to-validator.ts:417:5)
    at <anonymous> (../../node_modules/.pnpm/effect@3.16.10/node_modules/effect/src/Option.ts:438:45)
    at <anonymous> (../../node_modules/.pnpm/effect@3.16.10/node_modules/effect/src/Function.ts:248:8)
    at pipe (../../node_modules/.pnpm/effect@3.16.10/node_modules/effect/src/Function.ts:1147:9)
    at handleTypeLiteral (../../node_modules/.pnpm/@rjdellecese+confect@0.0.26_@effect+platform@0.87.1_effect@3.16.10__convex@1.25.1_@cler_869f671a691d2bb251ee7ecc88f3221e/node_modules/@rjdellecese/confect/src/server/schema-to-validator.ts:411:19)
    at Object.evaluate (../../node_modules/.pnpm/@rjdellecese+confect@0.0.26_@effect+platform@0.87.1_effect@3.16.10__convex@1.25.1_@cler_869f671a691d2bb251ee7ecc88f3221e/node_modules/@rjdellecese/confect/src/server/schema-to-validator.ts:337:8)
    at add [as add] (../../node_modules/.pnpm/effect@3.16.10/node_modules/effect/src/internal/matcher.ts:77:56)
    at Arguments.<anonymous> (../../node_modules/.pnpm/effect@3.16.10/node_modules/effect/src/internal/matcher.ts:337:48)
    at pipe (../../node_modules/.pnpm/effect@3.16.10/node_modules/effect/src/Function.ts:1163:6)
    at compileAst

it doesnt seem to like the record

RJ•2mo ago

You're right! The addition of v.record() (the Convex validator for TS records) is fairly recent, and I don't support it yet. It should be pretty easy to so I can add that to the docket, but if you don't want to wait I'd recommend just using

Schema.Array(
  Schema.Struct({
    outletId: OutletId,
    retailerOutlet: RetailerOutletSchema
  })
)

Schema.Array(
  Schema.Struct({
    outletId: OutletId,
    retailerOutlet: RetailerOutletSchema
  })
)

Schema.Array(
  Schema.Tuple(
    OutletId,
    RetailerStateSchema
  )
)

Schema.Array(
  Schema.Tuple(
    OutletId,
    RetailerStateSchema
  )
)

instead.

devagrOP•2mo ago

ah thats nice i changed it to arrays and it works for now ideally this shouldn't even be an array, it should be a table in the database

RJ•2mo ago

Agreed, but I get the convenience

devagrOP•2mo ago

thanks a lot for your help and for building confect

RJ•2mo ago

No problem, thanks for your feedback!

devagrOP•2mo ago

okay one more issue i ran into a while ago that i am not happy with the workaround for i have certain functions that want to access the database, but if i only access confectquerycontext, it won't work in mutations, and if i access confectmutationcontext it wont work in queries so right now i have to manually pass in the db, auth, and storage objects into these functions instead of those being inferred from context

RJ•2mo ago

I get this 100%, it kind of sucks right now. v1 will break the contexts down further, such that this isn't an issue. I've actually been working on this recently, and it's just about done (but I don't think it will be released until the v1 release). I think passing the requisite pieces individually is the best workaround in the meantime.

devagrOP•2mo ago

good to know thanks i was wondering if i can use effect's layers to layer in a generic service, and the layer itself would depend on the mutation and query contexts, but i don't have enough experience with effect's DI system to pull this off

RJ•2mo ago

Hm Yeah you could probably do that actually

devagrOP•2mo ago

i'd also need to wrap your query and mutation functions to provide this layer

RJ•2mo ago

I'm not a top expert on services/layers myself, but I'm imagining a ConfectDatabaseReader service that you create, for which you can construct layers from either a DatabaseReader or a DatabaseWriter. Or actually from a ConfectMutationCtx or ConfectQueryCtx If you used those you I don't think you'd need to wrap anything, but you would need to construct the layer and provide it to your helper Effects every time you need to use it

devagrOP•2mo ago

the layer itself would need to somehow depend on ConfectMutationCtx | ConfectQueryCtx instead of both or just one yeah but i have a lot of them and i'd rather just provide the service at the query and mutation layer rather than providing the service to every function individually

RJ•2mo ago

I was imagining that you have two different methods of constructing the layer, one which takes ConfectMutationCtx and one which takes ConfectQueryCtx

devagrOP•2mo ago

hmm that could work yeah two services that provide the same generic context but depend on different ones themselves

RJ•2mo ago

Yeah, but don't "depend" on in the sense of being a part of the service dependency graph "depend" on in the sense that you can construct the layer for it using regular functions which accept one or the other At least that's what I had in mind (and maybe that was already clear) Anyways I'm not exactly sure but if you play around with it and find a good workaround for now I'd love to see it And happy to share feedback/thoughts if I can be helpful

devagrOP•2mo ago

i'll let you know how it goes when i get around to it, right now it works by passing in those dependencies directly

RJ•2mo ago

Sounds good!

devagrOP•2mo ago

i've also noticed that since i added confect, it takes much longer to build and deploy the functions, making it slower to iterate have you noticed this or am i doing something wrong

RJ•2mo ago

Using Confect will definitely add overhead, but I haven't tested rigorously how much. I would guess that the type-checking is more heavyweight than the runtime building process—you could try running convex dev without type-checking and see if that helps things. I'm definitely interested in hearing about your experience with Confect's performance impact, as I'd like to minimize it as much as possible. You could also try Convex's static code generation feature: https://docs.convex.dev/production/project-configuration#using-static-code-generation-beta

devagrOP•2mo ago

i think ultimately ts-go should solve a lot of that

RJ•2mo ago

I'm hoping so too!

devagrOP•2mo ago

you could try running convex dev without type-checking

how?

RJ•2mo ago

convex dev --typecheck=disable I believe

Confect: Branded types don't work

Did you find this page helpful?