ampp Posts - Convex Community

ampp

•Created by ampp on 12/18/2024 in #support-community

Acceptable circular reference counts?

Currently within convex i have 20 circular references with 14 that appear with a _generated/api.d.ts (34 total).. Do the references related to api.d.ts have the potential to cause problems? I still have some refactor ptsd after having to redesign a lot to get it down from over 60 in october. Its not easy to tell where the tipping point is until stuff breaks and it can get very weird. Its also been keeping me from redesigning the final execution part of our event system for easier use. The biggest issue right now is composing events, as a event can spawn sub-events within the same mutation (within any ts function atm) and one simple hack is to use the new sub-mutations via ctx.runMutation which will break the chain. I'm just really unclear about the performance downsides, or other issues. Especially since i haven't hardly seen them mentioned, it feels like a cheap hack..

2 replies

CCConvex Community

•Created by ampp on 11/24/2024 in #support-community

dashboard log performance

9 replies

CCConvex Community

•Created by ampp on 11/14/2024 in #support-community

Looping over tables with normalizeId?

I've looked all over at posted examples of normalizeId and all related posts and i can't figure out if there is any other way to get a proper runtime tableName that isn't saved in the db. Is there any real downside to this, no performance hit right? When looping through say 10 tables for the one that might have the id. I'd like to be clear what all the options are. I know ents doesn't have ctx.db.get(id) but i havent seen that done either. Working code:

export const testString = sessionQuery({
  args: { id: v.id('dataPool') },
  handler: async (ctx, args) => {
    const dataPool = await ctx.table('dataPool').get(args.id)

    const test = await queryIdString(ctx, dataPool.idUnionFieldOrString as string)

    return test
  },
})

async function queryIdString(ctx: SessionQueryCtx, idString: string) {
  const tableNames = ['users', 'members', 'events']

  const tableResults = await Promise.all(
    tableNames.map(async (tableName) => {
      const test = ctx.table(tableName as any).normalizeId(idString)
      return test ? { tableName, test } : null
    })
  )
  const result = tableResults.filter((result) => result !== null)

  if (result.length === 0) {
    return null
  }

  return ctx.table(result[0].tableName as any).get(result[0].test)
}

export const testString = sessionQuery({
  args: { id: v.id('dataPool') },
  handler: async (ctx, args) => {
    const dataPool = await ctx.table('dataPool').get(args.id)

    const test = await queryIdString(ctx, dataPool.idUnionFieldOrString as string)

    return test
  },
})

async function queryIdString(ctx: SessionQueryCtx, idString: string) {
  const tableNames = ['users', 'members', 'events']

  const tableResults = await Promise.all(
    tableNames.map(async (tableName) => {
      const test = ctx.table(tableName as any).normalizeId(idString)
      return test ? { tableName, test } : null
    })
  )
  const result = tableResults.filter((result) => result !== null)

  if (result.length === 0) {
    return null
  }

  return ctx.table(result[0].tableName as any).get(result[0].test)
}

14 replies

CCConvex Community

•Created by ampp on 11/14/2024 in #support-community

Multiple monorepo app deployments using the same convex backend?

What is the general consensus about this in production? If its doable then what are the possible downsides? Assuming we have several apps deployed all using the same backend, i'm worried if one deploy fails while the other builds and updates the database the version mismatch. We do plan to track code versions and db versions eventually anyway which could have safeguards.

9 replies

CCConvex Community

•Created by ampp on 8/26/2024 in #support-community

rate limit error catching in convex helpers

I'm not sure the usefulness of this function

ts 
export function isRateLimitError(
  error: unknown,
): error is { data: RateLimitError } {
  return (
    error instanceof ConvexError &&
    "kind" in error.data &&
    error.data.kind === "RateLimited"
  );
}

ts 
export function isRateLimitError(
  error: unknown,
): error is { data: RateLimitError } {
  return (
    error instanceof ConvexError &&
    "kind" in error.data &&
    error.data.kind === "RateLimited"
  );
}

.catch((err) => { if (isRateLimitError(err)) { as it causes a typescript error "TypeError: cannot use 'in' operator to search for "kind" in " if non-rate limit error gets triggered.

1 replies

CCConvex Community

•Created by ampp on 7/27/2024 in #support-community

Advanced index usage

I don't understand why I can't do any of this with a index. I didn't see in the docs a explicit explanation that you couldn't do something like this using the available .eq,gt,gte,lt,lte so i tried a few things.

assume ['by_index1_index2_index3']
(q) => q
  .gt('index1', null
  .eq('index2', index2)
  .gt('index3', '2')
  .lt('index3', '4')

(q) => q // just ignore index1 enitrely
  .eq('index2', index2)
  .gt('index3', '2')
  .lt('index3', '4')

(q) => q 
  .eq('index1', index1)
// just ignore index2 but use to sort. 
  .gt('index3', '2')
  .lt('index3', '4')

(q) => q 
  .eq('index1', index1)
  .gt('index2, '1'
  .gt('index3', '2')
  .lt('index3', '4')

(q) => q 
  .eq('index1', arrayOfOptions)
  .eq('index2, '1'
  .gt('index3', '2')
  .lt('index3', '4')

assume ['by_index1_index2_index3']
(q) => q
  .gt('index1', null
  .eq('index2', index2)
  .gt('index3', '2')
  .lt('index3', '4')

(q) => q // just ignore index1 enitrely
  .eq('index2', index2)
  .gt('index3', '2')
  .lt('index3', '4')

(q) => q 
  .eq('index1', index1)
// just ignore index2 but use to sort. 
  .gt('index3', '2')
  .lt('index3', '4')

(q) => q 
  .eq('index1', index1)
  .gt('index2, '1'
  .gt('index3', '2')
  .lt('index3', '4')

(q) => q 
  .eq('index1', arrayOfOptions)
  .eq('index2, '1'
  .gt('index3', '2')
  .lt('index3', '4')

neq would solve some of this.. but how can just getting all of a index break things too. Is any of it going to be supported or just via that eventual sql style "adapter" In one real world example i have a events table with ['lastStatus', 'userid', 'eventType'] where i also have have a index [ 'userid', 'eventType'] when i want all statuses from that user. A gte null on lastStatus index would keep me from duplicating effort. Probably the most annoying one is to get the last 10 results since a certain time while getting a range of eventTypes for a user ['recordedAt', 'userid', 'eventType'] I fail to see how anyone could ever use more than a few indexes if all but the last one requires .eq usage. I certainly cant imagine complex use cases, I'd love to see some real world work arounds. Or how others here are creating separate tables etc to get more specific.

6 replies

CCConvex Community

•Created by ampp on 7/17/2024 in #support-community

Scheduled function data is no longer compatible makeFunctionReference

I think this happened with 1.13 but haven't gone to any effort to dig into it as its not exactly a big deal. I'm not sure how i could have caused this either, i had to go fix a previously working bit of code i wrote a couple weeks ago. const query = scheduledFunction.name i had to change to const query = scheduledFunction.name.replace('.js', '') otherwise i get:

Error: [CONVEX A(engine/events:retryScheduledEventHandlerFunctionReference)] [Request ID: 9ae1b87f138cd112] Server Error
Uncaught Error: Couldn't resolve api.engine.events.js.batchRegisteredMutations

Error: [CONVEX A(engine/events:retryScheduledEventHandlerFunctionReference)] [Request ID: 9ae1b87f138cd112] Server Error
Uncaught Error: Couldn't resolve api.engine.events.js.batchRegisteredMutations

I think the function should accept whatever is saved to the db. But I'd almost argue that the name output from _scheduled_functions shouldn't have a extension as its possibly incoherent because of ts/js. I haven't quite started storing these things in the db myself.

3 replies

CCConvex Community

•Created by ampp on 7/15/2024 in #support-community

Pagination, Take on a edge [ents]

I've been browsing and trying to figure out how to handle this for a few days. Basically i have a many to many between wallets and transactions. The only efficient way to get the entire transaction history is to do something like .table('wallets') .getX(wallet._id) .edge('transactions') But i cant do a pagination or a take or anything on that on a edge... so i assume anything i do is going to be a full read of all results. And can be thousands of transactions. The problem is that transactions has a array of to's and a array of from's with a walletid/amount pairing. Which would be bad to index and we want to keep transactions as small as possible. If i'm not missing something painfully simple, then i'm thinking directly accessing the linking (wallet_to_transaction) table would be smarter. Perhaps then i could do manual pagination or something, or at least get arrays of ids to use? Thanks

83 replies

CCConvex Community

•Created by ampp on 7/11/2024 in #support-community

Stopping infinite pagination

What is the best way to keep a paginated query from expanding in length forever as new results come in? I have a couple ideas but i couldn't find this talked about anywhere.

29 replies

CCConvex Community

•Created by ampp on 7/3/2024 in #support-community

mutations and try catch errors.

I've been wanting to catch errors from my event system that can run sets of mutations. From what i understood is that mutations wrapped with a try/catch no longer have any benefits of mutations. Basically it will execute/write everything up till the error right? I tested this. in

 try {
await saveEventMutation(
} catch (err)
{
await saveEventMutationError(
}

 try {
await saveEventMutation(
} catch (err)
{
await saveEventMutationError(
}

both get ran. I only want to log a error of the event. I'm guessing the only way to do this without both functions saving data is to do it via a try catch within a action? Then that forces me to schedule actions via mutations in many cases.

3 replies

CCConvex Community

•Created by ampp on 6/26/2024 in #support-community

makeFunctionReference function type

I want to get the type when i run this, i know ctx.scheduler doesn't care about type but i want to run the function in the action through the right function ctx.runQuery or runMutation/action. without having to know the function type. Wondering if there is a trick with typescript?

7 replies

CCConvex Community

•Created by ampp on 6/14/2024 in #support-community

Session Wrapper with sessionId w/Clerk

I think this is another situation where i have a simple need but most examples are more complex with local storage etc . Now that i've progressed past just using the getIdentity/viewer in most use cases, i need the sessionId in a ton of UI calls to convex. I just want to have a useSessionQuery that passes in the session id from the clerk useAuth(). The whole app is already wrapped in a ConvexProviderWithClerk and that is my session provider basically? Maybe id want to modify the ConvexProviderWithClerk instead of importing <SessionProvider> in this case? I was looking at convex-helpers but that seems to be primarily for local session storage. The stack articles don't reference if you are already using a auth provider. I would also like to use the queryWithSession wrapper side of things on the convex side but I'm guessing i have to do some customization compared to the docs as I'm forcing ent ergonomics on all functions. Thanks

15 replies

CCConvex Community

•Created by ampp on 6/13/2024 in #support-community

Deleting troubles. ents etc.

It seems like i'm in a type of dependency hell with this stuff, every time i have to track down a bug in our init system. I really want that delete all tables in the UI, there has been so much wasted time trying all this other stuff. We wrote an db wipe script to remember the empty state by exporting using npx convex export. But it is very slow some times because importing blank data takes forever for no great reason IMHO. But if we change the db layout it becomes out of date and misses the new tables with data. So you have to keep track of that. Then we accidently had a export with new tables with data that didn't match the new schema. So then we made a script that loops over the schema to delete everything but it follows the scheduled deletes so a lot of stuff is not actually deleted. With ents how do you delete something ignoring soft deletion/scheduled rules? Also, I feel like there must be a bug with ents on anything but scheduled deletes, the deletes do get scheduled. But nothing else ever gets completed immediately if it is a root table in a soft delete "tree". Im trying to defer debugging that as i dont care but its making all the above harder. Maybe I'm reading the documentation wrong, but i only see the deletionTime and it never gets deleted. Do i need to execute some other command?

20 replies

CCConvex Community

•Created by ampp on 6/10/2024 in #support-community

Uncaught Error: System tables can only be accessed from db.system.normalizeId(). with Ent

I have listScheduledFunctions with table _scheduled_functions that works great, but getScheduledMessage gives that error.

export const listScheduledFunctions = query({
    args: { 
      paginationOpts: paginationOptsValidator,
    },
    handler: async (ctx, args) => 
    {   
      await hasAdminRole(ctx);

      return await ctx.table.system("_scheduled_functions").order("desc").paginate(args.paginationOpts);
    },
  });
  
export const getScheduledMessage = query({
args: {
    id: v.id("_scheduled_functions"),
},
handler: async (ctx, args) => {
    return await ctx.table.system("_scheduled_functions").get(args.id);
},
});

export const listScheduledFunctions = query({
    args: { 
      paginationOpts: paginationOptsValidator,
    },
    handler: async (ctx, args) => 
    {   
      await hasAdminRole(ctx);

      return await ctx.table.system("_scheduled_functions").order("desc").paginate(args.paginationOpts);
    },
  });
  
export const getScheduledMessage = query({
args: {
    id: v.id("_scheduled_functions"),
},
handler: async (ctx, args) => {
    return await ctx.table.system("_scheduled_functions").get(args.id);
},
});

I tried to change .get to .normalizeId but no luck with that. The same error. Any ideas. I did check to maek sure my query is being called from functions.ts etc..

3 replies

CCConvex Community

•Created by ampp on 6/7/2024 in #support-community

Server side event workers

I've been through the ai-town, llama farm, and the work stealing stack and still unclear on all available options. Since we building more of a turn-based style slower game with lots of roles, permissions, acl it fits well in-between any two examples. In llama farm the /worker/client.ts is being run by the client via the console. Our goal is to generate events into a events table most of these process fairly quick, analytics and such but will get more compute heavy as time goes on, some events we want to run immediately . Its not something where we need a remote worker. Our worker needs to process the events and do often many mutations etc. The base question is how would you best run this worker on the hosting server? Examples like AI-town seem to push us towards just doing as much as possible within convex. But will we regret putting too much compute on convex from a cost standpoint eventually. Our db interactions are just going to be unavoidably heavy. I don't exactly the idea of a 1 second tick type thing as with ai-town. Is it possible to have server side(webhost) event listeners that respond immediately to pushed update from convex? Are there examples? Seems like we might end up with a bit of everything.

34 replies

CCConvex Community

•Created by ampp on 5/23/2024 in #support-community

Hard to trace error db.normalizeId

Error: [Request ID: 50fea7d41796b0d6] Server Error
Uncaught Error: Uncaught Error: Invalid arguments for `db.normalizeId`: invalid type: null, expected a string
    at Promise.retrieve (../../../../node_modules/.pnpm/convex-ents@0.7.6_convex@1.12.0/node_modules/convex-ents/src/functions.ts:722:24)
    at doc [as doc] (../../../../node_modules/.pnpm/convex-ents@0.7.6_convex@1.12.0/node_modules/convex-ents/src/functions.ts:1263:21)
    at then [as then] (../../../../node_modules/.pnpm/convex-ents@0.7.6_convex@1.12.0/node_modules/convex-ents/src/functions.ts:1312:8)

    at async handler (../../convex/clerkActions.ts:114:56)

Error: [Request ID: 50fea7d41796b0d6] Server Error
Uncaught Error: Uncaught Error: Invalid arguments for `db.normalizeId`: invalid type: null, expected a string
    at Promise.retrieve (../../../../node_modules/.pnpm/convex-ents@0.7.6_convex@1.12.0/node_modules/convex-ents/src/functions.ts:722:24)
    at doc [as doc] (../../../../node_modules/.pnpm/convex-ents@0.7.6_convex@1.12.0/node_modules/convex-ents/src/functions.ts:1263:21)
    at then [as then] (../../../../node_modules/.pnpm/convex-ents@0.7.6_convex@1.12.0/node_modules/convex-ents/src/functions.ts:1312:8)

    at async handler (../../convex/clerkActions.ts:114:56)

Not really a bug report but i was running a action that called a ctx.runMutation that then called a mutation function which by mistake had a ctx.viewerId that passed into another async function that probably failed at the getX for that user. This orginally worked via the web so it didn't get caught because they were authed. Using ents. I tried to run the function from console and a init script and got errors every time. But if i checked the act as user in console it still errored. What made it weird was line 114 was a empty line before the bad ctx.runMutation And every time i narrowed it down it was a empty line before the function running. And running the function directly that ctx.runMutation called gave the same error. Maybe this can save someone else the headache.

2 replies

CCConvex Community

•Created by ampp on 5/20/2024 in #support-community

Importing environment variables into tests.

This might not be a convex issue, but i sure couldn't get it working after a lot of searching and gpt's. I want to run tests on convex actions that call the clerk backend sdk. And i get a missing key error. Wondering if anyone else has managed to get this working.

3 replies

CCConvex Community

•Created by ampp on 5/20/2024 in #support-community

Convex tests w/ ents with action identity issue.

Basically the recent updates seemed to mess with our tests. And seeing the docs were also expanded upon, and offering a bit more control so we refactored a bit, merging the ideas from the tests page on the main page and what we had working before with ents and auth. We changed some code to use actions for the first time with a test, so the test was now breaking, and I verified the action that has a identity created - so asSarah.action(): that function then calls a ctx.runMutation() where ctx.viewerId is null within that. However the actions ctx.auth.getIdentity() does return a value.
Just ctx.viewerId is empty in the child mutations, when its not in normal convex use. I understand that ctx on actions is not designed to get "middleware". Wondering if this is likely a setup issue or a bug.? Wishlist: It's happened a couple of times but when you accidently call a t.mutation and its a action on the other side it would be nice if the error was more specific. I know I should work naming functions differently if they are been changed to action 🙃 . Its especially easy to mix up as most actions have ctx.runMutation inside..

5 replies

CCConvex Community

•Created by ampp on 5/15/2024 in #support-community

convex environment variable is not listed as a dependency in turbo.json

At some point we triggered this turbo warning with our monorepo https://stackoverflow.com/questions/74547130/error-variable-is-not-listed-as-a-dependency-in-turbo-json just curious if there is some way to handle this besides adding everything to globalEnv. I couldn't see any significant differences in our setup compared to the mono-repo template.

2 replies

CCConvex Community

•Created by ampp on 5/6/2024 in #support-community

Type instantiation is excessively deep and possibly infinite.

The last day or two ive had to disable typescript checking or tsignore as i have been getting this within the convex folder for the first time. Needing typecheck=disable to get around it. I had complained about it appearing in the app directory before as a annoyance. I recently added a lot of actions and some scheduled actions or mutations. I managed to get the same errors in the fullstack template by moving folders into the similar structure. I was seeing this was not suggested.

convex
├── projects
│   └── activities.ts
└── projects.ts

convex
├── projects
│   └── activities.ts
└── projects.ts

https://discord.com/channels/1019350475847499849/1187401985301950574 But that's old and i followed the design of the convex ents example. Im doing things 2 deep.

convex
├── projects
│   └──- tasks
│   └ tasks.ts
└── projects.ts

convex
├── projects
│   └──- tasks
│   └ tasks.ts
└── projects.ts

When I was playing around with the fullstack-convex template in its own environment it was triggered when i moved tasks.ts and /tasks/ 1 level deeper Its possible the saas-starter template does not have enough api or internal calls to triggers this. It seems to require there to be more then 1 on a page or multiple files with more than one call. I also just noticed that --typecheck=disable is set on saas-starter so maybe that would avoid ever getting the error combined with the lack of api or internal calls. Tonight i really don't really like typecheck=disable as i made schema changes to indexs and somehow got no warnings of issues till my business partner ran the same code without that flag and several functions were broken. This fairly reproducible i'm curious if i should just change my file structure or try to spend time creating another fork example of it in a broken state if we want to go down the "this needs to get fixed" path. I'm also curious if anyone else is naming their stuff like this in a big project. Thanks

34 replies