WeamonZ
WeamonZ4mo ago

Caching system failure.

I already mentioned some of these issues in this GitHub post, but I’ll go into more detail here. I've spent days trying to debug Convex's caching behavior, and at this point, it seems increasingly likely that the problems aren't coming from my end. There are multiple issues with the caching system, including: * Paginated query caching only working sporadically — sometimes it only works once every few days. * Auth sessions refreshing too frequently, which seems to unnecessarily invalidate queries. According to the documentation, refreshes should happen every hour, but as shown in the screenshot, one occurred after just 40 minutes. (It even refreshed the authRefreshTokens >2000 times in less than a month. Even tho the tokens are not expired). * Queries getting invalidated without any mutations. As you can see, nothing in the app has changed, yet almost every query is being re-run. * Table updates invalidate all queries referencing that table, even if the updated field isn't part of the query result. This makes it really hard to rely on the caching layer to reduce bandwidth or improve performance — especially in production scenarios.
GitHub
Issues · get-convex/convex-backend
The open-source reactive database for app developers - Issues · get-convex/convex-backend
No description
178 Replies
WeamonZ
WeamonZOP4mo ago
I just refreshed my browser, and ... an other refreshSession, with the caching working this time. It's absolutely not idempotent
No description
WeamonZ
WeamonZOP4mo ago
(And you can see that this time the getImagesPaginated wasn't cached somehow, even tho nothing had changed, I didn't even scroll the page)
WeamonZ
WeamonZOP4mo ago
--- Now I refreshed the page and the getCover functions are invalidated
No description
WeamonZ
WeamonZOP4mo ago
Literally no mutations were made between the screenshots.
jamwt
jamwt4mo ago
can you share the code in the query?
WeamonZ
WeamonZOP4mo ago
import { authQuery } from "../../procedures";
import { internal } from "../../_generated/api";
import { v } from "convex/values";
import { internalQuery } from "../../_generated/server";

type Result = null | {
url: string,
width: number,
height: number,
size: number,
isAlpha: boolean,
}

export const _internal = internalQuery({
args: { projectId: v.id('projects') },
handler: async (ctx, args): Promise<Result> => {
const project = await ctx.db.get(args.projectId);
if (!project) { throw new Error("Project not found"); }

const resultId = project.resultIds[project.resultIds.length - 1] ?? null;
const result = !resultId ? null :
await ctx.runQuery(internal.router.storage.getImage._internal, { id: resultId })

if (!result) { return null; }

return {
url: result.sizes.thumbnail.url,
width: result.width,
height: result.height,
size: result.size,
isAlpha: result.hasAlpha,
};
}
})

export const client = authQuery({
args: { projectId: v.id('projects') },
handler: async (ctx, args): Promise<Result> =>
ctx.runQuery(internal.router.projects.getCover._internal, args)
})
import { authQuery } from "../../procedures";
import { internal } from "../../_generated/api";
import { v } from "convex/values";
import { internalQuery } from "../../_generated/server";

type Result = null | {
url: string,
width: number,
height: number,
size: number,
isAlpha: boolean,
}

export const _internal = internalQuery({
args: { projectId: v.id('projects') },
handler: async (ctx, args): Promise<Result> => {
const project = await ctx.db.get(args.projectId);
if (!project) { throw new Error("Project not found"); }

const resultId = project.resultIds[project.resultIds.length - 1] ?? null;
const result = !resultId ? null :
await ctx.runQuery(internal.router.storage.getImage._internal, { id: resultId })

if (!result) { return null; }

return {
url: result.sizes.thumbnail.url,
width: result.width,
height: result.height,
size: result.size,
isAlpha: result.hasAlpha,
};
}
})

export const client = authQuery({
args: { projectId: v.id('projects') },
handler: async (ctx, args): Promise<Result> =>
ctx.runQuery(internal.router.projects.getCover._internal, args)
})
that's the getCover query and the getImages internal query
import { v } from "convex/values";
import { internalQuery } from "../../_generated/server";
import { Doc } from "../../_generated/dataModel";
import { internal } from "../../_generated/api";
import { SanitizedImage, sanitizeImage } from "../../lib/image/sanitizeImage";
import { publicQuery, serverQuery } from "../../procedures";

const args = {
id: v.id('images'),
}

type BasicImage = Omit<Doc<'images'>, 'sizes'> &
{ sizes: Doc<'imageSizes'>[], objects: Doc<'imageObjects'>[] }

type Image = SanitizedImage<BasicImage>

export type GetImageReturnType = Image | null;

export const _internal = internalQuery({
args, handler: async (ctx, { id }): Promise<Image | null> => {
const [image, sizes, objects] = await Promise.all([
ctx.db.get(id),
ctx.db.query("imageSizes").withIndex("by_image_id", q => q.eq("imageId", id)).collect(),
ctx.db.query("imageObjects").withIndex("by_image_id", q => q.eq("imageId", id)).collect(),
]);

if (!image) return null;

return sanitizeImage({
...image,
sizes,
objects,
});
}
})

export const client = publicQuery({
args, handler: (ctx, args): Promise<Image | null> =>
ctx.runQuery(internal.router.storage.getImage._internal, args)
})

export const server = serverQuery({
args, handler: (ctx, args): Promise<Image | null> =>
ctx.runQuery(internal.router.storage.getImage._internal, args)
})
import { v } from "convex/values";
import { internalQuery } from "../../_generated/server";
import { Doc } from "../../_generated/dataModel";
import { internal } from "../../_generated/api";
import { SanitizedImage, sanitizeImage } from "../../lib/image/sanitizeImage";
import { publicQuery, serverQuery } from "../../procedures";

const args = {
id: v.id('images'),
}

type BasicImage = Omit<Doc<'images'>, 'sizes'> &
{ sizes: Doc<'imageSizes'>[], objects: Doc<'imageObjects'>[] }

type Image = SanitizedImage<BasicImage>

export type GetImageReturnType = Image | null;

export const _internal = internalQuery({
args, handler: async (ctx, { id }): Promise<Image | null> => {
const [image, sizes, objects] = await Promise.all([
ctx.db.get(id),
ctx.db.query("imageSizes").withIndex("by_image_id", q => q.eq("imageId", id)).collect(),
ctx.db.query("imageObjects").withIndex("by_image_id", q => q.eq("imageId", id)).collect(),
]);

if (!image) return null;

return sanitizeImage({
...image,
sizes,
objects,
});
}
})

export const client = publicQuery({
args, handler: (ctx, args): Promise<Image | null> =>
ctx.runQuery(internal.router.storage.getImage._internal, args)
})

export const server = serverQuery({
args, handler: (ctx, args): Promise<Image | null> =>
ctx.runQuery(internal.router.storage.getImage._internal, args)
})
jamwt
jamwt4mo ago
gotcha. and these are all the same logged in user in the logs?
WeamonZ
WeamonZOP4mo ago
I'm in a dev environment, I only have my browser signed in
jamwt
jamwt4mo ago
got it. and none of these libraries rely on randomness?
WeamonZ
WeamonZOP4mo ago
type Context = Parameters<Parameters<typeof customCtx>[0]>[0] & {
db: DatabaseReader,
auth: Auth;
};
export const userCheck = async (ctx: Context) => {
// Look up the logged in user
const userId = await getAuthUserId(ctx);
if (!userId) throw new Error("Authentication required");
const user = await ctx.db.get(userId);
const userExtended = await ctx.db.query("usersExtended")
.withIndex('by_user_id', (q) => q.eq("userId", userId)).first();

if (!user || !userExtended) throw new Error("User not found");
// Pass in a user to use in evaluating rules,
// which validate data access at access / write time.
// const db = wrapDatabaseReader({ user, userExtended }, ctx.db, {});

const finalUser = {
...userExtended,
_idExtended: userExtended._id,
...user,
}

// This new ctx will be applied to the function's.
// The user is a new field, the db replaces ctx.db
return { user: finalUser, userId };
}

// The base function we're extending
// Here we're using a `customCtx` helper because our modification
// only modifies the `ctx` argument to the function.
export const authQuery = customQuery(query, customCtx(userCheck));
type Context = Parameters<Parameters<typeof customCtx>[0]>[0] & {
db: DatabaseReader,
auth: Auth;
};
export const userCheck = async (ctx: Context) => {
// Look up the logged in user
const userId = await getAuthUserId(ctx);
if (!userId) throw new Error("Authentication required");
const user = await ctx.db.get(userId);
const userExtended = await ctx.db.query("usersExtended")
.withIndex('by_user_id', (q) => q.eq("userId", userId)).first();

if (!user || !userExtended) throw new Error("User not found");
// Pass in a user to use in evaluating rules,
// which validate data access at access / write time.
// const db = wrapDatabaseReader({ user, userExtended }, ctx.db, {});

const finalUser = {
...userExtended,
_idExtended: userExtended._id,
...user,
}

// This new ctx will be applied to the function's.
// The user is a new field, the db replaces ctx.db
return { user: finalUser, userId };
}

// The base function we're extending
// Here we're using a `customCtx` helper because our modification
// only modifies the `ctx` argument to the function.
export const authQuery = customQuery(query, customCtx(userCheck));
My auth query (basically the tutorial one) I'm not using any external library My sanitizeImage function just sanitize the data
type PartialImage = {
url: string,
width: number,
height: number,
size: number,
}

type MainImage = PartialImage & { sizes: PartialImage[] }
type Image<T extends MainImage> = T;

export const sanitizeImage = <T extends MainImage>(image: Image<T>): SanitizedImage<T> => {
const sizes = image.sizes.sort((a, b) => a.size - b.size).map((s) => ({ ...s, dimension: Math.max(s.width, s.height) }));
const sanitizeSize = ({ url, width, height, size }: { url: string, width: number, height: number, size: number }) => ({ url, width, height, size });
return {
...image,
_sizes: image.sizes as typeof image['sizes'],
sizes: {
thumbnail: sanitizeSize(sizes.find((s) => s.dimension <= 200) ?? image),
small: sanitizeSize(sizes.find((s) => s.dimension <= 600 && s.dimension > 200) ?? image),
original: sanitizeSize(image),
}
}
}

export type SanitizedImage<T extends MainImage> =
Omit<T, 'sizes'> & {
_sizes: T['sizes'];
sizes: {
thumbnail: PartialImage;
small: PartialImage;
original: PartialImage;
};
};
type PartialImage = {
url: string,
width: number,
height: number,
size: number,
}

type MainImage = PartialImage & { sizes: PartialImage[] }
type Image<T extends MainImage> = T;

export const sanitizeImage = <T extends MainImage>(image: Image<T>): SanitizedImage<T> => {
const sizes = image.sizes.sort((a, b) => a.size - b.size).map((s) => ({ ...s, dimension: Math.max(s.width, s.height) }));
const sanitizeSize = ({ url, width, height, size }: { url: string, width: number, height: number, size: number }) => ({ url, width, height, size });
return {
...image,
_sizes: image.sizes as typeof image['sizes'],
sizes: {
thumbnail: sanitizeSize(sizes.find((s) => s.dimension <= 200) ?? image),
small: sanitizeSize(sizes.find((s) => s.dimension <= 600 && s.dimension > 200) ?? image),
original: sanitizeSize(image),
}
}
}

export type SanitizedImage<T extends MainImage> =
Omit<T, 'sizes'> & {
_sizes: T['sizes'];
sizes: {
thumbnail: PartialImage;
small: PartialImage;
original: PartialImage;
};
};
`
jamwt
jamwt4mo ago
in the log, I see a lot of queries all executing at the same time -- those are all the same projectId ?
WeamonZ
WeamonZOP4mo ago
They do not have the same projectId Let's say I have a bunch of projects with cover images. I removed the images from the getProjects query, and created a custom getCover query to retrieve the covers one by one, This prevent the getProjects query to reload ALL the projects when a cover changes
jamwt
jamwt4mo ago
final question -- no code changes in between as well, right? any code change invalidates all the caches
WeamonZ
WeamonZOP4mo ago
no code changes like mutations you mean ? oh, no, I didn't push any updates
WeamonZ
WeamonZOP4mo ago
No description
WeamonZ
WeamonZOP4mo ago
If you're referring to this: nop this doesn't happen between the screenshots
WeamonZ
WeamonZOP4mo ago
No description
WeamonZ
WeamonZOP4mo ago
Mhh, right click open link to see it better There is no mutation in between the cache invalidations the stripe get products doesn't mutate anything by the way
jamwt
jamwt4mo ago
okay I downloaded this. which particular timestamps do you want me to focus on? I see a cluster of requests at 1:34:29, a mutation at 1:45:00, then more queries at 1:45:00 and 1:45:01
WeamonZ
WeamonZOP4mo ago
I think the 1:34:29 seems interesting since it starts with a getCover that is not cached, but the other ones are
jamwt
jamwt4mo ago
how big are these results, btw? like # of KB. and is this the cloud product, or self-hosted image?
WeamonZ
WeamonZOP4mo ago
I'll check that, (Im using your cloud solution)
jamwt
jamwt4mo ago
also, free account or convex pro? just trying to rule out cache eviction
WeamonZ
WeamonZOP4mo ago
It's even more interesting that the getAccount is cached at 1:34:29 and then uncached at 1:34:29 later on free until pushed in production I saved logs that include : - getTasks - getResults - getProject - getImages All of this is around 84kb
jamwt
jamwt4mo ago
I see auth:store being called -- that probably invalidates the account, I would guess ah. so images are just paths or something. no image data. this should all fit in the cache no problem just making sure
WeamonZ
WeamonZOP4mo ago
Yeah,but if you look at 1:45:00 it's being called too and yet it DOESNT invalidate the getAccount query Yes i'm using my own S3 solution
WeamonZ
WeamonZOP4mo ago
No description
jamwt
jamwt4mo ago
the getAccount may already have been in flight. I'll ask one of our backend experts to take a look at this thread to help clarify. the timestamp may be when the request started, not when it ended.
WeamonZ
WeamonZOP4mo ago
correct, it seems to be when it started (I made a github post about it) https://github.com/get-convex/convex-backend/issues/88#issuecomment-2854644235
jamwt
jamwt4mo ago
not sure. let me stop speculating and get an engineer to look at this I see indy answered the post there. to elaborate on your follow up question invalidation is done purely on the basis of index ranges it doesn't care what fields you do or do not use within the document so yes, if you wanted to avoid invalidation, you'd have to break that record out into a separate table/record the system doesn't introspect the javascript runtime to determine what fields mattered. it's purely based on index ranges visited by database calls
jamwt
jamwt4mo ago
How Convex Works
A deep dive on how Convex's reactive database works internally.
jamwt
jamwt4mo ago
I'm guessing the log line stuff is just due to the timeline and the "sequential" nature of the loglines being a little misleading since many things are running concurrently, especially when tightly grouped but I'll ping someone to take a look at this
WeamonZ
WeamonZOP4mo ago
I'll look into that thank you very much ! As much as I have read there is no performance cost in spliting tables and merging them in a Promise.all(). Am I right ? https://github.com/get-convex/convex-backend/issues/95#issuecomment-2892478284 Like in this example I mean I dont promiseall in this example but you got me 🤣
jamwt
jamwt4mo ago
yeah, the speed will be around the same if you make the fetches concurrent, and the cache invalidation benefits will apply
WeamonZ
WeamonZOP4mo ago
---Oh, and one thing that might help: the auth:store is regularly pinging authRefreshTokens (in the health tab), causing a conflict. I don't think that's part of Convex Auth's normal behavior--- Irrelevant Thanks again
Wosca
Wosca4mo ago
Sorry to butt in, but I am having a very similar issue with the cache. This example is simply a mutation with a query updating its state due to the mutation, however me changing the score from 10 to 9 back to 10 doesn't cache the result of 10 at all. I have checked that the data being sent when the score is set to 10 are both the exact same, yet still no cache.
Wosca
Wosca4mo ago
Another extremely weird state which may suggest that cache is working but isn't being reported in the logs is that I can change these scores, quickly, and get some of the requests to presumably be out of sync, once the state is then changed (By changing another subject's score) it reverts to what seems like a cached version? However in the logs there is no sign of any cache from any of those queries.
WeamonZ
WeamonZOP4mo ago
Mhhhh It looks like you're mutating onValueChange. It looks like to me that it has some useEffect update behavior where the mutation is sent, the data is changed and an other mutation is sent Can you not use an input, but a button instead and click the button to trigger the mutation? And see if it produces the same problem Whenever you have multiple clients connected, the value will change and therefore every client might trigger the onValueChange which will trigger the mutation and breaks everything Use a defautValue instead of a value, and a key={currentValue} This will force the input to rerender when the value change but won't trigger the onValueChange (if the problem comes from there)
Wosca
Wosca4mo ago
Well yeah of course its not the most efficient way of doing things I really do need to fix it and thanks for the help with that. But when it is in the broken state, my point is that it is pulling that data from somewhere, it isn't my browser's cache as it still happens with cache disabled. And when looking at the websocket response from convex, it's showing a seemingly cached value, but no cache was hit?
No description
Wosca
Wosca4mo ago
I am so confused... All the debug button is doing is similar to what you said to do and just manually calling the mutation, in this case setting score 1 to 85, yet the score 1 from a completely different subject, is being reverted to a cached state? I didn't even need to spam requests to make it break
Wosca
Wosca4mo ago
Meanwhile from just this amount of debugging:
No description
WeamonZ
WeamonZOP4mo ago
OK I just wanted to check if it was react triggering unwanted mutations, but it indeed seems to come from convex caching layer If you refresh your browser the value that went from 17 to 16 goes back to 17? Or it stays at 16
Wosca
Wosca4mo ago
Yep..
Wosca
Wosca4mo ago
This is so bizare It's been an hour as well since I last tried but I guess the invisible cache is still there
WeamonZ
WeamonZOP4mo ago
Yep it stays at 16? Or it correctly retrieves the value from convex db
Wosca
Wosca4mo ago
Sorry, it flicks back to the incorrect value. This is a new recording after a refresh
WeamonZ
WeamonZOP4mo ago
So it doesn't retrieve the value from convex db?
Wosca
Wosca4mo ago
Looks like the initial load is cached but after changing it isn't cached
WeamonZ
WeamonZOP4mo ago
To be honest your logs seem all good, you're mutating so your queries change therefore they won't be cached, that's normal behavior to me
Wosca
Wosca4mo ago
Well it is supposedly, in the logs it isn't indicating a cache and I looked at the websocket messages and it's returning it as if it's the "correct" value
WeamonZ
WeamonZOP4mo ago
Wait, what is your mutation code Are you reverting changes in your mutation? Maybe your mutation triggers an other row in the table?
Wosca
Wosca4mo ago
export const updateSubjectData = mutation({
args: {
_id: v.id("saved_atar_data"),
subjectIndex: v.number(),
field: v.string(),
value: v.union(v.string(), v.number()),
subjects: v.array(
v.object({
currentRawPercentage: v.float64(),
currentScaledScore: v.float64(),
goal1: v.float64(),
goal2: v.float64(),
goal3: v.float64(),
goal4: v.float64(),
goalRawPercentage: v.optional(v.float64()),
goalScaledScore: v.float64(),
maxRawPercentage: v.optional(v.float64()),
maxScaledScore: v.float64(),
name: v.string(),
scaledScore: v.union(v.null(), v.float64()),
score: v.optional(v.float64()),
score1: v.float64(),
score2: v.float64(),
score3: v.float64(),
score4: v.float64(),
totalScore: v.float64(),
})
),
currentAtar: v.string(),
goalAtar: v.string(),
maxAtar: v.string(),
},
handler: async (ctx, args) => {
const { _id, subjects, currentAtar, goalAtar, maxAtar } = args;

// Verify user is authorized
const user = await ctx.auth.getUserIdentity();
if (!user) {
throw new Error("Unauthorized");
}

// Get the existing saved data
const savedData = await ctx.db.get(_id);
if (!savedData) {
throw new Error("Saved data not found");
}

// Update the data
return await ctx.db.patch(_id, {
subjects,
current_atar: currentAtar,
goal_atar: goalAtar,
max_atar: maxAtar,
// updated_at: new Date().toISOString(),
});
},
});
export const updateSubjectData = mutation({
args: {
_id: v.id("saved_atar_data"),
subjectIndex: v.number(),
field: v.string(),
value: v.union(v.string(), v.number()),
subjects: v.array(
v.object({
currentRawPercentage: v.float64(),
currentScaledScore: v.float64(),
goal1: v.float64(),
goal2: v.float64(),
goal3: v.float64(),
goal4: v.float64(),
goalRawPercentage: v.optional(v.float64()),
goalScaledScore: v.float64(),
maxRawPercentage: v.optional(v.float64()),
maxScaledScore: v.float64(),
name: v.string(),
scaledScore: v.union(v.null(), v.float64()),
score: v.optional(v.float64()),
score1: v.float64(),
score2: v.float64(),
score3: v.float64(),
score4: v.float64(),
totalScore: v.float64(),
})
),
currentAtar: v.string(),
goalAtar: v.string(),
maxAtar: v.string(),
},
handler: async (ctx, args) => {
const { _id, subjects, currentAtar, goalAtar, maxAtar } = args;

// Verify user is authorized
const user = await ctx.auth.getUserIdentity();
if (!user) {
throw new Error("Unauthorized");
}

// Get the existing saved data
const savedData = await ctx.db.get(_id);
if (!savedData) {
throw new Error("Saved data not found");
}

// Update the data
return await ctx.db.patch(_id, {
subjects,
current_atar: currentAtar,
goal_atar: goalAtar,
max_atar: maxAtar,
// updated_at: new Date().toISOString(),
});
},
});
I removed all other rows in the table Could it be the JSON just disallowing the cache? I used to have an updated_at which I removed because I thought it was the cause of the no cache
WeamonZ
WeamonZOP4mo ago
What are you doing with the subjects field? I don't think there is such thing as disallowing the cache. When you change a row in convex and query that said row, every queries using that row will automatically update and send the response (as uncached) to any client alive. The following requests (such as when you refresh the browser, and until you mutate the row again, will be cached)
Wosca
Wosca4mo ago
Oh interesting So would what I'm experiencing be normal then?
WeamonZ
WeamonZOP4mo ago
Well what I see is that when you mutate a field, an other field that isn't supposed to change changes?
Wosca
Wosca4mo ago
Yes, that's a bug I found along the way when trying to find out why my bandwidth usage was so high.
WeamonZ
WeamonZOP4mo ago
I think you're just not sending the right values in your array, log the args in your mutation And as for your bandwidth usage, you HAVE to split your queries to the LOWEST level possible What I mean by that, ONLY QUERY the bare minimum you need And as for your tables SPLIT THEM as much as you can So that if you update a row, only queries that need this row will rerender Example: We might want to do something like this : Project(Id, name, progress) BUT since the progress often changes we should do this instead Project(id, name) ProjectProgress(projectId, progress) So only queries that need the progress will update when the progress changes
import { defineSchema, defineTable } from 'convex/server';

export default defineSchema({
projects: defineTable({
name: 'string',
}),

projectProgress: defineTable({
projectId: 'string', // Foreign key to `projects._id`
progress: 'number', // e.g. 0 to 100
}).index('by_projectId', ['projectId']),
});
import { defineSchema, defineTable } from 'convex/server';

export default defineSchema({
projects: defineTable({
name: 'string',
}),

projectProgress: defineTable({
projectId: 'string', // Foreign key to `projects._id`
progress: 'number', // e.g. 0 to 100
}).index('by_projectId', ['projectId']),
});
Wosca
Wosca4mo ago
Yeah I should've made more of an effort migrating from my other database, I could just query it all at once and be using 0.1% of the bandwidth per month Very true
WeamonZ
WeamonZOP4mo ago
// convex/queries/getProjectAndProgress.ts
import { query } from 'convex/server';
import { v } from 'convex/values';

export const getProjectAndProgress = query({
args: { projectId: v.id('projects') },
handler: async (ctx, { projectId }) => {
const [project, progress] = await Promise.all([
ctx.db.get(projectId),
ctx.db
.query('projectProgress')
.withIndex('by_projectId', (q) => q.eq('projectId', projectId))
.order('desc')
.first(),
]);

return {
project,
progress,
};
});
// convex/queries/getProjectAndProgress.ts
import { query } from 'convex/server';
import { v } from 'convex/values';

export const getProjectAndProgress = query({
args: { projectId: v.id('projects') },
handler: async (ctx, { projectId }) => {
const [project, progress] = await Promise.all([
ctx.db.get(projectId),
ctx.db
.query('projectProgress')
.withIndex('by_projectId', (q) => q.eq('projectId', projectId))
.order('desc')
.first(),
]);

return {
project,
progress,
};
});
Wosca
Wosca4mo ago
I just had a look at what's being sent through the websocket link and you're right, it's sending through the old data, I'll try and find out why now.
WeamonZ
WeamonZOP4mo ago
Yeah I was having the same issue, migrating from prisma. I'm rewriting it all because I'm having the same issue as you have. The doc isnt clear on things Only update the fields you need. And query the db to fill the other fields. You can also just not send the field key and it won't change the value This will also reduce the bandwidth, cause if the user click on a button it doesn't send ALL the page inputs
Wosca
Wosca4mo ago
Yeah, just crazy that the tiniest amount of data is causing the bandwidth to skyrocket. I'll probably add debouncing to it as well to limit requests.
WeamonZ
WeamonZOP4mo ago
Adding denounce might slow your app too... If a user clicks a button, it will wait for 300ms before sending the update and then 300ms-700ms to get the update. So whenever a user clicks on a button it will take 1s to update the UI You should even add optimistic update which will indeed reduce the bandwidth BUT will add a complexity layer
WeamonZ
WeamonZOP4mo ago
That's a bit different in here You're querying not mutating Wait you're right Wait I am too
WeamonZ
WeamonZOP4mo ago
Throttling Requests by Single-Flighting
For write-heavy applications, use single flighting to dynamically throttle requests. See how we implement this with React hooks for Convex.
WeamonZ
WeamonZOP4mo ago
You indeed have to have some optimistic update Let's say you click 3 times within a second (1,2,3) You only want to send 3, but you want the user to show (1,2,3) So you have to unlink the input from convex (by using defaultValue instead of value) Use a debounce as you say And set a key={value} to update the input once the data changes
Wosca
Wosca4mo ago
Oh wow great find
WeamonZ
WeamonZOP4mo ago
Because if an other tab is opened and you're using defaultValue it won't update that tab once the value changes It will only update if the key=value (that comes from convex, changes) You'll have a 1s delay between your tabs but you will drastically reduce the bandwidth
Wosca
Wosca4mo ago
Yeah I don't mind the delay at all, it could be 10s for all I care. It just greatly improves the UX for my site as I implement it as an "Auto Save" feature.
WeamonZ
WeamonZOP4mo ago
Yeah obviously and I feel like not having to invalidate all the queries when performing mutation is a huge win But, the trade-off is that we REALLY have to code better
Wosca
Wosca4mo ago
Yeah, I was hoping for a smooth copy/paste and change of some functions for it to operate normally. Definitely learnt my lesson after this though
WeamonZ
WeamonZOP4mo ago
@Jamie Now you made me wonder if this function can even be cached at all by your system ?
import { v } from "convex/values";
import { authQuery } from "../../procedures";
import { internal } from "../../_generated/api";
import type { ReturnType } from './getTask';
import { internalQuery } from "../../_generated/server";
import { notEmpty } from "@utils/typescript.helpers";

type Result = Awaited<ReturnType>;

export type GetTasksReturnType = Result[];

const args = {
projectId: v.optional(v.id("projects")),
runIds: v.optional(v.array(v.string())),
ids: v.optional(v.array(v.id('tasks'))),
initialNumItems: v.optional(v.number()),
}

export const _internal = internalQuery({
args,
handler: async (ctx, args): Promise<Result[]> => {
const project = args.projectId ? (await ctx.db.query("tasks")
.withIndex("by_project_id", (q) => q.eq("projectId", args.projectId))
.order("desc")
.take(args.initialNumItems ?? 100)) : null;

const ids = project?.map((t) => t._id) ?? args.ids;

if (ids) return (await Promise.all(ids.map((id) => ctx.runQuery(internal.router.tasks.getTask._internal, { id }).catch(() => null)))).filter(notEmpty);
if (args.runIds) return (await Promise.all(args.runIds.map((runId) => ctx.runQuery(internal.router.tasks.getTask._internal, { runId }).catch(() => null)))).filter(notEmpty);

throw new Error("Please provide either a projectId, runIds or ids");
}
})

export const authed = authQuery({
args,
handler: async (ctx, args): Promise<Result[]> => {
return ctx.runQuery(internal.router.tasks.getTasks._internal, args)
}
})
import { v } from "convex/values";
import { authQuery } from "../../procedures";
import { internal } from "../../_generated/api";
import type { ReturnType } from './getTask';
import { internalQuery } from "../../_generated/server";
import { notEmpty } from "@utils/typescript.helpers";

type Result = Awaited<ReturnType>;

export type GetTasksReturnType = Result[];

const args = {
projectId: v.optional(v.id("projects")),
runIds: v.optional(v.array(v.string())),
ids: v.optional(v.array(v.id('tasks'))),
initialNumItems: v.optional(v.number()),
}

export const _internal = internalQuery({
args,
handler: async (ctx, args): Promise<Result[]> => {
const project = args.projectId ? (await ctx.db.query("tasks")
.withIndex("by_project_id", (q) => q.eq("projectId", args.projectId))
.order("desc")
.take(args.initialNumItems ?? 100)) : null;

const ids = project?.map((t) => t._id) ?? args.ids;

if (ids) return (await Promise.all(ids.map((id) => ctx.runQuery(internal.router.tasks.getTask._internal, { id }).catch(() => null)))).filter(notEmpty);
if (args.runIds) return (await Promise.all(args.runIds.map((runId) => ctx.runQuery(internal.router.tasks.getTask._internal, { runId }).catch(() => null)))).filter(notEmpty);

throw new Error("Please provide either a projectId, runIds or ids");
}
})

export const authed = authQuery({
args,
handler: async (ctx, args): Promise<Result[]> => {
return ctx.runQuery(internal.router.tasks.getTasks._internal, args)
}
})
I'd better split it into getByRunIds, getByIds, etc
ian
ian3mo ago
I know you've been going deep on this, but let's recap: The function above is cached based on: - Its arguments. If one function is called with runIds and another with projectId, they're cached separately. - What ranges of documents it reads - not only the documents it gets back, but the range. So if you ask for the "first 100" and one is inserted at the beginning, it's invalidated. - What auth token is retrieving it (iff it reads the ctx.auth) - Including any runQuery it runs. Doing ctx.runQuery from within a query doesn't currently do sub-query caching (iirc). So when one of the documents you query for updates, it will invalidate and re-run. If you're iterating and going through many auth tokens, it will invalidate and re-run. These are all requirements for correctness ("perfect" caching). You're smart to separate your high frequency updates so you don't invalidate big queries. Also limiting the "page size" so when one document changes you only have to fetch a smaller number of documents. I cover some of this in this Stack post A couple notes: - You already fetched the tasks in project above from the tasks table, so there's no need to do an individual ctx.runQuery. - If there is, you should factor out the function to get a task. Calling it via ctx.runQuery has no benefit currently. - One benefit of calling the function directly is I believe the db.get(taskId) will get a cached value since it was already read by the project query. - Another benefit is it doesn't spin up a new isolate (isolated JS container) - If you return IDs to a client and it subscribes to them, those queries won't be invalidated unless the task changes. However, if the query that gets the IDs is reading all the documents to get the IDs, that one is still invalidated and doing all the reads, so doing super granular queries on the client doesn't buy you much unless you have a bunch of sidecar data for each task. - If you don't want query subscriptions on the client and want to have an explicit refresh button or polling or some other inconsistent-but-maybe-cheaper approach, you can always use await client.query instead of useQuery.
Queries that scale
As your app grows from tens to hundreds to thousands of users, there are some techniques that will keep your database queries snappy and efficient. I’...
ian
ian3mo ago
To recap your initial concerns: - Paginated query caching only working sporadically — sometimes it only works once every few days., - The cache helper's paginated query solves the cache busting behavior, and I assume most of this was due to refresh tokens / other invalidation mentioned above. - Auth sessions refreshing too frequently, which seems to unnecessarily invalidate queries. According to the documentation, refreshes should happen every hour, but as shown in the screenshot, one occurred after just 40 minutes. (It even refreshed the authRefreshTokens >2000 times in less than a month. Even tho the tokens are not expired)., - Have you gotten to the bottom of this? I recall Convex Auth used to refresh the token on each page load, so if you're constantly refreshing in a dev environment, this would track. Not a production issue I've heard of for any large-scale current customers, so this would be surprising. - Queries getting invalidated without any mutations. As you can see, nothing in the app has changed, yet almost every query is being re-run., - Is this explained by auth invalidation / args / changes to unrelated fields? - Table updates invalidate all queries referencing that table, even if the updated field isn't part of the query result. - As discussed above, a document update doesn't invalidate a whole table, but it does invalidate queries that read the document even if the field isn't queried I do agree that selective field querying is a good optimization to add btw - but currently the query ranges are stored in a highly efficient data structure that is at the document range granularity to compute invalidations. One thing to do about the progress query: just have the query from the client for the progress alone, and have a separate query for the progress. By the look of the getProjectAndProgress query above, the whole thing is invalidated when the progress changes.
WeamonZ
WeamonZOP3mo ago
Hey @Ian, Just to make sure: - I understand how the caching works (based on the user ctx, and the args) - I refactored the getTasks into multiples queries - as you mention, the user account gets refreshed frequently. Which seems to be normal in dev mode based on your saying - BUT, as shown in the screenshot, queries don't get cached EVEN when the user is not refreshed The database doesn't change AT ALL, yet, when I refresh the page sometimes it gets the cached version, sometimes not Also the "Auth caching invalidation thing", would be correct IF all oh my queries were invalidated But it's not the case, some are, some not, then they are, then they are not, both are using the user ctx
ian
ian3mo ago
And the getCover wasn't being called for different projects on each invocation?
WeamonZ
WeamonZOP3mo ago
It is called for different projects, but the project didn't change Nor any table in my database And they were cached just 10s before
ian
ian3mo ago
And you didn't have many tabs open with different auth tokens?
WeamonZ
WeamonZOP3mo ago
Can It be a nextjs setup error? I believe that if the args sent and the user token are the same there is no reason to un cache things, so it doesn't come from next?
ian
ian3mo ago
When your auth token changes, each project query will be a cache miss
WeamonZ
WeamonZOP3mo ago
1 client, 1 browser, no action, no mutation BUT why my other queries using user ctx are correctly cached when the Auth token changes then? Also here, covers are cached even on token refresh As I mention, Caching makes no sense at all, there are no recognizable pattern in my project
ian
ian3mo ago
What I would do personally would be to start simple and try to investigate what the difference is. It seems that doing many project fetches for different project IDs accounts for why there are many projects being called after an auth invalidation. You can also try commenting out the auth parts temporarily to see if it reproduces without auth
WeamonZ
WeamonZOP3mo ago
It's not only this query, ALL my queries are invalidating for no reason Even actions Cached actions I mean
ian
ian3mo ago
Actions are never cached
WeamonZ
WeamonZOP3mo ago
With the helper, inner cached functions Used in actions
ian
ian3mo ago
I understand your frustration with not being able to make sense of caching. I have recommendations for how to diagnose it, if you're willing to try them.
WeamonZ
WeamonZOP3mo ago
Just take a look at this, pick any query (getAccount, getCover, getProjects, etc) and just check each time they are being called. Try to guess whether it will be cached or not, you will fail each time 🥲 Yep Tell me 🙂 Also, publicQueries (not using any user ctx) are getting uncached too for no reason
ian
ian3mo ago
That image was downsampled by Discord and is very hard to read, but I believe you that without more context it's hard to figure out why some things are cached and some things aren't
WeamonZ
WeamonZOP3mo ago
right click : open link this will help !
ian
ian3mo ago
it did! thanks. One thing to note that I think will help explain what we're seeing is that when you call ctx.runQuery from an action I believe it will cache the sub-query. So one action could have seeded all of the images. And the sub-query isn't touching auth, so it was cached for all users at once. This is different from ctx.runQuery within a query/mutation, if I'm recalling correctly
WeamonZ
WeamonZOP3mo ago
what I tried: - removing inner queries - removing authQueries in favor of publicQueries (so no user context) - removing ags (making no args queries to check if caching works) surely many other things that if you mention, I think I would have already tried ok, well, actually my action is calling stripe and nothing else, so... it doesn't touch any other part of the app
ian
ian3mo ago
How many of these are being run from Next.js server-side vs. client-side? I'm curious how many are using the websocket client vs. http client
WeamonZ
WeamonZOP3mo ago
mhhh id say 90% are websocket
ian
ian3mo ago
Are the getCover calls over HTTP?
WeamonZ
WeamonZOP3mo ago
nop using, useQuery
ian
ian3mo ago
Have you tried console.log of the args to see if each getCover is for the same vs. different args/ documents it scans?
WeamonZ
WeamonZOP3mo ago
I even tried queries with no args they are getting uncached too example: the getProfile query ALWAYS retrieves the same profile (and has a plain args text in it that NEVER changes) I have my projects A,B,C and it retrieves my covers for projects A,B,C so it's not the same args, but when my queries encounter the same args they should retrieve the cached version rigth ?
ian
ian3mo ago
If they're calling query->query, there isn't sub-query caching last I checked
WeamonZ
WeamonZOP3mo ago
we cannot use internal queries ? if so, why is it cached sometimes even with internal queries ?
ian
ian3mo ago
So you could have many queries calling the same query and it wouldn't be cached. I generally strongly advise against doing ctx.runQuery from within mutations or queries
WeamonZ
WeamonZOP3mo ago
then why is it cached sometimes ?
ian
ian3mo ago
If it's called from actions that might populate the cache maybe? I wouldn't over-optimize here and just do the query itself. The bandwidth will likely cost less than the function call
WeamonZ
WeamonZOP3mo ago
also my gest cost is not using ANY sub queries, nor authQueries, nor anything
import { publicQuery } from "../../../procedures";
import { COSTS } from "./credits.config";

export const unauth = publicQuery({
args: {},
handler: () => {
return COSTS;
}
})
import { publicQuery } from "../../../procedures";
import { COSTS } from "./credits.config";

export const unauth = publicQuery({
args: {},
handler: () => {
return COSTS;
}
})
it's getCost:client in the screenshot (I renamed publicQueries to unauth, and the default query({}) to publicQuery({}) for better naming) it just returns plain JSON
WeamonZ
WeamonZOP3mo ago
I have no action other than the stripe one in the screenshot, and the stripe one is only doing a stripe api call Yes but it says that it runs in the same context/transaction, therefore it should be cached ?
ian
ian3mo ago
no that's not the behavior - the same transaction means it'll read a consistent view of the database, but any in-memory caching is not shared between isolates btw, just to step back here, since I know some of this frustration is coming from a hesitation in launching to production. One thing that can help take the pressure off the current investigation would be to figure out the impact of the caching behavior on doing a launch as-is. What's the actual risk / cost we're talking about here? Do you expect nontrivial users / usage on day one? Would it exceed the included pro limits and be a nontrivial amount of usage pricing? I share your curiosity in understanding the internal caching logic, as well as understand there's a desire to predict pricing. But for most customers this ends up being the least of their concerns. It may end up that these hours of debugging cost more in salary than you're saving with any usage pricing above the included limits. And then the investigation can be more paced - making a simple repo so you can play around with different behaviors and do any optimization you need based on real user usage
WeamonZ
WeamonZOP3mo ago
Based on my single usage, our monthly cost that we expected to be 30$ would go up to 200$ - 600$ actually I used 1GB bandwidth in 4 days, 5mb write, 1gb read
ian
ian3mo ago
Ah so this is projecting your dev usage - and then multiplying by some number of users?
WeamonZ
WeamonZOP3mo ago
low average paid users, we made a huge update, so I cannot really compare how the free users will behave
ian
ian3mo ago
Ah, you have many existing users. That makes more sense
WeamonZ
WeamonZOP3mo ago
I have around 3,000 users
ian
ian3mo ago
Congrats!
WeamonZ
WeamonZOP3mo ago
thank you 🙂 but yeah, if they none of my users are hitting the cache the bandwidth cost will be insane I had redis and supabase and we were around 6gb of bandwidth per month FOR ALL USERS but here I used 1gb in 4 days JUST ME so I'm gonna check the internalQueries and mutations things, will extract them into helpers functions, but as mentioned this is not using any internal queries, mutations, etc and yet, not cached
ian
ian3mo ago
Have you investigated the usage graphs in the dashboard btw? Under project settings That can give you a better rundown of usage in the past day, since you've been making optimizations around project progress invalidation etc. And more info about which functions are most worth optimizing
WeamonZ
WeamonZOP3mo ago
at first I thought I lowered the bandwidth usage, but there are no real improvements the usePaginated ones were the biggest problems, now that we have a usePaginatedCached helper in the front end, it might help, but still, all the other functions will add up and with 3000 users it will burn our bill
ian
ian3mo ago
I would: 1. Reduce the _internal pattern to make helper functions instead 2. Split out the project progress query to not also read the project and be queried straight from the client, not from other queries, if this isn't already the case. 3. Make the project progress a public query, provided there's no secret data in there. In general if there are endpoints that can be shared between users that don't expose sensitive data, you'll benefit more from caching without them being authenticated. And also a reminder from above in case it got lost: - If you don't want query subscriptions on the client and want to have an explicit refresh button or polling or some other inconsistent-but-cheaper approach, you can always use await client.query instead of useQuery and choose how often to fetch it.
WeamonZ
WeamonZOP3mo ago
the project progress has been extracted since then ! 🙂
ian
ian3mo ago
And that's been sent to the client directly, ya? not via a runQuery?
WeamonZ
WeamonZOP3mo ago
what's the point of not using authenticated queries for the project if only the user can read it ?
ian
ian3mo ago
If it's user-specific and not shared between users, not much. just possible caching between auth refresh
WeamonZ
WeamonZOP3mo ago
yeah I'm already using queries in server components for that using useQuery for this, since I want the browser to be updated uppon changes well that's not the problem here since in my screenshot even after auth refresh it's still cached........ AND sometimes not 🥲 And sometimes no auth refreshed an not cached 🥲 Also, I'm doing this for a public library (of images), and it doesn't get cached too....... whatever i'm gonna check tomorrow (it's 00am), i'll extract the internal queries into helpers, and I'll update you we were planning to make the migration during this week... let's see if it fixes everything (spoiler, I have HUGE DOUBTS, given I have tried not using internal queries, using public queries, and it still didnt work) Can it be on the nextJS side ? (a <Provider/> problem) ? do you have logs of the data sent to your server for my queries ?
ian
ian3mo ago
There are log streams for granular usage data, but to log inputs/outputs, you need to add logging yourself. I'd recommend adding extensive logging, dumping it to a text file, and doing some investigation on that. NextJS could certainly be a culprit here in some way.
WeamonZ
WeamonZOP3mo ago
What would be the things to log ? - args - user token that's it ? I'll remove any unecessary providers (posthog, etc) and check
ian
ian3mo ago
and just sanity checking - you don't have streaming import set up / modifying things directly from the dashboard / etc? Good luck getting to the bottom of it tomorrow - it's a holiday here and I'm also keen to get back to it. If you have a simple repro in a repo we could play with, that would help. We have a lot of tests and dashboards around caching behavior, so an insidious nondeterministic bug there feels unlikely. But there well may be something unexpected, like running into LRU cache size limits, or possible optimizations around not caching things based on some heuristics that I'm unaware of. Understanding how that affects a project like yours is valuable. Hopefully you get enough confidence to start rolling it out soon - that's always exciting
WeamonZ
WeamonZOP3mo ago
Mhhh, nop I didn't dig into the streaming stuff so i don't think I have anything like that setup, I also didn't interaect with convex dashboard during the screenshots
ian
ian3mo ago
Axiom is pretty slick and my platform of choice for log stream data. You can build nice dashboards & alerting just from logging JSON from your code / string parsing.
WeamonZ
WeamonZOP3mo ago
Enjoy your holidays! I'll try to dig into it (and hopefully find the problem). I'll keep you updated with it, I hope this can help people Minor update : - I extracted most (if not ALL) of my _internal queries into functions helpers. - I didn't extract my _internal mutations since they ... mutate anyway... I'm gonna run like this for the next few days and see if there are noticeable cache issues
ian
ian3mo ago
I was thinking about helpers to make it easier to separate the args/handler definitions from exposing something publicly... What I do today:
const fooArgs = v.object({ bar: v.string() });

export const foo = query({ args: fooArgs, handler: fooHandler });

async function fooHandler(ctx: QueryCtx, args: Infer<typeof fooArgs>) {
...
}
const fooArgs = v.object({ bar: v.string() });

export const foo = query({ args: fooArgs, handler: fooHandler });

async function fooHandler(ctx: QueryCtx, args: Infer<typeof fooArgs>) {
...
}
which allows calling things directly and sharing the args types. But it means writing the types explicitly for the handler. I wonder how useful this would be to folks:
const fooSpec = querySpec({
args: { bar: v.string() },
handler: async (ctx, args) => {...},
});

export const foo = query(fooSpec);

// somewhere else
await fooSpec.handler(ctx, { bar: "baz" });
const fooSpec = querySpec({
args: { bar: v.string() },
handler: async (ctx, args) => {...},
});

export const foo = query(fooSpec);

// somewhere else
await fooSpec.handler(ctx, { bar: "baz" });
no typescript typing, with full type safety! I made a GitHub Issue for it, for anyone who wants to chime in Oh @WeamonZ I just remembered another source of caching that could be making a big difference in Dev for you! The cache is also invalidated if the function changes - so args, auth, and function definition. (oh, and also if it reads an environment variable that changes - basically anything it reads + the code it runs) So during a dev loop you might be iterating on code and it'll cause re-execution of queries every time npx convex dev syncs the code. Obv. this is less of an issue in prod. And I don't know if we have the cache key be a hash of the function and all its dependencies, or if we just invalidate all queries after a push.
WeamonZ
WeamonZOP3mo ago
Interesting ! I extracted everything into a functions folder for each "services" (auth, subscription, projects, etc) And I kept my queries into a "router" folder. In the end I think I prefer extracting the helpers into their one files so that they don't share too much with the router (which handles auth, etc) 🤔 I think enforcing a way to code is better for beginners to be fair. I'm more of the opinionated side, rather that giving 304903 ways of doing the same thing. I prefer the first way, It explicitly allows to extract the helper from the query. Also, the second format has downsides such has exposing/requiring custom query args
ian
ian3mo ago
yeah i'd use this for places where you're expecting to expose them as functions, aside from other helper functions. defining args when they're not actually validated is funky
WeamonZ
WeamonZOP3mo ago
Example:
export const getProfile = authQuery({
args:{},
// helper
handler: async (ctx) => getUserProfile(ctx, ctx.userId)
})

export const getUserProfile = publicQuery({
args: { id: v.id("profile"), },
handler: async (ctx, args) => getProfileByName(ctx, args.id)
})
export const getProfile = authQuery({
args:{},
// helper
handler: async (ctx) => getUserProfile(ctx, ctx.userId)
})

export const getUserProfile = publicQuery({
args: { id: v.id("profile"), },
handler: async (ctx, args) => getProfileByName(ctx, args.id)
})
I prefer to explicitly require a userId in my helper rather than relying on the user context this helps dissociating the helper from the user ctx And this reduces the amount of code from getUserProfile, getCurrentUserProfile, to just getProfile for example, this deduplicates code Mhh ok I'll keep that in mind, but, in my tests I didn't make any changes on the codebase to make sure no cache was revalidated
ian
ian3mo ago
@WeamonZ hey wanted to follow up - we've been doing some investigation and you're right there's something odd going on here. It seems to especially be prevalent with endpoints that use the built-in .paginate invalidating the query cache.. we're digging in and don't have a root cause but wanted to say sorry & thanks for hanging in there. Hopefully the cost is still acceptable for shipping, and the good news will be it'll only get cheaper once we get a pagination fix. I haven't looked into it, but I strongly suspect that using the paginator / stream helpers will not have this issue as they don't use the built-in .paginate under the hood. It's not gapless / perferct pagination though - so I'd do some testing with it. e.g. you might want to be explicitly setting endCursor on the client-side to avoid inserts/deletes that happen at the page boundary
WeamonZ
WeamonZOP3mo ago
Aaaah! I'm glad you found something! Indeed, I used to have a lot of paginated queries at first (especially to display the user library), but the caching was WAY WORSE than the current one (that I think is still bugged... Or at least in dev..). I reached the 1Gb bandwidth in like 3-4 days... I decided to remove most of them in my app until it's fixed because the costs would have been absurd. Now as for the rest, I really hope my app won't act like in dev. I still have 200mb of reads in 2 days.. So let's say I only have around 600 users visiting during the month and using 30% of the bandwidth. It would be : 600x0,3x100x30 = 540 000mb So around 540gb? When my current app (which uses the same system, just based on postgre, but invalidates things at the same time, only uses 6gb for thousands of users) 540-50=490x0.2=98$ I would have to pay 120$ each month for such a few amount of users I don't quite understand how the bandwidth is calculated, how can I go from 6gb to 500gb. I'm using indexes everywhere... I read an article about the pagination system but I didn't quite understand it all all ahaha
ian
ian3mo ago
Have you set up Log Streams? That has per-function call db bandwidth information. And the homepage tries to surface slow / heavy queries, and the usage graphs try to break it down by function, but I agree knowing the performance is important here
WeamonZ
WeamonZOP3mo ago
Not yet, you mentioned Axiom and I'll check up on that. I refactored some code and I'm seeing more and more cached functions. I had a getUserSubscription querying the current subscription at runtime (like expiration date > Date.now()), I'm now relying to some scheduler to update it, so it's hitting the cache more Merged some mutations into a big one (when the user uploads files, I now update the database all at once instead of per files). This reduces the amount of cache invalidations too As for the the paginated queries it literally never hits the cache Actually, the biggest improvements to caching have been related to payments. There was some mention of cache eviction, which might have been part of the issue — or maybe I'm just imagining things
WeamonZ
WeamonZOP3mo ago
By the way, when I try to modify the date to like start at 26-29 IT WONT LET ME, instead it will set it to 24-26. I can't modify the startAt. So... it's pretty hard for me to see the changes in bandwidth
No description
WeamonZ
WeamonZOP3mo ago
But the read/write ratio is still pretty insane 😭😭😭😭.
No description
WeamonZ
WeamonZOP3mo ago
Also, weirdly, now I don't see the cache invalidation uppon session refresh in dev. As we mentioned previously, this would/could cause queries invalidation when the token gets refreshed, but it doesn't seem to be the case anymore 🤔 Which is a good thing, cause it's unwanted Nevermind, it still occurs 🤣😭 Why tie queries to the refreshToken rather than the sessionAccount tho 🤔
ian
ian3mo ago
yeah it's the auth jwt (the refresh token isn't sent - I think is saved as an http-only cookie?). the rationale being that when the JWT expires we technically don't know if you still have access. But I'd LOVE to reconcile the JWT validation, so if it results in the same user & fields, it will be a cache hit. The challenge is when the sub field is a session ID vs. just a user ID. So because the session ID changes, the query is invalidated b/c that's an argument.
ian
ian3mo ago
Jamie Turner (@jamwt)
Mistake I just made: I didn't give our power users the benefit of the doubt. I swore our caching system was tested within an inch of its life, and was skeptical of reports of a regression where paginated queries aren't caching. Turns out I was wrong. Will fix ASAP.
From Jamie Turner (@jamwt)
X
WeamonZ
WeamonZOP3mo ago
🎊🎊🎊🎊🎊🎊🎊🎊🎊🎊 So glad you found the source of the problem !! Thank you for trusting me and for being so open to discussion!
ian
ian3mo ago
👏 team effort
jabra
jabra3mo ago
@ian is the fix in the latest convex ?
jabra
jabra3mo ago
Thanks @WeamonZ @ian this thread is very useful since i have a big spike last sunday and monday in db bandwidth
No description
jabra
jabra3mo ago
@WeamonZ do you mind shring your usePaginatedCached? Thanks in advance!
ian
ian3mo ago
It's in convex-helpers@0.1.90-alpha.2 currently if you use its query cache helper. I'm going to ship it soon (pr here: ) but I'm thinking through deeper pagination behavior to make sure it won't be a footgun. One thing folks should understand is that when you use the query cache provider, some of your queries may end up being very large if the query range grows - e.g. a paginated query could sit in the background getting 10, then 11, then ...100 items if it's subscribed to the end of a table and (order "desc" cursor "null") a lot are being added before it expires. I'm thinking through if/when to drop paginated subscriptions from the cache
jabra
jabra3mo ago
Better safe than sorry
jabra
jabra3mo ago
I discovered https://million.dev/ today and it is helping not only optimize react but rethink my convex queries to break them down to avoid a few extra invalidations
Million
Speed up your website by 70%
WeamonZ
WeamonZOP3mo ago
Oh I knew about react-scan but not this Paginated queries can consume bandwidth very quickly. For now, I’ve removed most of them and only kept pagination in the user library, where it’s necessary to access all content. When I need to display recent items, I use a getLatestItems query instead of a paginated query, which fetches only the latest 10–30 items. This approach significantly reduces database and bandwidth usage. Of course, the downside is that users can no longer browse their entire item history (which is a regression) but it's a necessary trade-off for now to limit load.. The issue with paginated queries is that they don’t handle dynamic data well. For example, if you fetch pages like 0–20, 20–40, and 40–60, then add a new item at the top, your ranges shift to 0–1, 1–21, 21–41, and 41–61. Because these new ranges no longer match the previously cached ones, Convex will invalidate all of them. Ideally, it would just prepend the new item and keep the rest cached — but I’m not sure if that’s even feasible, as there might be edge cases where this approach breaks.
jabra
jabra3mo ago
I hope they find a better solution for this soon , paginated queries are needed for me
ian
ian3mo ago
For example, if you fetch pages like 0–20, 20–40, and 40–60, then add a new item at the top, your ranges shift to 0–1, 1–21, 21–41, and 41–61. Because these new ranges no longer match the previously cached ones, Convex will invalidate all of them.
This is not true, thankfully. If you're within the same context, the query will change to 40-61, and the others will stay the same. Eventually it will page-split only replacing one page with two pages. That being said, if another user / in another tab comes along they could start their session for 41-61 and not have cache hits for the other pages. but you won't see the waterfall of invalidations like you'd get with naive limit/offset-based pagination If you only fetch the first page, the bandwidth is the same (with the exception that the first page can grow, vs. being limited like .take. I have a proposal internally where you can get the first page to not grow until you explicitly load more, at which point the first page is "pinned" to the results it has so far and the second page picks up where it left of
WeamonZ
WeamonZOP3mo ago
@ian Mhhhh that's not the current behavior in my app, or maybe I don't understand what you say. But the current behavior in my app is : - Page 1 (0-20) cached - Page 2 (0-20) cached - Page 3 (0-20) cached Add an element to the top - Page 1 (0-20) uncached - Page 2 (0-20) uncached - Page 3 (0-20) uncached I suspected that since each page doesn't have the same "key" now that it has shift because of the new element, it was an expected behavior that ALL pages were uncached, but it shouldnt ? In this example: I have my three pages (so the first one with cursor null, and the two others) I then upload an image (and its sizes (thumbnail,small,etc) that's why you see 2 mutations) The first page gets invalidated, whatever, ok Then I refresh the browser (because i'm using usePaginatedCached, so it doesn't ask for the update on the server, it processes it automatically). Since it's in realtime, the first page has already been cached, ok, great But the following pages are NOT cached
WeamonZ
WeamonZOP3mo ago
The page looks like this, just a simple scroll gallery with an upload button
No description
WeamonZ
WeamonZOP3mo ago
No description
ian
ian3mo ago
Yes if you full refresh, it will start at the start and go how far you ask it to. if you're on the same page, with or without the cached query helper, it won't re-request later pages. that's the point I was clarifying
WeamonZ
WeamonZOP3mo ago
Ohhh okay ! but overall it invalidates everything in the end
ian
ian3mo ago
The next set of requests have different arguments if they started in a different place in your screenshot I only see the first page being re-loaded (cursor: null) during the mutation I have to step away for now, sorry
WeamonZ
WeamonZOP3mo ago
... until the user re-render the component My ranges always are from 0-20 20-40 etc but the cursor changes because the array changed because of the new data. So it always invalidates every page I agree that everything is cached until the component is re-rendered, but adding an item to the results of a paginated query ultimately ends up invalidating all the pages.
ian
ian3mo ago
When you say your ranges are always 0-20, 20-40, do you mean you do cursor: null, numItems: 20, then take that cursor and do numItems: 20? or are you passing explicit parameters of 20/40? When a new item is added, the null query grows to 21 items, and the endCursor is the same (if you're using the default built-in pagination - I'm working on a hook that works well for streams/ paginator). If you refresh the page, it will not guarantee returning the same endCursor. And by invalidated I think you mean that when you refresh, you'll get a different endCursor (true), so requesting the next page will have different arguments than the first time - so it's a new query, so has new results. But it's not going around invalidating the cache for anyone else who had subscribed to the old page boundary. If you could distill this into a bug / feature request and file it, please do. I don't see what other behavior would make sense here
WeamonZ
WeamonZOP3mo ago
I'm using the built in pagination system
// component
const images = usePaginatedQuery(api.router.storage.getImagesPaginated.authed, {}, { initialNumItems: 20 });
// [....]
images.loadMore(20);
// component
const images = usePaginatedQuery(api.router.storage.getImagesPaginated.authed, {}, { initialNumItems: 20 });
// [....]
images.loadMore(20);
import { paginationOptsValidator } from "convex/server";
import { authQuery } from "../../procedures";
import { getImage } from "../../functions/images";

export const authed = authQuery({
args: { paginationOpts: paginationOptsValidator },
handler: async (ctx, args) => {
const workspaceId = ctx.user.currentWorkspaceId;
const workspaceImages = await ctx.db.query("workspaceImages")
.withIndex('by_workspace_id', q => q.eq('workspaceId', workspaceId))
.order("desc")
.paginate(args.paginationOpts)
//[...]
import { paginationOptsValidator } from "convex/server";
import { authQuery } from "../../procedures";
import { getImage } from "../../functions/images";

export const authed = authQuery({
args: { paginationOpts: paginationOptsValidator },
handler: async (ctx, args) => {
const workspaceId = ctx.user.currentWorkspaceId;
const workspaceImages = await ctx.db.query("workspaceImages")
.withIndex('by_workspace_id', q => q.eq('workspaceId', workspaceId))
.order("desc")
.paginate(args.paginationOpts)
//[...]
If you refresh the page, it will not guarantee returning the same endCursor.
Why that if you just said and the endCursor is the same
And by invalidated I think you mean that when you refresh, you'll get a different endCursor
Exactly, so you're telling me I should store locally the cursors or smthg as a good practice, there is a workaround ? btw take your time, it's not blocking my app for now, but as for @jabra idk
jabra
jabra3mo ago
it is not blocking me even though i'm getting a lot of bandwidth consumed, i will try the usePaginatedCached next
ian
ian3mo ago
and the endCursor is the same
This is for when you don't refresh the page (btw why is the user refreshing the page so frequently if Convex is keeping it up to date?) I would not recommend storing the cursors or diverging from the default behavior in general - re-using old cursors will mean loading every document that has been added since then. One thing that makes a big difference is to have small pages if data is being inserted frequently - especially initialNumItems < 10, maybe even as low as 1 or 2. Then request more data after that - potentially in bigger pages if the later data is not changing. Another thing to note: using the caching helpers will increase your bandwidth usage in order to improve UI/UX. They are strictly less efficient than the regular queries because they hold open idle subscriptions. Stale queries will continue to get updated in the background when data changes. The benefit is user experience, so they can come back to a page and the up-to-date data is already there. If your goal is to merely not pass the cache-busting parameter, that will be in the standard client soon, and you can copy my implementation with another new feature from here As a final suggestion, you can keep a field like a document order in a thread and have an index on [threadId, order] and paginate on known boundaries, like every 10 messages, to get more cache hits. But I'm also skeptical that optimizing for bandwidth from page refreshes makes a big difference in practice for queries that are already cached per-user due to auth. There's more work going on for pagination in general: - The streams and paginator helpers in convex-helpers@0.1.91 will split pages more eagerly, which will make a big difference if your data changes frequently. However, make sure to use them with a usePaginatedQuery hook like the one I linked above, or the standard one that will come out soon. - We're going to move away from the QueryJournal so old requests won't grow by default if you haven't loaded more data - it'll just be a rolling N-sized window, which may help with cache hits, and also bandwidth in the default case. That is the behavior added in https://github.com/get-convex/convex-helpers/commit/dc60a828e8741957aeec04ce35a376d5127e546f I've been spending a lot of time on talking this through - so I'll leave it to y'all from here to figure out what makes sense and what problems you're running into. If you have an issue and are a Pro customer, email support@ I hope this has been helpful!
kkcoms
kkcoms3mo ago
I can say the same about the paginated query caching. I actually never joined Discord in the past because I've been a Convex user for the past year and a half. However, the paginated query caching is not working at all, and that's why I'm here - my database bandwidth is extremely high, and it doesn't make sense why. And for most that I have tried to do it I haven't found a way to make it work. Is there any way to reduce the bandwidth? For me, it doesn't make sense. Even if I use a paginated query, which is catch, or even if I use normal queries, still the bandwidth is too high. And for sure it's not being triggered by the client. I feel like it's somehow a convex goes into this pattern of reauthenticating the user and then re-triggering all the queries.
WeamonZ
WeamonZOP3mo ago
Yep once the account token is refreshed they uncache all the queries... I don't really know why they opted for this instead of just checking for the account
WeamonZ
WeamonZOP3mo ago
"using the caching helpers will increase your bandwidth usage in order to improve UI/UX. They are strictly less efficient than the regular queries because they hold open idle subscriptions."
So this is the normal behavior ? https://github.com/get-convex/convex-backend/issues/116
GitHub
useQuery is still active for deleted (unmounted) component · Iss...
Hey, I&#39;m having an issue with useQuery still being active for deleted (unmounted) component. Description: I&#39;m seeing unexpected behavior where a useQuery is still firing for a deleted proje...
WeamonZ
WeamonZOP3mo ago
Ok it seems like that's what you're talking about. I expected the cached helpers to REDUCE the hits with the server to be honest Ok indeed it updates all of my previously hit hooks. That's interesting... So actually to reduce the bandwidth we should not use the cache helper 🤔 It would be great to have the queries cached (so that they don't hit the server for no reason when not updated) by default on the standard client. Is this what you're referring to ?
that will be in the standard client soon
ian
ian3mo ago
Yes your understanding is correct. The way that we know that nothing has been updated is to keep the subscription alive, which means re-running the queries. There is not currently a protocol-level signal for "this query has changed FYI but I'm not going to re-run it" I responded this in the issue:
the "convex-helpers/react/cache" behavior is to keep open the subscription in the background for some amount of time, so when you go back it's perfectly cached - not just the stale value but the consistent up-to-date value, which is accomplished by keeping open the subscription
Keeping a local stale cache is an application use-case decision that probably makes sense for many, but isn't the behavior of anything I've made yet. I'm going to write up a blog post about pagination. I agree the ability to understand pagination behavior and therefore predict and optimize around caching makes a lot of sense for folks whose apps are particularly bandwidth-sensitive. Thanks for engaging. Have you upgraded your convex version? We made some optimizations for pagination caching behavior recently, which improves the cache hit rate
kkcoms
kkcoms3mo ago
Thanks Ian. Yes, I've been a paid Convex customer for a year or so. I totally agree and would appreciate a blog post explaining in detail this because I love so many things about Convex and that's why I've been paying for this solution for a year. But the bandwidth it doesn't make sense, and sometimes we see (I see at least) crazy spikes that for most of adjudicate the errors to some client components that I have. It's almost unlikely that's the case, and for more that I do things to optimize the bandwidth, I have never been able to get it to a point where I say "okay, perfect, this makes sense" for a small application of like 4,000 users. Which is similar to what @jabra and @WeamonZ have mentioned.
ian
ian3mo ago
I mean the version of convex you're running, not your subscription plan 😅
jabra
jabra3mo ago
it definitively improved for me but still need to look into deeper and investigate every call cached or not. still a lot of calls are happening because of invalidations might need to break down queries and restructure some tables
No description
jabra
jabra3mo ago
I need to reduce the number of queries for 300 DAU
No description
ian
ian3mo ago
Have you set up log streams / investigated which functions are causing it? You might find that it's just some query scanning a ton of data due to a filter (instead of index) etc. For 90% of customers who write in, that's the culprit. https://stack.convex.dev/queries-that-scale
Queries that scale
As your app grows from tens to hundreds to thousands of users, there are some techniques that will keep your database queries snappy and efficient. I’...
jabra
jabra3mo ago
I didn't set log streams yet (on the todo list), your article is my go to and I removed most of the filters, you can see that my bandwidth is manageable. I need to understand well how this will scale (calls and cost) when I onboard 10 more communities with +1000 users each. My main concern this is the invalidation that causing everyone to refresh their queries. I onboarded the first collab group on my platform and they are more than 370 people that see the same page and they all can upload images, comment and react to each other.
kkcoms
kkcoms3mo ago
Ups... hahaha this one Convex v1.24.0
jabra
jabra3mo ago
i just setup log streams to axiom, will monitor and see where is the bottleneck
ian
ian3mo ago
I believe it was fixed in the 1.14.8 release, so if you update it should fix it

Did you find this page helpful?