filter vs withIndex
I'm encountering a very confusing querying bug, and I'm not sure whether it's an issue with Convex or my misunderstanding of the API 🤔
26 Replies
Given a schema that looks like this:
these two queries do not return the same results:
Should they?
Is the ordering the difference? The former will be in position order, then creation time order. The second will be in creation time order
Interesting! That's good to know, but no, it's not. I'm getting a different number of documents between the two queries.
In fact, unless I'm misreading something, it looks like I'm just getting
step
s for the wrong docId
Which one is incorrect? Are they both running in the same query? Mutation? Or run separately? This is very surprising
withIndex
is incorrect
In a query which initially used the stepsWithIndex
version
I switched to the stepsFilter
version to test whether this would fix the issues I'd been observing in the UI code, after narrowing the problem down to unexpected results from the particular Convex function which contained it
So I've never tested running them at the same time, in the same query. But I will now!
They produce different results (differing by number of documents) in the same queryIs withIndex one returning more? There’s a max number of rows scanned currently, which includes rows scanned that don’t match the filter. So the filter query may be returning a subset of results. But I thought it’d fail in that case rather than silently return the subset. I haven’t seen the code on it though
If the withIndex results have the wrong docId that would be very surprising
No, the difference is between 0 and n, where n seems, based on my observations so far, arbitrarily large
I'll triple-check that the
docId
is wrong in the withIndex
version of the query
(I was also very surprised!)Some sample data that reproduces this would be awesome, so we could tease this out
Hmm, how best to do that? I could share an export of the tables and copy the full code of the query?
I don't mind sharing access to the project, either
And sorry—also triple-checked that the
docId
is wrong for the withIndex
query, it isEither exporting the data or adding a teammate if the data is in prod should work.
it could also help debug on our side if you share your deployment name (https://dashboard.convex.dev/ -> click project (and pick production or development) -> settings -> deployment url)
The data is in the dev environment, and here is the deployment name: https://healthy-opossum-607.convex.cloud
May I have an email to add one (or both) of you to the project? Seems easier than copy+pasting a bunch of stuff(?)
to the team*, I guess?
presley@convex.dev
Sent
I only see the prod deployment (currently dev deployments are private for each member). Can you repro this there?
What query you are calling that is observing the bug?
Ah right
Yeah I can
getStepsSince
One sec
Actually, will be more than one sec; here are exports of the data in the meantimeThere's a nice Parcel (the JS bundler I'm using) bug which is preventing my production deploys from working at the moment. Are the above exports sufficient for reproduction?
We are working ion it
Should be sufficient
Ok, let me know if not, I'm sure I can figure something out if necessary!
@RJ, to confirm, the
doc
field on docs
is a string that's a JSON encoded document?Correct @sujayakar
Same with the
step
field on steps
okay! we've got a repro and have found the bug. we'll have a fix up shortly and will update the thread here when it's out. thanks for the detailed report @RJ 🙂
Fantastic! Thanks @sujayakar
You can also kick me out of your project. We don't have a button to leave yet.
Ok, issue should be fixed now. Sorry for the trouble @RJ and thank you for the report.
No worries, glad to have found that one sooner rather than later 🙂