Deleting troubles. ents etc.
It seems like i'm in a type of dependency hell with this stuff, every time i have to track down a bug in our init system.
I really want that delete all tables in the UI, there has been so much wasted time trying all this other stuff.
We wrote an db wipe script to remember the empty state by exporting using npx convex export. But it is very slow some times because importing blank data takes forever for no great reason IMHO.
But if we change the db layout it becomes out of date and misses the new tables with data. So you have to keep track of that.
Then we accidently had a export with new tables with data that didn't match the new schema.
So then we made a script that loops over the schema to delete everything but it follows the scheduled deletes so a lot of stuff is not actually deleted.
With ents how do you delete something ignoring soft deletion/scheduled rules?
Also, I feel like there must be a bug with ents on anything but scheduled deletes, the deletes do get scheduled. But nothing else ever gets completed immediately if it is a root table in a soft delete "tree". Im trying to defer debugging that as i dont care but its making all the above harder. Maybe I'm reading the documentation wrong, but i only see the deletionTime and it never gets deleted. Do i need to execute some other command?
9 Replies
i did try to schedule delete from my members table and got a error, that is a many to many users with a .edge('sphere') :
functions:scheduledDelete - is created by scheduledDeleteFactory
failure 48ms Uncaught Error: Tried to query index users_to_members.membersId but the query didn't use the index fields in order. Index fields: [membersId", "usersId", "_creationTime"] Query fields: ["membersId", "_creationTime"] First incorrect field: "_creationTime" the inverse happens too deleting from users. Im not defining deletion behavior to associated tables or anything. and docs say it doesnt cross many to many anyway etc. If you just sent some random table to deletion("soft") and delete a row, using .delete it updates the deletion time to now() and never schedules anything. Is that expected, and its expecting me to delete a linked row that would actually cascade? i did get schedule delete to work with a table with no edges
failure 48ms Uncaught Error: Tried to query index users_to_members.membersId but the query didn't use the index fields in order. Index fields: [membersId", "usersId", "_creationTime"] Query fields: ["membersId", "_creationTime"] First incorrect field: "_creationTime" the inverse happens too deleting from users. Im not defining deletion behavior to associated tables or anything. and docs say it doesnt cross many to many anyway etc. If you just sent some random table to deletion("soft") and delete a row, using .delete it updates the deletion time to now() and never schedules anything. Is that expected, and its expecting me to delete a linked row that would actually cascade? i did get schedule delete to work with a table with no edges
@ampp hey! sorry for the issues. it sounds like there are two things here, if I'm following you correctly. maybe three things. let me know if I have it right
1. A request that we have a "delete all tables" on the dashboard
2. Importing an empty dataset is too slow to delete tables via that method (using
npx convex import, I'm assuming)
and 3. ents seems to have some bugs about processing cascading deletes
is that right?Yeah, clear all tables essentially, the one simple improvement lessens the impact of the others. I've been relying on npx convex import to avoid dealing with what seems like a bug with ent delete and writing a custom function. I think its attempting to cascade on things it shouldn't, as many to many should be avoided as the documentation says. I'm also curious how to bypass any soft/scheduled delete behavior and just do a hard delete.
It's all a bit interwoven. 😅
gotcha. well, @Michal Srb has all the info on ents. on the dashboard delete all tables, was just chatting with @ari about it
I think you mean deleting the documents inside the tables, but not the tables themselves? you want empty tables back, basically?
Yeah thats 99% of it, i have npx convex dev running so it handles any shape changes to the schema once the tables are empty.
Hey @ampp, while we don't have a way to clear all your tables from the dashboard right now, you could achieve this by importing an "empty" snapshot with the
--replace flag
Steps:
1. Clear your tables manually once
2. Run a Snapshot Export on the dashboard (https://docs.convex.dev/database/import-export/export). This will allow you to download a zip file specifying an empty snapshot for each table, to be used in the next step
3. Run npx convex import —replace SNAPSHOT_NAME.zip
More on the import command: https://docs.convex.dev/database/import-export/import
Ah, I'm just catching up on this thread now and it looks like you've already tried this!Yeah, the issue is keeping a snapshot up to date with my complete table list that is empty. If my schema changes much then i got to do some manual intervention. Maybe a flag to get a empty snapshot matching your table structure would be a hacky way to get something a little more helpful.
Yeah gotcha, that case is really not handled well with this method! I'm going to file this issue and bubble it up to the folks that work on how the backend processes imports. Ideally we just offer a cleaner way to do this type of wipe instead of something hacky around imports 🙂
I agree we should have an easy way to reset the deployment data. This is in the same bucket of requests we get for dev workflows around branching.
I’ll look at the specific issue in Ents, but to answer your more general question, you can delete ignoring your ent rules via the built-in ctx.db. You can also get the full list of tables from your schema.