What are the .collect() limitations? How can I handle large amounts of data querying?
Right now in my application the tables are quite small and running
.collect() does not cause any problems. However I have concerns for the future. If my table has over 25k documents, how does .collect react()? In my specific case, I want a cron job to read a whole table and delete data if x condition is meet once a week. If the .collect() method only reads x amount of documents what happens to the query? Will it re-run and finish the other results or will I have to re-run the cron again? What should I do when I expect to process a large amount of data at once?
