39 replies

Fetching a random subset of documents without having to collect() over the entire set?

Hi team, I'm implementing an app where a core user flow is to be able to answer a subset of questions from a collection of documents I have stored in Convex. Right now, I have it implemented in a way where I fetch ALL of the documents (this can bloat to thousands eventually), and then apply filters related to which questions I'd like to exclude, and then return only a random subset of these documents.

This makes my db bandwidth usage skyrocket over time and I'd really like to optimize this. Is there any simpler way to get a random subset of docs from Convex? If not, is there a more efficient way of implementing this in general? I've attached a code snippet below.

export const getRandomQuestions = query({
  args: {
    quantity: v.number(),
    exclusionIds: v.optional(v.array(v.id("templateQuestions"))),
  },
  handler: async (ctx, args) => {
    const { exclusionIds, quantity } = args;

    const questions = await ctx.db.query("templateQuestions").collect();
    const filteredQuestions = exclusionIds
      ? questions.filter((q) => !exclusionIds.includes(q._id))
      : questions;

    // Use current timestamp as seed for reproducible randomness within same millisecond
    const seed = Date.now();
    const random = mulberry32(seed);
    for (let i = filteredQuestions.length - 1; i > 0; i--) {
      const j = Math.floor(random() * (i + 1));

      // Modern swap syntax
      [filteredQuestions[i], filteredQuestions[j]] = [
        filteredQuestions[j],
        filteredQuestions[i],
      ];
    }

    return filteredQuestions.slice(0, quantity);
  },
});

export const getRandomQuestions = query({
  args: {
    quantity: v.number(),
    exclusionIds: v.optional(v.array(v.id("templateQuestions"))),
  },
  handler: async (ctx, args) => {
    const { exclusionIds, quantity } = args;

    const questions = await ctx.db.query("templateQuestions").collect();
    const filteredQuestions = exclusionIds
      ? questions.filter((q) => !exclusionIds.includes(q._id))
      : questions;

    // Use current timestamp as seed for reproducible randomness within same millisecond
    const seed = Date.now();
    const random = mulberry32(seed);
    for (let i = filteredQuestions.length - 1; i > 0; i--) {
      const j = Math.floor(random() * (i + 1));

      // Modern swap syntax
      [filteredQuestions[i], filteredQuestions[j]] = [
        filteredQuestions[j],
        filteredQuestions[i],
      ];
    }

    return filteredQuestions.slice(0, quantity);
  },
});

Convex Community•12mo ago•

39 replies

pez

Fetching a random subset of documents without having to collect() over the entire set?

export const getRandomQuestions = query({
  args: {
    quantity: v.number(),
    exclusionIds: v.optional(v.array(v.id("templateQuestions"))),
  },
  handler: async (ctx, args) => {
    const { exclusionIds, quantity } = args;

    const questions = await ctx.db.query("templateQuestions").collect();
    const filteredQuestions = exclusionIds
      ? questions.filter((q) => !exclusionIds.includes(q._id))
      : questions;

    // Use current timestamp as seed for reproducible randomness within same millisecond
    const seed = Date.now();
    const random = mulberry32(seed);
    for (let i = filteredQuestions.length - 1; i > 0; i--) {
      const j = Math.floor(random() * (i + 1));

      // Modern swap syntax
      [filteredQuestions[i], filteredQuestions[j]] = [
        filteredQuestions[j],
        filteredQuestions[i],
      ];
    }

    return filteredQuestions.slice(0, quantity);
  },
});

export const getRandomQuestions = query({
  args: {
    quantity: v.number(),
    exclusionIds: v.optional(v.array(v.id("templateQuestions"))),
  },
  handler: async (ctx, args) => {
    const { exclusionIds, quantity } = args;

    const questions = await ctx.db.query("templateQuestions").collect();
    const filteredQuestions = exclusionIds
      ? questions.filter((q) => !exclusionIds.includes(q._id))
      : questions;

    // Use current timestamp as seed for reproducible randomness within same millisecond
    const seed = Date.now();
    const random = mulberry32(seed);
    for (let i = filteredQuestions.length - 1; i > 0; i--) {
      const j = Math.floor(random() * (i + 1));

      // Modern swap syntax
      [filteredQuestions[i], filteredQuestions[j]] = [
        filteredQuestions[j],
        filteredQuestions[i],
      ];
    }

    return filteredQuestions.slice(0, quantity);
  },
});

Fetching a random subset of documents without having to collect() over the entire set?

Similar Threads

Fetching a random subset of documents without having to collect() over the entire set?

Similar Threads

Similar Threads

Similar Threads