Starlord Posts - Convex Community

Starlord

•Created by Starlord on 2/9/2025 in #support-community

Index query with "OR" statement

hello, is there a way to optimize this query using index because "or" statement is not possible in index? this way full table data is fetched and only filtered afterwards.

    
// Get variations for filtered products
const allProductVariations = await ctx.db
    .query("productVariations")
    .filter(q => 
        q.or(...results.page.map(product => 
            q.eq(q.field("productId"), product._id)
        ))
    )
    .collect();

    
// Get variations for filtered products
const allProductVariations = await ctx.db
    .query("productVariations")
    .filter(q => 
        q.or(...results.page.map(product => 
            q.eq(q.field("productId"), product._id)
        ))
    )
    .collect();

7 replies

CCConvex Community

•Created by Starlord on 1/25/2025 in #support-community

Join tables query / paginate

An important limitation in Convex that affects how we need to structure queries, especially for paginated results with joins. Let me explain the problem and show hacky workarounds: The Problem In my current code, i am trying to: 1. Paginate products 2. Then fetch variations for those products 3. Then filter based on those variations (colors, sizes) This approach is problematic because: 1. The pagination happens BEFORE the joins and filters on variations 2. This means I might get fewer results than requested after filtering 3. I can't do true SQL-like JOINs in a single query For example, if I request 10 products: - I might get 10 products from the initial pagination - After filtering by variations, I might end up with only 3-4 products that actually match - The user sees fewer items than expected Possible Solutions 1. Denormalization Approach

// Store available colors and sizes directly on the product document
interface Product {
  _id: Id<"products">;
  // ... other fields ...
  availableColors: Id<"productColors">[];
  availableSizes: Array<{
    sizeCategoryId: Id<"sizeCategories">;
    values: string[];
  }>;
  hasInventory: boolean;
}

// Store available colors and sizes directly on the product document
interface Product {
  _id: Id<"products">;
  // ... other fields ...
  availableColors: Id<"productColors">[];
  availableSizes: Array<{
    sizeCategoryId: Id<"sizeCategories">;
    values: string[];
  }>;
  hasInventory: boolean;
}

2. Materialized Views Approach

// Create a separate table that pre-joins product data
interface ProductView {
  _id: Id<"productViews">;
  productId: Id<"products">;
  availableColors: Id<"productColors">[];
  availableSizes: Array<{
    sizeCategoryId: Id<"sizeCategories">;
    values: string[];
  }>;
  // ... other fields needed for filtering
}

// Create a separate table that pre-joins product data
interface ProductView {
  _id: Id<"productViews">;
  productId: Id<"products">;
  availableColors: Id<"productColors">[];
  availableSizes: Array<{
    sizeCategoryId: Id<"sizeCategories">;
    values: string[];
  }>;
  // ... other fields needed for filtering
}

3. Batch Loading Approach

// Load more products than needed initially, then filter
const BATCH_MULTIPLIER = 3; // Load 3x more products than requested
const batchSize = paginationOpts.numItems * BATCH_MULTIPLIER;

// Modified pagination options
const batchPaginationOpts = {
  ...paginationOpts,
  numItems: batchSize
};

// Load more products than needed initially, then filter
const BATCH_MULTIPLIER = 3; // Load 3x more products than requested
const batchSize = paginationOpts.numItems * BATCH_MULTIPLIER;

// Modified pagination options
const batchPaginationOpts = {
  ...paginationOpts,
  numItems: batchSize
};

74 replies

CCConvex Community

•Created by Starlord on 1/16/2025 in #support-community

Not authenticated after successful sign in request

8 replies

CCConvex Community

•Created by Starlord on 1/5/2025 in #support-community

Suboptimal Caching Behavior for Public Queries

Current Behavior - Query results are only cached per user session - Cache is not shared between different users or sessions - Even public queries with stable parameters require fresh DB hits for each new user/session - Example: Product listing queries with same filters/pagination hit DB for each new visitor Expected Behavior - Public queries with identical parameters should be cached across users/sessions - Queries that don't depend on user-specific data (like ctx.auth.getUserId()) should leverage shared cache - Cache invalidation should only occur when underlying data changes Impact - Increased database bandwidth consumption - Higher latency for common queries - Unnecessary load on database for identical queries - Cost implications for high-traffic applications

74 replies

CCConvex Community

•Created by Starlord on 12/20/2024 in #support-community

Debug Production

Hello, I have run into a situation where a user has a problem / bug in production environment. Is there a way to copy production environment into development? what would be the best way to login as this user and to simulate his connection?

12 replies

CCConvex Community

•Created by Starlord on 12/18/2024 in #support-community

pagination query 2 server requests on start

hello. please explain why paginated query is executing 2 times on first execution even it is exhausted? it think the problem is with convex source code: First execution: Gets the initial page of results Second execution: Checks if there are more items after the current page its doing second request even if first is already 'Exhausted'

export default function NewProductPage() {
  const result = usePaginatedQuery(
    api.admin.listOwnPendingProducts,
    {},
    { initialNumItems: 10 }
  );      

  useEffect(() => {
    console.log('Query state changed:', {
      status: result.status,
      productsLength: result.results?.length,
      isLoading: result.isLoading,
      results: result.results
    });
  }, [result]);

  return null;
}

export default function NewProductPage() {
  const result = usePaginatedQuery(
    api.admin.listOwnPendingProducts,
    {},
    { initialNumItems: 10 }
  );      

  useEffect(() => {
    console.log('Query state changed:', {
      status: result.status,
      productsLength: result.results?.length,
      isLoading: result.isLoading,
      results: result.results
    });
  }, [result]);

  return null;
}

Query state changed: {status: 'LoadingFirstPage', productsLength: 0, isLoading: true, results: Array(0)}
Query state changed: {status: 'LoadingFirstPage', productsLength: 0, isLoading: true, results: Array(0)}
eruda.js?v=374bd0d1:3361 [CONVEX Q(admin:listOwnPendingProducts)] [LOG] 'listOwnPendingProducts executed on backend'
Query state changed: {status: 'Exhausted', productsLength: 1, isLoading: false, results: Array(1)}
eruda.js?v=374bd0d1:3361 [CONVEX Q(admin:listOwnPendingProducts)] [LOG] 'listOwnPendingProducts executed on backend'
Query state changed: {status: 'Exhausted', productsLength: 1, isLoading: false, results: Array(1)}

Query state changed: {status: 'LoadingFirstPage', productsLength: 0, isLoading: true, results: Array(0)}
Query state changed: {status: 'LoadingFirstPage', productsLength: 0, isLoading: true, results: Array(0)}
eruda.js?v=374bd0d1:3361 [CONVEX Q(admin:listOwnPendingProducts)] [LOG] 'listOwnPendingProducts executed on backend'
Query state changed: {status: 'Exhausted', productsLength: 1, isLoading: false, results: Array(1)}
eruda.js?v=374bd0d1:3361 [CONVEX Q(admin:listOwnPendingProducts)] [LOG] 'listOwnPendingProducts executed on backend'
Query state changed: {status: 'Exhausted', productsLength: 1, isLoading: false, results: Array(1)}

12 replies

CCConvex Community

•Created by Starlord on 12/18/2024 in #support-community

Force logout user / invalidate JWT token

Hello, is it possible to force logout user / invalidate JWT Token on the backend side to make player need to login again?

35 replies

CCConvex Community

•Created by Starlord on 12/13/2024 in #support-community

mutation doesnt rollback query state on failure

22 replies