sbkl•6mo ago

convex-react-query state and plan on useInfiniteQuery hook support

What is the plan for convex-react-query? Is it here to stay or just experimentation? Also any visibility on when useInfiniteQuery might be supported? Saw it was under the TODO section in the github repo. Started to have a look myself to the source code to see if I can help but still unsure how this would work and unfamiliar with the mechanism of the convex client.

11 Replies

ballingt•6mo ago

It's here to stay! The slow start has been because the motivation for convex-react-query was TanStack Start. I wrote the library early in the summer when it was more of an experiment, but we've since sponsored Start and decided this will be a supported way to use Convex going forward. convex-react-query is part of that, although using it doesn't require using Start. Now that TanStack Start is nearing Beta it's a good time to be getting back to this.

TanStack | High Quality Open-Source Software for Web Developers

Headless, type-safe, powerful utilities for complex workflows like Data Management, Data Visualization, Charts, Tables, and UI Components.

ballingt•6mo ago

I just updated with the current plan for useInfinteQuery, this suggestion is to use usePaginatedQuery from Convex instead. https://github.com/get-convex/convex-react-query/issues/1 I'm interested in what we could do here long term, useInifiniteQuery doesn't quite mesh with Reactive Convex paginated queries today and potentially uniting them needs some more thought.

GitHub

react-query useInfiniteQuery and paginated queries · Issue #1 · get...

Is it possible to use convexQuery with tanstack useInfiniteQuery and a paginated convex query? I have infinite scroll working with the following react-query / convex integration, but it does not su...

ballingt•6mo ago

@sbkl I appreciate your feedback so far, feel free to open more issues (or post here).

sbklOP•6mo ago

Thank you for the insights on this. I am currently using usePaginatedQuery which works great but get no cache on the client so the loading state always comes back when navigating back to a page which doesn't happen with tanstack query useQuery.

ballingt•6mo ago

Cool, yeah makes sense, we know we need something here. @sbkl there's an interesting API design thing here, of course TanStack Query can hold on to stale results but as a rule Convex results aren't stale, they're always up to date and consistent with other queries. So our "cache" is staying subscribed to these queries. Both of these behaviors are useful, stale cached data for an endpoint and live consistent data, so it should be clear what behavior you're getting — ideally this gets figured out before convex-react-query leaves alpha. and then there's pagination, which is extra interesting because usePaginatedQuery in Convex holds the pagination cursors in the hook, not in a global query cache.

sbklOP•6mo ago

One the UX issue I see right now with usePaginatedQuery is that when you navigate from the list to one of the item page and navigate back, you lost the scroll position because the list completely rerenders. So definitely not ideal from this perspective. Unless I am doing something wrong? If there is a workaround on this, would definitely love to hear it.

hasanaktasTR•6mo ago

@sbkl The temporary solution you can use right now is to keep a global instance of the usePaginatedQuery query you used on the page running. In this way, I think you will be able to access the current data immediately when you come back because the websocket connection continues.

ballingt•6mo ago

This is the thing we could move out of the React state tree for this reason (and so you can have two separate hooks that share these queries), but yeah for now the solution is to raise that state up somewhere where it will persist. Currently pagination is probably tied too closely to the React state,when it could exist outside of it. Similar to raising up a useQuery() in the React component tree to keep a query subscription live when using the standard Convex React hooks.

sbklOP•5mo ago

Was looking into valtio Might be a good match to achieve this. Will make a few test and report back @ballingt @hasanaktasTR Landed on this with valtio which keeps the state global and subcription keeping going. Created new items on a item page and there were showing directly when navigating back. Need to make some test on keeping the scroll position. But should be good enough.

"use client";

import * as React from "react";
import { useConvexPaginatedQuery } from "@convex-dev/react-query";
import { proxy, useSnapshot } from "valtio";

import { api } from "../convex/_generated/api";

type DocumentItem =
  (typeof api.documents.query.paginatedList)["_returnType"]["page"][number];

export type UsePaginatedQueryResult<Item> = {
  results: ReadonlyArray<Item>;
  loadMore?: (numItems: number) => void;
} & (
  | {
      status: "LoadingFirstPage";
      isLoading: true;
    }
  | {
      status: "CanLoadMore";
      isLoading: false;
    }
  | {
      status: "LoadingMore";
      isLoading: true;
    }
  | {
      status: "Exhausted";
      isLoading: false;
    }
);

interface ConvexStore {
  documents: UsePaginatedQueryResult<DocumentItem>;
}

const convexStore = proxy<ConvexStore>({
  documents: {
    isLoading: true,
    status: "LoadingFirstPage",
    results: [],
    loadMore: undefined,
  },
});

const ConvexStoreContext = React.createContext(convexStore);

export const ConvexStoreProvider = ({
  children,
}: {
  children: React.ReactNode;
}) => {
  const paginatedDocumentQuery = useConvexPaginatedQuery(
    api.documents.query.paginatedList,
    {},
    { initialNumItems: 24 },
  );
  const state = React.useRef(convexStore).current;

  React.useEffect(() => {
    state.documents = paginatedDocumentQuery;
  }, [paginatedDocumentQuery]);

  return (
    <ConvexStoreContext.Provider value={state}>
      {children}
    </ConvexStoreContext.Provider>
  );
};

export function useConvexStore() {
  const state = React.useContext(ConvexStoreContext);
  const snap = useSnapshot(state);
  return snap;
}

"use client";

import * as React from "react";
import { useConvexPaginatedQuery } from "@convex-dev/react-query";
import { proxy, useSnapshot } from "valtio";

import { api } from "../convex/_generated/api";

type DocumentItem =
  (typeof api.documents.query.paginatedList)["_returnType"]["page"][number];

export type UsePaginatedQueryResult<Item> = {
  results: ReadonlyArray<Item>;
  loadMore?: (numItems: number) => void;
} & (
  | {
      status: "LoadingFirstPage";
      isLoading: true;
    }
  | {
      status: "CanLoadMore";
      isLoading: false;
    }
  | {
      status: "LoadingMore";
      isLoading: true;
    }
  | {
      status: "Exhausted";
      isLoading: false;
    }
);

interface ConvexStore {
  documents: UsePaginatedQueryResult<DocumentItem>;
}

const convexStore = proxy<ConvexStore>({
  documents: {
    isLoading: true,
    status: "LoadingFirstPage",
    results: [],
    loadMore: undefined,
  },
});

const ConvexStoreContext = React.createContext(convexStore);

export const ConvexStoreProvider = ({
  children,
}: {
  children: React.ReactNode;
}) => {
  const paginatedDocumentQuery = useConvexPaginatedQuery(
    api.documents.query.paginatedList,
    {},
    { initialNumItems: 24 },
  );
  const state = React.useRef(convexStore).current;

  React.useEffect(() => {
    state.documents = paginatedDocumentQuery;
  }, [paginatedDocumentQuery]);

  return (
    <ConvexStoreContext.Provider value={state}>
      {children}
    </ConvexStoreContext.Provider>
  );
};

export function useConvexStore() {
  const state = React.useContext(ConvexStoreContext);
  const snap = useSnapshot(state);
  return snap;
}

Then can access the documents query like so:

export function DocumentList() {
  const { documents } = useConvexStore();

// render results and implement the infinite scroll logic
}

export function DocumentList() {
  const { documents } = useConvexStore();

// render results and implement the infinite scroll logic
}

And make sure to wrap the whole thing with the provider.

export default function PageLayout({
  children,
}: {
  children: React.ReactNode;
}) {
  return (
    <ConvexStoreProvider>
        {children}
    </ConvexStoreProvider>
  );
}

export default function PageLayout({
  children,
}: {
  children: React.ReactNode;
}) {
  return (
    <ConvexStoreProvider>
        {children}
    </ConvexStoreProvider>
  );
}

Actually the scroll position is kept without additional effort and it works like a charm! Also tested the loadMore methods and it works great. Thank you for the suggestion

ballingt•5mo ago

Beautiful!

sbklOP•3d ago

@ballingt Gave up on that solution. Works well for a single simple paginated query, but for more complex queries with filters, I quickly realised I need a queryKey mechanism just like tanstack query and was finding myself trying to recreate the way the cache works with tanstack query lol. So now I am able to use useInfiniteQuery hook from tanstack query like so:

export const list = protectedQuery({
  args: {
    paginationOptions: v.object({
      cursor: v.union(v.string(), v.null()),
      numItems: v.optional(v.number()),
    }),
  },
  handler: async (ctx, { paginationOptions: { cursor, numItems } }) => {
    const user = await getCurrentUserOrThrow(ctx);
    const results = await ctx.db
      .query("documents")
      .withIndex("userId", (q) => q.eq("userId", user._id))
      .order("desc")
      .paginate({
        cursor,
        numItems: numItems ?? 24,
      });

    const { page, isDone, continueCursor } = results;

    return {
      items: page,
      nextCursor: isDone ? undefined : continueCursor,
    };
  },
});

export const list = protectedQuery({
  args: {
    paginationOptions: v.object({
      cursor: v.union(v.string(), v.null()),
      numItems: v.optional(v.number()),
    }),
  },
  handler: async (ctx, { paginationOptions: { cursor, numItems } }) => {
    const user = await getCurrentUserOrThrow(ctx);
    const results = await ctx.db
      .query("documents")
      .withIndex("userId", (q) => q.eq("userId", user._id))
      .order("desc")
      .paginate({
        cursor,
        numItems: numItems ?? 24,
      });

    const { page, isDone, continueCursor } = results;

    return {
      items: page,
      nextCursor: isDone ? undefined : continueCursor,
    };
  },
});

Then:

export function useDocumentInfiniteQuery() {
  const convex = useConvex();

  const infiniteQuery = useInfiniteQuery({
    queryKey: ["documents"],
    async queryFn({ pageParam }: { pageParam: string | null }) {
      const result = await convex.query(api.documents.query.list, {
        paginationOptions: {
          cursor: pageParam,
        },
      });
      return result;
    },
    getNextPageParam: (lastPage) => lastPage.nextCursor,
    initialPageParam: null,
  });

  return {
    ...infiniteQuery,
    items: infiniteQuery.data?.pages.flatMap((page) => page.items) ?? [],
  };
}

export function useDocumentInfiniteQuery() {
  const convex = useConvex();

  const infiniteQuery = useInfiniteQuery({
    queryKey: ["documents"],
    async queryFn({ pageParam }: { pageParam: string | null }) {
      const result = await convex.query(api.documents.query.list, {
        paginationOptions: {
          cursor: pageParam,
        },
      });
      return result;
    },
    getNextPageParam: (lastPage) => lastPage.nextCursor,
    initialPageParam: null,
  });

  return {
    ...infiniteQuery,
    items: infiniteQuery.data?.pages.flatMap((page) => page.items) ?? [],
  };
}

So from a pagination and cache point of view, I get everything out of the box from tanstack query and the paginate method in the convex backend and it is working properly BUT I obviously lost the reactivity. What would the way to subscribe to changes on a specific table and refetch with tanstack query? Was thinking to create a pagination table with a documentType, documentCount, create a record each time there is an update on a specific table (add, update, delete) and have a useQuery convex query listening for changes and use refetch each there is a change. I know all that is really not what convex is trying to achieve but that's all I can come up with to get pagination cached and working as expected while trying to still keep reactivity... or I can simply use tanstack query to invalidate the queries related for each mutation which seems more natural since I have been using tanstack query but not really the convex way. Ok I finally made it. The valtio store manages multiple paginated queries and query keys for a given query based on args. Placeholder data are there out of the box when changing query params. use-infinite-query.ts

"use client";

import type {
  PaginatedQueryArgs,
  PaginatedQueryReference,
  UsePaginatedQueryReturnType,
} from "convex/react";
import * as React from "react";
import { usePaginatedQuery } from "convex/react";
import { getFunctionName } from "convex/server";
import { proxy, useSnapshot } from "valtio";

type InfiniteQueryStore<Query extends PaginatedQueryReference> = Record<
  string,
  {
    queries: Record<string, UsePaginatedQueryReturnType<Query>>;
    lastKey: string | null;
  }
>;

const store = proxy<InfiniteQueryStore<any>>({});

export function useInfiniteQuery<Query extends PaginatedQueryReference>(
  query: Query,
  args: PaginatedQueryArgs<Query>,
  options: { initialNumItems: number },
) {
  const queryName = getFunctionName(query);
  const queryKey = JSON.stringify(args);
  const paginatedQuery = usePaginatedQuery(query, args, options);
  const state = React.useRef(store).current;
  const snap = useSnapshot(state);
  React.useEffect(() => {
    if (
      paginatedQuery.status !== "LoadingFirstPage" &&
      paginatedQuery.status !== "LoadingMore"
    ) {
      state[queryName] = {
        queries: state[queryName]
          ? {
              ...state[queryName].queries,
              [queryKey]: paginatedQuery,
            }
          : {
              [queryKey]: paginatedQuery,
            },
        lastKey: queryKey,
      };
    }
  }, [paginatedQuery, queryKey, queryName]);
  return (snap[queryName]?.queries[queryKey] ??
    (snap[queryName]?.lastKey
      ? snap[queryName]?.queries[snap[queryName]?.lastKey as string]
      : {
          results: [],
          status: "LoadingFirstPage",
          isLoading: true,
          loadMore: (_numItems: number) => {},
        })) as UsePaginatedQueryReturnType<Query>;
}

"use client";

import type {
  PaginatedQueryArgs,
  PaginatedQueryReference,
  UsePaginatedQueryReturnType,
} from "convex/react";
import * as React from "react";
import { usePaginatedQuery } from "convex/react";
import { getFunctionName } from "convex/server";
import { proxy, useSnapshot } from "valtio";

type InfiniteQueryStore<Query extends PaginatedQueryReference> = Record<
  string,
  {
    queries: Record<string, UsePaginatedQueryReturnType<Query>>;
    lastKey: string | null;
  }
>;

const store = proxy<InfiniteQueryStore<any>>({});

export function useInfiniteQuery<Query extends PaginatedQueryReference>(
  query: Query,
  args: PaginatedQueryArgs<Query>,
  options: { initialNumItems: number },
) {
  const queryName = getFunctionName(query);
  const queryKey = JSON.stringify(args);
  const paginatedQuery = usePaginatedQuery(query, args, options);
  const state = React.useRef(store).current;
  const snap = useSnapshot(state);
  React.useEffect(() => {
    if (
      paginatedQuery.status !== "LoadingFirstPage" &&
      paginatedQuery.status !== "LoadingMore"
    ) {
      state[queryName] = {
        queries: state[queryName]
          ? {
              ...state[queryName].queries,
              [queryKey]: paginatedQuery,
            }
          : {
              [queryKey]: paginatedQuery,
            },
        lastKey: queryKey,
      };
    }
  }, [paginatedQuery, queryKey, queryName]);
  return (snap[queryName]?.queries[queryKey] ??
    (snap[queryName]?.lastKey
      ? snap[queryName]?.queries[snap[queryName]?.lastKey as string]
      : {
          results: [],
          status: "LoadingFirstPage",
          isLoading: true,
          loadMore: (_numItems: number) => {},
        })) as UsePaginatedQueryReturnType<Query>;
}

Usage:

  const { results, status, loadMore } = useInfiniteQuery(
    // change with your paginated convex query path
    api.myPaginatedQuery,
    // Put the related args of your query excluding the paginationOptions
    args,
    { initialNumItems: 24 },
  );

  const { results, status, loadMore } = useInfiniteQuery(
    // change with your paginated convex query path
    api.myPaginatedQuery,
    // Put the related args of your query excluding the paginationOptions
    args,
    { initialNumItems: 24 },
  );

Not perfect yet as it doesn't account various args types. Like array of string that can hold same values but in different order but the core mechanics works. Types can be improved too.

convex-react-query state and plan on useInfiniteQuery hook support

Did you find this page helpful?