Rayy
Rayy•15mo ago

What is the best approach to call query when they depend on scheduled functions?

I am calling useMutation from client where it inserts data in the db for that particular chat Id, and then run certain scheduled functions based on the data, and I am returning the chat Id to the client when the data has been inserted to the db. My application is working fine, but I feel like I am making unnecessary query requests, when all the scheduled functions have not yet been completed. I am currently showing a loading state to the user till the embeddingId is undeifned, since the last scheduled function will populate the embeddingId. 
export const useQueryChatProps = ({ chatId }: { chatId: Id<"chatbook"> }) => {
  const data = useQuery(api.chatbook.getEmbeddingId, {
    chatId,
  });

  const loading = data === undefined || data?.embeddingId === undefined;

  return {
    data,
    loading,
  };
};
export const useQueryChatProps = ({ chatId }: { chatId: Id<"chatbook"> }) => {
  const data = useQuery(api.chatbook.getEmbeddingId, {
    chatId,
  });

  const loading = data === undefined || data?.embeddingId === undefined;

  return {
    data,
    loading,
  };
};
Is this okay? Or is there a better way to achieve this? Here is my code for inserting data to the db and calling the scheuled functions 
   const user = await ctx.db
      .query("users")
      .filter((q) => q.eq(q.field("_id"), args.userId));
    if (!user) {
      throw new ConvexError("You need to login first.");
    }

    const url = args.storageId
      ? await ctx.storage.getUrl(args.storageId)
      : args.url;

    if (!url) {
      throw new ConvexError("No URL found.");
    }

    const chatId = await ctx.db.insert("chatbook", {
      userId: args.userId,
      url,
    });

      await ctx.scheduler.runAfter(0, internal.github.getFilesFromRepo, {
        repoUrl: url,
        filePath: extractPathFromUrl(url),
        chatId,
      });

    return chatId;
   const user = await ctx.db
      .query("users")
      .filter((q) => q.eq(q.field("_id"), args.userId));
    if (!user) {
      throw new ConvexError("You need to login first.");
    }

    const url = args.storageId
      ? await ctx.storage.getUrl(args.storageId)
      : args.url;

    if (!url) {
      throw new ConvexError("No URL found.");
    }

    const chatId = await ctx.db.insert("chatbook", {
      userId: args.userId,
      url,
    });

      await ctx.scheduler.runAfter(0, internal.github.getFilesFromRepo, {
        repoUrl: url,
        filePath: extractPathFromUrl(url),
        chatId,
      });

    return chatId;
The scheduled function is dependant on other scheduled functions.
8 Replies
Michal Srb
Michal Srb•15mo ago
If you don't need to show the chat until the embedding is there you can return null on the backend: 
export const getEmbeddingId = query({
  args: {chatId: v.id("chatbook")},
  handler: async (ctx, {chatId}) => {
    const chat = ctx.get(chatId);
    if (chat?.embeddingId === undefined) {
      return null;
    }
   },
});
export const getEmbeddingId = query({
  args: {chatId: v.id("chatbook")},
  handler: async (ctx, {chatId}) => {
    const chat = ctx.get(chatId);
    if (chat?.embeddingId === undefined) {
      return null;
    }
   },
});
The query will rerun when the chatbook document changes, but it won't be pushed to the client if the result is the same (null in my example). Does that help?
Rayy
RayyOP•15mo ago
Yeah I get that, but won't the query still be called multiple times till it has an embeddingId? Or is it okay for it to be like that?
Michal Srb
Michal Srb•15mo ago
I would say this is OK. You could avoid it if your scheduled functions didn't touch the data getEmbeddingId relies on. You could have a separate document on which the scheduled functions "work" (ie write progress to), and then when they are done they write to the "chatbook" document only.
Rayy
RayyOP•15mo ago
Ah, yes. That would work. I'll look into that or just use this. Thanks.
Michal Srb
Michal Srb•14mo ago
@Rayy for the approach: Say getEmbeddingId reads from "chatbook" table You could have scheduled functions which work on a document in "chatbookInProgress" table The "chatbookInProgress" document can include v.id("chatbook"). The scheduled functions can read from "chatbook", and write to "chatbookInProgress". When your offline processing is done, your final scheduled function can write into "chatbook". Only then getEmbeddingId will recompute.
Rayy
RayyOP•14mo ago
I was able to implement it, and now my getEmbeddingId query will be called only after it has all the embeddings. Thanks about that @Michal Srb . I just had one more query, I don't want to call a query, everytime a component re-renders. I do not want to use One-off queries, since I would want to listen to the changes in the query and if any, then update the document accordingly. Whenever my Chatbot re-renders, my query is being called which I want to avoid. I do not want to add memoization to the Chatbot component either. Is there any other way to achieve this? 
const useQueryEmbeddingProps = ({
  chatId,
}: {
  chatId: Id<"chatbook">;
}) => {
  const data = useQuery(api.embedding.getEmbeddingId, { chatId });

  const loading = data === undefined;

  return {
    data,
    loading,
  };
};

const ChatId = ({ params }: { params: { chatId: string } }) => {
  const chatId = params.chatId as Id<"chatbook">;
  const { isLoading, isAuthenticated } = useConvexAuth();
  const chat = useQueryEmbeddingProps({ chatId });

  return (
    <div className="h-[calc(100vh-56px)] mt-14 font-sans flex">
      <div className="h-full w-[60%] border-r border-r-border">
        {chat?.data?.type === "code" && (
          <CodeViewer url={chat?.data?.url} chatId={chat?.data?._id} />
        )}
        {chat?.data?.type === "video" && <VideoViewer url={chat?.data?.url} />}
        {chat?.data?.type === "doc" && <PDFViewer url={chat?.data?.url!} />}
      </div>
      <div className="flex w-[40%] justify-between flex-col h-full min-w-[450px]">
        <ChatBot
          chatId={chat?.data?._id!}
          title={chat?.data?.title!}
          type={chat?.data?.type ?? "doc"}
        />
      </div>
    </div>
  );
};
const useQueryEmbeddingProps = ({
  chatId,
}: {
  chatId: Id<"chatbook">;
}) => {
  const data = useQuery(api.embedding.getEmbeddingId, { chatId });

  const loading = data === undefined;

  return {
    data,
    loading,
  };
};

const ChatId = ({ params }: { params: { chatId: string } }) => {
  const chatId = params.chatId as Id<"chatbook">;
  const { isLoading, isAuthenticated } = useConvexAuth();
  const chat = useQueryEmbeddingProps({ chatId });

  return (
    <div className="h-[calc(100vh-56px)] mt-14 font-sans flex">
      <div className="h-full w-[60%] border-r border-r-border">
        {chat?.data?.type === "code" && (
          <CodeViewer url={chat?.data?.url} chatId={chat?.data?._id} />
        )}
        {chat?.data?.type === "video" && <VideoViewer url={chat?.data?.url} />}
        {chat?.data?.type === "doc" && <PDFViewer url={chat?.data?.url!} />}
      </div>
      <div className="flex w-[40%] justify-between flex-col h-full min-w-[450px]">
        <ChatBot
          chatId={chat?.data?._id!}
          title={chat?.data?.title!}
          type={chat?.data?.type ?? "doc"}
        />
      </div>
    </div>
  );
};
Michal Srb
Michal Srb•14mo ago
Whenever my Chatbot re-renders, my query is being called which I want to avoid.
This is not the case. The query on the server only runs when the data it depends on changes. The React client caches the result. useQuery doesn't force the query to run again.
Rayy
RayyOP•14mo ago
I might be doing something wrong then, I’ll look into it. Hey @Michal Srb , I figured the error, I was updating the conversation in the "chatbook" table itself, making it call the embeddingId every time a message is patched to the database. So, in order to fix that, I created a new "conversations" table which has the messages filed, along with the chatId. When I insert the conversationId in the respective chatId in the "chatbook" table, the embeddingId is still called everytime there is a change in the "conversations" table. But without inserting the conversationId, the embeddingId is not called every time there is a change in the "conversations" table. But I would like that conversationId in the "chatbook" table. One way I found this to work is to pass the conversationId as a string. I was wondering if there are any better ways? It was becoming a real pain, so I just refactored my schema. Now the “chatbook” table only depends on userId. And everything looks good. 🙂

Did you find this page helpful?