esselitow•3h ago

withSearchIndex multiword not working

I cannot get this to work, I've tried reading the docs, using the AI and also reading up on Tantivy's SimpleTokenizer If I have a searchIndex on a field that I have specifically created for search, if I use a space I get no results... So my index:

  orders: defineTable(orderSchema)
    .searchIndex("searchText", {
      searchField: "searchText",
    })

  orders: defineTable(orderSchema)
    .searchIndex("searchText", {
      searchField: "searchText",
    })

my search example:

.withSearchIndex("searchText", (q) =>
        q.search("searchText", "17992 doe")
      );

.withSearchIndex("searchText", (q) =>
        q.search("searchText", "17992 doe")
      );

one of my orders has a searchText value of:

17992 john doe

17992 john doe

Searching for each individually works (17992 or john or doe) and searching in the correct order works, like 17992 john or john doe but not when searching for multiple words that are not in the correct order, but I thought that each space would split and tokenize the words making them searchable as the query in is also tokenized the same so they should match? It makes the search much less useful and very unreliable, so I assume I am doing something wrong FYI This happens in Development and Production, on Free and Pro tier

3 Replies

Convex Bot•3h ago

Thanks for posting in <#1088161997662724167>. Reminder: If you have a Convex Pro account, use the Convex Dashboard to file support tickets. - Provide context: What are you trying to achieve, what is the end-user interaction, what are you seeing? (full error message, command output, etc.) - Use search.convex.dev to search Docs, Stack, and Discord all at once. - Additionally, you can post your questions in the Convex Community's <#1228095053885476985> channel to receive a response from AI. - Avoid tagging staff unless specifically instructed. Thank you!

Clever Tagline•2h ago

I ran into some issues when I first used search indices, and the response that I got from Convex support was that "the keyword matching only performs prefix matching for typeahead search". That's why it works when the search text exactly matches part of the saved text (including spaces), but not when searching with words in a different order. You might be able to set up something using query streams where you tokenize the search string yourself, perform one query per token, then merge the streams.

Merging Streams of Convex data

New convex-helpers are available now for fetching streams of documents, merging them together, filtering them them out, and paginating the results. Wi...

esselitowOP•2h ago

that is very unfortunate but thank you for the docs I think this can solve my problem 🙂 stream does not support withSearchIndex though :/

withSearchIndex multiword not working

Did you find this page helpful?