Syncing with Elasticsearch or any search engine
Hi. Is there any guide or blog of how to sync Convex data with Elasticsearch?
I have being developing an Ecommerce App to learn Convex and I would like to add some analytics and facet filters.
For example:
Most sell items on a date range.
Orders with more items.
Most active users per country.
I have being using raw JavaScript with some indexes for the queries, but some of them need "Join-Like".
I know there is an Airbyte Connector but I'm not very familiar with Airbyte.
9 Replies
How do you want to connect it? Convex has a search feature. https://docs.convex.dev/text-search
Full Text Search | Convex Developer Hub
Run search queries over your Convex documents
(in case your use case isn't supported by full text search)
i don't think we wrote up a walkthrough of adding airbyte or fivetran syncing to elasticsearch, but it is possible. We should make one, good idea. The only docs i know of are here https://docs.airbyte.com/integrations/destinations/convex#setup-guide
Convex | Airbyte Documentation
This page contains the setup guide and reference information for the Convex destination connector.
I'm currently working around some indexes and the full text search, but I want to add some more complex aggregations and I understand that Convex isn't optimized for that
@Gary, el Pingüino Artefacto can you say more about the complex aggregations?
meaning, you want to search over aggregations? or just created aggregated values in general based on your records?
search over aggregation
for example: orders with more than 5 items sell
only items sell during weekends
idk
i'm just playing to see how far i can get with just convex
@Gary, el Pingüino Artefacto from your initial question it sounded like you might want this for offline analytics, not as part of your product?
If it's for offline analytics then Airbyte or Fivetran to, say, Databricks, or some other analytics DB, is probably what you need
Yes, Im trying to implement real time analytics. Im working on this in my free time, I will clean the code and show you what Im trying to do
Basically, Im trying to implement analytics with some filters, prefer on realtime
Beware that there is no way to prioritize some function calls over other ones, and so your analytics could negatively affect your product performance (but this sounds more like a learning exercise).
It is a learning exercise, not a real product