Chad Maycumber•3y ago

GEO JSON queries

I assume this isn’t currently supported. This would be a great feature for apps that heavily leverage location! In the meantime, would a supported architecture look something like a sync to elastic search where it is supported?

24 Replies

ian•3y ago

I would use Uber’s H6 library and do indexed queries on the various granularities, stored in Convex. I haven’t done it yet but it should work just fine. What kind of query in particular are you thinking?

Chad MaycumberOP•3y ago

I think this would work! We're working with locations around a particular users within a radius

ian•3y ago

Yeah, I've become more a fan of H6 H3 over DB-specific features, since generally you can just over-fetch hexagons, and it's a more predictable query index. Lmk how it works for you

Coffee11•15mo ago

@Chad Maycumber did you guys make this work? how was it?

Gerald•15mo ago

do you have code snippets?

Tiger 🐅•14mo ago

same I NEED CODE PLZ 😂

v•14mo ago

Is it h6 or h3

ian•14mo ago

- A repo doing this with H3 directly: https://github.com/sujayakar/geospatial-convex/ - We have started work on a geospatial component that will provide an API to insert locations & a user-defined key, then query for within a polygon. Under the hood it also uses H3, which I increasingly see as the right API for most of these sorts of queries.

GitHub

GitHub - sujayakar/geospatial-convex

Contribute to sujayakar/geospatial-convex development by creating an account on GitHub.

ian•14mo ago

whoops edited it - thanks for catching that

Tiger 🐅•14mo ago

h - good luck figuring it out :tenkaSmug:

v•14mo ago

Sum like that

Tiger 🐅•14mo ago

ok it might be a skill issue and a learning curve but man that code is complex ill use convex for another day :Sweat: literally sweating like a dog ok maybe thats not bc of the code but rather bc its super hot here in germany 💀 ill be honest

v•14mo ago

Just say fuck it and try anyways. Can't learn if ur scared of it right Worst thing that can happen is you give up Not that bad. The code doesn't seem very complex, maybe it's worth it to give it a shot anyways

ian•14mo ago

Here's what I would do for the simple version: 1. Pick a single granularity (H3 level) that you care about - you can look around but usually you want something that covers a few city blocks, unless you're showing a full country of data 2. When you're inserting something into the database, use the H3 library to turn your lat/long into a string at that level (the string encodes what "cell" it's in) 3. When you're querying for things, find all the cells you care about for that level - there are H3 functions for this. Do an indexed lookup for each cell - querying your table where the cell string matches exactly. If you want multiple layers of granularity without the gnarly math to do a range query, just have multiple fields that each denote the cell ID at that level of granularity. Then do a query at the level you care about

Tiger 🐅•14mo ago

im building something to get users mongodb/monogoose got me covered already its funny, ive not used it for like decades now never thought i'd use it again, but here we are i got my next project after this one, not involving geospatial stuff ill use convex for that project oh no, i still need real-time stuff for some pages shit yeah maybe i should just go this way with convex

ian•14mo ago

You can also do this with Convex doing basic queries - just do an indexed query on a longitude range, then filter that down based on a latitude range. For an MVP or app that doesn't get millions of users, you'll be fine.

Tiger 🐅•14mo ago

true i might not even get users also

ian•14mo ago

It turns out longitude does a pretty good job of segmenting. S. America is pretty far east of the US, the dense part of Australia is east of China & Japan, etc.

Tiger 🐅•14mo ago

altho, i could use polling here actually 🤔 and still go with mongodb/mongoose let me do some research into lng and lat without that h3 lib cuz that github repo seems to overcomplicate things

ian•14mo ago

Another trick is to also store the rounded longitude, then do an indexed query doing an equality on longitude and a tight range on latitude, then do a tighter bound on longitude Yeah I'd say doing vanilla queries should get you to your first 100,000 users, at which point we'll have a component for that 🤘

Tiger 🐅•14mo ago

:kumikosmug: if i get any users hahaha ty for all the help @ian ok i think i figured someth out

// app/routes/products.tsx
import { json, LoaderFunction } from "@remix-run/node";
import { useLoaderData } from "@remix-run/react";
import { getProducts } from "~/convex/getProducts";

const calculateDistance = (lat1: number, lon1: number, lat2: number, lon2: number) => {
  const earthRadius = 6371000; // Earth's radius in meters
  const latDifference = (lat2 - lat1) * (Math.PI / 180); // Convert latitude difference to radians
  const lonDifference = (lon2 - lon1) * (Math.PI / 180); // Convert longitude difference to radians
  
  const haversine = 
    Math.sin(latDifference / 2) * Math.sin(latDifference / 2) +
    Math.cos(lat1 * (Math.PI / 180)) * Math.cos(lat2 * (Math.PI / 180)) *
    Math.sin(lonDifference / 2) * Math.sin(lonDifference / 2);
  
  const arcDistance = 2 * Math.atan2(Math.sqrt(haversine), Math.sqrt(1 - haversine));
  const distance = earthRadius * arcDistance; // Distance in meters
  return distance;
};

export const loader: LoaderFunction = async ({ request }) => {
  const url = new URL(request.url);
  const lat = parseFloat(url.searchParams.get("lat") || "0");
  const lon = parseFloat(url.searchParams.get("lon") || "0");
  const radius = parseFloat(url.searchParams.get("radius") || "0");
  const search = url.searchParams.get("search") || "";

  const products = await getProducts();

  const filteredProducts = products.filter((product) => {
    const distance = calculateDistance(lat, lon, product.latitude, product.longitude);
    const nameMatch = product.name.toLowerCase().includes(search.toLowerCase());
    return distance <= radius && nameMatch;
  });

  return json(filteredProducts);
};

export default function Products() {
  const products = useLoaderData<typeof loader>();

  return (
    <div>
      <h1>Products within Radius</h1>
      <ul>
        {products.map((product) => (
          <li key={product._id}>{product.name}</li>
        ))}
      </ul>
    </div>
  );
}

// app/routes/products.tsx
import { json, LoaderFunction } from "@remix-run/node";
import { useLoaderData } from "@remix-run/react";
import { getProducts } from "~/convex/getProducts";

const calculateDistance = (lat1: number, lon1: number, lat2: number, lon2: number) => {
  const earthRadius = 6371000; // Earth's radius in meters
  const latDifference = (lat2 - lat1) * (Math.PI / 180); // Convert latitude difference to radians
  const lonDifference = (lon2 - lon1) * (Math.PI / 180); // Convert longitude difference to radians
  
  const haversine = 
    Math.sin(latDifference / 2) * Math.sin(latDifference / 2) +
    Math.cos(lat1 * (Math.PI / 180)) * Math.cos(lat2 * (Math.PI / 180)) *
    Math.sin(lonDifference / 2) * Math.sin(lonDifference / 2);
  
  const arcDistance = 2 * Math.atan2(Math.sqrt(haversine), Math.sqrt(1 - haversine));
  const distance = earthRadius * arcDistance; // Distance in meters
  return distance;
};

export const loader: LoaderFunction = async ({ request }) => {
  const url = new URL(request.url);
  const lat = parseFloat(url.searchParams.get("lat") || "0");
  const lon = parseFloat(url.searchParams.get("lon") || "0");
  const radius = parseFloat(url.searchParams.get("radius") || "0");
  const search = url.searchParams.get("search") || "";

  const products = await getProducts();

  const filteredProducts = products.filter((product) => {
    const distance = calculateDistance(lat, lon, product.latitude, product.longitude);
    const nameMatch = product.name.toLowerCase().includes(search.toLowerCase());
    return distance <= radius && nameMatch;
  });

  return json(filteredProducts);
};

export default function Products() {
  const products = useLoaderData<typeof loader>();

  return (
    <div>
      <h1>Products within Radius</h1>
      <ul>
        {products.map((product) => (
          <li key={product._id}>{product.name}</li>
        ))}
      </ul>
    </div>
  );
}

` this can work for first iteration eventually if i do get users i can deal with proper fuzzy string search and improve the radius calc and stuff

ian•11mo ago

Here's a component released last week that provides geospatial search 🪄 https://www.convex.dev/components/geospatial

vini•8mo ago

I really really appreciate this and will try it out today in a side project. I've been really enjoying using convex and been convincing some friends and clients to try it. Thanks a lot!!

MrDSA•3d ago

Hey this is my code and i'd wanna use search Index after doing the geospatial query, this is my function now, can you please show me how to do that?

export const searchBookswithLocation = query({
  args: {
    searchTerm: v.string(),
    latitude: v.number(),
    longitude: v.number(),
  },
  handler: async (ctx, args) => {
    const maxResults = 16;
    const maxDistance = 10000;
    const result = await geospatial.queryNearest(
      ctx,
      { latitude: args.latitude, longitude: args.longitude },
      maxResults,
      maxDistance,
    )
    ;
    return result;
  },
});

export const searchBookswithLocation = query({
  args: {
    searchTerm: v.string(),
    latitude: v.number(),
    longitude: v.number(),
  },
  handler: async (ctx, args) => {
    const maxResults = 16;
    const maxDistance = 10000;
    const result = await geospatial.queryNearest(
      ctx,
      { latitude: args.latitude, longitude: args.longitude },
      maxResults,
      maxDistance,
    )
    ;
    return result;
  },
});

GEO JSON queries

Did you find this page helpful?