cyremur•2y ago

How to understand database bandwidth

So in my use case database bandwidth is likely going to be the most crucial factor. However, I'm unsure what exactly would be considered database bandwidth? Updating data via function? Most certainly Reading data in a function? Probably Subscription triggers of a useQuery on changed data? No idea

15 Replies

Indy•2y ago

The simplest way to think about this is everytime a db.blah is actually called it counts towards database bandwidth So a re-trigger of query function via a useQuery react hook because the underlying data has changed will count.

jamwt•2y ago

One complexity is: not if it’s a cache hit. So if your query is cached, there’s no need for us to talk to the database, so you’ll have no db bandwidth.

cyremurOP•2y ago

So I assume read, update, and broadcast to all subscribers counts as 3x the touched data?

jamwt•2y ago

yes, because only one subscriber will re-run the query. the rest will just pull the cached result generally speaking, if you're working with fairly small records (1-2kb), database bandwidth is probably a non-factor for you. that's the intention. if you're working with large records though (10s of KB), then this may be a factor worth paying attention to @ballingt do you want to share your calculator you put together?

cyremurOP•2y ago

So I like convex as an idea but for my use case, I have 25kb*3 on every game Action. Let's say I optimize that and go down to some 10kb gamestate, that's still a lot of data for every game Action update. With some like 100 actions per game I'm still at like 3mb per game. So unless I really optimize around the way convex handles data, I think I need to roll my own server at some point? Sorry for taking this general question to a specific use case tangent

jamwt•2y ago

no problem. tom is updating this calculator he made yesterday to help answer this question, as it's a common and reasonable one it will both tell you how many games you could run on the free plan, and how many games would run on the pro plan ($25/mo) without using any metered pricing per month it sounds like your goal is to run a certain expectation of games per month on the free plan?

ballingt•2y ago

Here's something pretty rough, hopefully helpful for the conversation @cyremur https://observablehq.com/@ballingt/convex-game-cost-estimator-draft-only-function-executions-and-db-bandwidth

Observable

Convex game cost estimator DRAFT

Say I've got a multiplayer game app. Each game has x players and takes y moves. Each time a player makes a move in a game that's one mutation plus the query function running once per player. The queries might be cached if they return the full game state. But if there's hidden information different for each player, the queries might need to be ex...

jamwt•2y ago

thanks @ballingt so looks like on the free plan, cyremur, you could do 480 games per month (16 per day) within the built-in db bandwith limits more than 22,000 games per month on pro plan each additional 1,000 games per month would cost about 50 cents

cyremurOP•2y ago

well I certainly love the team behind convex, thanks for this top-class supportt It's a really interesting question and I'm honestly not sure. I had some ideas towards monetization in the long run but on the other hand fronting even like $50/months in freeplay alpha for a while seems ok. I was just hoping that this is a future me issue, but on the other hand I'd also like the game to be popular enough that it's more a soon me issue. One of my big goals is to make this a "fair" CCG where you only straight up pay for boosters and don't have intrusive freemium daily quest addiction whale milking bs or an additional subscription to play with the cards you "own". So I was hoping to monetize via selling cards but give basically unlimited access for trading and playing, and for that model the server costs to support gameplay feel like they should be as low as possible. To go further off-topic, since the gamestate is well defined and everything is designed with Query/Mutation model, I think convex has really good on- and offboarding story for me. Plus it gives me server(less)-side game authority and hidden information (hand cards) that are lacking in the current implementation, which will definitely come in handy towards a more public alpha/beta stage. So it looks like convex is a really good choice to build on for the early stages of the game, and I will have to re-evaluate rolling my own backend server infra if and when the game gets big enough that it matters.

Indy•2y ago

Do talk to us if you start considering rolling your own backend. We'd love to understand the setup at the time. Good luck with the game! Looking forward to seeing how it does 🙂

ian•2y ago

I could imagine decomposing the game state into smaller chunks would allow you to read and write less data, unless you actually need all of that data at every stage. Storing materialized normalized state into documents could reduce it further. One goal of Convex is to be so useful and time saving and powerful that you're willing to pay for the time and effort and maintenance it saves you. If there's something that we do (or don't do) that is worth paying for, let us know 🙏 .

cyremurOP•2y ago

I am considering that a lot but I'm afraid not having access to all gamestate easily will slow down prototyping velocity so it's definitely a "later" type optimization. Actually one relevant-ish question for that is: does it matter if I put the whole state in one column or if I split it up into multiple columns? Like if I mutate "drawCard" and have deckOfCards and handCards in separate columns, does that mean the websocket subscription can do a partial update instead of sending a full row? Or is that too much dream magic? Also one optimization I've been thinking about is a minification layer. Defeats the purposes of full-stack typesafety and readability but I could save some bytes if I only save single letters instead of "gameStarted" or "waitingOpponent", right?

ian•2y ago

the websocket subscription is a function of what you return from your query. When you fetch a document from the DB it grabs all fields (columns). Whatever you return goes over the websocket. So your query could reshape the data and just return a subset. I agree with the prototype velocity angle. I am speaking more to your longer term price math, where you won't need to pass the whole object. I personally wouldn't minifiy. You could make enums with number -> string mappings, and possibly have those number constants still have typesafety, but I haven't dealt with enums much in TS

cyremurOP•2y ago

I actually did code card "Properties" as enums and considered them like String constants while coding and turned out my convex port automatically made them into numbers, that was cool. So I wrote a minification layer to deal with the most egregious waste of DB space...

const convertStateForDB = (state: GameState) => {
  const decks = state.decks.map((deck) => {
    return { ...deck, cards: deck.cards.map((card) => card.id) };
  });
  const hands = state.hands.map((hand) => hand.map((card) => card.id));
  const entities = state.entities.map((entity) => {
    return { ...entity, source: entity.source.id };
  });
  return { ...state, decks, hands, entities };
};

const convertStateForUserSpace = (state: DBGameState) => {
  const decks = state.decks.map((deck) => {
    return {
      ...deck,
      cards: deck.cards.map(
        (card) => allCards[card] ?? raise(`Card with Id ${card} doesn't exist.`)
      ),
    };
  });
  const hands = state.hands.map((hand) =>
    hand.map(
      (card) => allCards[card] ?? raise(`Card with Id ${card} doesn't exist.`)
    )
  );
  const entities = state.entities.map((entity) => {
    return {
      ...entity,
      source:
        allCards[entity.source] ??
        raise(`Card with Id ${entity.source} doesn't exist.`),
    };
  });
  return { ...state, decks, hands, entities };
};

const convertStateForDB = (state: GameState) => {
  const decks = state.decks.map((deck) => {
    return { ...deck, cards: deck.cards.map((card) => card.id) };
  });
  const hands = state.hands.map((hand) => hand.map((card) => card.id));
  const entities = state.entities.map((entity) => {
    return { ...entity, source: entity.source.id };
  });
  return { ...state, decks, hands, entities };
};

const convertStateForUserSpace = (state: DBGameState) => {
  const decks = state.decks.map((deck) => {
    return {
      ...deck,
      cards: deck.cards.map(
        (card) => allCards[card] ?? raise(`Card with Id ${card} doesn't exist.`)
      ),
    };
  });
  const hands = state.hands.map((hand) =>
    hand.map(
      (card) => allCards[card] ?? raise(`Card with Id ${card} doesn't exist.`)
    )
  );
  const entities = state.entities.map((entity) => {
    return {
      ...entity,
      source:
        allCards[entity.source] ??
        raise(`Card with Id ${entity.source} doesn't exist.`),
    };
  });
  return { ...state, decks, hands, entities };
};

I still feel like there's a clash between full typesafety and traffic efficiency Like I'm itching to write so much boilerplate right now to take this:

  controller: 1,
  currentAction: 1,
  currentMove: 3,
  currentPower: 2,
  currentToughness: -3,
  hexId: 64,
  isDead: true,
  source: 7,

  controller: 1,
  currentAction: 1,
  currentMove: 3,
  currentPower: 2,
  currentToughness: -3,
  hexId: 64,
  isDead: true,
  source: 7,

to this:

  a: 1,
  b: 1,
  c: 3,
  d: 2,
  e: -3,
  f: 64,
  g: true,
  h: 7,

  a: 1,
  b: 1,
  c: 3,
  d: 2,
  e: -3,
  f: 64,
  g: true,
  h: 7,

to save like 70 extra bytes, cause with 10+ creatures on the field stupid stuff like this can easily shave off another kB but then data in dashboard wouldn't be legible anymore whatsoever... that's why it would be super cool to have a minification layer as middleware so that the "decryption" or "de-minification" can run between DB and dashboard as well

rebecca•2y ago

One thing to note is that database bandwidth and database storage get rounded up to the nearest KB per document so it's probably not worth optimizing at the bytes level.