sbkl
sbkl•5d ago

streaming api upsert for json fields send bad request response

I've got my streaming api import working on first upload. But not working on subsequent upload for the exact same data. My table schema got a json field of type: 
  allocation: v.record(
    v.string(),
    v.record(v.string(), v.union(v.number(), v.literal("x"))),
  )
  allocation: v.record(
    v.string(),
    v.record(v.string(), v.union(v.number(), v.literal("x"))),
  )
When I set this field for the existing record as {} (empty object), then the update happens no problem. But if the json field got data in it, then I get a bad request response. Sounds like a bug? Hopefully, not a limitation of the api. Sample data: 
  {
    BR: {
      "5101": 5,
      "5102": 3,
      "5104": 2,
      "5105": 3,
      "5107": 3,
    },
    CA: {
      "5001": 3,
      "5003": 2,
      "5004": 1,
      "5006": 2,
      "5007": 3,
      "5011": 3,
      "5013": 2,
      "5015": 2,
      "5016": 1,
    },
  }
  {
    BR: {
      "5101": 5,
      "5102": 3,
      "5104": 2,
      "5105": 3,
      "5107": 3,
    },
    CA: {
      "5001": 3,
      "5003": 2,
      "5004": 1,
      "5006": 2,
      "5007": 3,
      "5011": 3,
      "5013": 2,
      "5015": 2,
      "5016": 1,
    },
  }
In term of update, I can change the numbers, or remove entries entirely, or remove nested attributes. So the data above can become like the below where I removed the CA market entirely and kept until two locations within the BR market. 
  {
    BR: {
      "5101": 10,
      "5102": 5,
    },
  }
  {
    BR: {
      "5101": 10,
      "5102": 5,
    },
  }
The reason I am using a json field is that otherwise I would end up with hundreds of thousands of documents in the table for a single collection (fashion world). And I will have multiple collections for multiple clients. So the json field seems appropriate especially it is easier to update a single field for a few thousands rows than hundred k rows each time an update happen. And it works well with the way I display the data in the UI.
2 Replies
Convex Bot
Convex Bot•5d ago
Thanks for posting in <#1088161997662724167>. Reminder: If you have a Convex Pro account, use the Convex Dashboard to file support tickets. - Provide context: What are you trying to achieve, what is the end-user interaction, what are you seeing? (full error message, command output, etc.) - Use search.convex.dev to search Docs, Stack, and Discord all at once. - Additionally, you can post your questions in the Convex Community's <#1228095053885476985> channel to receive a response from AI. - Avoid tagging staff unless specifically instructed. Thank you!
sbkl
sbklOP•5d ago
my bad. was getting an error with my schema/update. Used a sample dataset with regular mutation to identify it.

Did you find this page helpful?