All notes
Backend3 min read

Stray hex numbers in my JSON: chunked transfer encoding

The ESP32 downloads the student whitelist from a Google Apps Script endpoint. Occasionally parsing failed because small hex tokens had ended up inside the JSON stream.

The cause

With HTTP chunked transfer encoding, the body is split into pieces, each preceded by its size in hexadecimal. Reading the raw stream meant those size lines were being treated as data.

The fix

I wrote a small three-state filter that tracks whether it is reading a chunk-size line, chunk data, or the line ending after a chunk, and only passes real data on to the JSON parser.

A related catch: card UIDs that looked like numbers were being parsed as numbers. Sanitising values before sending them as JSON kept them as exact strings.