Skip to content

Conversation

jonfriesen
Copy link
Contributor

Our original goal of the http1 parser was to offload as much as possible to the Go standard library. Unfortunately, this was rather difficult to do because the std lib http package is designed around being a consumer or producer of http messages. This means it does makes some changes to wire traffic that work for it. While most of these changes were fine, a few were problematic when it came to a out-of-band observability use case. It also became difficult to handle traffic that was unhealthy, which we still wanted for observability purposes but had to hack around to get pass validity checks within the standard library.

As a note, the Go http package is great, it was not intended to do what we were doing and using the wrong tool is not going to have favourable outcomes.

This PR introduces a new HTTP1.0 and HTTP1.1 parser. I designed it following the previous parsers high level streaming operations, but reimplementing the internals using small pieces of the Go http package (eg. Headers struct, and chunk reader implementation), but left the rest of it to my own devices. I focused pretty heavily on satisfying the RFC 9112 spec.

Along with this implementation, I wanted to be a bit more confident in this change as well as future changes to the parser and add some more thorough unit testing. I wrote a handful of tests for the different specific internal tests. Then wrote up some detail specs on how to use the tools and had a combination of Claude 4.1 and Gemini 2.5 help me build a list of important tests and RFC spec related tests. Claude 3.5 Sonnet help with implementation of these tests.

@jonfriesen jonfriesen force-pushed the jon/eng-647-add-rfc-compliant-http1-parser branch 2 times, most recently from 44fc1dd to 9b7b848 Compare September 16, 2025 21:48
@jonfriesen jonfriesen marked this pull request as ready for review September 16, 2025 22:51
@jonfriesen jonfriesen force-pushed the jon/eng-647-add-rfc-compliant-http1-parser branch from 9b7b848 to 35b986a Compare September 16, 2025 23:18
@jonfriesen jonfriesen force-pushed the jon/eng-647-add-rfc-compliant-http1-parser branch 6 times, most recently from ab6e99c to 143bd7f Compare September 18, 2025 22:36
@tylerflint tylerflint force-pushed the jon/eng-647-add-rfc-compliant-http1-parser branch from f23af5c to f2f3fe5 Compare October 2, 2025 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants