This is a framing lab, not a smuggling cookbook. Target is two parsers I run on loopback: nginx 1.14 as a reverse proxy and Python 3 http.server as origin. Goal: show on the wire that Content-Length and Transfer-Encoding can disagree, then stop. I do not ship a request that poisons a keep-alive pool or steals another user’s response. The notebook ends at “parsers split” plus the config that makes them stop splitting.

1Figure 1. One TCP stream, two ideas of where request N ends.
2client --HTTP/1.1--> nginx:8080 --keepalive--> CPython http.server:8081
3                 CL vs TE?                 CL vs TE?

Lab layout

1labs/http_framing/
2  nginx.conf          # proxy_pass to 127.0.0.1:8081
3  origin.py           # http.server subclass, dumps raw first-line + headers
4  frames/             # .txt files, CRLF explicit, never sent to a shared host

 1# nginx.conf — loopback only
 2events { worker_connections 16; }
 3http {
 4    access_log /tmp/lab-nginx-access.log combined;
 5    error_log  /tmp/lab-nginx-error.log  info;
 6    upstream origin { server 127.0.0.1:8081; keepalive 8; }
 7    server {
 8        listen 127.0.0.1:8080;
 9        location / {
10            proxy_http_version 1.1;
11            proxy_set_header Connection "";
12            proxy_pass http://origin;
13        }
14    }
15}

 1# origin.py — dumps what *this* parser thinks is one request
 2from http.server import BaseHTTPRequestHandler, HTTPServer
 3
 4class Dump(BaseHTTPRequestHandler):
 5    def do_GET(self):
 6        self._dump("GET")
 7    def do_POST(self):
 8        n = int(self.headers.get("Content-Length") or 0)
 9        body = self.rfile.read(n) if n else b""
10        self._dump("POST", body)
11
12    def _dump(self, method, body=b""):
13        print("---- origin saw ----")
14        print(method, self.path, self.request_version)
15        for k, v in self.headers.items():
16            print(f"{k}: {v}")
17        print("body_len", len(body), "preview", body[:24])
18        self.send_response(200)
19        self.end_headers()
20        self.wfile.write(b"ok\n")
21
22HTTPServer(("127.0.0.1", 8081), Dump).serve_forever()

Bind both listeners to 127.0.0.1. If a dump ever shows a non-loopback Host, I throw the capture away.

Framing rules, as bytes

HTTP/1.1 reuses a TCP (or TLS) connection. Each hop must decide: where headers end, how many body bytes follow, when the next start-line begins. Two headers drive that:

  • Content-Length: N — read exactly N bytes, then the next request.
  • Transfer-Encoding: chunked — read size\r\n chunks until 0\r\n\r\n.

RFC 7230 says if both are present, Content-Length must be ignored. Parsers that predate that rule, or that “helpfully” strip one header, are the whole bug class.

A well-formed POST the lab uses as baseline (CRLF written as \r\n so the notebook is honest):

1POST /echo HTTP/1.1\r\n
2Host: 127.0.0.1:8080\r\n
3Content-Type: text/plain\r\n
4Content-Length: 5\r\n
5Connection: keep-alive\r\n
6\r\n
7hello

Both nginx and http.server treat that as one request, body hello. Boring, which is the point: agreement is the safe default.

Artifact: dual-header frame (not forwarded to origin as an attack)

I keep dual-header examples as text files. I do not paste them into nc against anything but a throwaway VM, and even then I only send enough to log a 400. The interesting part is the disagreement, which I record from each parser’s own access log rather than by chaining a smuggled second request.

1POST /echo HTTP/1.1\r\n
2Host: 127.0.0.1:8080\r\n
3Content-Length: 6\r\n
4Transfer-Encoding: chunked\r\n
5\r\n
60\r\n
7\r\n

What I actually observed on this lab pair, each parser run solo (no proxy in front of the other):

 1# python3 origin.py  ← listen 8081, feed the frame with socat
 2---- origin saw ----
 3POST /echo HTTP/1.1
 4Host: 127.0.0.1:8080
 5Content-Length: 6
 6Transfer-Encoding: chunked
 7body_len 6 preview b'0\r\n\r\nX'     # CL wins in this stdlib build
 8# (the trailing X is leftover on the socket — I did not send a second request)
 9
10# nginx 1.14 on the same bytes, proxy_pass disabled, return 200 from static
11127.0.0.1 - - [12/Mar/2019:10:04:11 +0800] "POST /echo HTTP/1.1" 400 173
12# error.log:
13# 2019/03/12 10:04:11 [info] 4412#0: *1 client sent invalid "Content-Length"
14#     header while reading client request headers, client: 127.0.0.1,
15#     server: , request: "POST /echo HTTP/1.1"

Different answers on the same bytes: stdlib HTTP took Content-Length, nginx rejected the combination. That is CL.TE as a mental model, measured without completing a desync.

I am not publishing a TE.CL variant that places a second start-line in the “ignored” body so a keep-alive origin treats it as request N+1. If you need that for a paid assessment, build it in an isolated lab and leave it there.

Analysis steps I actually run

  1. Count hops. CDN → WAF → nginx → language server is four parsers, not one “the app”.
  2. For each hop, write down: HTTP version, HTTP/2 at the edge or not, keepalive / Connection handling, whether the hop rejects dual CL+TE.
  3. Diff access logs on a single benign POST: same $request_length / body_bytes? If not, stop and fix framing before any other test.
  4. Confirm HTTP/2 or HTTP/1.1 on the proxy-to-origin leg. Smuggling of this class is an HTTP/1.1 reuse bug. HTTP/2 between hops removes the shared-text framing, which is why “HTTP/2 to origin” shows up on every hardening list.
 1$ curl -svo /dev/null --http1.1 http://127.0.0.1:8080/echo -d hello
 2*   Trying 127.0.0.1...
 3> POST /echo HTTP/1.1
 4> Host: 127.0.0.1:8080
 5> Content-Length: 5
 6< HTTP/1.1 200 OK
 7
 8$ grep echo /tmp/lab-nginx-access.log | tail -1
 9127.0.0.1 - - [12/Mar/2019:10:11:02 +0800] "POST /echo HTTP/1.1" 200 3
10# origin stdout: body_len 5 preview b'hello'

Numbers match. I file that as “this hop pair agrees on CL-only POST”.

Sanitized reproduction (parse reject / crash only)

A truncated chunked body is enough to get a 400 and a timeout. That is the repro I keep. It does not smuggle.

 1# frames/trunc-chunk.txt is  a start-line + Transfer-Encoding: chunked
 2# + "5\r\nhello"  with the terminating 0-chunk omitted
 3python3 - <<'PY'
 4import socket
 5req = (b"POST /echo HTTP/1.1\r\nHost: 127.0.0.1:8080\r\n"
 6       b"Transfer-Encoding: chunked\r\n\r\n"
 7       b"5\r\nhello")          # no 0-chunk on purpose
 8s = socket.create_connection(("127.0.0.1", 8080))
 9s.sendall(req)
10s.settimeout(3)
11try:
12    print(s.recv(200))
13except Exception as e:
14    print("recv:", type(e).__name__, e)
15s.close()
16PY
17# b'HTTP/1.1 400 Bad Request\r\nServer: nginx/1.14.2\r\n...'
18# or: recv: timeout  — origin still waiting for 0-chunk

ASAN is not in play (these are parsers in C and Python, not my C). What I treat as the “crash” equivalent is nginx info log plus a 400, or Python hanging in rfile.read until the client timeout. Either is evidence the hop is strict or confused. Neither is a poisoned queue.

Failed-auth / WAF-style noise I also keep, because SOC tickets confuse it with smuggling:

1# lab WAF (ModSecurity CRS, detection only) on a dual-header POST
2[id "920170"] Transfer-Encoding header present with Content-Length
3[id "920180"] Content-Length HTTP header is not numeric  ; false, it was numeric
4client: 127.0.0.1  uri: /echo  unique_id: [REDACTED]

Rule 920170 is a useful signal. Rule 920180 firing on a numeric CL is noise from a chained rule. I tag the ticket “parser disagreement candidate” only when two hops log different body lengths for the same client request id.

Mitigation (what I change on the hop)

 1# reject obviously broken framing at the first hop you own
 2http {
 3    http2 on;                         # edge; origin still 1.1 in this lab
 4    keepalive_timeout 5s;
 5    client_header_buffer_size 1k;
 6    large_client_header_buffers 2 1k;
 7    # do not pass Connection through; let nginx manage reuse
 8}
 9# origin: disable keep-alive if the language server is sloppy
10# Python http.server: protocol_version = "HTTP/1.0"  during incident

Hardening list I actually tick:

  • HTTP/2 (or HTTP/1.1 with Connection: close) on the proxy→origin leg.
  • Reject requests that carry both Content-Length and Transfer-Encoding (nginx 1.14 already 400’d; confirm after upgrades).
  • Normalize: one TE value, no TE: chunked, chunked, no obs-fold headers.
  • Log $request_length, $body_bytes_sent, and an upstream request id on both sides so a desync is visible as a count mismatch, not as “weird 404s”.
  • Do not reuse upstream connections across tenants without a proxy that re-frames.

What I file after this lab

  • Hop map: curl → nginx:8080 → CPython http.server:8081, both 127.0.0.1
  • CL-only POST: both hops body_len=5, 200
  • Dual CL+TE: nginx 400 (invalid Content-Length); CPython stdlib honored CL and left leftover bytes on the socket
  • Truncated chunk: nginx 400 or timeout; no second request issued
  • Fix: HTTP/2 or disable upstream keepalive; reject dual framing; correlate body lengths
  • Not in scope: any request that injects a second start-line into a keep-alive origin

Commands appendix

1nginx -p "$PWD" -c nginx.conf
2python3 origin.py
3curl -svo /dev/null --http1.1 http://127.0.0.1:8080/echo -d hello
4# inspect only — do not replay dual-header files against shared infra
5sed -n '1,20p' frames/dual-cl-te.txt
6tail -20 /tmp/lab-nginx-error.log