short-pinned tcp windows under load
Hit a weird one tracing latency on a busy box: SO_RCVBUF was being clamped well
below the configured value because the autotuner got confused by a burst of tiny RST replies
from a broken peer. Pinning the buffer at setsockopt time wasn’t enough —
the kernel still let it shrink. Workaround in the end was just disabling auto-tuning per-socket,
which has its own tradeoffs but at least the floor stays where I put it.
Reproducer was easier than I feared: spin up a peer that opens TCP, immediately RSTs a thousand times in a tight loop, then opens a real connection. The legit connection inherits a stunted window for a few seconds. Whether that matters in production depends on what the burst-replier looks like — in our case it was a half-broken health-check on a different box that everyone forgot about.
The kernel-side path is in tcp_rcv_space_adjust if anyone wants to chase it. Not
sure if “teach the autotuner to recognise floor pinning” is worth a patch upstream
versus “just don’t use auto-tuning when you care.” Probably the latter.
One thing I want to remember: ss -tin shows the actual current rcvbuf and the
unacked window. If you suspect the autotuner, just watch those two numbers across the regression
period. Beats reading kernel source.