← Back to Writing

I Built TCP Until curl Worked — Then the Interesting Problems Started

The first successful curl against WireStack proved that packets could travel from a Linux TAP device through my Ethernet, ARP, IPv4, TCP, and HTTP code. It did not prove that the TCP implementation behaved well once the network stopped being polite.

What the first curl actually proved

A short request on a clean loopback path may never lose, reorder, or significantly delay a segment. It can avoid the conditions that make TCP a transport protocol rather than a sequence-numbered byte pipe.

WireStack had to track unacknowledged ranges, retransmit them, reassemble out-of-order receive data, close both directions correctly, and respect the peer's advertised window. FIN consumes sequence space. A duplicate ACK is evidence, but not proof, of loss. A zero receive window pauses data without ending the connection.

The retransmission timer cannot be a constant

A fixed retransmission timeout either fires too aggressively on a slow path or waits too long on a fast one. WireStack samples eligible round trips, maintains a smoothed estimate and variance, then derives an adaptive RTO with backoff.

Why retransmissions complicate RTT measurement

Retransmitted segments create ambiguity: an ACK might correspond to the original transmission or the retry. Using that sample would corrupt the estimator, so the implementation follows the rule that ambiguous acknowledgements do not update RTT.

Three limits on one sender

I originally treated data waiting to be sent as one number. TCP made me separate application demand, the peer's advertised receive window, and the congestion window. The usable flight size is bounded by both windows.

When the receiver advertises zero, the connection is not closed. The sender pauses and uses zero-window persist behavior so a later window update cannot disappear forever.

Loss changes how much may be sent

Reliability alone would retransmit missing bytes. Congestion control also changes the sender's allowed flight size. WireStack implements Reno-style behavior and NewReno partial-ACK recovery, with negotiated SACK information used to describe holes more precisely.

This required separating three quantities I had initially blurred together: bytes the application wants to send, bytes the receiver permits, and bytes congestion control permits. The effective send limit is constrained by both the receive window and the congestion window.

Linux was the other implementation

The useful test was not two instances of my own stack agreeing on the same bug. The repository's isolated-network-namespace harness connects WireStack to the Linux kernel, drives a real curl, introduces packet conditions, and captures traffic for inspection.

That interoperability exposed byte-order mistakes, option negotiation details, retransmission behavior, and close-state edges that unit tests alone could miss.

Interoperability was the useful referee

I tested the stack against Linux rather than only connecting WireStack to itself. Real curl requests exercised passive open and HTTP serving; a Python server exercised active open from the other direction. Packet captures let me compare sequence numbers, acknowledgements, retransmissions, and close states when the endpoint disagreed with my assumptions. That does not make the implementation RFC-complete. It makes the supported subset observable against an independent stack.

Packet-level evidence

For the timeout-retransmission scenario, I first let Linux complete a real connection to WireStack. Then the harness applied tc netem loss 100% only on the WireStack-to-client direction and sent the HTTP request. The TAP capture sat before the drop point, so it recorded the response WireStack transmitted even though Linux never received it.

After the retransmission timer fired, I removed the loss rule. The eventual response had to be byte-identical to the segment visible in the upstream capture, and curl had to complete with the expected body. Seeing both capture points mattered: one showed what my stack sent; the other showed what the Linux side could actually receive.

Linux curl → request → WireStack
WireStack → response → [netem drops it]
                     ↑ TAP capture sees transmission
RTO expires → retransmit → Linux receives response

The repository also has a deterministic reordering scenario: a multi-segment request arrives out of order, cumulative ACK stays at the missing range, and WireStack advertises a SACK block for bytes it retained. Those traces turned retransmission and reassembly from diagrams into state I could inspect. They still cover a bounded subset, not RFC completeness.

TIME_WAIT was not idle cleanup

Releasing a connection tuple immediately after the final ACK can let delayed segments from an old connection collide with a new one. The state looks inactive to the application, but it still protects protocol history.

What the first request came to mean

WireStack is a correctness-focused learning stack, not a performance competitor to Linux. It does not implement simultaneous open, DSACK, CUBIC, or BBR.

The first HTTP response was still an important milestone. What changed was what I believed it demonstrated. It proved the vertical path existed. The work after it tested whether the path remained coherent under delay, loss, reordering, backpressure, and shutdown.

Related project: WireStack case study →