24.1 TCP/IP Protocol Stack
TCP/IP Protocol Stack
The Transmission Control Protocol (TCP) is the core transport layer protocol in the Internet Protocol Suite, and its software implementation architecture is called the TCP stack (organized using a layered structure, hence the term "stack"). Vint Cerf and Bob Kahn first proposed the core ideas of TCP in their 1974 paper "A Protocol for Packet Network Intercommunication" (at that time, it was a single protocol combining transport and network forwarding). After iterations, it was decided around 1978 to 1979 to split TCP and IP into two independent protocols, which were published as RFC 791 (IP) and RFC 793 (TCP) in September 1981. RFC 9293 has superseded RFC 793 as of August 2022, and is the current latest standard specification for the TCP protocol.
The TCP stack provides key functions such as end-to-end reliable data transmission, congestion control, and flow control. Unlike other mainstream operating systems, FreeBSD innovatively implements a multi-TCP stack coexistence architecture, which allows the system to simultaneously load multiple TCP protocol stack implementations and select different TCP stacks for different network connections or system-wide use.
Current main development and maintenance efforts are focused on the RACK stack (the RACK algorithm was originally from Google, and FreeBSD's tcp_rack stack implementation is by Randall Stewart of Netflix) and the base stack (evolved from the classic 4.4BSD stack implementation, with CUBIC as the default congestion algorithm).
Using the RACK Stack
To enable the RACK stack, first load the kernel module tcp_rack.ko and immediately switch the default TCP stack:
# kldload tcp_rack
# sysctl net.inet.tcp.functions_default=rackTo persist the configuration across reboots, add the following to /boot/loader.conf and /etc/sysctl.conf respectively:
# echo 'tcp_rack_load="YES"' >> /boot/loader.conf
# echo 'net.inet.tcp.functions_default=rack' >> /etc/sysctl.confAfter rebooting the system or loading the kernel module, use the following command to display the list of available TCP stacks:
# sysctl net.inet.tcp.functions_available
net.inet.tcp.functions_available:
Stack D Alias PCB count
freebsd freebsd 3
rack * rack 0The stack marked with * in the output is the current system default TCP protocol stack.
BBR Congestion Control Algorithm
The goals of BBR (Bottleneck Bandwidth and Round-trip propagation time) include:
Fully utilizing available network bandwidth.
Minimizing network transmission latency.
BBR does not use packet loss as a congestion signal; instead, it dynamically adjusts the sending rate based on detected bandwidth and latency, providing advantages in high-bandwidth, high-latency network environments. BBR is implemented as a congestion control module in Linux, but in FreeBSD it is implemented as an independent TCP stack (tcp_bbr), switched via net.inet.tcp.functions_default.
BBR Implementation and Application in FreeBSD
Load the tcp_bbr module and set it as the default TCP stack:
Persistent configuration:
After rebooting the system, check the current default TCP stack:
If the output shows net.inet.tcp.functions_default: bbr, then TCP BBR has been successfully enabled.
Troubleshooting and Unresolved Issues
In testing environments, RACK and BBR performed well in local area networks, but bandwidth significantly decreased in internet environments: RACK was approximately one-third of the default stack, and BBR approximately one-sixth of the default stack. These tests were conducted under specific network conditions (such as high-latency cross-border links), and actual performance may vary depending on the network environment. It should be noted that the design goals of RACK and BBR are to improve rather than reduce throughput; the above results may stem from systematic bias in specific packet loss/latency environments and do not represent the performance of these stacks in typical deployment scenarios.
References
Netflix. netflix/tcplog_dumper[EB/OL]. [2026-03-26]. https://github.com/netflix/tcplog_dumper. An open-source project by Netflix that provides TCP log dumping tools for TCP protocol analysis.
Cheng Y, Cardwell N, Dukkipati N, et al. The RACK-TLP Loss Detection Algorithm for TCP: RFC 8985[S/OL]. (2021-02)[2026-04-17]. https://www.rfc-editor.org/rfc/rfc8985. The IETF standard document for the RACK-TLP loss detection algorithm, written by Yuchung Cheng, Neal Cardwell, et al. from Google.
FreeBSD Project. tcp -- Internet Transmission Control Protocol[EB/OL]. [2026-04-14]. https://man.freebsd.org/cgi/man.cgi?query=tcp&sektion=4. TCP protocol stack manual page, describing the Transmission Control Protocol implementation and socket options.
FreeBSD Project. ip -- Internet Protocol[EB/OL]. [2026-04-14]. https://man.freebsd.org/cgi/man.cgi?query=ip&sektion=4. IP protocol manual page, describing the Internet Protocol implementation and socket options.
Exercises
Enable the FreeBSD default stack, RACK stack, and BBR stack in sequence, and use
iperf3to test throughput and latency in both LAN and internet environments. Record the performance differences of the three stacks in different network scenarios, and analyze the causes of the differences from the perspective of congestion control algorithm principles.Review the source code of the
tcp_bbrmodule, identify the key parameters that control the behavior of the BBR algorithm, modify two of them and reload the module, and record the impact of parameter changes on transmission performance.Analyze the implementation mechanism of FreeBSD's multi-TCP stack coexistence architecture, design a test scenario to specify different TCP stacks for different network connections within the same system, and record the configuration method and verification results.
Last updated