Friday, April 28, 2017

Enabling BBR on Ubuntu Xenial

BBR (Bottleneck Bandwidth and RTT) is a novel congestion control scheme for TCP. It was developed by several Google engineers and has been in use on and Youtube. See the PERT KB for more background.
Here's how you can enable it on a machine running Ubuntu 16.04 LTS "Xenial Xerus".
Note: This is mostly useful for systems that send fairly large TCP flows. And the configuration is incompatible with other packet schedulers such as fq_codel, so consider the tradeoff.

Upgrade Linux Kernel to ≥4.9

The BBR implementation was merged into the master branch of the Linux kernel before the 4.9 release. Xenial uses the 4.4 kernel by default, but you can install a newer one (4.10 as of this writing) from a preview repository of the "hardware enablement" kernel provided by Canonical:
sudo apt install -y --install-recommends \

Enable fq Packet Scheduler

BBR fundamentally requires a packet scheduler that can "pace" outgoing packets per flow.  Enable this by adding an "up" clause (shown in boldface below) to each interface in its corresponding configuration file, e.g. /etc/network/interfaces.d/eth0.cfg:

auto eth0
iface eth0 inet dhcp
  up tc qdisc replace dev $IFACE root fq pacing
iface eth0 inet6 auto

Make BBR the Default Congestion Control Algorithm for TCP

echo net.ipv4.tcp_congestion_control=bbr | \
  sudo tee /etc/sysctl.d/90-bbr

Activate by Rebooting

A reboot should now enable all necessary components: the recent kernel, a pacing-capable packet scheduler, and BBR as the default.
You can check whether BBR is actually in effect by using the ss -i command while having at least one TCP connection open—which is already guaranteed to be the case if you are logged in over SSH.
$ ss --tcp -i
State       Recv-Q Send-Q                                               Local Address:Port                                                                Peer Address:Port
ESTAB       0      1848                         2001:620:5ca1:2f0:f816:3eff:fe9c:71af:ssh                                                           2001:620:0:69::107:57795
  bbr wscale:5,7 rto:208 rtt:6.078/0.237 ato:48 mss:1182 cwnd:12 bytes_acked:156685 bytes_received:10033 segs_out:851 segs_in:977 send 18.7Mbps lastrcv:4 lastack:4 pacing_rate 87.2Mbps unacked:4 retrans:0/13 rcv_rtt:12 rcv_space:28560
ESTAB       0      0                            2001:620:5ca1:2f0:f816:3eff:fe9c:71af:ssh                                                           2001:620:0:69::107:61315
  bbr wscale:5,7 rto:220 rtt:18.674/25.752 ato:40 mss:1182 cwnd:11 bytes_acked:8025 bytes_received:5125 segs_out:86 segs_in:122 send 5.6Mbps lastsnd:470176 lastrcv:480332 lastack:470068 pacing_rate 60.5Mbps rcv_rtt:12 rcv_space:28560
The "-i" part of the output mentions "bbr" and includes some BBR-specific metrics as well as the pacing rate applied to each outbound TCP flow.

No comments: