24.3 Link Aggregation and Failover

FreeBSD provides the lagg(4) interface, which can aggregate multiple network interfaces into a virtual interface to provide failover and link aggregation. As long as at least one aggregated interface's link is active, failover ensures that traffic continues to flow. Link aggregation works best with switches that support LACP, as the protocol distributes traffic bidirectionally and can respond to individual link failures.

The aggregation protocol supported by the lagg interface determines which ports are used to send traffic and whether specific ports are allowed to receive traffic. The following protocols are supported by lagg(4):

Protocol	Description
failover	This mode only sends and receives traffic through the primary port. If the primary port is unavailable, the next active port is used. The first interface added to the virtual interface is the primary port, and all subsequently added interfaces will be used as failover devices. If a failover occurs to a non-primary port, the original port will become the primary port again once it recovers
loadbalance	Provides a static setup without negotiating aggregation with the peer or exchanging frames to monitor links. If the switch supports LACP, the LACP protocol should be used instead
lacp	The IEEE 802.3ad Link Aggregation Control Protocol (LACP) (the standard is now maintained by IEEE 802.1AX) negotiates a set of aggregatable links with the peer to form one or more Link Aggregation Groups (LAGs). Each LAG consists of ports with the same speed operating in full duplex, and traffic is balanced across ports based on the maximum total speed of the LAG. Typically, only one LAG contains all ports. LACP quickly converges to a new configuration when physical connections change. LACP balances outbound traffic based on hashed protocol header information and accepts inbound traffic on any active port. The hash includes Ethernet source and destination addresses, VLAN tags if available, and IPv4 or IPv6 source and destination addresses
roundrobin	This mode distributes outbound traffic across all active ports using a round-robin scheduler and accepts inbound traffic on any active port. This mode should be used with caution as it may cause Ethernet frames to arrive out of order
broadcast	This mode sends outbound traffic to all ports configured on the lagg interface and receives frames from any port

Configuration Examples

This section demonstrates how to configure a switch and FreeBSD system for LACP load balancing. It then describes how to configure two Ethernet interfaces in failover mode, and how to configure failover mode between an Ethernet interface and a wireless interface.

Example 1. LACP Aggregation with a Switch

This example connects two Ethernet interfaces on a FreeBSD machine to the first two Ethernet ports on a switch as a load-balanced and fault-tolerant link. More interfaces can be added to increase throughput and fault tolerance. Replace the switch ports, Ethernet devices, and IP addresses in the example with actual configurations.

Ethernet links require frames to maintain order, so the same traffic between two sites always passes through the same physical link, and the maximum rate of a single flow is limited by the bandwidth of a single interface. The transmission algorithm uses as much information as possible to distinguish different flows and balance traffic across available interfaces.

On the FreeBSD system, create a lagg(4) interface using physical interfaces ue0 and ue1:

# ifconfig lagg0 create
# ifconfig lagg0 up laggproto lacp laggport ue0 laggport ue1
# ifconfig ue0 up
# ifconfig ue1 up

And assign an IP address to the interface:

# dhclient lagg0 

Next, verify the status of the virtual interface:

# ifconfig lagg0
lagg0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        options=8000b<RXCSUM,TXCSUM,VLAN_MTU,LINKSTATE>
        ether f8:e4:3b:5a:b2:04
        hwaddr 00:00:00:00:00:00
        inet 192.168.31.21 netmask 0xffffff00 broadcast 192.168.31.255
        laggproto lacp lagghash l2,l3,l4
        laggport: ue0 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
        laggport: ue1 flags=1c<ACTIVE,COLLECTING,DISTRIBUTING>
        groups: lagg
        media: Ethernet autoselect
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

Ports marked as ACTIVE are the LAG members negotiated with the remote switch. Traffic will be sent and received through these active ports. -v can be added to the above command to view the LAG identifiers.

To make this configuration persist across reboots, add the following entries to the /etc/rc.conf file on the FreeBSD system:

ifconfig_ue0="up"
ifconfig_ue1="up"
cloned_interfaces="lagg0"
ifconfig_lagg0="laggproto lacp laggport ue0 laggport ue1 DHCP"

Example 2. Failover Mode

Failover mode can be used to switch to a backup interface when the link on the primary interface is lost. To configure failover, ensure that the underlying physical interfaces are enabled, then create the lagg(4) interface. In this example, em0 is the primary interface, em1 is the backup interface, and the virtual interface is assigned an IP address via DHCP:

# ifconfig lagg0 create
# ifconfig lagg0 up laggproto failover laggport em0 laggport em1
# dhclient lagg0
# ifconfig em0 up
# ifconfig em1 up

The virtual interface status should appear as follows:

# ifconfig lagg0
lagg0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
	options=4e504bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
	ether 00:50:56:29:a9:45
	hwaddr 00:00:00:00:00:00
	inet 192.168.5.5 netmask 0xffffff00 broadcast 192.168.5.255
	laggproto failover lagghash l2,l3,l4
	laggport: em0 flags=5<MASTER,ACTIVE>
	laggport: em1 flags=0<>
	groups: lagg
	media: Ethernet autoselect
	status: active
	nd6 options=829<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL,STABLEADDR>

Traffic will be sent and received on em0. If the link on em0 is lost, em1 will become the active link. If the primary interface's link recovers, it will become the active link again.

To make this configuration persist across reboots, add the following entries to the /etc/rc.conf file:

ifconfig_em0="up"
ifconfig_em1="up"
cloned_interfaces="lagg0"
ifconfig_lagg0="laggproto failover laggport em0 laggport em1 DHCP"

Example 3. Failover Mode Between Ethernet and Wireless Interfaces

For laptop users, it is often necessary to configure the wireless device as a backup interface, used only when the Ethernet connection is unavailable. Using lagg(4), failover mode can be configured to prefer the Ethernet connection for performance and security reasons, while maintaining the ability to transmit data over the wireless connection.

This configuration works by overwriting the MAC address of the Ethernet interface with the MAC address of the wireless interface.

Note

In theory, either the Ethernet or wireless MAC address can be changed to match the other. However, some common wireless interfaces do not support overriding the MAC address, so it is recommended to overwrite the Ethernet MAC address.

Note

The wireless interface driver must be added to the kld_list in /etc/rc.conf to load the driver, followed by a reboot. Otherwise, the driver will not be loaded when setting up the lagg(4) interface, which will affect normal operation.

In this example, em0 is the primary interface, and wlan0 is the failover interface. The wlan0 interface is created from the rtwn0 physical wireless interface, and the Ethernet interface will be configured with the wireless interface's MAC address. First, enable the wireless interface, replacing rtwn0 with the system's wireless interface name:

# ifconfig wlan0 create wlandev rtwn0

Now determine the MAC address of the wireless interface:

# ifconfig wlan0 ether
wlan0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=200001<RXCSUM,RXCSUM_IPV6>
	ether 20:0d:b0:c4:ab:59

The ether line will display the MAC address of the specified interface. Now, change the MAC address of the Ethernet interface to the wireless card's MAC address:

# ifconfig em0 ether 20:0d:b0:c4:ab:59

Ensure the em0 interface is enabled, then create the lagg(4) interface with em0 as the primary interface and failover to wlan0:

# ifconfig em0 up
# ifconfig lagg0 create
# ifconfig lagg0 up laggproto failover laggport em0 laggport wlan0

Then, start the DHCP client to obtain an IP address:

# dhclient lagg0

The virtual interface status should appear as follows:

# ifconfig lagg0
lagg0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
	options=4e504bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
	ether 20:0d:b0:c4:ab:59
	hwaddr 00:00:00:00:00:00
	inet 192.168.5.7 netmask 0xffffff00 broadcast 192.168.5.255
	laggproto failover lagghash l2,l3,l4
	laggport: em0 flags=5<MASTER,ACTIVE>
	groups: lagg
	media: Ethernet autoselect
	status: active
	nd6 options=829<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL,STABLEADDR>

To make this configuration persist across reboots, add the following entries to the /etc/rc.conf file:

ifconfig_em0="ether 20:0d:b0:c4:ab:59"
wlans_rtwn0="wlan0"
ifconfig_wlan0="WPA"
cloned_interfaces="lagg0"
ifconfig_lagg0="up laggproto failover laggport em0 laggport wlan0 DHCP"