Received: by 2002:a05:6a10:16a7:0:0:0:0 with SMTP id gp39csp1532587pxb; Fri, 6 Nov 2020 12:07:50 -0800 (PST) X-Google-Smtp-Source: ABdhPJyxfbcm5iQYFUpmTW1C4xrVhlmG30b7sct504xNQiOxbMiQTpUx3C5MWVqaFgoHDX3koRbh X-Received: by 2002:a17:906:9459:: with SMTP id z25mr3872550ejx.88.1604693270656; Fri, 06 Nov 2020 12:07:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1604693270; cv=none; d=google.com; s=arc-20160816; b=IiqAmh06NOcHvxaFORTGWcfk0bWFwGlSVq5vv/Z3d+r9DULYanviU6K2BRPXz0sqH3 +RoW3DN4J+pIhCcGrfwEbVd2hWcZDWBpgXV0VE8eClR0cykpvDYxrziRAVpGYf6dACD2 4wKjmaEvVRBKcxS4Bo8MFXfTCtqlEqkIlzIUPoWU/PuG6363pjDeB1SyyT1mI3MTXiiO bNUnxmTCWVtJn6UZtVnscroABngUKoWry1mBKs3KpefA30L/Q/Xq/LpGdwJHAkANgIs7 VtgNh+BYthw8pjh4UVhXaHL18cxnem/l6HeOTf7n/X0wyN+zIppF6gwcQJKo5lai3SWj nt8w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=XNXA/CeQS7zyJo2u4lKUSyDZW3GAa/bFqKiyxW4Ayu8=; b=bOpxrBUCbZ14H/tfCx866UjJTnJxxgmJV8jlQMzYM7Dc0jL51Gc8GrS9K8zsq3sTo4 UEWhe9pNQ64VRr6mrHe5KtI1yqYnb9IXR2GLkTVnPgaVcQvnkLzLkMaPRaFS9iTZ3S+f IyM88fbQrylqusC9L5uh5g6OIt43wTIc9yXqmE5XDL/xHr5pZTy4C99AtuwsqE6NJn8F 7T1uvSvhxlD5dNekgSVmqaOCuiKdvssyOLQ/wo/A1AsdhvaCC2LohBuh9eRSSSjKpmxQ eG5y87+16HfsM6d7/MR7gkDXyB2nRzdNEfZFO0MaTDxR4jjH/dD5+V1/u+weXG+DJl6q ZbEw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=hqfba4Se; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id cq11si1678598edb.332.2020.11.06.12.07.26; Fri, 06 Nov 2020 12:07:50 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=hqfba4Se; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728402AbgKFUFW (ORCPT + 99 others); Fri, 6 Nov 2020 15:05:22 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:27665 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728413AbgKFUFQ (ORCPT ); Fri, 6 Nov 2020 15:05:16 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1604693110; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=XNXA/CeQS7zyJo2u4lKUSyDZW3GAa/bFqKiyxW4Ayu8=; b=hqfba4SepikL4W2GMfogXiMmWcTeTv7jzPp429f20FWkVcCGu4atKMU6XifH0q0hDx+Yhd gVkHfS449ahnFH1MbYacg57gpOV2Nn2oisDVWvq/aUaLhqg6a0X/8gb5dRy2mbB471RU5l pX2htZ3hRH1KZNhn2mEZVwsWVLSOuEc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-433-fwN__yRINzO_zKIkvdxLNg-1; Fri, 06 Nov 2020 15:05:05 -0500 X-MC-Unique: fwN__yRINzO_zKIkvdxLNg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 48D80803652; Fri, 6 Nov 2020 20:05:03 +0000 (UTC) Received: from f33vm.wilsonet.com.wilsonet.com (dhcp-17-185.bos.redhat.com [10.18.17.185]) by smtp.corp.redhat.com (Postfix) with ESMTP id AE1AB7366D; Fri, 6 Nov 2020 20:05:01 +0000 (UTC) From: Jarod Wilson To: linux-kernel@vger.kernel.org Cc: Jarod Wilson , Jay Vosburgh , Veaceslav Falico , Andy Gospodarek , "David S. Miller" , Jakub Kicinski , Thomas Davis , netdev@vger.kernel.org Subject: [PATCH 5/5] bonding: update Documentation for port/bond terminology Date: Fri, 6 Nov 2020 15:04:36 -0500 Message-Id: <20201106200436.943795-6-jarod@redhat.com> In-Reply-To: <20201106200436.943795-1-jarod@redhat.com> References: <20201106200436.943795-1-jarod@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Swap in port/bond terminology where appropriate, leaving all legacy sysfs and procfs interface mentions in place, but marked as deprecated. Additionally, add more netlink/iproute2 documentation, and note that this is the preferred method of interfacing with the bonding driver. While we're at it, also make some mention of NetworkManager's existence. Cc: Jay Vosburgh Cc: Veaceslav Falico Cc: Andy Gospodarek Cc: "David S. Miller" Cc: Jakub Kicinski Cc: Thomas Davis Cc: netdev@vger.kernel.org Signed-off-by: Jarod Wilson --- Documentation/networking/bonding.rst | 581 ++++++++++++++++----------- 1 file changed, 348 insertions(+), 233 deletions(-) diff --git a/Documentation/networking/bonding.rst b/Documentation/networking/bonding.rst index adc314639085..9641add9fb32 100644 --- a/Documentation/networking/bonding.rst +++ b/Documentation/networking/bonding.rst @@ -21,6 +21,15 @@ Added Sysfs information: 2006/04/24 - Mitch Williams +Major terminology rework done late 2020, to start to remove the use of +the socially problematic terms "master" and "slave" from the code. The +"master" device is now referred to as simply the "bond" device and the +"slave" devices as "ports", but all sysfs, procfs and module options +have been retained as-is for userspace compatibility. The sysfs and +procfs interfaces are deprecated though, in favor of userspace making +use of netlink and iproute for any and all bonding configuration and +information-gathering work. + Introduction ============ @@ -53,10 +62,11 @@ who to ask for help, please follow the links at the end of this file. 3.2.2 Configuring Multiple Bonds with Initscripts 3.3 Configuring Bonding Manually with Ifenslave 3.3.1 Configuring Multiple Bonds Manually - 3.4 Configuring Bonding Manually via Sysfs - 3.5 Configuration with Interfaces Support - 3.6 Overriding Configuration for Special Cases - 3.7 Configuring LACP for 802.3ad mode in a more secure way + 3.4 Configuring Bonding Manually via netlink + 3.5 Configuring Bonding Manually via Sysfs + 3.6 Configuration with Interfaces Support + 3.7 Overriding Configuration for Special Cases + 3.8 Configuring LACP for 802.3ad mode in a more secure way 4. Querying Bonding Configuration 4.1 Bonding Configuration @@ -134,8 +144,8 @@ Build and install the new kernel and modules. 1.2 Bonding Control Utility --------------------------- -It is recommended to configure bonding via iproute2 (netlink) -or sysfs, the old ifenslave control utility is obsolete. +It is recommended to configure bonding via iproute2 (netlink). +The sysfs interfaces and the old ifenslave control utility are obsolete. 2. Bonding Driver Options ========================= @@ -167,22 +177,23 @@ or, for backwards compatibility, the option value. E.g., The parameters are as follows: -active_slave +active_port +active_slave (DEPRECATED) - Specifies the new active slave for modes that support it + Specifies the new active port for modes that support it (active-backup, balance-alb and balance-tlb). Possible values - are the name of any currently enslaved interface, or an empty - string. If a name is given, the slave and its link must be up in order - to be selected as the new active slave. If an empty string is - specified, the current active slave is cleared, and a new active - slave is selected automatically. + are the name of any currently aggregated interface, or an empty + string. If a name is given, the port and its port must be up in order + to be selected as the new active port. If an empty string is + specified, the current active port is cleared, and a new active + port is selected automatically. Note that this is only available through the sysfs interface. No module parameter by this name exists. The normal value of this option is the name of the currently - active slave, or the empty string if there is no active slave or - the current mode does not use an active slave. + active port, or the empty string if there is no active port or + the current mode does not use an active port. ad_actor_sys_prio @@ -199,8 +210,8 @@ ad_actor_system protocol packet exchanges (LACPDUs). The value cannot be NULL or multicast. It is preferred to have the local-admin bit set for this mac but driver does not enforce it. If the value is not given then - system defaults to using the masters' mac address as actors' system - address. + system defaults to using the bonds' mac address as actors' + system address. This parameter has effect only in 802.3ad mode and is available through SysFs interface. @@ -216,8 +227,8 @@ ad_select bandwidth. Reselection of the active aggregator occurs only when all - slaves of the active aggregator are down or the active - aggregator has no slaves. + ports of the active aggregator are down or the active + aggregator has no ports. This is the default value. @@ -226,18 +237,18 @@ ad_select The active aggregator is chosen by largest aggregate bandwidth. Reselection occurs if: - - A slave is added to or removed from the bond + - A port is added to or removed from the bond - - Any slave's link state changes + - Any port's link state changes - - Any slave's 802.3ad association state changes + - Any port's 802.3ad association state changes - The bond's administrative state changes to up count or 2 The active aggregator is chosen by the largest number of - ports (slaves). Reselection occurs as described under the + ports (ports). Reselection occurs as described under the "bandwidth" setting, above. The bandwidth and count selection policies permit failover of @@ -265,7 +276,8 @@ ad_user_port_key This parameter has effect only in 802.3ad mode and is available through SysFs interface. -all_slaves_active +all_ports_active +all_slaves_active (DEPRECATED) Specifies that duplicate frames (received on inactive ports) should be dropped (0) or delivered (1). @@ -281,10 +293,10 @@ arp_interval Specifies the ARP link monitoring frequency in milliseconds. - The ARP monitor works by periodically checking the slave + The ARP monitor works by periodically checking the port devices to determine whether they have sent or received traffic recently (the precise criteria depends upon the - bonding mode, and the state of the slave). Regular traffic is + bonding mode, and the state of the port). Regular traffic is generated via ARP probes issued for the addresses specified by the arp_ip_target option. @@ -327,50 +339,50 @@ arp_validate active or 1 - Validation is performed only for the active slave. + Validation is performed only for the active port. backup or 2 - Validation is performed only for backup slaves. + Validation is performed only for backup ports. all or 3 - Validation is performed for all slaves. + Validation is performed for all ports. filter or 4 - Filtering is applied to all slaves. No validation is + Filtering is applied to all ports. No validation is performed. filter_active or 5 - Filtering is applied to all slaves, validation is performed - only for the active slave. + Filtering is applied to all ports, validation is performed + only for the active port. filter_backup or 6 - Filtering is applied to all slaves, validation is performed - only for backup slaves. + Filtering is applied to all ports, validation is performed + only for backup ports. Validation: Enabling validation causes the ARP monitor to examine the incoming - ARP requests and replies, and only consider a slave to be up if it + ARP requests and replies, and only consider a port to be up if it is receiving the appropriate ARP traffic. - For an active slave, the validation checks ARP replies to confirm - that they were generated by an arp_ip_target. Since backup slaves + For an active port, the validation checks ARP replies to confirm + that they were generated by an arp_ip_target. Since backup ports do not typically receive these replies, the validation performed - for backup slaves is on the broadcast ARP request sent out via the - active slave. It is possible that some switch or network - configurations may result in situations wherein the backup slaves + for backup ports is on the broadcast ARP request sent out via the + active port. It is possible that some switch or network + configurations may result in situations wherein the backup ports do not receive the ARP requests; in such a situation, validation - of backup slaves must be disabled. + of backup ports must be disabled. - The validation of ARP requests on backup slaves is mainly helping - bonding to decide which slaves are more likely to work in case of - the active slave failure, it doesn't really guarantee that the - backup slave will work if it's selected as the next active slave. + The validation of ARP requests on backup ports is mainly helping + bonding to decide which ports are more likely to work in case of + the active port failure, it doesn't really guarantee that the + backup port will work if it's selected as the next active port. Validation is useful in network configurations in which multiple bonding hosts are concurrently issuing ARPs to one or more targets @@ -387,11 +399,11 @@ arp_validate Enabling filtering causes the ARP monitor to only use incoming ARP packets for link availability purposes. Arriving packets that are not ARPs are delivered normally, but do not count when determining - if a slave is available. + if a port is available. Filtering operates by only considering the reception of ARP packets (any ARP packet, regardless of source or destination) when - determining if a slave has received traffic for link availability + determining if a port has received traffic for link availability purposes. Filtering is useful in network configurations in which significant @@ -405,26 +417,26 @@ arp_validate arp_all_targets Specifies the quantity of arp_ip_targets that must be reachable - in order for the ARP monitor to consider a slave as being up. - This option affects only active-backup mode for slaves with + in order for the ARP monitor to consider a port as being up. + This option affects only active-backup mode for ports with arp_validation enabled. Possible values are: any or 0 - consider the slave up only when any of the arp_ip_targets + consider the port up only when any of the arp_ip_targets is reachable all or 1 - consider the slave up only when all of the arp_ip_targets + consider the port up only when all of the arp_ip_targets are reachable downdelay Specifies the time, in milliseconds, to wait before disabling - a slave after a link failure has been detected. This option + a port after a link failure has been detected. This option is only valid for the miimon link monitor. The downdelay value should be a multiple of the miimon value; if not, it will be rounded down to the nearest multiple. The default @@ -432,8 +444,8 @@ downdelay fail_over_mac - Specifies whether active-backup mode should set all slaves to - the same MAC address at enslavement (the traditional + Specifies whether active-backup mode should set all ports to + the same MAC address at connection (the traditional behavior), or, when enabled, perform special handling of the bond's MAC address in accordance with the selected policy. @@ -442,16 +454,16 @@ fail_over_mac none or 0 This setting disables fail_over_mac, and causes - bonding to set all slaves of an active-backup bond to - the same MAC address at enslavement time. This is the + bonding to set all ports of an active-backup bond to + the same MAC address at connection time. This is the default. active or 1 The "active" fail_over_mac policy indicates that the MAC address of the bond should always be the MAC - address of the currently active slave. The MAC - address of the slaves is not changed; instead, the MAC + address of the currently active port. The MAC + address of the ports is not changed; instead, the MAC address of the bond changes during a failover. This policy is useful for devices that cannot ever @@ -478,12 +490,12 @@ fail_over_mac The "follow" fail_over_mac policy causes the MAC address of the bond to be selected normally (normally - the MAC address of the first slave added to the bond). - However, the second and subsequent slaves are not set + the MAC address of the first port added to the bond). + However, the second and subsequent ports are not set to this MAC address while they are in a backup role; a - slave is programmed with the bond's MAC address at - failover time (and the formerly active slave receives - the newly active slave's MAC address). + port is programmed with the bond's MAC address at + failover time (and the formerly active port receives + the newly active port's MAC address). This policy is useful for multiport devices that either become confused or incur a performance penalty @@ -491,11 +503,11 @@ fail_over_mac address. - The default policy is none, unless the first slave cannot + The default policy is none, unless the first port cannot change its MAC address, in which case the active policy is selected by default. - This option may be modified via sysfs only when no slaves are + This option may be modified via sysfs only when no ports are present in the bond. This option was added in bonding version 3.2.0. The "follow" @@ -526,7 +538,7 @@ max_bonds miimon Specifies the MII link monitoring frequency in milliseconds. - This determines how often the link state of each slave is + This determines how often the link state of each port is inspected for link failures. A value of zero disables MII link monitoring. A value of 100 is a good starting point. The use_carrier option, below, affects how the link state is @@ -558,22 +570,22 @@ mode balance-rr or 0 Round-robin policy: Transmit packets in sequential - order from the first available slave through the + order from the first available port through the last. This mode provides load balancing and fault tolerance. active-backup or 1 - Active-backup policy: Only one slave in the bond is - active. A different slave becomes active if, and only - if, the active slave fails. The bond's MAC address is + Active-backup policy: Only one port in the bond is + active. A different port becomes active if, and only + if, the active port fails. The bond's MAC address is externally visible on only one port (network adapter) to avoid confusing the switch. In bonding version 2.6.2 or later, when a failover occurs in active-backup mode, bonding will issue one - or more gratuitous ARPs on the newly active slave. - One gratuitous ARP is issued for the bonding master + or more gratuitous ARPs on the newly active port. + One gratuitous ARP is issued for the bonding device interface and each VLAN interfaces configured above it, provided that the interface has at least one IP address configured. Gratuitous ARPs issued for VLAN @@ -588,7 +600,7 @@ mode XOR policy: Transmit based on the selected transmit hash policy. The default policy is a simple [(source MAC address XOR'd with destination MAC address XOR - packet type ID) modulo slave count]. Alternate transmit + packet type ID) modulo port count]. Alternate transmit policies may be selected via the xmit_hash_policy option, described below. @@ -596,17 +608,17 @@ mode broadcast or 3 - Broadcast policy: transmits everything on all slave + Broadcast policy: transmits everything on all port interfaces. This mode provides fault tolerance. 802.3ad or 4 IEEE 802.3ad Dynamic link aggregation. Creates aggregation groups that share the same speed and - duplex settings. Utilizes all slaves in the active + duplex settings. Utilizes all ports in the active aggregator according to the 802.3ad specification. - Slave selection for outgoing traffic is done according + Port selection for outgoing traffic is done according to the transmit hash policy, which may be changed from the default simple XOR policy via the xmit_hash_policy option, documented below. Note that not all transmit @@ -619,7 +631,7 @@ mode Prerequisites: 1. Ethtool support in the base drivers for retrieving - the speed and duplex of each slave. + the speed and duplex of each port. 2. A switch that supports IEEE 802.3ad Dynamic link aggregation. @@ -634,20 +646,20 @@ mode In tlb_dynamic_lb=1 mode; the outgoing traffic is distributed according to the current load (computed - relative to the speed) on each slave. + relative to the speed) on each port. In tlb_dynamic_lb=0 mode; the load balancing based on current load is disabled and the load is distributed only using the hash distribution. - Incoming traffic is received by the current slave. - If the receiving slave fails, another slave takes over - the MAC address of the failed receiving slave. + Incoming traffic is received by the current port. + If the receiving port fails, another port takes over + the MAC address of the failed receiving port. Prerequisite: Ethtool support in the base drivers for retrieving the - speed of each slave. + speed of each port. balance-alb or 6 @@ -658,7 +670,7 @@ mode The bonding driver intercepts the ARP Replies sent by the local system on their way out and overwrites the source hardware address with the unique hardware - address of one of the slaves in the bond such that + address of one of the ports in the bond such that different peers use different hardware addresses for the server. @@ -668,24 +680,24 @@ mode IP information from the ARP packet. When the ARP Reply arrives from the peer, its hardware address is retrieved and the bonding driver initiates an ARP - reply to this peer assigning it to one of the slaves + reply to this peer assigning it to one of the ports in the bond. A problematic outcome of using ARP negotiation for balancing is that each time that an ARP request is broadcast it uses the hardware address of the bond. Hence, peers learn the hardware address of the bond and the balancing of receive traffic - collapses to the current slave. This is handled by + collapses to the current port. This is handled by sending updates (ARP Replies) to all the peers with their individually assigned hardware address such that the traffic is redistributed. Receive traffic is also - redistributed when a new slave is added to the bond - and when an inactive slave is re-activated. The + redistributed when a new port is added to the bond + and when an inactive port is re-activated. The receive load is distributed sequentially (round robin) - among the group of highest speed slaves in the bond. + among the group of highest speed ports in the bond. - When a link is reconnected or a new slave joins the + When a link is reconnected or a new port joins the bond the receive traffic is redistributed among all - active slaves in the bond by initiating ARP Replies + active ports in the bond by initiating ARP Replies with the selected MAC address to each of the clients. The updelay parameter (detailed below) must be set to a value equal or greater than the switch's @@ -695,16 +707,16 @@ mode Prerequisites: 1. Ethtool support in the base drivers for retrieving - the speed of each slave. + the speed of each port. 2. Base driver support for setting the hardware address of a device while it is open. This is - required so that there will always be one slave in the + required so that there will always be one port in the team using the bond hardware address (the - curr_active_slave) while having a unique hardware - address for each slave in the bond. If the - curr_active_slave fails its hardware address is - swapped with the new curr_active_slave that was + curr_active_port) while having a unique hardware + address for each port in the bond. If the + curr_active_port fails its hardware address is + swapped with the new curr_active_port that was chosen. num_grat_arp, @@ -712,7 +724,7 @@ num_unsol_na Specify the number of peer notifications (gratuitous ARPs and unsolicited IPv6 Neighbor Advertisements) to be issued after a - failover event. As soon as the link is up on the new slave + failover event. As soon as the link is up on the new port (possibly immediately) a peer notification is sent on the bonding device and each VLAN sub-device. This is repeated at the rate specified by peer_notif_delay if the number is @@ -726,10 +738,11 @@ num_unsol_na are generated by the ipv4 and ipv6 code and the numbers of repetitions cannot be set independently. -packets_per_slave +packets_per_port +packets_per_slave (DEPRECATED) - Specify the number of packets to transmit through a slave before - moving to the next one. When set to 0 then a slave is chosen at + Specify the number of packets to transmit through a port before + moving to the next one. When set to 0 then a port is chosen at random. The valid range is 0 - 65535; the default value is 1. This option @@ -747,11 +760,11 @@ peer_notif_delay primary - A string (eth0, eth2, etc) specifying which slave is the + A string (eth0, eth2, etc) specifying which port is the primary device. The specified device will always be the - active slave while it is available. Only when the primary is + active port while it is available. Only when the primary is off-line will alternate devices be used. This is useful when - one slave is preferred over another, e.g., when one slave has + one port is preferred over another, e.g., when one port has higher throughput than another. The primary option is only valid for active-backup(1), @@ -759,41 +772,41 @@ primary primary_reselect - Specifies the reselection policy for the primary slave. This - affects how the primary slave is chosen to become the active slave - when failure of the active slave or recovery of the primary slave + Specifies the reselection policy for the primary port. This + affects how the primary port is chosen to become the active port + when failure of the active port or recovery of the primary port occurs. This option is designed to prevent flip-flopping between - the primary slave and other slaves. Possible values are: + the primary port and other ports. Possible values are: always or 0 (default) - The primary slave becomes the active slave whenever it + The primary port becomes the active port whenever it comes back up. better or 1 - The primary slave becomes the active slave when it comes - back up, if the speed and duplex of the primary slave is + The primary port becomes the active port when it comes + back up, if the speed and duplex of the primary port is better than the speed and duplex of the current active - slave. + port. failure or 2 - The primary slave becomes the active slave only if the - current active slave fails and the primary slave is up. + The primary port becomes the active port only if the + current active port fails and the primary port is up. The primary_reselect setting is ignored in two cases: - If no slaves are active, the first slave to recover is - made the active slave. + If no ports are active, the first port to recover is + made the active port. - When initially enslaved, the primary slave is always made - the active slave. + When initially connected, the primary port is always made + the active port. Changing the primary_reselect policy via sysfs will cause an - immediate selection of the best active slave according to the new + immediate selection of the best active port according to the new policy. This may or may not result in a change of the active - slave, depending upon the circumstances. + port, depending upon the circumstances. This option was added for bonding version 3.6.0. @@ -803,7 +816,7 @@ tlb_dynamic_lb mode. The value has no effect on any other modes. The default behavior of tlb mode is to shuffle active flows across - slaves based on the load in that interval. This gives nice lb + ports based on the load in that interval. This gives nice lb characteristics but can cause packet reordering. If re-ordering is a concern use this variable to disable flow shuffling and rely on load balancing provided solely by the hash distribution. @@ -822,7 +835,7 @@ tlb_dynamic_lb updelay Specifies the time, in milliseconds, to wait before enabling a - slave after a link recovery has been detected. This option is + port after a link recovery has been detected. This option is only valid for the miimon link monitor. The updelay value should be a multiple of the miimon value; if not, it will be rounded down to the nearest multiple. The default value is 0. @@ -851,7 +864,7 @@ use_carrier xmit_hash_policy - Selects the transmit hash policy to use for slave selection in + Selects the transmit hash policy to use for port selection in balance-xor, 802.3ad, and tlb modes. Possible values are: layer2 @@ -860,10 +873,10 @@ xmit_hash_policy field to generate the hash. The formula is hash = source MAC XOR destination MAC XOR packet type ID - slave number = hash modulo slave count + port number = hash modulo port count This algorithm will place all traffic to a particular - network peer on the same slave. + network peer on the same port. This algorithm is 802.3ad compliant. @@ -879,13 +892,13 @@ xmit_hash_policy hash = hash XOR source IP XOR destination IP hash = hash XOR (hash RSHIFT 16) hash = hash XOR (hash RSHIFT 8) - And then hash is reduced modulo slave count. + And then hash is reduced modulo port count. If the protocol is IPv6 then the source and destination addresses are first hashed using ipv6_addr_hash. This algorithm will place all traffic to a particular - network peer on the same slave. For non-IP traffic, + network peer on the same port. For non-IP traffic, the formula is the same as for the layer2 transmit hash policy. @@ -901,8 +914,8 @@ xmit_hash_policy This policy uses upper layer protocol information, when available, to generate the hash. This allows for traffic to a particular network peer to span multiple - slaves, although a single connection will not span - multiple slaves. + ports, although a single connection will not span + multiple ports. The formula for unfragmented TCP and UDP packets is @@ -910,7 +923,7 @@ xmit_hash_policy hash = hash XOR source IP XOR destination IP hash = hash XOR (hash RSHIFT 16) hash = hash XOR (hash RSHIFT 8) - And then hash is reduced modulo slave count. + And then hash is reduced modulo port count. If the protocol is IPv6 then the source and destination addresses are first hashed using ipv6_addr_hash. @@ -968,16 +981,16 @@ resend_igmp This option is useful for bonding modes balance-rr (0), active-backup (1), balance-tlb (5) and balance-alb (6), in which a failover can - switch the IGMP traffic from one slave to another. Therefore a fresh + switch the IGMP traffic from one port to another. Therefore a fresh IGMP report must be issued to cause the switch to forward the incoming - IGMP traffic over the newly selected slave. + IGMP traffic over the newly selected port. This option was added for bonding version 3.7.0. lp_interval Specifies the number of seconds between instances where the bonding - driver sends learning packets to each slaves peer switch. + driver sends learning packets to each ports peer switch. The valid range is 1 - 0x7fffffff; the default value is 1. This Option has effect only in balance-tlb and balance-alb modes. @@ -986,11 +999,11 @@ lp_interval ============================== You can configure bonding using either your distro's network -initialization scripts, or manually using either iproute2 or the -sysfs interface. Distros generally use one of three packages for the -network initialization scripts: initscripts, sysconfig or interfaces. -Recent versions of these packages have support for bonding, while older -versions do not. +initialization scripts, or manually using either iproute2, NetworkManager, +or the sysfs interface. Distros generally use one of four packages for +the network initialization scripts: initscripts, sysconfig, interfaces, or +NetworkManager. Any remotely modern version of these packages should have +support for bonding, but very old versions may not. We will first describe the options for configuring bonding for distros using versions of initscripts, sysconfig and interfaces with full @@ -1004,15 +1017,18 @@ Determining this is fairly straightforward. First, look for a file called interfaces in /etc/network directory. If this file is present in your system, then your system use interfaces. See -Configuration with Interfaces Support. +Configuration with Interfaces Support. Many modern distros now use +NetworkManager, for which you should find the directory /etc/NetworkManager +on your system, the NetworkManager GUI, and nmcli for command-line based +configuration of network components, including bonds. Else, issue the command:: $ rpm -qf /sbin/ifup It will respond with a line of text starting with either -"initscripts" or "sysconfig," followed by some numbers. This is the -package that provides your network initialization scripts. +"initscripts", "sysconfig" or "NetworkManager", followed by some numbers. +This is the package that provides your network initialization scripts. Next, to determine if your installation supports bonding, issue the command:: @@ -1020,7 +1036,14 @@ issue the command:: $ grep ifenslave /sbin/ifup If this returns any matches, then your initscripts or -sysconfig has support for bonding. +sysconfig has support for bonding. Note however, that ifenslave has been +deprecated for quite some time, any many distributions no longer use it +as a primary means of configuring bonds. In some cases, ifup is simply a +wrapper script that calls NetworkManager's nmcli, which has had support +for bonding since it's earliest days being deployed as a distribution +default network management infrastructure. + +Note: classic ifenslave binaries are considered deprecated. 3.1 Configuration with Sysconfig Support ---------------------------------------- @@ -1034,9 +1057,9 @@ front end does not provide any means to work with bonding devices. Bonding devices can be managed by hand, however, as follows. First, if they have not already been configured, configure the -slave devices. On SLES 9, this is most easily done by running the +port devices. On SLES 9, this is most easily done by running the yast2 sysconfig configuration utility. The goal is for to create an -ifcfg-id file for each slave device. The simplest way to accomplish +ifcfg-id file for each port device. The simplest way to accomplish this is to configure the devices for DHCP (this is only to get the file ifcfg-id file created; see below for some issues with DHCP). The name of the configuration file for each device will be of the form:: @@ -1047,8 +1070,8 @@ Where the "xx" portion will be replaced with the digits from the device's permanent MAC address. Once the set of ifcfg-id-xx:xx:xx:xx:xx:xx files has been -created, it is necessary to edit the configuration files for the slave -devices (the MAC addresses correspond to those of the slave devices). +created, it is necessary to edit the configuration files for the port +devices (the MAC addresses correspond to those of the port devices). Before editing, the file will contain multiple lines, and will look something like this:: @@ -1111,7 +1134,7 @@ The possible values are: ======== ====================================================== The line BONDING_MASTER='yes' indicates that the device is a -bonding master device. The only useful value is "yes." +bonding aggregator device. The only useful value is "yes." The contents of BONDING_MODULE_OPTS are supplied to the instance of the bonding module for this device. Specify the options @@ -1119,9 +1142,9 @@ for the bonding mode, link monitoring, and so on here. Do not include the max_bonds bonding parameter; this will confuse the configuration system if you have multiple bonding devices. -Finally, supply one BONDING_SLAVEn="slave device" for each -slave. where "n" is an increasing value, one for each slave. The -"slave device" is either an interface name, e.g., "eth0", or a device +Finally, supply one BONDING_SLAVEn="port device" for each +port. where "n" is an increasing value, one for each port. The +"port device" is either an interface name, e.g., "eth0", or a device specifier for the network device. The interface name is easier to find, but the ethN names are subject to change at boot time if, e.g., a device early in the sequence has failed. The device specifiers @@ -1129,7 +1152,7 @@ a device early in the sequence has failed. The device specifiers network device, and will not change unless the device's bus location changes (for example, it is moved from one PCI slot to another). The example above uses one of each type for demonstration purposes; most -configurations will choose one or the other for all slave devices. +configurations will choose one or the other for all port devices. When all configuration files have been modified or created, networking must be restarted for the configuration changes to take @@ -1162,7 +1185,7 @@ Under sysconfig, configuring a device with BOOTPROTO='dhcp' will cause it to query DHCP for its IP address information. At this writing, this does not function for bonding devices; the scripts attempt to obtain the device address from DHCP prior to adding any of -the slave devices. Without active slaves, the DHCP requests are not +the port devices. Without active ports, the DHCP requests are not sent to the network. 3.1.2 Configuring Multiple Bonds with Sysconfig @@ -1237,6 +1260,14 @@ place the following text:: Be sure to change the networking specific lines (IPADDR, NETMASK, NETWORK and BROADCAST) to match your network configuration. +Note that recent Red Hat and Fedora releases (at least Red Hat Enterprise +Linux 8 and Fedora 30 or newer) prefer NetworkManager over the network +initscripts, but the two can coexist on a system. By default, NetworkManager +will try to handle all ifcfg-* files, including bonding ones, unless the +following line is added to tell classis initscripts to handle them:: + + NM_CONTROLLED=no + For later versions of initscripts, such as that found with Fedora 7 (or later) and Red Hat Enterprise Linux version 5 (or later), it is possible, and, indeed, preferable, to specify the bonding options in the ifcfg-bond0 @@ -1416,8 +1447,85 @@ RHEL 4 as well. On kernels exhibiting this problem, it will be impossible to configure multiple bonds with differing parameters (as they are older kernels, and also lack sysfs support). -3.4 Configuring Bonding Manually via Sysfs ------------------------------------------- +3.4 Configuring Bonding Manually via netlink +-------------------------------------------- + +The iproute2 utility can manage bonding devices over the bonding driver's +netlink interface. This is the officially endorsed and current method via +which any bonding device interactions from userspace should be done. The +iproute2 v4.x and later are known to fully support bonding. + +This interface allows dynamic configuration of all bonds in the system +without unloading the module. It also allows for adding and removing +bonds at runtime. The ifenslave utility is no longer relevant, though it +may still exist in some distributions as a wrapper script simply calling +various "ip link ..." invocations underneath. + +Use of the netlink/iproute2 interface allows you to use multiple bonds +with different configurations without having to reload the module. It also +allows you to use multiple, differently configured bonds when bonding +is compiled into the kernel. + +Creating and Destroying Bonds +----------------------------- +To add a new bond foo:: + + # ip link add foo type bond + +To remove an existing bond bar:: + + # ip link del bar + +To show all existing bonds:: + + # ip link show type bond + +Adding and Removing Ports +------------------------- +Bond component port interfaces must be in a down state to be added to a +bond. + +To connect interface eth0 to bond bond0:: + + # ip link set eth0 down + # ip link set eth0 master bond0 + +To free port eth0 from bond bond0:: + + # ip link set eth0 nomaster + +Once ports are added to the bond, bring it up:: + + # ip link set bond0 up + +When an interface eth1 is connected to a bond, among its flags in "ip link +show" output:: + + # ip link show eth1 + 3: eth1: mtu 1500 qdisc mq master bond0 state UP mode DEFAULT group default qlen 1000 + link/ether 00:11:22:33:44:55 brd ff:ff:ff:ff:ff:ff + +...we see SLAVE, indicating this interface is part of another aggregate +device (also present for team and brige ports), and we see 'master bond0', +which lets us know that it's part of the device named bond0, which is +presumably, a bonding device. We can confirm this by looking for bond0 +in the output of:: + + # ip link show type bond + +All current components of bond0 can quickly be identified by:: + + # ip link show master bond0 + +Further iproute2 enhancements may be in order to give a more detailed +overview of bonds in the system, similar to /proc/net/bonding/. + + +3.5 Configuring Bonding Manually via Sysfs (DEPRECATED) +------------------------------------------------------- + +The bonding sysfs interface is deprecated, all users are urged to use +iproute2 which manages bonds via netlink. Starting with version 3.0.0, Channel Bonding may be configured via the sysfs interface. This interface allows dynamic configuration @@ -1444,7 +1552,7 @@ To add a new bond foo:: To remove an existing bond bar:: - # echo -bar > /sys/class/net/bonding_masters + # echo -bar > /sys/class/net/bonding_master To show all existing bonds:: @@ -1456,30 +1564,30 @@ To show all existing bonds:: truncated if you have more than a few hundred bonds. This is unlikely to occur under normal operating conditions. -Adding and Removing Slaves --------------------------- -Interfaces may be enslaved to a bond using the file +Adding and Removing Ports +------------------------- +Interfaces may be connected to a bond using the file /sys/class/net//bonding/slaves. The semantics for this file are the same as for the bonding_masters file. -To enslave interface eth0 to bond bond0:: +To connect interface eth0 to bond bond0:: # ifconfig bond0 up # echo +eth0 > /sys/class/net/bond0/bonding/slaves -To free slave eth0 from bond bond0:: +To free port eth0 from bond bond0:: # echo -eth0 > /sys/class/net/bond0/bonding/slaves -When an interface is enslaved to a bond, symlinks between the +When an interface is connected to a bond, symlinks between the two are created in the sysfs filesystem. In this case, you would get -/sys/class/net/bond0/slave_eth0 pointing to /sys/class/net/eth0, and +/sys/class/net/bond0/lower_eth0 pointing to /sys/class/net/eth0, and /sys/class/net/eth0/master pointing to /sys/class/net/bond0. This means that you can tell quickly whether or not an -interface is enslaved by looking for the master symlink. Thus: +interface is connected by looking for the master symlink. Thus: # echo -eth0 > /sys/class/net/eth0/master/bonding/slaves -will free eth0 from whatever bond it is enslaved to, regardless of +will free eth0 from whatever bond it is connected to, regardless of the name of the bond interface. Changing a Bond's Configuration @@ -1536,7 +1644,7 @@ To configure the interval between learning packet transmits:: .. note:: the lp_interval is the number of seconds between instances where - the bonding driver sends learning packets to each slaves peer switch. The + the bonding driver sends learning packets to each port's peer switch. The default interval is 1 second. Example Configuration @@ -1570,7 +1678,7 @@ your init script:: echo +eth2 > /sys/class/net/bond1/bonding/slaves echo +eth3 > /sys/class/net/bond1/bonding/slaves -3.5 Configuration with Interfaces Support +3.6 Configuration with Interfaces Support ----------------------------------------- This section applies to distros which use /etc/network/interfaces file @@ -1589,7 +1697,7 @@ Example Configurations ---------------------- In /etc/network/interfaces, the following stanza will configure bond0, in -active-backup mode, with eth0 and eth1 as slaves:: +active-backup mode, with eth0 and eth1 as ports:: auto bond0 iface bond0 inet dhcp @@ -1623,7 +1731,7 @@ For a full list of ``bond-*`` supported options in /etc/network/interfaces and some more advanced examples tailored to you particular distros, see the files in /usr/share/doc/ifenslave-2.6. -3.6 Overriding Configuration for Special Cases +3.7 Overriding Configuration for Special Cases ---------------------------------------------- When using the bonding driver, the physical port which transmits a frame is @@ -1645,7 +1753,7 @@ tx_queues can be used to change this value. There is no sysfs parameter available as the allocation is done at module init time. The output of the file /proc/net/bonding/bondX has changed so the output Queue -ID is now printed for each slave:: +ID is now printed for each slave port:: Bonding Mode: fault-tolerance (active-backup) Primary Slave: None @@ -1667,18 +1775,20 @@ ID is now printed for each slave:: Permanent HW addr: 00:1a:a0:12:8f:cc Slave queue ID: 2 -The queue_id for a slave can be set using the command:: +Note: the procfs interface is considered deprecated. + +The queue_id for a port can be set using the command:: # echo "eth1:2" > /sys/class/net/bond0/bonding/queue_id Any interface that needs a queue_id set should set it with multiple calls like the one above until proper priorities are set for all interfaces. On distributions that allow configuration via initscripts, multiple 'queue_id' -arguments can be added to BONDING_OPTS to set all needed slave queues. +arguments can be added to BONDING_OPTS to set all needed port queues. These queue id's can be used in conjunction with the tc utility to configure a multiqueue qdisc and filters to bias certain traffic to transmit on certain -slave devices. For instance, say we wanted, in the above configuration to +port devices. For instance, say we wanted, in the above configuration to force all traffic bound to 192.168.1.100 to use eth1 in the bond as its output device. The following commands would accomplish this:: @@ -1695,16 +1805,16 @@ selection policy to be overridden, selecting instead qid 2, which maps to eth1. Note that qid values begin at 1. Qid 0 is reserved to initiate to the driver that normal output policy selection should take place. One benefit to simply -leaving the qid for a slave to 0 is the multiqueue awareness in the bonding +leaving the qid for a port to 0 is the multiqueue awareness in the bonding driver that is now present. This awareness allows tc filters to be placed on -slave devices as well as bond devices and the bonding driver will simply act as -a pass-through for selecting output queues on the slave device rather than +port devices as well as bond devices and the bonding driver will simply act as +a pass-through for selecting output queues on the port device rather than output port selection. This feature first appeared in bonding driver version 3.7.0 and support for -output slave selection was limited to round-robin and active-backup modes. +output port selection was limited to round-robin and active-backup modes. -3.7 Configuring LACP for 802.3ad mode in a more secure way +3.8 Configuring LACP for 802.3ad mode in a more secure way ---------------------------------------------------------- When using 802.3ad bonding mode, the Actor (host) and Partner (switch) @@ -1759,7 +1869,8 @@ few bonding parameters: Each bonding device has a read-only file residing in the /proc/net/bonding directory. The file contents include information -about the bonding configuration, options and state of each slave. +about the bonding configuration, options and state of each port. +However, note that the procfs interface is considered deprecated. For example, the contents of /proc/net/bonding/bond0 after the driver is loaded with parameters of mode=0 and miimon=1000 is @@ -1788,14 +1899,14 @@ bonding configuration, state, and version of the bonding driver. ------------------------- The network configuration can be inspected using the ifconfig -command. Bonding devices will have the MASTER flag set; Bonding slave +command. Bonding devices will have the MASTER flag set; Bonding port devices will have the SLAVE flag set. The ifconfig output does not -contain information on which slaves are associated with which masters. +contain information on which ports are associated with which bonds. -In the example below, the bond0 interface is the master -(MASTER) while eth0 and eth1 are slaves (SLAVE). Notice all slaves of +In the example below, the bond0 interface is the bond device +(MASTER) while eth0 and eth1 are ports (SLAVE). Notice all ports of bond0 have the same MAC address (HWaddr) as bond0 for all modes except -TLB and ALB that require a unique MAC address for each slave:: +TLB and ALB that require a unique MAC address for each port:: # /sbin/ifconfig bond0 Link encap:Ethernet HWaddr 00:C0:F0:1F:37:B4 @@ -1868,29 +1979,29 @@ For reasons of simplicity, and to support the use of adapters that can do VLAN hardware acceleration offloading, the bonding interface declares itself as fully hardware offloading capable, it gets the add_vid/kill_vid notifications to gather the necessary -information, and it propagates those actions to the slaves. In case +information, and it propagates those actions to the ports. In case of mixed adapter types, hardware accelerated tagged packets that should go through an adapter that is not offloading capable are "un-accelerated" by the bonding driver so the VLAN tag sits in the regular location. VLAN interfaces *must* be added on top of a bonding interface -only after enslaving at least one slave. The bonding interface has a -hardware address of 00:00:00:00:00:00 until the first slave is added. -If the VLAN interface is created prior to the first enslavement, it -would pick up the all-zeroes hardware address. Once the first slave +only after enslaving at least one port. The bonding interface has a +hardware address of 00:00:00:00:00:00 until the first port is added. +If the VLAN interface is created prior to the first port binding, it +would pick up the all-zeroes hardware address. Once the first port is attached to the bond, the bond device itself will pick up the -slave's hardware address, which is then available for the VLAN device. +port's hardware address, which is then available for the VLAN device. -Also, be aware that a similar problem can occur if all slaves +Also, be aware that a similar problem can occur if all ports are released from a bond that still has one or more VLAN interfaces on -top of it. When a new slave is added, the bonding interface will -obtain its hardware address from the first slave, which might not +top of it. When a new port is added, the bonding interface will +obtain its hardware address from the first port, which might not match the hardware address of the VLAN interfaces (which was -ultimately copied from an earlier slave). +ultimately copied from an earlier port). There are two methods to insure that the VLAN device operates -with the correct hardware address if all slaves are removed from a +with the correct hardware address if all ports are removed from a bond interface: 1. Remove all VLAN interfaces then recreate them @@ -1907,7 +2018,7 @@ mode, which might not be what you want. ================== The bonding driver at present supports two schemes for -monitoring a slave device's link state: the ARP monitor and the MII +monitoring a port device's link state: the ARP monitor and the MII monitor. At the present time, due to implementation restrictions in the @@ -1927,8 +2038,8 @@ The ARP monitor relies on the device driver itself to verify that traffic is flowing. In particular, the driver must keep up to date the last receive time, dev->last_rx. Drivers that use NETIF_F_LLTX flag must also update netdev_queue->trans_start. If they do not, then the -ARP monitor will immediately fail any slaves using that driver, and -those slaves will stay down. If networking monitoring (tcpdump, etc) +ARP monitor will immediately fail any ports using that driver, and +those ports will stay down. If networking monitoring (tcpdump, etc) shows the ARP requests and replies on the network, then it may be that your device driver is not updating last_rx and trans_start. @@ -1987,10 +2098,10 @@ up. 8.1 Adventures in Routing ------------------------- -When bonding is configured, it is important that the slave -devices not have routes that supersede routes of the master (or, +When bonding is configured, it is important that the port +devices not have routes that supersede routes of the bond (or, generally, not have routes at all). For example, suppose the bonding -device bond0 has two slaves, eth0 and eth1, and the routing table is +device bond0 has two ports, eth0 and eth1, and the routing table is as follows:: Kernel IP routing table @@ -2013,9 +2124,9 @@ as an unsolicited ARP reply (because ARP matches replies on an interface basis), and is discarded. The MII monitor is not affected by the state of the routing table. -The solution here is simply to insure that slaves do not have +The solution here is simply to insure that ports do not have routes of their own, and if for some reason they must, those routes do -not supersede routes of their master. This should generally be the +not supersede routes of their bond. This should generally be the case, but unusual configurations or errant manual or automatic static route additions may cause trouble. @@ -2037,12 +2148,12 @@ For example, given a modules.conf containing the following:: alias eth2 e1000 alias eth3 e1000 -If neither eth0 and eth1 are slaves to bond0, then when the +If neither eth0 and eth1 are ports to bond0, then when the bond0 interface comes up, the devices may end up reordered. This -happens because bonding is loaded first, then its slave device's +happens because bonding is loaded first, then its port device's drivers are loaded next. Since no other drivers have been loaded, when the e1000 driver loads, it will receive eth0 and eth1 for its -devices, but the bonding configuration tries to enslave eth2 and eth3 +devices, but the bonding configuration tries to connect eth2 and eth3 (which may later be assigned to the tg3 devices). Adding the following:: @@ -2099,7 +2210,7 @@ before any network drivers participating in a bond. This requirement is due to the interface index (ipAdEntIfIndex) being associated to the first interface found with a given IP address. That is, there is only one ipAdEntIfIndex for each IP address. For example, if eth0 and -eth1 are slaves of bond0 and the driver for eth0 is loaded before the +eth1 are ports of bond0 and the driver for eth0 is loaded before the bonding driver, the interface for the IP address will be associated with the eth0 interface. This configuration is shown below, the IP address 192.168.1.1 has an interface index of 2 which indexes to eth0 @@ -2146,25 +2257,25 @@ When running network monitoring tools, e.g., tcpdump, it is common to enable promiscuous mode on the device, so that all traffic is seen (instead of seeing only traffic destined for the local host). The bonding driver handles promiscuous mode changes to the bonding -master device (e.g., bond0), and propagates the setting to the slave +bond device (e.g., bond0), and propagates the setting to the port devices. For the balance-rr, balance-xor, broadcast, and 802.3ad modes, -the promiscuous mode setting is propagated to all slaves. +the promiscuous mode setting is propagated to all ports. For the active-backup, balance-tlb and balance-alb modes, the -promiscuous mode setting is propagated only to the active slave. +promiscuous mode setting is propagated only to the active port. -For balance-tlb mode, the active slave is the slave currently +For balance-tlb mode, the active port is the port currently receiving inbound traffic. -For balance-alb mode, the active slave is the slave used as a -"primary." This slave is used for mode-specific control traffic, for +For balance-alb mode, the active port is the port used as a +"primary." This port is used for mode-specific control traffic, for sending to peers that are unassigned or if the load is unbalanced. For the active-backup, balance-tlb and balance-alb modes, when -the active slave changes (e.g., due to a link failure), the -promiscuous setting will be propagated to the new active slave. +the active port changes (e.g., due to a link failure), the +promiscuous setting will be propagated to the new active port. 11. Configuring Bonding for High Availability ============================================= @@ -2464,7 +2575,7 @@ balance-tlb: special switch configuration is required. On the down side, in this mode all incoming traffic arrives over a single interface, this mode requires certain ethtool support in the - network device driver of the slave interfaces, and the ARP + network device driver of the port interfaces, and the ARP monitor is not available. balance-alb: @@ -2571,7 +2682,7 @@ help. Note that when a bonding interface has no active links, the driver will immediately reuse the first link that goes up, even if the updelay parameter has been specified (the updelay is ignored in this -case). If there are slave interfaces waiting for the updelay timeout +case). If there are port interfaces waiting for the updelay timeout to expire, the interface that first went into that state will be immediately reused. This reduces down time of the network if the value of updelay has been overestimated, and since this occurs only in @@ -2594,9 +2705,9 @@ It is not uncommon to observe a short burst of duplicated traffic when the bonding device is first used, or after it has been idle for some period of time. This is most easily observed by issuing a "ping" to some other host on the network, and noticing that the -output from ping flags duplicates (typically one per slave). +output from ping flags duplicates (typically one per port). -For example, on a bond in active-backup mode with five slaves +For example, on a bond in active-backup mode with five ports all connected to one switch, the output may appear as follows:: # ping -n 10.0.4.2 @@ -2618,7 +2729,7 @@ traffic to all ports until its MAC forwarding table is updated. Since the interfaces attached to the bond may occupy multiple ports on a single switch, when the switch (temporarily) floods the traffic to all ports, the bond device receives multiple copies of the same packet -(one per slave device). +(one per port device). The duplicated packet behavior is switch dependent, some switches exhibit this, and some do not. On switches that display this @@ -2752,22 +2863,22 @@ Any Ethernet type cards (you can even mix cards - a Intel EtherExpress PRO/100 and a 3com 3c905b, for example). For most modes, devices need not be of the same speed. -Starting with version 3.2.1, bonding also supports Infiniband -slaves in active-backup mode. +Starting with version 3.2.1, bonding also supports Infiniband ports, +but only in active-backup mode and with exactly 2 ports per bond. 3. How many bonding devices can I have? ---------------------------------------- There is no limit. -4. How many slaves can a bonding device have? +4. How many ports can a bonding device have? ---------------------------------------------- This is limited only by the number of network interfaces Linux supports and/or the number of network cards you can place in your -system. +system, outside of the Infiniband case. -5. What happens when a slave link dies? +5. What happens when a port link dies? ---------------------------------------- If link monitoring is enabled, then the failing device will be @@ -2819,14 +2930,14 @@ The active-backup mode should work with any Layer-II switch. 8. Where does a bonding device get its MAC address from? --------------------------------------------------------- -When using slave devices that have fixed MAC addresses, or when +When using port devices that have fixed MAC addresses, or when the fail_over_mac option is enabled, the bonding device's MAC address is -the MAC address of the active slave. +the MAC address of the active port. For other configurations, if not explicitly configured (with ifconfig or ip link), the MAC address of the bonding device is taken from -its first slave device. This MAC address is then passed to all following -slaves and remains persistent (even if the first slave is removed) until +its first port device. This MAC address is then passed to all following +ports and remains persistent (even if the first port is removed) until the bonding device is brought down or reconfigured. If you wish to change the MAC address, you can set it with @@ -2837,19 +2948,23 @@ ifconfig or ip link:: # ip link set bond0 address 66:77:88:99:aa:bb The MAC address can be also changed by bringing down/up the -device and then changing its slaves (or their order):: +device and then changing its ports (or their order):: # ifconfig bond0 down ; modprobe -r bonding # ifconfig bond0 .... up # ifenslave bond0 eth... + -- or -- + # ip link set bond0 down ; modprobe -r bonding + # ip link set bond0 .... up + # ip link set eth... master bond0 This method will automatically take the address from the next -slave that is added. +port that is added. -To restore your slaves' MAC addresses, you need to detach them -from the bond (``ifenslave -d bond0 eth0``). The bonding driver will -then restore the MAC addresses that the slaves had before they were -enslaved. +To restore your ports' MAC addresses, you need to detach them +from the bond (``ifenslave -d bond0 eth0`` or ``ip link set eth0 nomaster``). +The bonding driver will then restore the MAC addresses that the ports had +before they were connected. 16. Resources and Links ======================= -- 2.28.0