How to configure network bonding in linux
We will configure active-backup bonding
Then will do testing, just to disable one interface and check. we should be able to access our machine using the bond ip.
[root@server ~]# vi /etc/sysconfig/network-scripts/ifcfg-bond0
DEVICE=bond0
IPADDR=192.168.159.164
NETMASK=255.255.255.0
GATEWAY=192.168.5.1
USERCTL=no
BOOTPROTO=static
ONBOOT=yes
BONDING_OPTS="mode=1 miimon=100"
[root@server ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none
[root@server ~]#
[root@server ~]# vi /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=eth1
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none
[root@server ~]# vi /etc/modprobe.conf
alias bond0 bonding
[root@server ~]#
[root@server ~]# modprobe bonding
[root@server ~]# /etc/init.d/network restart
Shutting down interface bond0: [ OK ]
Shutting down loopback interface: [ OK ]
Bringing up loopback interface: [ OK ]
Bringing up interface bond0: [ OK ]
[root@server ~]#
[root@server ~]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.6.0 (September 26, 2009)
Bonding Mode: fault-tolerance (active-backup)
Primary Slave: None
Currently Active Slave: eth0
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Slave Interface: eth0
MII Status: up <============== Showing status up eth0
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: 00:0c:29:16:ad:b4
Slave queue ID: 0
Slave Interface: eth1
MII Status: up <============== Show status up for eth1
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: 00:0c:29:16:ad:be
Slave queue ID: 0
[root@server ~]#
[root@server ~]# ethtool eth0
Settings for eth0:
Current message level: 0x00000007 (7)
Link detected: yes
[root@server ~]# ethtool eth1
Settings for eth1:
Current message level: 0x00000007 (7)
Link detected: yes
[root@server ~]#
===============================================================================
Now we do some testing by disabling link for eth 1 and then check bond0 status
===============================================================================
[root@server ~]# grep link /var/log/messages
Jun 22 18:29:59 server kernel: eth1: link down
Jun 22 18:29:59 server kernel: bonding: bond0: link status definitely down for interface eth1, disabling it
[root@server ~]#
[root@server ~]#
[root@server ~]# ethtool eth1
Settings for eth1:
Current message level: 0x00000007 (7)
Link detected: no
[root@server ~]#
[root@server ~]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.6.0 (September 26, 2009)
Bonding Mode: fault-tolerance (active-backup)
Primary Slave: None
Currently Active Slave: eth0 <======== show link status up for eth0
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Slave Interface: eth0
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: 00:0c:29:16:ad:b4
Slave queue ID: 0
Slave Interface: eth1
MII Status: down <======== show link status down for eth1, and we still connected to our machine
Speed: Unknown hence we confirm bonding is working well
Duplex: Unknown
Link Failure Count: 1
Permanent HW addr: 00:0c:29:16:ad:be
Slave queue ID: 0
[root@server ~]#
[root@server ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UNKNOWN qlen 1000
link/ether 00:0c:29:16:ad:b4 brd ff:ff:ff:ff:ff:ff
3: eth1: <NO-CARRIER,BROADCAST,MULTICAST,SLAVE,UP> mtu 1500 qdisc pfifo_fast master bond0 state DOWN qlen 1000
link/ether 00:0c:29:16:ad:b4 brd ff:ff:ff:ff:ff:ff
4: pan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
link/ether e6:79:4f:de:c4:c1 brd ff:ff:ff:ff:ff:ff
5: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether 00:0c:29:16:ad:b4 brd ff:ff:ff:ff:ff:ff
inet 192.168.159.164/24 brd 192.168.159.255 scope global bond0
inet6 fe80::20c:29ff:fe16:adb4/64 scope link
valid_lft forever preferred_lft forever
=====================================================================================================================
Note: output of eth1 output there is LOWER_UP, flag is missing which we will match again once eth1 network recoverd
=====================================================================================================================
=========================================================
check bond0 status after connection recoverd for eth1
=========================================================
[root@server ~]# grep link /var/log/messages
Jun 22 18:32:15 server kernel: eth1: link up
Jun 22 18:32:15 server kernel: bond0: link status definitely up for interface eth1, 4294967295 Mbps full duplex.
[root@server ~]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.6.0 (September 26, 2009)
Bonding Mode: fault-tolerance (active-backup)
Primary Slave: None
Currently Active Slave: eth0
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Slave Interface: eth0
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: 00:0c:29:16:ad:b4
Slave queue ID: 0
Slave Interface: eth1
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 1
Permanent HW addr: 00:0c:29:16:ad:be
Slave queue ID: 0
[root@server ~]# ethtool eth1
Settings for eth1:
Current message level: 0x00000007 (7)
Link detected: yes
[root@server ~]#
=====================================================================================================================
Note: output of eth1 output there is LOWER_UP, flag is back which we will match again once eth1 network recoverd
=====================================================================================================================
[root@server ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UNKNOWN qlen 1000
link/ether 00:0c:29:16:ad:b4 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP qlen 1000
link/ether 00:0c:29:16:ad:b4 brd ff:ff:ff:ff:ff:ff
4: pan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
link/ether e6:79:4f:de:c4:c1 brd ff:ff:ff:ff:ff:ff
5: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether 00:0c:29:16:ad:b4 brd ff:ff:ff:ff:ff:ff
inet 192.168.159.164/24 brd 192.168.159.255 scope global bond0
inet6 fe80::20c:29ff:fe16:adb4/64 scope link
valid_lft forever preferred_lft forever
[root@server ~]#
=========================================================
Different Modes that can be used in /etc/modprobe.conf
=========================================================
balance-rr or 0 — round-robin mode for fault tolerance and load balancing.
active-backup or 1 — Sets active-backup mode for fault tolerance.
balance-xor or 2 — Sets an XOR (exclusive-or) mode for fault tolerance and load balancing.
broadcast or 3 — Sets a broadcast mode for fault tolerance. All transmissions are sent on all slave interfaces.
802.3ad or 4 — Sets an IEEE 802.3ad dynamic link aggregation mode. Creates aggregation groups that share the same speed & duplex settings.
balance-tlb or 5 — Sets a Transmit Load Balancing (TLB) mode for fault tolerance & load balancing.
balance-alb or 6 — Sets an Active Load Balancing (ALB) mode for fault tolerance & load balancing.
=====================================================
#Now following is explanation about LOWER_UP:
=====================================================
LOWER_UP is a physical layer link flag (the layer below the network layer, where IP is generally located).
LOWER_UP indicates that an Ethernet cable is plugged in device is connected to the network.
LOWER_UP differs from UP, which additionally requires the network interface to be enabled.
We will configure active-backup bonding
Then will do testing, just to disable one interface and check. we should be able to access our machine using the bond ip.
[root@server ~]# vi /etc/sysconfig/network-scripts/ifcfg-bond0
DEVICE=bond0
IPADDR=192.168.159.164
NETMASK=255.255.255.0
GATEWAY=192.168.5.1
USERCTL=no
BOOTPROTO=static
ONBOOT=yes
BONDING_OPTS="mode=1 miimon=100"
[root@server ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none
[root@server ~]#
[root@server ~]# vi /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=eth1
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none
[root@server ~]# vi /etc/modprobe.conf
alias bond0 bonding
[root@server ~]#
[root@server ~]# modprobe bonding
[root@server ~]# /etc/init.d/network restart
Shutting down interface bond0: [ OK ]
Shutting down loopback interface: [ OK ]
Bringing up loopback interface: [ OK ]
Bringing up interface bond0: [ OK ]
[root@server ~]#
[root@server ~]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.6.0 (September 26, 2009)
Bonding Mode: fault-tolerance (active-backup)
Primary Slave: None
Currently Active Slave: eth0
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Slave Interface: eth0
MII Status: up <============== Showing status up eth0
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: 00:0c:29:16:ad:b4
Slave queue ID: 0
Slave Interface: eth1
MII Status: up <============== Show status up for eth1
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: 00:0c:29:16:ad:be
Slave queue ID: 0
[root@server ~]#
[root@server ~]# ethtool eth0
Settings for eth0:
Current message level: 0x00000007 (7)
Link detected: yes
[root@server ~]# ethtool eth1
Settings for eth1:
Current message level: 0x00000007 (7)
Link detected: yes
[root@server ~]#
===============================================================================
Now we do some testing by disabling link for eth 1 and then check bond0 status
===============================================================================
[root@server ~]# grep link /var/log/messages
Jun 22 18:29:59 server kernel: eth1: link down
Jun 22 18:29:59 server kernel: bonding: bond0: link status definitely down for interface eth1, disabling it
[root@server ~]#
[root@server ~]#
[root@server ~]# ethtool eth1
Settings for eth1:
Current message level: 0x00000007 (7)
Link detected: no
[root@server ~]#
[root@server ~]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.6.0 (September 26, 2009)
Bonding Mode: fault-tolerance (active-backup)
Primary Slave: None
Currently Active Slave: eth0 <======== show link status up for eth0
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Slave Interface: eth0
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: 00:0c:29:16:ad:b4
Slave queue ID: 0
Slave Interface: eth1
MII Status: down <======== show link status down for eth1, and we still connected to our machine
Speed: Unknown hence we confirm bonding is working well
Duplex: Unknown
Link Failure Count: 1
Permanent HW addr: 00:0c:29:16:ad:be
Slave queue ID: 0
[root@server ~]#
[root@server ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UNKNOWN qlen 1000
link/ether 00:0c:29:16:ad:b4 brd ff:ff:ff:ff:ff:ff
3: eth1: <NO-CARRIER,BROADCAST,MULTICAST,SLAVE,UP> mtu 1500 qdisc pfifo_fast master bond0 state DOWN qlen 1000
link/ether 00:0c:29:16:ad:b4 brd ff:ff:ff:ff:ff:ff
4: pan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
link/ether e6:79:4f:de:c4:c1 brd ff:ff:ff:ff:ff:ff
5: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether 00:0c:29:16:ad:b4 brd ff:ff:ff:ff:ff:ff
inet 192.168.159.164/24 brd 192.168.159.255 scope global bond0
inet6 fe80::20c:29ff:fe16:adb4/64 scope link
valid_lft forever preferred_lft forever
=====================================================================================================================
Note: output of eth1 output there is LOWER_UP, flag is missing which we will match again once eth1 network recoverd
=====================================================================================================================
=========================================================
check bond0 status after connection recoverd for eth1
=========================================================
[root@server ~]# grep link /var/log/messages
Jun 22 18:32:15 server kernel: eth1: link up
Jun 22 18:32:15 server kernel: bond0: link status definitely up for interface eth1, 4294967295 Mbps full duplex.
[root@server ~]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.6.0 (September 26, 2009)
Bonding Mode: fault-tolerance (active-backup)
Primary Slave: None
Currently Active Slave: eth0
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Slave Interface: eth0
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 0
Permanent HW addr: 00:0c:29:16:ad:b4
Slave queue ID: 0
Slave Interface: eth1
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 1
Permanent HW addr: 00:0c:29:16:ad:be
Slave queue ID: 0
[root@server ~]# ethtool eth1
Settings for eth1:
Current message level: 0x00000007 (7)
Link detected: yes
[root@server ~]#
=====================================================================================================================
Note: output of eth1 output there is LOWER_UP, flag is back which we will match again once eth1 network recoverd
=====================================================================================================================
[root@server ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UNKNOWN qlen 1000
link/ether 00:0c:29:16:ad:b4 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP qlen 1000
link/ether 00:0c:29:16:ad:b4 brd ff:ff:ff:ff:ff:ff
4: pan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
link/ether e6:79:4f:de:c4:c1 brd ff:ff:ff:ff:ff:ff
5: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
link/ether 00:0c:29:16:ad:b4 brd ff:ff:ff:ff:ff:ff
inet 192.168.159.164/24 brd 192.168.159.255 scope global bond0
inet6 fe80::20c:29ff:fe16:adb4/64 scope link
valid_lft forever preferred_lft forever
[root@server ~]#
=========================================================
Different Modes that can be used in /etc/modprobe.conf
=========================================================
balance-rr or 0 — round-robin mode for fault tolerance and load balancing.
active-backup or 1 — Sets active-backup mode for fault tolerance.
balance-xor or 2 — Sets an XOR (exclusive-or) mode for fault tolerance and load balancing.
broadcast or 3 — Sets a broadcast mode for fault tolerance. All transmissions are sent on all slave interfaces.
802.3ad or 4 — Sets an IEEE 802.3ad dynamic link aggregation mode. Creates aggregation groups that share the same speed & duplex settings.
balance-tlb or 5 — Sets a Transmit Load Balancing (TLB) mode for fault tolerance & load balancing.
balance-alb or 6 — Sets an Active Load Balancing (ALB) mode for fault tolerance & load balancing.
=====================================================
#Now following is explanation about LOWER_UP:
=====================================================
LOWER_UP is a physical layer link flag (the layer below the network layer, where IP is generally located).
LOWER_UP indicates that an Ethernet cable is plugged in device is connected to the network.
LOWER_UP differs from UP, which additionally requires the network interface to be enabled.
