IPMP has the following features:

  1. It eliminates a single network adapter as a single point of failure in these cases, ( a. Network adapter failure b. Network link failure)
  2. It enables interfaces to fail over within approximately 10 seconds when using the default configuration.
  3. It can be configured by adjusting the parameters in the “/etc/default/mpathd” file.
  4. It can be configured for use with both IPv4 and IPv6.
  5. It enables interfaces to be configured as standby interfaces.

{IPMP failure detection:
Probe­-based failure detection
-detects network error by sending ICMP ECHO_REQUEST messages
Link based failure detection
-detects network error by checking the IFF_RUNING

How to Configure IPMP on Solaris

*PROBE-BASED IPMP*

{Probe-­Based IPMP Requirements:

  • The Solaris 8 10/00 OS, as a minimum, must be installed.
  • Unique MAC addresses must be configured on each network interface.
  • Multiple network adapter interfaces must be connected on each subnet.
  • An IPMP group name must be assigned to the group of interfaces.
  • A test address is assigned to an interface.
  • Additional hosts or devices must exist on the same subnet.

{Configuring Probe­-Based IPMP:

Sample:

IPMP FAILOVER WITH 2 FLOATING/PUBLIC IP ADDRESS:

{2 NIC address:
e1000g0 ip : 192.168.1.1
e1000g1 ip : 192.168.1.2

{2 floating/public address:
192.168.1.3
192.168.1.4

{edit /etc/hosts with your 4 IP address:

192.168.1.3    server-1    server  loghost
192.168.1.4    server-2 loghost
192.168.1.1    server-e1000g0  loghost
192.168.1.2    server-e1000g1  loghost

Here are the step:

  1. # cat /etc/release
  2. # eeprom “local-­mac-­address?=true”
  3. # ifconfig e1000g0 192.168.1.1 netmask + broadcast + group ipmpserver deprecated -failover up
  4. # ifconfig e1000g0 addif 192.168.1.3 netmask + broadcast + failover up
  5. # ifconfig e1000g1 192.168.1.2 netmask + broadcast + group ipmpserver deprecated -failover standby up
  6. # ifconfig e1000g1 addif 192.168.1.4 netmask + broadcast + ­failover up
  7. # ifconfig ­-a

{Using configuration files:
1. # cat /etc/hostname.e1000g0
server-e1000g0 netmask + broadcast + group ipmpserver deprecated -failover up \
addif server-1 netmask + broadcast + failover up

2. # cat /etc/hostname.e1000g1
server-e1000g1 netmask + broadcast + group ipmpserver deprecated -failover standby up \
addif server-2 netmask + broadcast + failover up

3. # init 6

IPMP FAILOVER WITH 1 FLOATING/PUBLIC IP ADDRESS:

{2 NIC address:
e1000g0 ip : 192.168.1.1
e1000g1 ip : 192.168.1.2

{1 floating/public address:
192.168.1.3

{edit /etc/hosts with your 3 IP address:

192.168.1.3    server   loghost
192.168.1.1    server-e1000g0
192.168.1.2    server-e1000g1

{Using configuration files:
1. # cat /etc/hostname.e1000g0
server-e1000g0 netmask + broadcast + group ipmpserver deprecated -failover up \
addif server netmask + broadcast + failover up

2. # cat /etc/hostname.e1000g1
server-e1000g1 netmask + broadcast + group ipmpserver deprecated -failover standby up

3. # init 6

*LINK-BASED IPMP*

{Link­-Based IPMP Requirements:

  • Solaris 9 12/02 OS, at a minimum, must be installed.
  • Network interfaces must use any of the following drivers: hme, eri, ce, ge, bge, qfe, dmfe, e1000g, ixgb, nge, nxge, rge, xge
  • Unique MAC addresses must be configured on each of the interfaces.
  • An IPMP group name must be assigned to interfaces

{Configuring Link­-Based IPMP

  1. # cat /etc/release
  2. # eeprom “local­mac­address=true”
  3. # ifconfig e1000g0 192.168.1.1 netmask + broadcast + group <groupname> up
  4. # ifconfig e1000g1 192.168.1.2 netmask + broadcast + group <groupname> standby up
  5. # ifconfig ­a

{Configuring Link­-IPMP – Administration Guide – IP ServicesBased IPMP using configuration files
1. # cat /etc/hostname.e1000g0
server-e1000g0 netmask + broadcast + group <groupname> up

2. # cat /etc/hostname.e1000g1
server-e1000g1 netmask + broadcast + group <groupname> standby up

3. # init 6

{Edit file “/etc/default/mpathd” to adjust failover detection timeout.

find the line “FAILURE_DETECTION_TIME=10000″.

the default config is 10000, it mean the failover will take around 10 seconds.

To test the interface failover and failback, use these commands:

#if_mpadm -d e1000g0     >> Dettach interface and force failover
#if_mpadm -r e1000g0      >> Reattach interface and failback

NOTE:

deprecated =the interface will not used for data traffic
-failover = the test interface (physical NIC) will not failover
+failover = the floating/public ip address will failover.


Reference:

IPMP – Administration Guide – IP Services

man ifconfig
man in.mpathd
docs.sun.com

——————————————————
COMMON ERROR Messages and Diagnostics
——————————————————
Test address address is not unique; disabling probe based failure detection on interface_name

Description:
For in.mpathd to perform probe-based failure detection, each test address in the group must be unique. Since the IPv6 test address is a link-local address derived from the MAC address, each IP interface in the group must have a unique MAC address.

——-
NIC interface_name of group group_name is not plumbed for IPv[4|6] and may affect failover capability

Description:
All NICs in a multipathing group must be homogeneously plumbed. For example, if a NIC is plumbed for IPv4, then all NICs in the group must be plumbed for IPv4. The streams modules pushed on all NICs must be identical.

——-
No test address configured on interface interface_name disabling probe-based failure detection on it

Description:
In order for in.mpathd to perform probe-based failure detection on a NIC, it must be configured with a test address: IPv4, IPv6, or both.

——-
The link has come up on interface_name more than 2 times in the last minute; disabling failback until it stabilizes.

Description:
In order to prevent interfaces with intermittent hardware, such as a bad cable, from causing repeated failovers and failbacks, in.mpathd does not failback to interfaces with frequently fluctuating link states.
——-

Invalid failure detection time assuming default 10000

Description:
An invalid value was encountered for FAILURE_DETECTION_TIME in the /etc/default/mpathd file.

——-
Too small failure detection time of time assuming minimum 100

Description:
The minimum value that can be specified for FAILURE_DETECTION_TIME is currently 100 milliseconds.
——-
Invalid value for FAILBACK value

Description:
Valid values for the boolean variable FAILBACK are yes or no.

——-
Invalid value for TRACK_INTERFACES_ONLY_WITH_GROUPS value

Description:
Valid values for the boolean variable TRACK_INTERFACES_ONLY_WITH_GROUPS are yes or no.
——-

Cannot meet requested failure detection time of time ms on (inet[6] interface_name) new failure detection time for group group_name is time ms

Description:
The round trip time for ICMP probes is higher than necessary to maintain the current failure detection time. The network is probably congested or the probe targets are loaded. in.mpathd automatically increases the failure detection time to whatever it can achieve under these conditions.
——-

Improved failure detection time time ms on (inet[6] interface_name) for group group_name

Description:
The round trip time for ICMP probes has now decreased and in.mpathd has lowered the failure detection time correspondingly.
——-

NIC failure detected on interface_name

Description:
in.mpathd has detected NIC failure on interface_name, and has set the IFF_FAILED flag on NIC interface_name.
——-

Successfully failed over from NIC interface_name1 to NIC interface_name2

Description:
in.mpathd has caused the network traffic to failover from NIC interface_name1 to NIC interface_name2, which is part of the multipathing group.
——-
NIC repair detected on interface_name

Description:

in.mpathd has detected that NIC interface_name is repaired and operational. If the IFF_FAILED flag on the NIC was previously set, it will be reset.

——-
Successfully failed back to NIC interface_name

Description:
in.mpathd has restored network traffic back to NIC interface_name, which is now repaired and operational.

——-
The link has gone down on interface_name

Description:
in.mpathd has detected that the IFF_RUNNING flag for NIC interface_name has been cleared, indicating the link has gone down.

——-
The link has come up on interface_name

Description:
in.mpathd has detected that the IFF_RUNNING flag for NIC interface_name has been set, indicating the link has come up.

 

========================

UPDATE 4 FEBRUARY 2011 >> Other example, its more simple and just applied on customer site:

=========================

CONFIGURE IPMP: 2 INTERFACE WITH 1 VIRTUAL IP ADDRESS:
——————————————————
setup >> local-mac-address?=true
——–
HOSTNAME: SOLARIS10
IP ADDRESS BGE0: 192.168.0.1
IP ADDRESS BGE1: 192.168.0.2

VIRTUAL IP ADDRESS: 192.168.0.3

GROUPNAME: IPMP
——–
[/etc/hostname.bge0:
SOLARIS10 netmask + broadcast + group ipmp \
deprecated -failover up addif 192.168.0.3 netmask + broadcast + failover up

[/etc/hostname.bge1:
192.168.0.2 netmask + broadcast + group ipmp deprecated -failover standby up

[/etc/host:
::1    localhost
127.0.0.1    localhost
192.168.0.1    SOLARIS10    loghost

REBOOT SERVER

RESULT:
[ifconfig -a:
bge0: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 2
inet 192.168.0.1 netmask ffffff00 broadcast 192.168.0.255
groupname ipmp
ether 0:21:22:74:1f:5a
bge0:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
inet 192.168.0.3 netmask ffffff00 broadcast 192.168.0.255
bge1: flags=39040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,FAILED,STANDBY> mtu 1500 index 3
inet 192.168.0.2 netmask ffffff00 broadcast 192.168.0.255
groupname ipmp
ether 0:21:22:74:1f:5b
sppp0: flags=10010008d1<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST,IPv4,FIXEDMTU> mtu 1500 index 4
inet 192.168.224.2 –> 192.168.224.1 netmask ffffff00
ether 0:0:0:0:0:0

##############################################
CONFIGURE IPMP: 4 INTERFACE WITH 2 VIRTUAL IP ADDRESS:
——————————————————
setup >> local-mac-address?=true
——–
HOSTNAME: SOLARIS11
IP ADDRESS NXGE0: 192.168.0.1
IP ADDRESS NXGE1: 192.168.0.2
IP ADDRESS NXGE2: 192.168.0.3
IP ADDRESS NXGE3: 192.168.0.4

VIRTUAL IP ADDRESS: 192.168.0.5
VIRTUAL IP ADDRESS: 192.168.0.6

GROUPNAME: IPMP AND IPMP1
———-

[/etc/hostname.nxge0:
SOLARIS11 netmask + broadcast + group ipmp \
deprecated -failover up addif 192.168.0.5 netmask + broadcast + failover up

[/etc/hostname.nxge1:
192.168.0.2 netmask + broadcast + group ipmp deprecated -failover standby up

[/etc/hostname.nxge2:
192.168.0.3 netmask + broadcast + group ipmp1 \
deprecated -failover up addif 192.168.0.6 netmask + broadcast + failover up

[/etc/hostname.nxge3:
192.168.0.4 netmask + broadcast + group ipmp1 deprecated -failover standby up

[etc/host:
::1    localhost
127.0.0.1    localhost
192.168.0.1    SOLARIS11    loghost

———-
REBOOT SERVER

RESULT

[ifconfig -a:
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
nxge0: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 2
inet 192.168.0.1 netmask ffffffe0 broadcast 192.168.0.255
groupname ipmp
ether 0:22:28:74:1c:ba
nxge0:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
inet 192.168.0.5 netmask ffffffe0 broadcast 192.168.0.255
nxge1: flags=69040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,STANDBY,INACTIVE> mtu 1500 index

3
inet 192.168.0.2 netmask ffffffe0 broadcast 192.168.0.255
groupname ipmp
ether 0:22:28:74:1c:bb
nxge3: flags=9040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 4
inet 192.168.0.3 netmask ffffff00 broadcast 192.168.0.255
groupname ipmp1
ether 0:22:28:74:bb:5a
nxge3:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 4
inet 192.168.0.6 netmask ffffff00 broadcast 192.168.0.255
nxge4: flags=69040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER,STANDBY,INACTIVE> mtu 1500 index

5
inet 192.168.0.4 netmask ffffff00 broadcast 192.168.0.255
groupname ipmp1
ether 0:22:28:74:bb:5b
sppp0: flags=10010008d1<UP,POINTOPOINT,RUNNING,NOARP,MULTICAST,IPv4,FIXEDMTU> mtu 1500 index 6
inet 192.168.224.2 –> 192.168.224.1 netmask ffffff00
ether 0:0:0:0:0:0

About these ads

  1. Nik

    thanks, perfect article!

  2. narayan

    Aziz your blog is awesome, it really helps lot, keep up the good work. thanks a lot

  3. Laszlo

    Have you encountered any issues with the IPMP failing over one ip from one interface to an other, but immediately detecting duplicate address, and this brings the entire group to Failed:
    May 16 15:31:43 node2 in.mpathd[235]: [ID 594170 daemon.error] NIC failure detected on igb0 of group Production
    May 16 15:31:43 node2 in.mpathd[235]: [ID 832587 daemon.error] Successfully failed over from NIC igb0 to NIC e1000g0
    May 16 15:31:43 node2 ip: [ID 567813 kern.warning] WARNING: e1000g0:2 has duplicate address 192.168.060.133 (in use by 00:21:28:a3:49:16); disabled
    May 16 15:31:46 node2 in.mpathd[235]: [ID 168056 daemon.error] All Interfaces in group Production have failed

    Any idea would be helpful.

    Laszlo

  4. Truly no matter if someone doesn’t understand then its up to other people that they will
    help, so here it happens.




Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s



Follow

Get every new post delivered to your Inbox.

Join 78 other followers

%d bloggers like this: