How to setup IP Multi Pathing (IPMP) public interface on Solaris for Oracle Grid Infrastructure 11.2

Using two network interfaces, vnet1 and vnet2 it is possible to create a redundant public interface that is used as one by Grid Infrastructure. If one of the interfaces (and depending on network design, links) is unavailable the public interface is still available.

Current configuration:

$ srvctl config nodeapps

Network exists: 1/165.156.22.240/255.255.255.240/vnet1, type static
VIP exists: /myserver01-vip/165.156.22.246/165.156.22.240/255.255.255.240/vnet1, hosting node myserver01
VIP exists: /myserver02-vip/165.156.22.247/165.156.22.240/255.255.255.240/vnet1, hosting node myserver02
GSD exists
ONS exists: Local port 6100, remote port 6200, EM port 2016

$ crsctl check cluster -all
**************************************************************
myserver01:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
myserver02:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************

Below are the network interfaces, you can see the VIP (.246) and SCANs (.248, .250) bound to vnet1. vnet2 has nothing running on it.

$ /usr/sbin/ifconfig -a
....
vnet1: flags=9040843 mtu 1500 index 3
        inet 165.156.22.251 netmask fffffff0 broadcast 165.156.22.255
        groupname topside
vnet1:1: flags=1000843 mtu 1500 index 3
        inet 165.156.22.244 netmask fffffff0 broadcast 165.156.22.255
vnet1:2: flags=1040843 mtu 1500 index 3
        inet 165.156.22.246 netmask fffffff0 broadcast 165.156.22.255
vnet1:4: flags=1040843 mtu 1500 index 3
        inet 165.156.22.248 netmask fffffff0 broadcast 165.156.22.255
vnet1:6: flags=1040843 mtu 1500 index 3
        inet 165.156.22.250 netmask fffffff0 broadcast 165.156.22.255
vnet2: flags=9040843 mtu 1500 index 4
        inet 165.156.22.252 netmask fffffff0 broadcast 165.156.22.255
        groupname topside
...
...

To perform the change an outage for all PUBLIC traffic is required:

$ srvctl stop listener
$ srvctl stop cvu
$ srvctl stop scan_listener
$ srvctl stop scan$ srvctl stop listener
$ srvctl stop cvu
$ srvctl stop scan_listener
$ srvctl stop scan

Check that nothing is bound to the interfaces (on all nodes in the cluster):

$ /usr/sbin/ifconfig -a

As root (SUDO setup in this case, below the SUBNET mask ends in .240, your network may be different!)

$ /opt/sfw/bin/sudo $ORACLE_HOME/bin/srvctl modify nodeapps\
         -n myserver01 -A 165.156.22.246/255.255.255.240/vnet1\|vnet2

$ srvctl config nodeapps
Network exists: 1/165.156.22.240/255.255.255.240/vnet1:vnet2, type static
VIP exists: /myserver01-vip/165.156.22.246/165.156.22.240/255.255.255.240/vnet1:vnet2, hosting node myserver01
VIP exists: /myserver02-vip/165.156.22.247/165.156.22.240/255.255.255.240/vnet1:vnet2, hosting node myserver02
GSD exists
ONS exists: Local port 6100, remote port 6200, EM port 2016

$ srvctl start nodeapps

Check the SCAN listener also:

$ srvctl config scan
SCAN name: mycluster-scan, Network: 1/165.156.22.240/255.255.255.240/vnet1:vnet2
SCAN VIP name: scan1, IP: /mycluster-scan/165.156.22.249
SCAN VIP name: scan2, IP: /mycluster-scan/165.156.22.250
SCAN VIP name: scan3, IP: /mycluster-scan/165.156.22.248
$ srvctl start scan
$ srvctl start scan_listener
$ srvctl start cvu