Unsegmented Network

In the default, unsegmented network in a Nutanix cluster, the Controller VM has two virtual network interfaces—eth0 and eth1. Interface eth0 is connected to the built-in external virtual switch, which is in turn connected to the external network through a bond or NIC team that contains the host’s physical uplinks. Interface eth1 is connected to an internal network that enables the CVM to communicate with the hypervisor. In this network, all traffic, whether backplane traffic or management traffic, uses interface eth0. These interfaces are on the default VLAN on the virtual switch.

Network segmentation is not supported in the following configurations:

• Clusters on which the CVMs have a manually created eth2 interface.
• Clusters on which the eth2 interface on one or more CVMs have been assigned an IP address manually. During an upgrade to an AOS release that supports network segmentation, an eth2 interface is created on each CVM in the cluster. Even though the cluster does not use these interfaces until you configure network segmentation, you must not manually configure these interfaces in any way.
• ESXi clusters in which the CVM is connected to a VMware distributed virtual switch.
• Clusters that have two (or more) vSwitches or bridges for CVM traffic isolation. In this release, the CVM management network (eth0), which carries user VM traffic, and the CVM backplane network (eth2) must reside on a single vSwitch or bridge. These CVM networks cannot be placed on separate vSwitches or bridges.

Segmented Network

In a segmented network, management traffic uses interface eth0 and the backplane traffic uses interface eth2. The backplane network uses either the default VLAN or, optionally, a separate VLAN that you specify when segmenting the network.

See Create a Backplane Network

Network segmentation is supported in the following environment:

• The hypervisor must be one of the following:
  • AHV
  • ESXi
  • Hyper-V
• The AOS version must be 5.5 or later.
• RDMA requirements:
  • Network segmentation is supported with RDMA for AHV and ESXi hypervisors only.
  • For the NX-9030-G5 platform, each node must have two Mellanox CX-3 Pro network cards.
  • For G6 platforms, each node must have two Mellanox CX-4 network cards. (For this reason, RDMA is not supported on platforms that have only one NIC per node.)
• The Controller VM interfaces eth0 and eth2 must be configured as follows:
  • On AHV, both interfaces must be on bridge br0.
  • On ESXi, both interfaces must be on vSwitch0 and must have the same physical network adapters (vmnic#) as uplinks.
• In all cases, the network adapters must be 10 GbE adapters.

You can segment the network on an existing cluster by using the Prism web console. The network segmentation process creates a separate network for backplane communications on the existing default virtual switch and places the eth2 interfaces (that are created on the CVMs during upgrade) and the host interfaces on the newly created network. From the specified subnet, IP addresses are assigned to each new interface. Two IP addresses are therefore required per node. If you specify the optional VLAN ID, the newly created interfaces are placed on the VLAN. A separate VLAN is highly recommended for the backplane network to achieve true segmentation.