Understanding VLAN, Trunk, NIC Teaming, Virtual Switch Configuration in Hyper-v Server 2012 R2

With Server virtualization you can run multiple server instances concurrently on a single physical host; yet servers are isolated from each other and operate independently. Similarly Network virtualization provides multiple virtual network infrastructures run on the same physical network with or without overlapping IP addresses. Each virtual network infrastructure operates as if they are the only virtual network running on the shared network infrastructure. Hyper-v Network Virtualization also decouples physical network from virtual network. Network virtualization can be achieved via System Center Virtual Machine Manager (SCVMM) managing multiple Hyper-v Servers, a single Hyper-v Server or clustered Hyper-v Servers. Microsoft Hyper-v Network Virtualization provides multi-tenant aware, multi-VLAN aware and non-hierarchical IP address assignment to virtual machines in conventional on-premises and cloud based data center.

Hyper-v Virtual Network Type

  • Private Virtual Network Switch allows communication between virtual machines connected to the same virtual switch. Virtual Machines connected to this type of virtual switch cannot communicate with Hyper-V Parent Partition. You can create any number of Private virtual switches.
  • Internal Virtual Network Switch can be used to allow communication between virtual machines connected to the same switch and also allow communication to the Hyper-V Parent Partition. You can create any number of internal virtual switches
  • External Virtual Network Switch allows communication between virtual machines running on the same Hyper-V Server, Hyper-V Parent Partition and Virtual Machines running on the remote Hyper-V Server. It requires a physical network adapter on the Hyper-V Host that is not mapped to any other External Virtual Network Switch. As a result, you can create External virtual switches as long as you have physical network adapters that are not mapped to any other external virtual switches.

Follow the guide lines to configure Virtual Networking in Windows Server 2012 R2 Hyper-v role installed. A highly available clustered Hyper-v server should have the following configuration parameters.

Example VLAN

Network Type VLAN ID IP Addresses
Default 1 10.10.10.1/24
Management 2 10.10.20.1/24
Live Migration 3 10.10.30.1/24
Prod Server 4 10.10.40.1/24
Dev Server 5 10.10.50.1/24
Test Server 6 10.10.60.1/24
Storage 7 10.10.70.1/24
DMZ 99 192.168.1.1/24

Example NIC Configuration with 8 network card (e.g. 2x quad NIC card)

Virtual Network Name Purpose Connected Physical Switch Port Virtual Switch Configuration
MGMT Management Network Port configured with VLAN 2 Allow Management Network ticked

Enable VLAN identification for management operating system ticked

LiveMigration Live Migration Port configured with VLAN 3 Allow Management Network un-ticked

Enable VLAN identification for management operating system ticked

iSCSI Storage Port configured with VLAN 7 Allow Management Network un-ticked

Enable VLAN identification for management operating system ticked

VirtualMachines Prod, Dev, Test, DMZ Port configured with Trunk Mode Allow Management Network un-ticked

Enable VLAN identification for management operating system un-ticked

Recommendation:

  • Do not assign VLAN ID in NIC Teaming Wizard instead assign VLAN ID in Virtual Switch Manager.
  • Configure virtual switch network as External Virtual Network.
  • Configure Physical Switch Port Aggregation using EtherChannel.
  • Configure Logical Network Aggregation using NIC Teaming Wizard.
  • Enable VLAN ID in Virtual Machine Settings.

Example Virtual Machine Network Configuration

Virtual Machine Type VLAN ID Tagged in VM>Settings>Network Adapter Enable VLAN identifier Connected Virtual Network
Prod VM 4 Ticked VirtualMachines
Dev VM 5 Ticked VirtualMachines
Test VM 6 Ticked VirtualMachines
DMZ VM with two NICs 4, 99 Ticked VirtualMachines

 

NIC Teaming with Virtual Switch

Multiple network adapters on a computer to be placed into a team for the following purposes:

  • Bandwidth aggregation
  • Traffic failover to prevent connectivity loss in the event of a network component failure

There are two basic configurations for NIC Teaming.

  • Switch-independent teaming. This configuration does not require the switch to participate in the teaming. Since in switch-independent mode the switch does not know that the network adapter is part of a team in the host, the adapters may be connected to different switches. Switch independent modes of operation do not require that the team members connect to different switches; they merely make it possible.
  • Switch-dependent teaming. This configuration that requires the switch to participate in the teaming. Switch dependent teaming require participating NIC to be connected in same physical switch. There are two modes of operation for switch-dependent teaming: Generic or static teaming (IEEE 802.3ad draft v1). Link Aggregation Control Protocol teaming (IEEE 802.1ax, LACP).

Load Balancing Algorithm

NIC teaming in Windows Server 2012 R2 supports the following traffic load distribution algorithms:

  • Hyper-V switch port. Since VMs have independent MAC addresses, the VM’s MAC address or the port it’s connected to on the Hyper-V switch can be the basis for dividing traffic.
  • Address Hashing. This algorithm creates a hash based on address components of the packet and then assigns packets that have that hash value to one of the available adapters. Usually this mechanism alone is sufficient to create a reasonable balance across the available adapters.
  • Dynamic. This algorithm takes the best aspects of each of the other two modes and combines them into a single mode. Outbound loads are distributed based on a hash of the TCP Ports and IP addresses. Dynamic mode also rebalances loads in real time so that a given outbound flow may move back and forth between team members. Inbound loads are distributed as though the Hyper-V port mode was in use.

NIC Teaming within Virtual Machine

NIC teaming in Windows Server 2012 R2 may also be deployed in a VM. This allows a VM to have virtual NICs connected to more than one Hyper-V switch and still maintain connectivity even if the physical NIC under one switch gets disconnected.

To enable NIC Teaming with virtual machine. In the Hyper-V Manager, in the settings for the VM, select the VM’s NIC and the Advanced Settings item, then enable the checkbox for NIC Teaming in the VM.

Physical Switch Configuration

  • In Trunk Mode, a virtual switch will listen to all the network traffic and forward the traffic to all the ports. In other words, network packets are sent to all the virtual machines connected to it. By default, a virtual switch in Hyper-V is configured in Trunk Mode, which means the virtual switch receives all network packets and forwards them to all the virtual machines connected to it. There is not much configuration needed to configure the virtual switch in Trunk Mode.
  • In Access Mode, the virtual switch receives network packets in which it first checks the VLAN ID tagged in the network packet. If the VLAN ID tagged in the network packet matches the one configured on the virtual switch, then the network packet is accepted by the virtual switch. Any incoming network packet that is not tagged with the same VLAN ID will be discarded by the virtual switch.

Cisco EtherChannel

EtherChannel provides automatic recovery for the loss of a link by redistributing the load across the remaining links. If a link fails, EtherChannel redirects traffic from the failed link to the remaining links in the channel without intervention. EtherChannel Negotiation Protocols are:

  • PAgP (Cisco Proprietary)
  • LACP (IEEE 802.3ad)

EtherChannel with Switch Independent NIC Teaming

This example shows how to configure an EtherChannel on a switch. It assigns two ports as static-access ports in VLAN 10 to channel 5 with the PAgP mode desirable:

1. To configure specific VLAN for teamed NIC

Switch# configure terminal
Switch(config)# interface range gigabitethernet0/1 -2
Switch(config-if-range)# switchport mode access
Switch(config-if-range)# switchport access vlan 10
Switch(config-if-range)# channel-group 5 mode desirable non-silent
Switch(config-if-range)# end

2. To configure Trunk for teamed NIC

Switch# configure terminal
Switch(config)# interface range gigabitethernet0/1 -2
Switch(config-if-range)# switchport mode Trunk
Switch(config-if-range)# channel-group 5 mode desirable non-silent
Switch(config-if-range)# end

EtherChannel with Switch dependent NIC Teaming

This example shows how to configure an EtherChannel on a switch. It assigns two ports as static-access ports in VLAN 10 to channel 5 with the LACP mode active:

Switch# configure terminal
Switch(config)# interface range gigabitethernet0/1 -2
Switch(config)#switchport
Switch(config-if-range)# switchport mode access
Switch(config-if-range)# switchport access vlan 10
Switch(config-if-range)# channel-group 5 mode active
Switch(config-if-range)# end
Switch# show port lacp-channel

This example shows how to configure a cross-stack EtherChannel. It uses LACP passive mode and assigns two ports on stack member 2 and one port on stack member 3 as static-access ports in VLAN 10 to channel 5:

Switch# configure terminal
Switch(config)# interface range gigabitethernet2/0/4 -5
Switch(config-if-range)# switchport mode access
Switch(config-if-range)# switchport access vlan 10
Switch(config-if-range)# channel-group 5 mode active
Switch(config-if-range)# exit
Switch(config)# interface gigabitethernet3/0/3
Switch(config-if)# switchport mode access
Switch(config-if)# switchport access vlan 10
Switch(config-if)# channel-group 5 mode active
Switch(config-if)# exit

Setup Dynamic Load Balance with 802.3ad NIC Teaming and load balance method: Automatic.

Switch#conf t
Switch(config)#int Gi2/0/23
Switch(config-if)#switchport
Switch(config-if)#switchport mode access
Switch(config-if)#switchport access vlan 100
Switch(config-if)#spanning-tree portfast
Switch(config-if)#channel-group 1 mode active
Switch(config)#port-channel load-balance src-mac
Switch(config)#end
Switch#show etherchannel 1 summary
Switch#show spanning-tree interface port-channel 1
Switch#show etherchannel load-balance

HP Switch Configuration

LACP Config:

PROCURVE-Core1#conf ter
PROCURVE-Core1# trunk PORT1-PORT2 (e.g. C1/C2) Trk<ID> (a.e. Trk99) LACP
PROCURVE-Core1# vlan <VLANID>
PROCURVE-Core1# untagged Trk<ID> (e.g. Trk99)
PROCURVE-Core1# show lacp
PROCURVE-Core1# show log lacp

Trunk Config:

PROCURVE-Core1#conf ter
PROCURVE-Core1# trunk PORT1-PORT2 (e.g. C1/C2) Trk<ID> (a.e. Trk99) TRUNK
PROCURVE-Core1# vlan <VLANID>
PROCURVE-Core1# untagged Trk<ID> (e.g. Trk99)
PROCURVE-Core1# show Trunk
PROCURVE-Core1# show log trunk

How to configure SMB 3.0 Multichannel in Windows Server 2012 Step by Step

SMB Multichannel

The SMB protocol follows the client-server model; the protocol level is negotiated by the client request and server response when establishing a new SMB connection. Windows Server 2012 introduces a feature called SMB 3.0 Multichannel. Multichannel provides link aggregation and fault tolerance.

SMB 3.0 introduces multipath I/O (MPIO) where multiple TCP connections can be established with given SMB session. Benefits include increase bandwidth, enable transparent network interface failover and load balancing per session.

SMB Encryption

Open following registry key

HKLM\System\CurrentControlSet\Services\LanmanServer\Parameters

  • If value of EncryptData DWORD is set to 0 then communication between SMB client and server is encrypted
  • If value of RejectUnencryptedAccess DWORD is set to 1 then communication between SMB client and server is rejected.

SMB Multichannel Requirement:

  • At least two computers that run on Windows Server 2012 R2, Windows Server 2012, or Windows 8 operating systems. No additional features have to be installed—SMB Multichannel is enabled by default.
  • Multiple network adapters in all hosts
  • One or more network adapters that support Receive Side Scaling (RSS)
  • One of more network adapters that are configured by using NIC Teaming
  • One or more network adapters that support remote direct memory access (RDMA)
  • Both NICs must be in different subnets
  • Enable NICs for client access
  • Dedicated subnets SMB storage
  • Dedicated Storage VLAN depending on if/how you do converged fabrics
  • VNX File OE version 7.1.65 and later or SMB 3.0 compliant storage
  • Port Channel Group configured in Cisco switch

TCP/IP session without Multichannel Session

  • No Automatic failover or Automatic failover if NICs are teamed
  • No Automatic failover if RDMA capability is not used
  • Only one NIC engaged
  • Only one CPU engaged
  • Can not use combined NIC bandwidth

TCP/IP session without Multichannel Session

  • Automatic failover or faster automatic failover if NICs are teamed
  • Automatic failover if RDMA capability is used. Multiple RDMA connection
  • All NICs engaged
  • CPU work load shared across all CPU cores
  • Combine NIC bandwidth

Which one to use, RDMA or RSS?

If you are looking fault tolerance and throughput then obvious choice is NIC teaming with RSS.

Adding a SMB Share in VNX Storage

  1. Create a network. Go to Settings -> Network -> Settings for File, Setup your network information
  2. Go to Storage -> Storage Configuration -> File Systems to create storage. Setup your storage configuration
  3. Go to CIFS Servers tab and create your Server configuration.
  4. Go back to your CIFS Share configuration and assign your CIFS Server as allowed and allow SMB protocol.
  5. Connect your CIFS Share with \\CIFSServer\CIFSShare and your new administrator password.

Adding a port channel group in Switch

Configuration of Cisco Switch with 2 network ports (If you have Cisco)

Switch#conf t
Switch(config)#Int PORT (a.e. Gi3/1)
Switch(config)#switchport mode access
Switch(config)#spanning-tree portfast
Switch(config)#channel-group <40> mode active
Switch(config)#Int port (a.e. Gi3/1)
Switch(config)#switchport mode access
Switch(config)#spanning-tree portfast
Switch(config)#channel-group <40> mode active

Configuration of HP Procurve with 2 network ports (If you have HP)

PROCURVE#conf ter
PROCURVE# trunk PORT1-PORT2 (a.e. C1/C2) Trk<ID> (a.e. Trk99) LACP
PROCURVE# vlan <VLANID>
PROCURVE# untagged Trk<ID> (a.e. Trk99)
PROCURVE# show lacp
PROCURVE# show log lacp

Adding SMB 3.0 Share in Hyper-v

  1. From Server Manager, click Tools and then click Hyper-V Manager
  2. Click Hyper-v Settings, Click Virtual Hard Disk, Type UNC path of SMB 3.0. Click Virtual Machine, Type UNC path of SMB 3.0
  3. Click Ok.
  4. Open PowerShell Prompt, Enable Multichannel using the following cmdlets.
  5. Configure SMB Multichannel using Windows PowerShell

Get-SmbClientConfiguration | Select EnableMultichannel

Get-SmbServerConfiguration | Select EnableMultichannel

    6. Enable Multichannel

Set-SmbServerConfiguration -EnableMultiChannel $true

Set-SmbClientConfiguration -EnableMultiChannel $true

   7. Verify Multichannel

Get-SmbConnection

Get-SmbMultichannelConnection