Traffic shaping and Network I/O Control (NIOC)

My current project is to move our network infrastructure within out virtual environment to the 10GB network. I will be removing twelve 1GB network cables per host.  Each host will have two 16GB Fiber Channel cards and two 10GB ethernet cards. I will have a total of four, not twelve cables connected to each host. To be able to use these features in vCenter you need to have the Enterprise Plus licensing.

First, because I do not want a single VM guest to be able to consume the full 20GB, I need to set up traffic shaping and NIOC to make sure they get the bandwidth they require.

Traffic Shaping had little use in traditional design(s) with a large amounts of 1 GbE links to a rack mount server. Each link carried specific traffic and was paired with a secondary link for failover. Now that 10 GB adapter and switches have become less expensive. With adding 10GB for multiple networks, it has becomes vital to control the bandwidth across multiple hosts.

Network I/O Control (NIOC) controls the ingress of traffic. This is better explained as traffic from the host. NIOC manages the host’s network traffic to ensure it does not exceed the defined network shares and limits, but it does not control traffic destined to the host. This is called the egress traffic. Traffic Shaping has the ability to artificially limit traffic based on both ingress and egress.

Ingress and Egress

The terms ingress and egress are very technical and hard to grasp the concept. Here is a little break down.

Ingress traffic is entering the switch, host, and/or guest.

Egress traffic is leaving the switch, host, and/or guest.

 

 

Configure Traffic Shaping

Open Inventory -> Networking -> Resources Allocation

Here are my current settings

These are the default setting for the resources pool.  The FT has been changed to show the difference between the Low Shares and the High Shares value.  You can right click on any of these Network resource pool and click on the Edit Settings…

The Physical adapter shares drop down you can choose low, normal, high, or custom.  Above you can see the number of shares.  If you choose custom you can put in a share number if you need a system to have more or less shares than any of these default settings.

 

Configure NOIC

We need to limit the bandwidth that vMotion is allocated. If the traffic is vMotion only, we need to configure the Egress Traffic Shaping on the vMotion port group.

Open Inventory -> Networking

Choose a port group from your configured vDS. Right click and Edit settings…

Here is an example of my configuration.

• Average Bandwidth: 1,024,000 Kbits/sec – this will be the average bandwidth. Any bandwidth below this amount will be allowed for bursting.
• Peak Bandwidth: 2,048,000 Kbits/sec – this is the maximum bandwidth allow to the port group.
• Burst Size: 64,000 Kbytes – this is a throughput measurement of how long you can go beyond the average bandwidth and burst to the peak bandwidth. If the peak value and average value to the same number there is no burst.

These are a basics of using both Traffic Shaping and NIOC to control the bandwidth on your network.

Here is a clip from Networking for VMware Administrators. You can purchase the book here.

Traffic Shaping Math

Here’s a concrete example showing how to calculate how long traffic will peak in a “best case” scenario:

• Let’s assume, for easy math, that you set the average bandwidth value to 1,000 Kbps.
• You also set the peak bandwidth to 2,000 Kbps, which is twice the value of the average bandwidth.
• Finally, you configure the burst size to 1,000 kilobytes (KB). Hint—don’t forget that there are 8 bits in a byte, which means that 1,000 KB is 8,000 Kb. Big “B” is for bytes and little “b” is for bits.

If the burst bonus is completely full, which would mean that it’s the full value of the burst size (8,000 Kb), then you could peak for 4 seconds:

8,000 Kb burst size / 2,000 Kbps peak bandwidth = 8 / 2 = 4 seconds