native-mac-learning-in-vsphere-6-7-removes-the-need-for-promiscuous-mode-for-nested-esxi

A new and again very helpful blog post from William Lam. Please find here William’s complete article with all comments.

Always a pleasure to read William’s great posts!

Over the years, several solutions have been developed here and here to help reduce the impact of promiscuous mode, which is a requirement for running Nested ESXi as a workload. Although these solutions worked extremely well, it however did require users to install additional software to enable this functionality. The most recent solution was a new Learnswitch VMkernel module (released as a VMware Fling) that enables MAC learning capabilities on ESXi.

Today, I am pleased to announce that with the release of vSphere 6.7, the MAC Learning functionality is now available as a native feature of the VMware Distributed Virtual Switch (VDS) and as some of you may have guessed from the title, promiscuous mode is also no longer a requirement for running Nested ESXi! I wanted to take a moment and thank Subin, Jobin, Sriram, Rajeev & Samuel from our Network and Security Business Unit (NSBU) at VMware who worked tirelessly to get this integrated and productized into ESXi. Not only will this benefit Nested ESXi workloads but also other solutions and use cases that have historically required the use of promiscuous mode. For customers who are still running ESXi 6.0 or 6.5, you should continue to use the Learnswitch Fling until you fully upgrade to vSphere 6.7.

To use the new MAC Learning functionality, you will of course need to upgrade to vSphere 6.7 (both vCenter and ESXi) but also upgrade to the latest VDS version which is 6.6. MAC Learning can be enabled on a per Distributed Virtual Portgroup bases and today, it is only available when using the vSphere API. For those that have used the VDS API to manage their VDS, you will simply use the existing ReconfigureDVPortgroup_Task() method and in 6.7, there now a new macManagementPolicy property which allows you to enable and define your MAC Learning settings. This new MAC Management Policy will also be the new preferred method for managing security policies going forward for a DV Portgroup and the previous security policy settings should no longer be used.

Disclaimer: Nested ESXi is still not officially supported by VMware. Please use at your own risk. 

To demonstrate the new MAC Learning APIs, I have created two small PowerCLI functions called Get-MacLearn and Set-MacLearn which you can download from here. You will need to make sure to download the latest PowerCLI 10.1.0release which adds support for vSphere 6.7

The Get-MacLearn function can be used to retrieve the current MAC Learning configuration for a given DV Portgroup, simple run the following command which can accept a one or more DV Portgroup names:

MAC Learning

Photo courtesy of William Lam

As we can see from the output, I currently do not have MAC Learning enabled on this DV Portgroup. We can also see new properties such the limit which defines maximum number of MAC Addresses that can be learned (4096 max) and limitPolicy which defines the switching policy (drop or accept) when exceeding the learned MAC Address limit. As mentioned earlier, the new Mac Management interface should be used to manage security policies and as part of the output, I have also include both the new and legacy security policy settings.

The Set-MacLearn function can be used to enable MAC Learning as well as specifying the security policies for a given DV Portgroup. For Nested ESXi usage, you will want to set the following:

  • MAC Learning: true
  • Promiscuous mode: False
  • Forged Transmit: True
  • MAC Changes: False
  • Limit: 4096 (optional, default is provided in the function)
  • Limit Policy: Drop (optional, default is provided in function)

MAC Learning

Photo courtesy of William Lam

Once the reconfiguration has completed, we can re-run the Get-MacLearn function to confirm our changes as shown in the screenshot below:

MAC Learning

Photo courtesy of William Lam

At this point, you are now ready to start deploying your Nested ESXi workloads to this DV Portgroup or if you performed this operation on one of your existing DVPortgroup, you have now disabled the need for promiscuous mode!

Lastly, I wanted to share one additional tool that can be useful get more information about the current learned MAC Addresses which is only available directly on the ESXi Shell. The utility is called netdebg and below are a few examples on how to use it.

Note: Please note, this tool is meant for debugging purposes and there are no guarantees this will continue to work the same way in future releases.

To list all switches both VSS and VDS, run the following command:

MAC Learning

Photo courtesy of William Lam

To check whether a given powered on VM’s DV Port has MAC Learning enabled, you can run the following and specify the DVPortID as well as the name of your VDS (which you need to use sxcfg-vswitch -l or sxcli network vswitch dvs vmware list to retrieve):

MAC Learning

Photo courtesy of William Lam

To retrieve all learned MAC Addresses on a given DV Port,you can run the following and specify the DVPortID as well as the name of your VDS (which you need to use esxcfg-vswitch -l or esxcli network vswitch dvs vmware list to retrieve):

MAC Learning

Photo courtesy of William Lam

In the screenshot above, the first address (d5:d6) is actually a VM running on top of my Nested ESXi VM and the second address (5c:98) is my Nested ESXi VM’s vmnic0. MAC Address entries will age out automatically between 10-20 minutes and no additional steps are required to clear out old learned entries.

Thanks again William for this great article, always very helpful!

CNIL
Metrics and Logs

(formerly, Opvizor Performance Analyzer)

VMware vSphere & Cloud
PERFORMANCE MONITORING, LOG ANALYSIS, LICENSE COMPLIANCE!

Monitor and Analyze Performance and Log files:
Performance monitoring for your systems and applications with log analysis (tamperproof using immudb) and license compliance (RedHat, Oracle, SAP and more) in one virtual appliance!

Subscribe to Our Newsletter

Get the latest product updates, company news, and special offers delivered right to your inbox.
Share on twitter
Share on linkedin
Share on facebook
Share on email

Subscribe to our newsletter

White Paper — Registration

We will send you the research paper via email.

CodeNotary — Webinar

White Paper — Registration

Please let us know who you are, so we can send you the CodeNotary Trusted Software Supply Chain white paper.

Become a partner

Start Your Trial

Please enter contact information to receive an email with the virtual appliance download instructions.

Start Free Trial

Please enter contact information to receive an email with the free trial details.