native-mac-learning-in-vsphere-6-7-removes-the-need-for-promiscuous-mode-for-nested-esxi

A new and again very helpful blog post from William Lam. Please find here William’s complete article with all comments.

Always a pleasure to read William’s great posts!

Over the years, several solutions have been developed here and here to help reduce the impact of promiscuous mode, which is a requirement for running Nested ESXi as a workload. Although these solutions worked extremely well, it however did require users to install additional software to enable this functionality. The most recent solution was a new Learnswitch VMkernel module (released as a VMware Fling) that enables MAC learning capabilities on ESXi.

Today, I am pleased to announce that with the release of vSphere 6.7, the MAC Learning functionality is now available as a native feature of the VMware Distributed Virtual Switch (VDS) and as some of you may have guessed from the title, promiscuous mode is also no longer a requirement for running Nested ESXi! I wanted to take a moment and thank Subin, Jobin, Sriram, Rajeev & Samuel from our Network and Security Business Unit (NSBU) at VMware who worked tirelessly to get this integrated and productized into ESXi. Not only will this benefit Nested ESXi workloads but also other solutions and use cases that have historically required the use of promiscuous mode. For customers who are still running ESXi 6.0 or 6.5, you should continue to use the Learnswitch Fling until you fully upgrade to vSphere 6.7.

To use the new MAC Learning functionality, you will of course need to upgrade to vSphere 6.7 (both vCenter and ESXi) but also upgrade to the latest VDS version which is 6.6. MAC Learning can be enabled on a per Distributed Virtual Portgroup bases and today, it is only available when using the vSphere API. For those that have used the VDS API to manage their VDS, you will simply use the existing ReconfigureDVPortgroup_Task() method and in 6.7, there now a new macManagementPolicy property which allows you to enable and define your MAC Learning settings. This new MAC Management Policy will also be the new preferred method for managing security policies going forward for a DV Portgroup and the previous security policy settings should no longer be used.

Disclaimer: Nested ESXi is still not officially supported by VMware. Please use at your own risk. 

To demonstrate the new MAC Learning APIs, I have created two small PowerCLI functions called Get-MacLearn and Set-MacLearn which you can download from here. You will need to make sure to download the latest PowerCLI 10.1.0release which adds support for vSphere 6.7

The Get-MacLearn function can be used to retrieve the current MAC Learning configuration for a given DV Portgroup, simple run the following command which can accept a one or more DV Portgroup names:

MAC Learning

Photo courtesy of William Lam

As we can see from the output, I currently do not have MAC Learning enabled on this DV Portgroup. We can also see new properties such the limit which defines maximum number of MAC Addresses that can be learned (4096 max) and limitPolicy which defines the switching policy (drop or accept) when exceeding the learned MAC Address limit. As mentioned earlier, the new Mac Management interface should be used to manage security policies and as part of the output, I have also include both the new and legacy security policy settings.

The Set-MacLearn function can be used to enable MAC Learning as well as specifying the security policies for a given DV Portgroup. For Nested ESXi usage, you will want to set the following:

  • MAC Learning: true
  • Promiscuous mode: False
  • Forged Transmit: True
  • MAC Changes: False
  • Limit: 4096 (optional, default is provided in the function)
  • Limit Policy: Drop (optional, default is provided in function)

MAC Learning

Photo courtesy of William Lam

Once the reconfiguration has completed, we can re-run the Get-MacLearn function to confirm our changes as shown in the screenshot below:

MAC Learning

Photo courtesy of William Lam

At this point, you are now ready to start deploying your Nested ESXi workloads to this DV Portgroup or if you performed this operation on one of your existing DVPortgroup, you have now disabled the need for promiscuous mode!

Lastly, I wanted to share one additional tool that can be useful get more information about the current learned MAC Addresses which is only available directly on the ESXi Shell. The utility is called netdebg and below are a few examples on how to use it.

Note: Please note, this tool is meant for debugging purposes and there are no guarantees this will continue to work the same way in future releases.

To list all switches both VSS and VDS, run the following command:

MAC Learning

Photo courtesy of William Lam

To check whether a given powered on VM’s DV Port has MAC Learning enabled, you can run the following and specify the DVPortID as well as the name of your VDS (which you need to use sxcfg-vswitch -l or sxcli network vswitch dvs vmware list to retrieve):

MAC Learning

Photo courtesy of William Lam

To retrieve all learned MAC Addresses on a given DV Port,you can run the following and specify the DVPortID as well as the name of your VDS (which you need to use esxcfg-vswitch -l or esxcli network vswitch dvs vmware list to retrieve):

MAC Learning

Photo courtesy of William Lam

In the screenshot above, the first address (d5:d6) is actually a VM running on top of my Nested ESXi VM and the second address (5c:98) is my Nested ESXi VM’s vmnic0. MAC Address entries will age out automatically between 10-20 minutes and no additional steps are required to clear out old learned entries.

Thanks again William for this great article, always very helpful!

CNIL
Metrics and Logs

(formerly, Opvizor Performance Analyzer)

VMware vSphere & Cloud
PERFORMANCE MONITORING, LOG ANALYSIS, LICENSE COMPLIANCE!

Monitor and Analyze Performance and Log files:
Performance monitoring for your systems and applications with log analysis (tamperproof using immudb) and license compliance (RedHat, Oracle, SAP and more) in one virtual appliance!

Subscribe to Our Newsletter

Get the latest product updates, company news, and special offers delivered right to your inbox.

Subscribe to our newsletter

Use Case - Tamper-resistant Clinical Trials

Goal:

Blockchain PoCs were unsuccessful due to complexity and lack of developers.

Still the goal of data immutability as well as client verification is a crucial. Furthermore, the system needs to be easy to use and operate (allowing backup, maintenance windows aso.).

Implementation:

immudb is running in different datacenters across the globe. All clinical trial information is stored in immudb either as transactions or the pdf documents as a whole.

Having that single source of truth with versioned, timestamped, and cryptographically verifiable records, enables a whole new way of transparency and trust.

Use Case - Finance

Goal:

Store the source data, the decision and the rule base for financial support from governments timestamped, verifiable.

A very important functionality is the ability to compare the historic decision (based on the past rulebase) with the rulebase at a different date. Fully cryptographic verifiable Time Travel queries are required to be able to achieve that comparison.

Implementation:

While the source data, rulebase and the documented decision are stored in verifiable Blobs in immudb, the transaction is stored using the relational layer of immudb.

That allows the use of immudb’s time travel capabilities to retrieve verified historic data and recalculate with the most recent rulebase.

Use Case - eCommerce and NFT marketplace

Goal:

No matter if it’s an eCommerce platform or NFT marketplace, the goals are similar:

  • High amount of transactions (potentially millions a second)
  • Ability to read and write multiple records within one transaction
  • prevent overwrite or updates on transactions
  • comply with regulations (PCI, GDPR, …)


Implementation:

immudb is typically scaled out using Hyperscaler (i. e. AWS, Google Cloud, Microsoft Azure) distributed across the Globe. Auditors are also distributed to track the verification proof over time. Additionally, the shop or marketplace applications store immudb cryptographic state information. That high level of integrity and tamper-evidence while maintaining a very high transaction speed is key for companies to chose immudb.

Use Case - IoT Sensor Data

Goal:

IoT sensor data received by devices collecting environment data needs to be stored locally in a cryptographically verifiable manner until the data is transferred to a central datacenter. The data integrity needs to be verifiable at any given point in time and while in transit.

Implementation:

immudb runs embedded on the IoT device itself and is consistently audited by external probes. The data transfer to audit is minimal and works even with minimum bandwidth and unreliable connections.

Whenever the IoT devices are connected to a high bandwidth, the data transfer happens to a data center (large immudb deployment) and the source and destination date integrity is fully verified.

Use Case - DevOps Evidence

Goal:

CI/CD and application build logs need to be stored auditable and tamper-evident.
A very high Performance is required as the system should not slow down any build process.
Scalability is key as billions of artifacts are expected within the next years.
Next to a possibility of integrity validation, data needs to be retrievable by pipeline job id or digital asset checksum.

Implementation:

As part of the CI/CD audit functionality, data is stored within immudb using the Key/Value functionality. Key is either the CI/CD job id (i. e. Jenkins or GitLab) or the checksum of the resulting build or container image.

White Paper — Registration

We will also send you the research paper
via email.

CodeNotary — Webinar

White Paper — Registration

Please let us know where we can send the whitepaper on CodeNotary Trusted Software Supply Chain. 

Become a partner

Start Your Trial

Please enter contact information to receive an email with the virtual appliance download instructions.

Start Free Trial

Please enter contact information to receive an email with the free trial details.