VMware vSphere ESX Host Virtual Switch Layer 2 Security Features
The virtual switch has the ability to enforce security policies to prevent virtual machines from impersonating other nodes on the network. There are three components to this feature. These should all be set to “REJECT” to enable the security feature.
•Promiscuous mode is disabled by default for all virtual machines. This prevents them from seeing unicast traffic to other nodes on the network.
•MAC address change lockdown prevents virtual machines from changing their own unicast addresses. This also prevents them from seeing unicast traffic to other nodes on the network, blocking a potential security vulnerability that is similar to but narrower than promiscuous mode.
•Forged transmit blocking, when you enable it, prevents virtual machines from sending traffic that appears to come from nodes on the network other than themselves.
Cisco Nexus 1000v Switch Layer 2 Security
MAC ACLs
MAC ACLs are ACLs that filter traffic using information in the Layer 2 header of each packet.
Port security lets you configure Layer 2 interfaces permitting inbound traffic from a restricted set of MAC addresses called secure MAC addresses. In addition, traffic from these MAC addresses is not allowed on another interface within the same VLAN. The number of MAC addresses that can be secured is configurable per interface.
DAI is used to validate ARP requests and responses as follows:
•Intercepts all ARP requests and responses on untrusted ports.
•Verifies that a packet has a valid IP-to-MAC address binding before updating the ARP cache or forwarding the packet.
•Drops invalid ARP packets.
DAI can determine the validity of an ARP packet based on valid IP-to-MAC address bindings stored in a Dynamic Host Configuration Protocol (DHCP) snooping binding database. This database is built by DHCP snooping when it is enabled on the VLANs and on the device. It may also contain static entries that you have created.
If an ARP packet is received on a trusted interface, the device forwards the packet without any checks. On untrusted interfaces, the device forwards the packet only if it is valid.
IP Source Guard is a per-interface traffic filter that permits IP traffic only when the IP address and MAC address of each packet matches the IP and MAC address bindings of dynamic or static IP source entries in the Dynamic Host Configuration Protocol (DHCP) snooping binding table.
You can enable IP Source Guard on Layer 2 interfaces that are not trusted by DHCP snooping. IP Source Guard supports interfaces that are configured to operate in access mode and trunk mode. When you initially enable IP Source Guard, all inbound IP traffic on the interface is blocked except for the following:
•DHCP packets, which DHCP snooping inspects and then forwards or drops, depending upon the results of inspecting the packet.
•IP traffic from static IP source entries that you have configured in the Cisco Nexus 1000V.
The device permits the IP traffic when DHCP snooping adds a binding table entry for the IP address and MAC address of an IP packet or when you have configured a static IP source entry.
The device drops IP packets when the IP address and MAC address of the packet do not have a binding table entry or a static IP source entry.
HyTrust recently celebrated its 3-year birthday. HyTrust was founded in October 2007 to bring secure access control and policy to virtual infrastructure, enabling wider adoption of virtualization throughout the enterprise — exactly the same focus that we have today.
It’s amazing to see what we have achieved in the last three years: great enterprise customers; solid partnerships with the major players in virtualization (VMware, Cisco, RSA, Intel and Symantec); numerous accolades, including Best of Show at VMworld; and, of course, several significant releases of HyTrust Appliance…
So we’re excited to let you know that HyTrust Appliance 2.1 is now generally available. It is chock-full of exciting new enterprise features, including protection for the control of Cisco Nexus 1000V, application-level high availability, and smart card support. As always, we have also made 2.1 available in the Community Edition form, which can be downloaded for free here:
http://info.hytrust.com/appliance.html
New HyTrust Appliance Capabilities At a Glance
Support for VMware vSphere 4.1
Integrated access control, policy and audit logging for Cisco Nexus 1000V CLI management (NX-OS command set)
Support for complex, multi-domain Active Directory environments
Single sign-on via Windows pass-through authentication with smart card integration
New ESX hardening templates including VMware Hardening Guide 4.0 and (Sarbanes Oxley) SOX hardening template
Application-level high availability (in addition to VMware HA/FT and federation)
For those of you currently evaluating HyTrust Appliance, we’d like to extend an added incentive to make your purchase in Q4: for a limited time, HyTrust is offering a free “jump-start” professional services package to help you get up and running quickly. Contact sales (sales@hytrust.com) for more information.
The vDS UI also allows a phased migration of vmnics from vSS to vDS without disruption to an operational environment. VMs can be migrated from a vSS to a vDS on the fly so long as the vDS and vSS have connectivity to the same network at the same time and the origin Port Group on the vSS and destination DV Port Group on the vDS are configured to the same VLAN.
Host Profiles provide a way to migrate multiple hosts at one time. Host Profiles use a golden profile from a migrated host to propagate a configuration to a number of other hosts.
When applying a Host Profile to a host, the host must be in Maintenance Mode. This requires VMs to be either powered down or migrated to another host.
Host Profiles are most appropriate for new installations of similarly configured hosts (i.e. same number of vmnics, same vmnic to physical switch configuration, no active VMS).
The table below summarizes the deployment situations and suggested methods for migration from vSS to vDS. Note: These are suggestions only; both methods will work within the guidelines mentioned above.
Summary of Migration Methods
Table 1 – Summary of vSS to vDS Migration Methods
DeploymentSituation
SuggestedMethod
Details
New servers, same vmnic config, no active VMs
vDS UI + HP
Migrate first host with vDS UI. Take host profile and apply to remaining hosts
<5 Existing Servers, no active VMs
vDS UI
Small number of servers. Can use host profiles, but possibly easier to continue with vDS UI
>5 Existing servers, same vmnic configs, no active VMs
vDS UI + HP
Larger number of servers with similar vmnic configuration. No active VMs so can enter maintenance mode and use Host Profiles
Existing Servers, active/operational VMs
vDS UI
Cannot use Maintenance Mode as VMs active. Phased vmnic migration suggested to ensurecontinuity of VM communications
Existing Servers, dissimilar vmnic configurations
vDS UI
Enables per host tailoring of vmnic to dvUplink PortGroup mapping
Ongoing Compliance Checking
HP
Non-disruptively check network settings are compliant with approved “golden” configuration
Note: vDS UI = Use vDS UI; HP = use Host Profiles; vDS + HP = use vDS UI to deploy first host and Host Profiles for remaining hosts.
Applying NIC Teaming Policies to DV Port Groups With a vSS, NIC teaming policies are defined on the virtual switch with an optional override on each Port Group definition. With vDS, NIC teaming policies are only defined on the DV Port Groups and apply to dvUplinks, not vmnics. The vmnics are mapped to the dvUplinks on a per host basis. This enables each host to have a different vmnic to physical host configuration and yet use the same NIC teaming policy over all hosts spanned by the vDS.
Monitoring Hash vmnic Selection in NIC Teams
The esxtop command from the ESX console can reveal the physical NIC (vmnic) used by virtual port or VM within a NIC team.
Use esxtop to see the following information:
PORT-ID represents an internal port number on the virtual switch
USED-BY column shows what that port number is used by (e.g. VMkernel, VM, etc)
TEAM-PNIC column shows what physical nic (vmnic) is being used for traffic from that virtual port (the result of the hash within the NIC team)
The remaining columns indicate the Receive and Transmit traffic rates on those ports.
To use esxtop, type esxtop from the ESX console and then type n.
A list of commands for the ESX command line interface is published in Chapter 6 of the ESX 4.0 Configuration Guide (available at http://www.vmware.com/support/pubs/). To control console output to one page at a time by adding the | more suffix to the commands. For example: esxcfg-vswitch –l | more
The amazing Dudley Smith, from VMware’s Technical Account Manager team has release a larger version of his vSphere Network Connections and Ports for ESX diagram and an accompanying excel spreadsheet listing all the TCP/IP ports for various communication purposes.
vmxnet3 – features and use information – tips and tricks
UPDATED for Windows 2008 Core
Glad to see this has been posted and we can talk about it now… please share your experiences and let us know if these tips work for you and what sort of performance benefits you’ve noticed when using this new driver.
We’ve been switching our Windows and Linux VMs to use “VMXNET Enhanced” for some time now and see public information on the new VMXNET3 NIC for guests…
This Thread has been started to help with procedures on the conversion of existing machines from older NIC to newer NIC as it is not 100% straightforward and there are some tricks to remove old hardware and change to new hardware. This would be similar in the physical world to changing from a 100 BaseT PCI Card to a GigE card. The old drivers need to be removed, new drivers installed, and IP Addresses moved over. If you just remove the old NIC and install the new one you may end up with a IP Address Conflict error saying the Address you are trying to use is already in use on another Network Interface. The problem is that when you open Device Manager the old NIC is hidden. See below for steps on how to overcome this.
Question: What is VMXNET3?
Answer: VMXNET3 builds upon VMXNET and Enhanced VMXNET as the third generation paravirtualized virtual networking NIC for guest operating systems.
New VMXNET3 features over previous version of Enhanced VMXNET include:
• MSI/MSI-X support (subject to guest operating system kernel support)
• Receive Side Scaling (supported in Windows 2008 when explicitly enabled through the device’s Advanced configuration tab)
• IPv6 checksum and TCP Segmentation Offloading (TSO) over IPv6
• VLAN off-loading
• Large TX/RX ring sizes (configured from within the virtual machine)
VMware vNetwork module that encompasses the vDS and VMXNET-3 enables inline monitoring and centralized firewall services and maintains the virtualmachine’s network run-time characteristics.
Flexible shows up in Windows Device Manager as an “VMware
Accelerated AMD PCNet Adapter” and Enhanced vmxnet show up as “VMware
PCI Ethernet Adapter”. http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1001805 Flexible — The Flexible network adapter
identifies itself as a Vlance adapter when a virtual machine boots, but
initializes itself and functions as either a Vlance or a vmxnet
adapter, depending which driver initializes it. VMware Tools versions
recent enough to know about the Flexible network adapter include the
vmxnet driver but identify it as an updated Vlance driver, so the guest
operating system uses that driver. When using the Flexible network
adapter, you can have vmxnet performance when sufficiently recent
VMware tools are installed. When an older version of VMware Tools is
installed, the Flexible adapter uses the Vlance adapter (with Vlance
performance) rather than giving no network capability at all when it
can’t find the vmxnet adapter. Enhanced vmxnet — The enhanced vmxnet adapter is
based on the vmxnet adapter but provides some high-performance features
commonly used on modern networks, such as jumbo frames. This virtual
network adapter is the current state-of-the-art device in virtual
network adapter performance, but it is available only for some guest
operating systems on ESX Server 3.5. This network adapter will become
available for additional guest operating systems in the future.
Networking Error, IP Address Already Assigned to Another Adapter
KB Article 1179
Updated Jan. 07, 2009
Why do I see an error message that “The IP address XXX.XXX.XXX.XXX…” is already assigned to another adapter?
Solution
Under certain conditions, you may see the following error message from a Windows guest operating system:
The IP address XXX.XXX.XXX.XXX you have entered for this network
adapter is already assigned to another adapter Name of adapter. Name of
adapter is hidden from the network and Dial-up Connections folder
because it is not physically in the computer or is a legacy adapter
that is not working. If the same address is assigned to both adapters
and they become active, only one of them will use this address. This
may result in incorrect system configuration. Do you want to enter a
different IP address for this adapter in the list of IP addresses in
the advanced dialog box?
In this message, XXX.XXX.XXX.XXX is an IP address that you are
trying to set and Name of adapter is the name of a network adapter that
is present in the registry but hidden in Device Manager.
This can occur when you change a network connection’s TCP/IP configuration from DHCP to a static IP address if:
You have upgraded VMware virtual network adapters (for example
when you migrate a virtual machine from an older to a new version of
VMware software.)
You have added and removed network adapters multiple times.
The cause of the error is that a network adapter with the same IP
address is in the Windows registry but is hidden in the Device Manager
(My Computer > Properties > Hardware > Device Manager). This
hidden adapter is called a ghosted network adapter.
Using the Show hidden devices option in the Device Manager (View
Show hidden devices) does not always show the old virtual NIC
(ghosted adapter) to which that IP Address is assigned
To resolve this problem, follow these steps to make the ghosted
network adapter visible in the Device Manager and uninstall the ghosted
network adapter from the registry:
1. Select Start > Run.
2. Enter cmd.exe and press Enter.
3. At the command prompt, run this command:
set devmgr_show_nonpresent_devices=1
4. Enter Start DEVMGMT.MSC and press Enter to start Device Manager.
5. Select View > Show Hidden Devices.
6. Expand the Network Adapters tree (select the plus sign next to the Network adapters entry).
7. Right-click the dimmed network adapter, and then select Uninstall.
8. Close Device Manager.
How to remove these “phantom” NICs from Windows 2008 Server Core
Copy devcon.exe over to the server core server (extract devcon.exe from \SUPPORT\TOOLS\SUPPORT.CAB on a Windows 2003 R2 x64 disc).
Run devcon.exe findall =net (this should list all NICs on the system, including the phantoms). Example output:
PCI\VEN_15AD&DEV_0720&SUBSYS_072015AD&REV_10\4&B70F118&0&0088: VMware PCI Ethernet Adapter #2
PCI\VEN_15AD&DEV_0720&SUBSYS_072015AD&REV_10\3&18D45AA6&0&88: VMware PCI Ethernet Adapter
PCI\VEN_15AD&DEV_07B0&SUBSYS_07B015AD&REV_01\FF565000EB16A3FE00: vmxnet3 Ethernet Adapter
3 matching device(s) found.
Observe that vmxnet3 was the active NIC and the others needed to be removed.
For UDP, use vmxnet3 to be able to configure a larger vNIC Rx ring size. Because UDP can be a lot more bursty (due to lack of flow-control), having a larger Rx ring size helps to provide buffering/elasticity to better absorb the bursts. The new vmxnet3 allows resizing the vNIC’s Rx ring size, up to around 1 to 2 thousand buffers. As a side note, there is some negative performance impact with larger ring size due to larger memory foot print. The new vxmnet3 vNIC is more efficient than the e1000 vNIC. Also in general, ESX 4 has some performance improvements over ESX 3.5.
The story only gets better with vSphere 4 and ESX 4 with the new Intel Nehalem processors. Initial tests from engineering show a staggering 30Gbps throughput.
Choosing a Network Adapter for Your Virtual Machine
KB Article
1001805
Updated
May 05, 2009
Details
The Virtual Machine wizard’s Choose Networks window allows you to specify a network and a network adapter. The network adapter choices available depend on these factors:
The version of the virtual machine, which depends on what host created it or most recently updated it
Whether or not the virtual machine has been updated to the latest version for the current host
The guest operating system
The Choose Networks window makes available only those network adapters that make sense for the virtual machine you are creating. Each adapter type is discussed in some detail in “Available Network Adapters,” below. Here is an overview of what Choose Networks might offer you:
For virtual machines native to VMware Workstation 4x, VMware GSX Server 3, or VMware ESX Server 2.x, you can explicitly choose between Vlance and vmxnet
For most 32bit virtual machines native to VMware Workstation 5 or 6, VMware Server 2, or VMware ESX Server 3, only the Flexible adapter is available
For most 64bit virtual machines and for 32bit Microsoft Windows Vista virtual machines, only the e1000 adapter is available
For certain guest operating systems on VMware ESX Server 3.5 and later, you can choose the Enhanced vmxnet adapter in addition to the Flexible or e1000 adapter mentioned for that guest type in the previous bullets
Solution
Available Network Adapters
The following network adapters might be available for your virtual machine, depending on the factors discussed above:
Vlance — Vlance (also called PCNet32) is a faithful virtual implementation of a common, if now somewhat aging, physical network adapter. Most 32bit guest operating systems, except for Windows Vista, have built-in support for this card so a virtual machine configured with this network adapter can use its network immediately.
vmxnet — The vmxnet virtual network adapter has no physical counterpart. VMware makes vmxnet available because Vlance, a faithful implementation of a physical card, is far from optimal for network performance in a virtual machine. Vmxnet is highly optimized for performance in a virtual machine. Because there is no physical card of type vmxnet, operating system vendors do not provide built-in drivers for this card. You must install VMware Tools to have a driver for the vmxnet network adapter available.
Flexible — The Flexible network adapter identifies itself as a Vlance adapter when a virtual machine boots, but initializes itself and functions as either a Vlance or a vmxnet adapter, depending which driver initializes it. VMware Tools versions recent enough to know about the Flexible network adapter include the vmxnet driver but identify it as an updated Vlance driver, so the guest operating system uses that driver. When using the Flexible network adapter, you can have vmxnet performance when sufficiently recent VMware tools are installed. When an older version of VMware Tools is installed, the Flexible adapter uses the Vlance adapter (with Vlance performance) rather than giving no network capability at all when it cannot find the vmxnet adapter.
e1000 — e1000 is a faithful virtual implementation of a physical network adapter that is broadly supported by newer operating systems, specifically most 64bit operating systems and both 32 and 64bit Windows Vista. e1000 performance is intermediate between Vlance and vmxnet.
Enhanced vmxnet — The enhanced vmxnet adapter is based on the vmxnet adapter but provides some high-performance features commonly used on modern networks, such as jumbo frames. This virtual network adapter is the current state-of-the-art device in virtual network adapter performance, but it is available only for some guest operating systems on ESX Server 3.5. This network adapter will become available for additional guest operating systems in the future.
32/64bit versions of Microsoft Windows 2003 (Enterprise and Datacenter Editions). You can use enhanced vmxnet adapters with other versions of the Microsoft Windows 2003 operating system, but a workaround is required to enable the option in the VI Client. For more information, see Enabling enhanced vmxnet adapters for Microsoft Windows Server 2003.
32bit version Microsoft Windows XP Professional
32/64bit versions Red Hat Enterprise Linux 5.0
32/64bit versions SUSE Linux Enterprise Server 10
64bit versions Red Hat Enterprise Linux 4.0
Enhanced VMXNET is supported only for a limited set of guest operating systems:
Adapter Caveats
This section discusses some potential issues you might have.
Migrating virtual machines that use enhanced vmxnet. Enhanced vmxnet is new with ESX Server 3.5. Virtual machines configured to have enhanced vmxnet adapters cannot migrate to older ESX Server hosts, even though virtual machines can usually migrate freely between ESX Server 3.0 and ESX Server 3.0.1.
Upgrading from ESX Server 2.x to ESX Server 3.x. When a virtual hardware upgrade operation transforms a virtual machine created on an ESX Server 2.x host to an ESX Server 3.x host, Vlance adapters are automatically upgraded to Flexible. In contrast, vmxnet adapters are not upgraded automatically because certain guest operating systems — specifically most or all Linux versions — do not reliably preserve network settings when a network adapter is replaced. Because the guest operating system thinks a Flexible adapter is still Vlance, it retains the settings in that case. If the upgrade were to replace a vmxnet adapter with a Flexible adapter, the guest operating system would erroneously discard the settings.
After the virtual hardware upgrade, the network adapter is still vmxnet, without the fallback compatibility of the Flexible adapter. Just as on the original older host, if VMware Tools is uninstalled on the virtual machine, it is unable to access its network adapters.
Network adapters on multi-boot Linux. The Virtual Machine Settings dialog box and New Virtual Machine wizard allow creation of only those virtual network adapters that are supported for the selected guest operating system. If you change the guest operating system, the existing network adapters are not affected. When you switch a multi-boot Linux system between 32bit mode and 64bit mode, a problem arises because most 32bit Linux versions do not support e1000 adapters while most 64bit Linux versions support only e1000 adapters. Consider configuring your virtual machine with one of each type of network adapter (e1000 and Flexible). You can then set up your guest operating system to use only the network adapter for which it has a driver in each mode.
You can add the second adapter any time the virtual machine is powered off, but you need to change the configured guest operating system type from 32bit to 64bit or vice-versa in order to be offered the other network adapter. Since changing that setting before rebooting into the other bit depth can potentially improve the efficiency of virtual machine scheduling, plan to change the guest operating system type setting before your first reboot into the other bit depth.
Adding virtual disks. Adding an existing older (ESX Server 2.x) virtual disk to an ESX Server 3.x virtual machine results in a de-facto downgrade of that virtual machine to ESX Server 2.x. If you are using ESX Server 3.x features, such as enhanced vmxnet or Flexible network adapters, the virtual machine becomes inconsistent. When you add an existing ESX Server 2.x virtual disk to an ESX Server 3.x machine, you should immediately use the Upgrade Virtual Hardware command to restore the virtual machine to the ESX Server 3 version.
Note: Executing Upgrade Virtual Hardware changes the ESX Server 2 virtual disk so it is no longer usable on an ESX Server 2 virtual machine. Consider making a copy of the disk before you upgrade one of the two copies to ESX Server 3 format.
If you must migrate a virtual machine between newer and older hosts, do not choose enhanced vmxnet but instead one of the older adapter types. Flexible or e1000 are offered whenever enhanced vmxnet is offered.
Our new digital products will take your workflow to all-new levels of high productivity. We know you'll find everything you need - and more! Start building with Stack.
