When we embarked on Red Hat Enterprise Linux 7 a few years ago, customers asked us for help obtaining consistent and persistent network device names in order to make locating and differentiating the interfaces easier. You can read the details here.
We have recently received input from customers running Red Hat Enterprise Linux 7 as a guest in VMware environments that they were not seeing the consistent device naming, so we dug into this a bit more. What we found is that Red Hat Enterprise Linux 7 virtual guests using the vmxnet3 driver on VMware hypervisors have names such as eno16780032 instead of an expected name such as ens192 (scheme 2). Unfortunately, this can lead to not only inconsistent network device names but sometimes network loss when the virtual guest machine or the hypervisor was rebooted.
Root cause of device naming issues
We dissected this a bit more to understand why our customers are not benefitting from the work in Red Hat Enterprise Linux 7.
The root cause was determined to be that the VMware firmware/bios was reporting to the Linux kernel meaningless data, such as the device's onboard index value, which the kernel then passed on to user-space. This resulted in unusually long name lengths of the interface name field, especially with VLANs which could result in exceeding the name limit.
When the virtual guest machine or the hypervisor was rebooted, the firmware could report new hardware information resulting in a complete change to the device interface name change, new devices being created, or no device created at all. The end result was a loss of networking on reboot and other issues for software that referenced the old device name. This behavior has so far been verified on VMware ESXi versions 5.5, 6.0 and 6.1, though other versions could be affected as well.
With the release of Red Hat Enterprise Linux 7.3, a fix was provided in systemd-219-30 that if the device had a questionable or incorrect index number reported by the firmware, then an alternate predictable network naming scheme would be used. For Red Hat Enterprise Linux virtual guests on VMware, these names are typically but not always ens192 or ens32. Note that these network device names have the prefixes of ens (PCI Express hotplug slot index numbers) versus eno (ethernet on-board index numbers) which adhere to the predictable network naming conventions. The resolution for persistent device naming in VMware environments was documented and explained in the Red Hat Enterprise Linux 7.3 Release Notes.
Errata and additional Information
We recently discovered that virtual guests that had technically incorrect long device names that were upgraded to Red Hat Enterprise Linux 7.3 inherited correct device naming, but could result in a network loss. In order to avoid this from happening, we created Bug Fix Advisory RHBA-2016:2698, which includes a pre-upgrade script to check for the long network device name using the eno (ethernet on-board) identifier. If the long network device name is detected, it will create a udev rule to maintain the current name so that the virtual guest will not experience network loss. While the longer network device name might be harmless on its own, it can be problematic with VLANs resulting in the device being renamed or removed on reboot so we recommend that customers strive for the use of one of the supported schemes identified in the Red Hat Enterprise Linux 7 Release Notes.
Additional detailed explanations, instructions to manually correct the network device name and restore networking has been documented in the Red Hat Knowledge Solution - Interface name changes after update to RHEL 7.3 and network does not start.
The errata update is available now so that customers using Red Hat Satellite or custom package repositories can simply pull this errata update into the repositories and proceed with normal upgrades to VMware guests. The Red Hat Enterprise Linux 7.3 DVD ISO images will not be updated to include the errata update as most virtual environments are using some other network repository for installs that are updated to include errata updates. Therefore, if you are upgrading VMware guests from the DVD ISO, be prepared to manually correct the network interface name.
This will affect systems that:
are running on a VMware hypervisor
and are Red Hat Enterprise Linux 7.0, 7.1, or 7.2 virtual guests upgrading to Red Hat Enterprise Linux 7.3
and are using the vmxnet3 driver
This issue does not affect new installs or upgrades of Red Hat Enterprise Linux running on other hypervisors such as Red Hat Virtualization, KVM, Microsoft Hyper-V or other, non-VMware hypervisors.
Terry Bowling is a Red Hat TAM in the NA Central region and is transitioning to Technical Product Manager. He has been migrating workloads from UNIX to Linux since 1998 and has supported business environments for major telecom and pharmaceutical companies. Most recently he has been focused on enabling our customers to migrate to container platforms using Red Hat Atomic/OpenShift, as well as adopting SAP HANA.
Twitter: https://twitter.com/terry_bowling #RHTAM
A Red Hat Technical Account Manager (TAM) is a specialized product expert who works collaboratively with IT organizations to strategically plan for successful deployments and help realize optimal performance and growth. The TAM is part of Red Hat’s world class Customer Experience and Engagement organization and provides proactive advice and guidance to help you identify and address potential problems before they occur. Should a problem arise, your TAM will own the issue and engage the best resources to resolve it as quickly as possible with minimal disruption to your business.
About the author
Terry Bowling has been designing and working with customers on UNIX and GNU/Linux environments since 1999. He brings this experience to the RHEL Product Management team to provide the best experience to assembling and deploying RHEL for customers. This includes the RHEL installer, image builder and related build services for RHEL being developed at Console.RedHat.com.