Single Root I/O Virtualization (SR-IOV) Primer

Many IT organizations have embraced virtualization technologies to leverage well known benefits such as server consolidation, reduced costs, scalability, high utilization rates and so on. Although these benefits are great, sometimes they can cause performance issues due to the overwhelming usage of I/O. One method of reducing performance issues is to enable SR-IOV capabilities that allow a PCIe device to appear to be multiple separate physical PCIe devices. The purpose of SR-IOV is to provide isolation of resources that enable the scalability of VMs and to enable a near-bare metal

performance for I/O operations. SR-IOV defines two important terms:

• Physical Functions (PF)- Full PCIe devices that include the SR-IOV capabilities and abilities to move data in and out of the device.
• Virtual Functions (VFs)- ’Lightweight’ PCIe functions that contain the resources necessary for data movement but have a carefully minimized set of configuration resources.

Since we are talking about virtualization, we will closely examine VFs. When connecting VFs to virtual machine, there are 2 options:

• Regular NICs -  This option allows the connection of the virtual NIC (vNic)  to a virtual bridge that resides on top of a physical nic. VFs can be used as if they were regular physical NICs and a bridge can be configured on top of them whereby a vNic is connected to this bridge.
• Direct attachment to VMs- This process entails detaching the VFs from the host and attaching them directly to the VM. The VM will be directly attached to the hardware and the VF will disappear from the host.

There are many advantages associated with directly attaching VFs to a VM including:

• VFs have near-native performance which is suitable for faster workloads.
• Low latency - when directly attaching a VF to a VM, the software process is bypassed since the VM is directly attached to a hardware component.
• Scalability of the host is improved - by directly attaching the VFs, the CPU is bypassed which means that more CPU is available to apps in VMs.
• VMs have direct access to the hardware.
• Data protection between guests is guaranteed because each VF belongs to its own IO memory management unit (IOMMU) group. Devices that share the same IOMMU group cannot be attached to different VMs. Each IOMMU group has isolated access to the memory, therefore enabling data protection between the VMs.
• VMs can share a single physical port- VFs that share the same PF can be attached to different VMs.

In our next blog post, we will explore how Red Hat Enterprise Virtualization handles SR-IOV to supercharge your network throughput.