libvirt is an open source toolkit to manage virtualization platforms. It supports multiple hypervisors including KVM, QMU, XEN, and bhyve and targets Linux, FreeBSD, and other operating systems. Open vSwitch (OVS) is a production quality, multilayer virtual switch that is designed to enable massive network automation through programmatic extension while supporting standard management interfaces and protocols.
When used on top of Linux, libvirt implements networks using Linux bridges by default. Although using a Linux bridge is sufficient for many workloads, it may impose limitations when using some advanced features. It may also be harder to use Linux bridges with some common network features, such as VLANs, requiring extra effort to make it work. To solve these potential limitations, you can use OVS to implement libvirt networks.
[ Cheat sheet: Get a list of Linux utilities and commands for managing servers and networks. ]
OVS has many advantages and features, including:
- Standard 802.1Q VLAN model with trunking
- Link Aggregation Control Protocol (LACP)
- Multicast snooping
- Spanning Tree Protocol (STP) and Rapid Spanning Tree Protocol (RSTP)
- Fine-grained quality-of-service control
- Per-virtual machine (VM) interface traffic policing
- Visibility into inter-VM communication through NetFlow, sFlow, IPFIX, SPAN, RSPAN, and GRE-tunneled mirrors
This article shows how to implement a libvirt network using OVS and configure a VM to use this network.
I am using a system based on Fedora 37 with libvirt and OVS installed. If you don't have OVS available, you can install it with DNF:
# dnf install -y openvswitch
libvirt supports using OVS since version 0.9.11, so if you are using any modern Linux distribution, you can reproduce these steps. In addition, I am using virsh to manipulate libvirt objects and VMs and
ovs-vsctl to manipulate OVS bridges. I also assume you have previous experience with libvirt.
Create the virtual switch
Before you can define a libvirt network based on OVS, you need to create a virtual switch in OVS. Virtual switches in OVS are controlled by the daemon
ovs-vswichd and usually start through the systemd unit
# systemctl status openvswitch ● openvswitch.service - Open vSwitch Loaded: loaded (/usr/lib/systemd/system/openvswitch.service; enabled; preset: disabled) Active: active (exited) since Wed 2023-03-01 22:08:07 -03; 19min ago Main PID: 1052 (code=exited, status=0/SUCCESS) CPU: 2ms
If the daemon is not running, you can run it and enable it to automatically start:
# systemctl enable --now openvswitch
Once the daemon is running you can create a virtual switch:
# ovs-vsctl add-br ovsbr0
Then validate it was created correctly:
# ovs-vsctl show 654f0d05-ff5e-4979-a3ac-5b82b5f237a2 Bridge ovsbr0 Port ovsbr0 Interface ovsbr0 type: internal ovs_version: "2.17.0"
Now you can start playing with libvirt.
[Cheat sheet: Old Linux commands and their modern replacements ]
Define the libvirt network
Because libvirt uses Linux bridges by default, any
virsh subcommand that creates new networks will create a network based on it, so you need to create an XML file with the definition of the OVS-based network and import it to libvirt:
# cat ovs-network.xml <network> <name>ovs</name> <uuid>f58cad29-0455-439a-b533-8362669cec92</uuid> <forward mode='bridge'/> <bridge name='ovsbr0'/> <virtualport type='openvswitch'/> </network>
Then import the network in libvirt:
# virsh net-define ovs-network.xml Network ovs defined from ovs-network.xml
Note: It is possible to use
uuidgen to generate a new Universal Unique Identifier (UUID) for the network.
Just defining the network in libvirt is not sufficient for the network to be available. You must start the network and enable auto-start on boot:
# virsh net-start ovs Network ovs started # virsh net-autostart ovs Network ovs marked as autostarted # virsh net-list Name State Autostart Persistent -------------------------------------------- default active yes yes ovs active yes yes
For this example, I already have a VM deployed with the default network. I will edit its configuration to change the network to the
ovs network and start the virtual machine:
# virsh list --all Id Name State ------------------------ - rhel8 shut off # virsh edit rhel8 … <interface type='network'> <mac address='52:54:00:84:c7:4c'/> <source network='ovs'/> <----- modify here <model type='virtio'/> <address type='pci' domain='0x0000' bus='0x01' slot='0x00' function='0x0'/> </interface> … # virsh start rhel8 Domain 'rhel8' started # virsh list Id Name State ----------------------- 1 rhel8 running
If everything works,
ovs creates a new interface in your virtual switch:
# ovs-vsctl show 654f0d05-ff5e-4979-a3ac-5b82b5f237a2 Bridge ovsbr0 Port ovsbr0 Interface ovsbr0 type: internal Port vnet0 Interface vnet0 ovs_version: "2.17.0"
[ Check out Network automation for everyone, a complimentary book from Red Hat. ]
Configure a VM to use the OVS network
Because my VM was configured to use the default network, now it has no IP address configured. This happens because the VM was configured to use Dynamic Host Configuration Protocol (DHCP), which obtained the IP address from the default network's integration with dnsmasq. For now, configure a static IP address for the VM by logging into the VM console and running:
# ip addr show 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:84:c7:4c brd ff:ff:ff:ff:ff:ff # ip addr add 10.0.0.10/24 dev enp1s0 # ip addr show dev enp1s0 2: enp1s0: <BROADCAST,MULTICAST,UP,LOWER\_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether 52:54:00:84:c7:4c brd ff:ff:ff:ff:ff:ff inet 10.0.0.10/24 scope global enp1s0 valid_lft forever preferred_lft forever
OVS creates a local interface with the name of the virtual switch you created. You can use this interface to validate that it's possible to communicate with the VM:
# ip addr show ovsbr0 5: ovsbr0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 72:85:87:ed:c8:41 brd ff:ff:ff:ff:ff:ff # ip addr add 10.0.0.20/24 dev ovsbr0 # ip link set up dev ovsbr0 # ip addr show ovsbr0 5: ovsbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 link/ether 72:85:87:ed:c8:41 brd ff:ff:ff:ff:ff:ff inet 10.0.0.20/24 scope global ovsbr0 valid_lft forever preferred_lft forever inet6 fe80::7085:87ff:feed:c841/64 scope link valid_lft forever preferred_lft forever
And now the moment of truth:
# ping -c 3 10.0.0.10 PING 10.0.0.10 (10.0.0.10) 56(84) bytes of data. 64 bytes from 10.0.0.10: icmp\_seq=1 ttl=64 time=1.54 ms 64 bytes from 10.0.0.10: icmp\_seq=2 ttl=64 time=0.318 ms 64 bytes from 10.0.0.10: icmp\_seq=3 ttl=64 time=0.320 ms --- 10.0.0.10 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss
Oh yeah, it works!!!
This is a good start. You've configured a OVS virtual switch and integrated it with libvirt network. For now, you cannot do much with this configuration because the OVS bridge is not connected to the rest of the network.
In my next article, I will show how to integrate the virtual switch with the physical network and use preexisting services, such as DHCP and DNS.