In the Newton release, the default vhostuser mode in Open vSwitch (OvS) is dpdkvhostuser. In Red Hat OpenStack Platform 10 GA, and subsequent updates till 29th June, the default vhostuser mode is dpdkvhostuser. With the latest update to OSP10 (post 29th June 2018), the default mode has been changed to dpdkvhostuserclient. This post provides the information on vhostuser migration and verifying the vhostuser modes of the VMs created with dpdkvhostuser mode.
In order to understand the difference between the two modes and the advantage of moving to dpdkvhostuserclient mode, read the OvS documentation onvhostuser modes. In short, vhostuser allows Qemu to fetch/put network data to OvS-DPDK without overloading Qemu with the translation. And the vhostuser socket is a UNIX domain socket, created to establish the communication between Qemu and OvS-DPDK. This communication follows a specific messaging format detailed in theQemu's vhost user document.
In the dpdkvhostuserclient mode, Qemu acts as the server and creates the vhostuser socket. And OvS acts as client which connects to the created socket. Using this mode, crashes/restarts on the OvS-DPDK application does not require to restart the VM, the client will establish the connection again after starting again.
The steps for migration existing VMs with server to client mode in OvS is detailed in the minor update documentation, along with the limitations. Let’s see how and where to check the vhostuser mode information in both Qemu and OvS. The following sections shows the difference that these two modes create on a compute node in a TripleO deployment. This is applicable since the Red Hat OpenStack Platform 10 release.
dpdkvhostuser
In the dpdkvhostuser mode, the socket will be created by OvS in the /var/run/openvswitch directory. The OvS service file should be patched to make OvS run with qemu as group ownership. This allows the Qemu, which is the client, to connect to the socket.
This snippet shows that the vhostuser socket is created with the ownership as root:qemu in the default directory.
$ ll /var/run/openvswitch/vhu1879c31a-fa
srwxrwxr-x. 1 root qemu 0 Jun 19 04:38 /var/run/openvswitch/vhu1879c31a-fa
This snippet shows that the vhostuser socket's port details attached to the br-int bridge, which also shows the mode of the socket.
# Partial output
$ ovs-vstl show
Bridge br-int
Port "vhu1879c31a-fa"
tag: 1
Interface "vhu1879c31a-fa"
type: dpdkvhostuser
The OvS process runs with the patched ownership. This snippet shows that the ovs-vswitchd process runs with the root:qemu ownership.
# Partial output
$ ps ax -o user,group,pid,command,args | grep -e ovsdb-server -e \
ovs-vswitchd
root root 11770 ovsdb-server /etc/openvswit
root qemu 11806 ovs-vswitchd unix:/var/run/
This snippet shows the extract from the virsh xml of a VM with vhostuser socket is connected in the client mode.
# Partial output
$ virsh dumpxml instance-00000002
<interface type='vhostuser'>
<mac address='fa:16:3e:4b:ea:c4'/>
<source type='unix' path='/var/run/openvswitch/vhu1879c31a-fa' \
mode='client'/>
<model type='virtio'/>
<alias name='net0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03' \
function='0x0'/>
</interface>
The libvirt logs shows the socket and its mode attached to the VM. If there is any failure in the migration, this log will help to understand the mode of the vhostuser socket attached to the VM.
# Log from /var/log/libvirt/qemu/instance-00000002.log file
-chardev socket,id=charnet0,path=/var/run/openvswitch/vhueacd1f04-7a
The process associated with the VM runs with qemu:qemu ownership.
# Partial output
$ ps ax -o user,group,pid,command,args | grep -e qemu-kvm
qemu qemu 166572 /usr/libexec/qemu-kvm -name \
/usr/libexec/qemu-kvm -name guest=instance-00000002,debug-threads=on
dpdkvhostuserclient
In the dpdkvhostuserclient mode, the socket will be created by Qemu in the /var/lib/vhost_sockets directory. A new group hugetlbfs has been created which will be used to share the socket between Qemu and OvS. The group value in the /etc/libvirt/qemu.conf will be configured with the new group hugetlbfs. All the VMs created then on will run with qemu:hugetlbfs ownership. OvS will run with the ownership openvswitch:hugetlbfs, which has been introduced from OvS 2.8 version onwards. The configuration OVS_USER_ID in the file /etc/sysconfig/openvswitch, will configure the ownership of the OvS process that will run on the node.
The ovsdb-server service file will change the ownership of the /var/run/openvswitch and ovs-vswitchd service file will change the ownership of /dev/hugepages to the value configured using OVS_USER_ID.
Below snippets demonstrate this behavior of change in the ownership of the files and process and other configuration compare to the older vhostuser mode.
$ ll /var/lib/vhost_sockets/vhu28b691d4-0e
srwxrwxr-x. 1 qemu hugetlbfs 0 Jun 21 12:00 \
/var/lib/vhost_sockets/vhu28b691d4-0e
# Partial output
$ ovs-vstl show
Port "vhu28b691d4-0e"
tag: 1
Interface "vhu28b691d4-0e"
type: dpdkvhostuserclient
options: {vhost-server-path="/var/lib/vhost_sockets/vhu28b691d4-0e"}
# Partial output
$ ps ax -o user,group,pid,command,args | grep -e ovsdb-server -e \
ovs-vswitchd
openvsw+ hugetlb+ 1691 ovsdb-server /etc/openvswit ovsdb-server
openvsw+ hugetlb+ 1773 ovs-vswitchd unix:/var/run/ ovs-vswitchd
This snippet shows the extract from the virsh XML of a VM with Qemu as server.
# Partial output
$ virsh dumpxml instance-00000003
<interface type='vhostuser'>
<mac address='fa:16:3e:6d:49:e8'/>
<source type='unix' \
path='/var/lib/vhost_sockets/vhu28b691d4-0e' mode='server'/>
<target dev='vhu28b691d4-0e'/>
<model type='virtio'/>
<alias name='net0'/>
<address type='pci' domain='0x0000' bus='0x00' \
slot='0x03' function='0x0'/>
</interface>
# Log from /var/log/libvirt/qemu/instance-00000003.log file
-chardev socket,id=charnet0,path=/var/lib/vhost_sockets/vhu28b691d4-0e,server
# Partial output
$ ps ax -o user,group,pid,command,args | grep -e qemu-kvm
qemu hugetlb+ 152317 /usr/libexec/qemu-kvm -name \
/usr/libexec/qemu-kvm -name guest=instance-00000003,debug-threads=on
A note about group IDs (GIDs)
In Newton-based deployment, all the OpenStack services are deployed as systemd services, whereas on Queens based deployment, all the OpenStack services will be deployed using the containers.
But Open vSwitch still runs as the systemd service, which requires the vhostuser socket created by Qemu (running as nova_libvirt container) to share the same group id with the OvS running the host.
In order to facilitate the sharing of vhostuser between host and Kolla-based nova_libvirt container, the GID of hugetlbfs in the host has been aligned with the GID value of the hugetlbfs in the Kolla container. Also this allows us to enable the Newton-based deployment to upgrade to Queens without much difficulty. The GID value of the hugetlbfs group is configured as 42477.
Allowing time to migrate
Red Hat OpenStack supports both vhost user client and server mode, as some users may need some time to migrate to vhost user client mode. That said, Red Hat is recommending vhost user client mode as it has only advantages compared to server mode, and the corresponding code still exist to allow some time for users to migrate:
lib/netdev-dpdk.c: VLOG_WARN_ONCE("dpdkvhostuser ports are considered deprecated; "
lib/netdev-dpdk.c: "please migrate to dpdkvhostuserclient ports.");
This warning, and the dpdkvhostuser code, are very likely to be removed from the OVS 2.11 release upstream sometime after spring 2019.
저자 소개
Saravanan KR is a Principal Software Engineer at Red Hat.
유사한 검색 결과
채널별 검색
오토메이션
기술, 팀, 인프라를 위한 IT 자동화 최신 동향
인공지능
고객이 어디서나 AI 워크로드를 실행할 수 있도록 지원하는 플랫폼 업데이트
오픈 하이브리드 클라우드
하이브리드 클라우드로 더욱 유연한 미래를 구축하는 방법을 알아보세요
보안
환경과 기술 전반에 걸쳐 리스크를 감소하는 방법에 대한 최신 정보
엣지 컴퓨팅
엣지에서의 운영을 단순화하는 플랫폼 업데이트
인프라
세계적으로 인정받은 기업용 Linux 플랫폼에 대한 최신 정보
애플리케이션
복잡한 애플리케이션에 대한 솔루션 더 보기
오리지널 쇼
엔터프라이즈 기술 분야의 제작자와 리더가 전하는 흥미로운 스토리
제품
- Red Hat Enterprise Linux
- Red Hat OpenShift Enterprise
- Red Hat Ansible Automation Platform
- 클라우드 서비스
- 모든 제품 보기
툴
체험, 구매 & 영업
커뮤니케이션
Red Hat 소개
Red Hat은 Linux, 클라우드, 컨테이너, 쿠버네티스 등을 포함한 글로벌 엔터프라이즈 오픈소스 솔루션 공급업체입니다. Red Hat은 코어 데이터센터에서 네트워크 엣지에 이르기까지 다양한 플랫폼과 환경에서 기업의 업무 편의성을 높여 주는 강화된 기능의 솔루션을 제공합니다.