[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [libvirt] [Qemu-devel] RFC decoupling VM NIC provisioning fromVM NIC connection to backend networks



> -----Original Message-----
> From: qemu-devel-bounces+benve=cisco com nongnu org
[mailto:qemu-devel-
> bounces+benve=cisco com nongnu org] On Behalf Of Markus Armbruster
> Sent: Monday, October 31, 2011 7:05 AM
> To: Daniel P. Berrange
> Cc: libvir-list redhat com; David Wang (dwang2); Ram Durairaj
> (radurair); qemu-devel nongnu org; Sumit Naiksatam (snaiksat)
> Subject: Re: [Qemu-devel] [libvirt] RFC decoupling VM NIC provisioning
> fromVM NIC connection to backend networks
> 
> "Daniel P. Berrange" <berrange redhat com> writes:
> 
> > On Fri, Oct 28, 2011 at 04:15:41PM -0700, Sumit Naiksatam (snaiksat)
> wrote:
> >> Hi,
> >>
> >> In its current implementation Libvirt makes sure that the network
> >> interfaces that it passes/provision to a VM (for example to qemu[-
> kvm])
> >> are already connected to its backend (interfaces/networks) by the
> time
> >> the VM starts its boot process. In a non virtualized setup it would
> be
> >> like booting a machine with the Ethernet cable already plugged into
> a
> >> router/switch port. While in a non virtualized setup you can boot a
> >> machine first (with no physical connection to a router/switch) and
> later
> >> connect its NIC/s to the switch/router, when you boot a VM via
> Libvirt
> >> it is not possible to decouple the two actions (VM boot, cable
> >> plug/unplug).
> >>
> >> An example of case where the capability of decoupling the two
> actions
> >> mentioned above is a requirement in Quantum/NetStack which is the
> >> network service leveraged by OpenStack. The modular design of
> OpenStack
> >> allows you to:
> >> - provision VMs with NIC/s
> >> - create networks
> >> - create ports on networks
> >> - plug/unplug a VM NIC into/from a given port on a network (at
> runtime)
> >>
> >> Note that this runtime plug/unplug requirement has nothing to do
> with
> >> hot plug/unplug of NICs.
> >> The idea is more that of decoupling the provisioning of a VM from
> the
> >> connection of the VM to the network/s.
> >> This would make it possible to change (at run-time too) the
networks
> the
> >> NIC/s of a given VM are connected to.
> >>
> >> For example, when a VM boots, its interfaces should be in link down
> >> state if the network admin has not connected the VM NIC/s to any
> >> "network" yet.
> >> Even though libvirt already provides a way to change the link state
> of
> >> an a VM NIC, link state and physical connection are two different
> things
> >> and should be manageable independently.
> >>
> >> Ideally the configuration syntax should be interface type and
> hypervisor
> >> type agnostic.
> >>
> >> Let's take QEMU[-kvm] as an example - when Libvirt starts a QEMU
VM,
> it
> >> passes to QEMU a number of file descriptors that map to host
backend
> >> interfaces (for example macvtap interfaces).
> >>
> >> In order to introduce this runtime plug/unplug capability, we need
a
> >> mechanism that permits to delay the binding between the host
macvtap
> >> interfaces and the guest taps (because you cannot know the fd of
the
> >> macvtap interfaces before you create them). This means you need a
> >> mechanism that allows you to change such fd/s at runtime:
> >>
> >> - you can close/reset an fd (ie, when you disconnect a VM NIC from
> its
> >> network)
> >> - you can open/set an fd (ie, when you connect a VM NIC to a
> network)
> >>
> >> This could probably be a libvirt command that translates to a QEMU
> >> monitor command.
> >>
> >> Can the runtime plug/unplug capability described above be achieved
> >> (cleanly) with another mechanism?
> >>
> >> Is anybody working on implementing something similar?
> >
> > No, but I've long thought about doing this & it is quite
> straightforward
> > todo really. Ordinarily when we start QEMU we do
> >
> >    qemu ...  -device e1000,id=nic0,netdev=netdevnic0 \
> >              -netdev user,id=netdevnic0
> >
> > Todo what you describe we need to be able to:
> >
> >  1. Start QEMU with a NIC, but no netdev
> >  2. Add a netdev to running QEMU.
> >  3. Remove a netdev from a running QEMU
> >  4. Associate a netdev with a NIC in running QEMU
> >
> > We can do 1:
> >
> >   $ qemu ...  -device e1000,id=nic0
> >
> > But QEMU prints an annoying warning
> >
> >   Warning: nic nic0 has no peer
> >
> > We can do 2 via the monitor:
> >
> >   (qemu) netdev_add type=user,id=netdevnic0
> >
> > We can do 3 via the monitor:
> >
> >   (qemu) netdev_del netdevnic0
> >
> >
> > The problem is 4 - AFAICT we can't connect the existing NIC upto the
> newly
> > hotplugged netdev, since we can't update the 'netdev' property in
the
> NIC
> > device.
> 
> Yes, that's the missing piece.

Can we go ahead and add this missing piece to QEMU?
Is there any concern on the QEMU side?

/Chris
 
> >         Also if we delete the netdev, we can't clear out the
'netdev'
> > property in the NIC, so its dangling to a netdev that no longer
> exists.
> 
> Err, this sounds like we had a dangling pointer there.  We don't.
> 
> Until commit a083a89d, netdev_del disconnected the backend netdev from
> the frontend NIC (assigning null to property "netdev"), then deleted
> the
> netdev.
> 
> Since then, we delay the actual deletion until the NIC goes away,
> because "removing host netdev peer while guest is active leads to
> guest-visible inconsistency and/or crashes".
> 
> I figure that needs to be fixed differently to enable dynamic network
> backend switching.
> 
> > The latter is fairly harmless, since we can just re-use the name if
> adding
> > a new backend later. The first problem is a bit of a pain, unless we
> plug
> > in a 'user' backend on the CLI, and immediately netdev_del it before
> starting
> > the CPUs. Ideally we'd have some way to set qdev properties for
> devices so we
> > can associate the NIC with the new netdev.
> >
> > eg when adding a netdev:
> >
> >    (qemu) netdev_add type=user,id=netdevnic0
> >    (qemu) set nic0 netdev=netdevnic0
> >
> > Or removing one
> >
> >    (qemu) netdev_add netdevnic0
> >    (qemu) unset nic0 netdev
> 
> Please work with Anthony to get this use case covered in QOM.
> 
> > WRT to libvirt XML config. Normally you specifiy a NIC like
> >
> >      <interface type='network'>
> >       <mac address='52:54:00:0f:7d:ad'/>
> >       <source network='default'/>
> >       <model type='virtio'/>
> >     </interface>
> >
> > To boot a guest without any netdev backend present, we'd introduce a
> > new network type="none". eg
> >
> >      <interface type='none'>
> >        <mac address='52:54:00:0f:7d:ad'/>
> >        <model type='virtio'/>
> >      </interface>
> >
> > The existing API  'virDomainUpdateDevice', can then be used to
change
> > the interface config on the fly, adding or removing the netdev by
> > passing in new XML with a different 'type' attribute & <source>
> > element.
> >
> > Finally, when adding & removing the netdev backends to a running
> guest,
> > we likely want to be able to set the NIC's  link carrier, so the
> guest
> > OS sees that it has lost / gain its network connection & will thus
> > retry DHCP / IPv6 autoconfig. There is already a QEMU montior
command
> > 'set_link' for changing the NIC link carrier. A minor problem is
> that,
> > AFAICT, we can't specify the link carrier state on the command line
> > when specifying the NIC hardware, eg we would want something like
> > this when starting a guest without a netdev back
> >
> >   qemu ... -device e1000,id=nic0,link=down
> 
> Should be easy enough to do.
> 
> > And when adding a netdev we would do
> >
> >   (qemu) netdev_add user,id=netdevnic0
> >   (qemu) set nic0 netdev=netdevnic0
> >   (qemu) set_link nic0 up
> >
> > Or when removing a netdev
> >
> >   (qemu) set_link nic0 down
> >   (qemu) unset nic0 netdev
> >   (qemu) netdev_del user,id=netdevnic0



[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]