[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [libvirt] [RFC] Vhost-user backends cross-version migration support





On 02/03/2017 11:11 AM, Daniel P. Berrange wrote:
On Fri, Feb 03, 2017 at 10:41:12AM +0100, Maxime Coquelin wrote:


On 02/03/2017 10:27 AM, Daniel P. Berrange wrote:
On Thu, Feb 02, 2017 at 08:27:18PM +0200, Michael S. Tsirkin wrote:
On Thu, Feb 02, 2017 at 06:21:55PM +0000, Daniel P. Berrange wrote:
On Thu, Feb 02, 2017 at 07:31:49PM +0200, Michael S. Tsirkin wrote:
On Thu, Feb 02, 2017 at 05:29:08PM +0000, Daniel P. Berrange wrote:
On Thu, Feb 02, 2017 at 07:20:35PM +0200, Michael S. Tsirkin wrote:
On Thu, Feb 02, 2017 at 05:10:28PM +0000, Daniel P. Berrange wrote:
On Thu, Feb 02, 2017 at 06:21:55PM +0200, Michael S. Tsirkin wrote:
On Thu, Feb 02, 2017 at 03:06:21PM +0000, Daniel P. Berrange wrote:
On Thu, Feb 02, 2017 at 03:14:01PM +0100, Maxime Coquelin wrote:


On 02/01/2017 12:41 PM, Daniel P. Berrange wrote:

It depends where / how in OVS it needs to be set. The only stuff libvirt
does with OVS is to run 'add-port' and 'del-port' commands via the ovs
cli tool. We pass through arguments from the port profile stored in the
XML config.

  <interface type='bridge'>
    <source bridge='ovsbr'/>
    <virtualport type='openvswitch'>
      <parameters profileid='menial' interfaceid='09b11c53-8b5c-4eeb-8f00-d84eaa0aaa4f'/>
    </virtualport>
  </interface>

eg those things in <parameters/> get passed as cli args to the 'add-port'
command. Soo if add-port needs this new version string, then we'd need
to add the version to the openvswitch virtualport XML.

If the version is provided to OVS in a different command, then it would
probably be outside scope of libvirt.

I think it would make sense to be a parameter of the add-port command.
But it would be for vhost-user related add-port command, I didn't find
where/if this is managed in libvirt XML.

For vhost-user, libvirt does not have any interaction with OVS at
all. If the thing that's using the vhost-user UNIX socket, in turn
connects to OVS, that's outside scope of libvirt. IOW, for vhost-user
OVS it seems like that job is for Nova / os-vif to solve.

I don't think they currently understand the issues involved in
cross-version migration though.  This is a complex subject and easy to
get wrong.  It would be significantly better to figure it out at libvirt
level since it already deals with cross-version migration issues.

Libvirt considers vhost-user to be a blackbox though - it just exposes
a UNIX socket, and whatever is on the other end is completely opaque.
The fact that the other end might plumb the data stream into openvswitch
is not something libvirt should know, as we don't want to end up having
to add custom code to libvirt for every different vhost-user server
impl.

IOW, if the version str can be passed to QEMU, and then onto vhost-user
backend in QEMU, then libvirt can be involved. If the version str has
to be given to openvswitch that's not for libvirt to deall with.

Regards,
Daniel

It's not just about passing it to QEMU. The following is needed:
- need to query version when creating VM/device
- need to query supported versions when migrating VM

Those are both things that nova can do, since it knows what the vhost-user
device in question is connected to, and so can query the versions, and check
versions before triggering migration with libvirt

It can, but then it will need to query libvirt or source for the version
string since it's in the XML.

No, it wouldn't be in the XML at all. Nova on the source queries what
vhostuser version it has and what the target host has, and can prevent
the migration if they're incompatible.

This is not sufficient. Exactly the same as qemu machine type,
this must be preserved from time of install and moved
wherever VM goes.

Nova has the ability todo that.

I dont think libvirt has to be
involved at all for this, as all the info can be obtained by nova/os-vif
from the vhostuser impl it has configured.

Given we are already confused at libvirt level, I find it
highly unlikely upper levels will do the right thing.

The only confusion is about what must consume the data. I mistakenly
thought it was being proposed that the qemu vhostuser backend was
being given a version string. Now it seems the version info must be
set against the 3rd party component that vhostuser connects to, and
that is outside scope of libvirt. That is entirely managed by Nova
today, so Nova is the right thing to manage this versioning info.

This is my understanding, with as example using Nova:
1. Before starting the VM, Nova queries source host's OVS and all
   possible destination hosts' OVS to retrieve supported versions
2. Nova chooses the first common version supported by all hosts
3. Nova create the dpdkvhostuser interfaces in OVS with passing the
   selected version as parameter

That should be enough, right? Or maybe I miss something?

It isn't realistic to check all possible destination hosts when first
starting QEMU, not least becasue they may not yet exist. New hardware
may be deployed *after* QEMU is first started.

When starting the VM, it should just default to using whatever the
latest version is on the host in question.

There can be an optional host configuration parameter that admins can
set to force use of an older version.

This is how Nova deals with machine type - it defaults to latest, but
you can set hw_machine_type in nova.conf to force an older version if
you need migration compatibility between hosts with varying versions.

Ok, I see.
I thought it could be automated somehow if a list of available host
is already present.

At time of migration, Nova need to request the version of the target
host. If this version is not compatible with what the VM is currently
using, then Nova should *not* start the migration, it should raise
an error such that the scheduler retries and picks a different target
host

Thanks for the clarification.

Maxime


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]