[libvirt] [Qemu-ppc] [RFC PATCH qemu] spapr_pci: Create PCI-express root bus by default

[Alexey Kardashevskiy]

On 22/11/16 00:08, Andrea Bolognani wrote:
> On Mon, 2016-11-21 at 13:12 +1100, Alexey Kardashevskiy wrote:
>>>>>     1) switch to PCI Express on newer machine types, and
>>>>>        expose some sort of capability through QMP so that
>>>>>        libvirt can know about the switch
>>>>>  
>>>>> [...]
>>>>> Option 1) would break horribly with existing libvirt
>>>>> versions, and so would Option 2) if we default to using
>>>>  
>>>> How exactly 1) will break libvirt? Migrating from pseries-2.7 to
>>>> pseries-2.8 does not work anyway, and machines are allowed to behave
>>>> different from version to version, what distinct difference will using
>>>> "pseries-pcie-X.Y" make?
>>>  
>>> Existing libvirt versions assume that pseries guests have
>>  
>> If libvirt is using just "pseries" (without a version), then having a
>> virtual PCIe-PCI bridge (and "pci.0" always available by default) will do it.
> 
> Please don't. Any device that is included in the guest
> by default and can't be turned off makes libvirt's life
> significantly more difficult (see below).
>
>> If libvirt is using a specific version of pseries, then it already knows
>> that <=2.7 has pci.0 as a root, pcie.0 otherwise. libvirt has a knowledge
>> what QEMU version has what, right?
> 
> It doesn't yet, that's the point :)
> 
> We *could* add such knowledge to libvirt[1], but *existing*
> libvirt versions would still not know about it, which means
> that upgrading QEMU withough upgrading libvirt will result
> in failure to create new guests.
>
> 
>> In what scenario will an additional machine type help?
> 
> Because then libvirt could learn that
> 
>   pseries-x.y       <->  pci.0
>   pseries-pcie-x.y  <->  pcie.0
> 
> the same way it already knows that
> 
>   pc-i440fx-x.y     <->  pci.0
>   pc-q35-x.y        <->  pcie.0
> 
> and choosing between one or the other would be, I think,
> much easier for the user as well.
>
>>> a legacy PCI root bus, and will base their PCI address
>>> allocation / PCI topology decisions on that fact: they
>>> will, for example, use legacy PCI bridges.
>>>  
>>> So if you used a new QEMU binary with a libvirt version
>>> that doesn't know about the change, new guests would end up
>>> using the wrong controllers. Existing guests would not be
>>> affected as they would stick with the older machine types,
>>> of course.
>>>  
>>>> I believe after we introduced the very first
>>>> pseries-pcie-X.Y, we will just stop adding new pseries-X.Y.
>>>  
>>> Isn't i440fx still being updated despite the fact that q35
>>> exists? Granted, there are a lot more differences between
>>> those two machine types than just the root bus type.
>>  
>> I do not know about i440<->q35 but in pseries the difference is going to be
>> very simple.
>>  
>> For example, we did not change the machine type when we switched from
>> default OHCI to XHCI, switching from PCI to PCIe does not sound like we
>> need a whole new machine type for this either.
> 
> The change from OHCI to XHCI only affected the *default* USB
> controller, which libvirt tries its best not to use anyway:
> instead, it will prefer to use '-M ...,usb=off' along with
> '-device ...' and set both the controller model and its PCI
> address explicitly, partially to shield its users from such
> changes in QEMU.


Ok. Always likes this approach really. We should start moving to this
direction with PHB - stop adding the default PHB at all when -nodefaults is
passed (or -machine pseries,pci=off ?) and let libvirt manage PHBs itself
(and provide another spapr-phb type like spapr-pcie-host-bridge or add a
"pcie_root_bus_type" property to the existing PHB type).

What will be wrong with this approach?



> 
> Let's not forget that libvirt is a management layer, and as
> such it needs to have as much control as possible over the
> virtual hardware presented to the guest, mostly for guest ABI
> compatibility purposes. Defaulting to XHCI instead of OHCI,
> same as pretty much all other defaults, is good for people who
> run QEMU directly, but libvirt needs to be able to control
> such knobs itself in order to effectively perform the task it
> was designed for.
> 
> Moreover, we're talking about a more fundamental change here:
> the PCI Root Bus is not just any random device, it's the one
> fundation upon which the entire PCI hierarchy is built. Based
> on whether the machine exposes a PCI Express Root Bus or a
> legacy PCI Express Root Bus, libvirt will create very
> different PCI hierarchies, eg. using several ioh3420 devices
> instead of a single pci-bridge device.
> 
> I'm still not sure if that enough to warrant an entirely new
> machine type, but it definitely has a more far-reaching impact
> than simply flipping the default USB controller from OHCI to
> XHCI.
> 
>>> Even if no newer pseries-x.y were to be added after
>>> introducing pseries-pcie, you could still easily create
>>> guests that use either root bus, so no loss in functionality.
>>  
>> I could do this with the existing pseries if the machine had a "root but
>> type" property.
> 
> That was indeed one of my original proposals ;)
> 
> Just note, once again, that the default for this property
> would have to be "off" or we would run into the same issues
> described above.
> 
> 
> [1] Even though, as I mentioned earlier in the thread,
>     performing version checks on machine types is frowned
>     upon as it turns into a minefield as soon as backports
>     and downstream machine types enter the picture... It's
>     much better to expose some flag through QMP for libvirt
>     to introspect
> -- 
> Andrea Bolognani / Red Hat / Virtualization
> 


-- 
Alexey