[vfio-users] Strange behavior with recent update. ACS(?) & VFIO

[Mark]

Thank you for your input Alex, here is a pastie with some of the info you
were interested in:
VM powered off: http://pastie.org/10927534
00:1b.0 is the onboard, 01:00.0 and 01:00.1 are the nvidia gpu/nvidia audio.

Notice in that pastie the driver in use is snd_hda_intel, yet in this lspci
-vvv (with the VM booted up) it is changed to vfio-pci, and they are still
in the same iommu group.
VM powered on: http://pastie.org/10927535

"Perhaps you made changes to configuration files before update that were
only incorporated when the initrd was built for the newly installed kernel."

I am thinking this was the case, as re-doing initramfs seemed to fix the
issue.  The kernel had been upgraded to 4.6.4-1.

On Tue, Aug 2, 2016 at 2:39 PM, Alex Williamson <alex.l.williamson at gmail.com
> wrote:

>
>
> On Tue, Aug 2, 2016 at 3:22 PM, Mark <kram321 at gmail.com> wrote:
>
>> Hi,
>>
>> Today I updated my arch linux system. It had only been a week since the
>> last update.  Not notices applicable to my update had been posted.
>>
>> My setup is a Intel i7-4790S, ASRock mobo supporting vt-d and all the
>> goodies.
>>
>> I have libvirt using virt-manager for my win7 VM, with nvidia 750 gpu and
>> hdmi audio device passed through to it, also the onboard intel audio
>> controller (non-hdmi).  These 3 devices are all on the same iommu group.
>>
>
> I have serious doubts that this was ever the case.  The GPU and GPU audio
> are always grouped together because there's not ACS on the endpoint.  The
> root port might get involved, which might drag in other root ports if ACS
> is missing there and it's part of a multifunction device.  But I have never
> seen the _onboard_ audio get wrapped into that since it's on a separate
> slot on the root complex.  Clearly 'sudo lspci -vvv' and listing of the
> iommu groups is going to be required here.
>
>
>> Before the update, while the VM was powered off, my host played audio
>> through the onboard intel audio controller.  When the VM was on, audio from
>> the VM played through the onboard audio controller and the host lost audio
>> output.  This is the desired behavior.  Another behavior in the past was to
>> only assign the gpu and hdmi audio on the gpu, but leave the onboard audio
>> unassigned to vfio.
>>
>> After the update, pulse audio no longer reports audio devices in the
>> host.  Trying to figure out why this is, I've removed all passthrough
>> devices from all VMs in virt-manager, removed the vfio module load options,
>> but vfio_pci still claims the 3 devices.  The only way I've been able to
>> get the audio devices back is to BLACKLIST the vfio module.
>>
>
> vfio-pci will not bind to any devices unless you tell it to via an ids=
> option or scripts to make use of driver_override, so it sounds a lot like
> you're missing some of the setup done previously to the system, perhaps not
> rebuilding the initramfs/initrd to actually make the changes take effect.
> Perhaps you made changes to configuration files before update that were
> only incorporated when the initrd was built for the newly installed kernel.
>
>
>> My questions are:
>>
>> 1. Is there any other way besides /etc/modules.d/vfio.conf that vfio
>> could be told to claim a device on boot?
>>
>
> The init process is just a bunch of scripts, so there's pretty much an
> infinite number of ways once you understand where things fit.
>
>
>> 2. Is ACS impacting my ability to have the host use a device claimed by
>> iommu because of a recent change?
>>
>
> You haven't actually said what kernels were new and old, but there haven't
> been any changes related to ACS on haswell for a long time.
>
>
>> 3. Is there another way of going about assigning a claimed device to the
>> host?
>>
>
> unbind from one driver, bind to another via sysfs.
>
>
>> 4. Can I no longer only assign 2 of 3 devices in an iommu group to vfio?
>>
>
> *endpoints* within an iommu group must all be detached from the host
> drivers, this has always been the case, nothing has changed here.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20160802/3f3fac45/attachment.htm>