[vfio-users] VM doesn't boot if I use GPU passthrough

[Will Marler]

On Mon, Jan 25, 2016 at 9:07 AM, Ryan Flagler <ryan.flagler at gmail.com>
wrote:

> Will, could you tell us the following?
>
> What Linux distribution on host?
>
Arch

> What kernel are you using on host?
> What libvirt version on host?
> What qemu version on host?
>
Will have to check when I'm home from work & the kids are asnooze, but it's
whatever's latest (and I'm not using the linux-vfio-lts kernel)

> What OS on guest?
>
Windows 10.

> What nvidia graphics driver version on guest?
>
Again, I'll have to check. But the latest or nearly latest.

> My machines gpu driver crashes constantly and I'm trying to narrow down
> why. Thanks!
>
How frustrating : (. I'll also get a pastebin of my XML for you, in case
that will help. I've been running "stable" since mid 2015. I use the quotes
because some things tripped me up (guest machine can't "sleep," can only
power on & power off; when host machine goes to sleep with guest running,
on host wake-up the guest is non-responsive and 100% CPU).

Will


>
> On Mon, Jan 25, 2016, 10:02 AM Will Marler <will at wmarler.com> wrote:
>
>> This is discussed in
>> http://vfio.blogspot.com/2015/05/vfio-gpu-how-to-series-part-4-our-first.html.
>> You have to do more than <kvm><hidden state='on'/></kvm>:
>>
>> "The GeForce card is nearly as easy, but we first need to work around
>> some of the roadblocks Nvidia has put in place to prevent you from using
>> the hardware you've purchased in the way that you desire (and by my reading
>> conforms to the EULA for their software, but IANAL).  For this step we
>> again need to run virsh edit on the VM.  Within the <features> section,
>> remove everything between the <hyperv> tags, including the tags
>> themselves.  In their place add the following tags:
>>
>>     <kvm>
>>       <hidden state='on'/>
>>     </kvm>
>>
>> Additionally, within the <clock> tag, find the timer named hypervclock,
>> remove the line containing this tag completely.  Save and exit the edit
>> session."
>>
>> I can confirm it works, I've been getting a lot of mileage from my
>> passed-through 750Ti lately since getting a Steam Link :-D.
>>
>> On Sun, Jan 24, 2016 at 7:32 AM, Ruben Felgenhauer <
>> 4felgenh at informatik.uni-hamburg.de> wrote:
>>
>>> Hi,
>>>
>>> finally I had time to this again. I tried out virt-manager and after a
>>> bit of playing around with it, it /somewhat/ worked:
>>>
>>> The machine is at least booting. I still have a standard vga card
>>> enabled in the virt-manager config window.
>>> After the machine has booted, I can see that the device gets recognized
>>> as 750ti.
>>> However, the gpu doesn't get used, because of 'Code 43'.
>>> Code 43 is a generic error, so any idea what it could mean in this case?
>>>
>>> Of course I added the <kvm><hidden state='on'/></kvm> lines at the
>>> associated position.
>>>
>>> Best regards,
>>> Ruben
>>>
>>>
>>> Am 18.01.2016 um 22:27 schrieb Will Marler:
>>>
>>> I'm not sure what correct command-line syntax is. Have you tried using
>>> libvirt and VirtManager to handle your VM rather than command line, and
>>> modifying the XML rather than the command line? I think that's generally
>>> the preferred method these days (it's certainly easier from my point of
>>> view, and the way I got my 750 Ti to pass through).
>>>
>>> On Mon, Jan 18, 2016 at 11:04 AM, Ruben Felgenhauer <
>>> 4felgenh at informatik.uni-hamburg.de> wrote:
>>>
>>>> Hi, Alex!
>>>>
>>>> Thanks for your reply!
>>>> My GPU indeed has a seperate audio device located at 01:00.1.
>>>>
>>>> However, just adding -device vfio-pci,host=01:00.1 doesn't seem to do
>>>> the trick.
>>>> Of course the corresponding device is already blacklisted and bound to
>>>> vfio.
>>>>
>>>> The Debian Wiki entry about VGA passthrough (
>>>> <https://wiki.debian.org/VGAPassthrough>
>>>> https://wiki.debian.org/VGAPassthrough) mentions QEMU arguments like
>>>> "-device
>>>> vfio-pci,host=01:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on,romfile=...
>>>> -device vfio-pci,host=01:00.1,bus=pcie.0" which seems to address GPUs with
>>>> audio devices, but if I try to do something similar, the buses 'root' and
>>>> 'pcie' couldn't be found. Maybe I missed something very important?
>>>>
>>>> On the same article, it says that the "HDMI soundcard [...] needs to be
>>>> unbound from its driver":
>>>> # echo '0000:01:00.1' | sudo tee
>>>> /sys/bus/pci/devices/0000:01:00.1/driver/unbind
>>>> I figured the vfio-bind script from the Arch Linux Forum thread (
>>>> https://bbs.archlinux.org/viewtopic.php?id=162768) would do exactly
>>>> this thing, so I didn't explicitly do so for the audio device. Is that okay?
>>>>
>>>> Best regards,
>>>> Ruben
>>>>
>>>>
>>>> Am 18.01.2016 um 08:31 schrieb Alexander Petrenz:
>>>>
>>>> Hi Ruben,
>>>>
>>>> I guess your 750ti also has some audio device. You should pass through
>>>> this too. It should be something like 01:00.1. There are many command line
>>>> examples you can find about that.
>>>> Also I´m not quite sure, if you should remove the x-vga=on.
>>>>
>>>> Regards
>>>> Alex
>>>>
>>>> On Sun, Jan 17, 2016 at 11:12 PM, Ruben Felgenhauer <
>>>> <4felgenh at informatik.uni-hamburg.de>4felgenh at informatik.uni-hamburg.de>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I am trying to pass my nVidia GTX 750ti to my QEMU guest.
>>>>>
>>>>> Problem is: After the QEMU monitor pops up, nothing happens. The GPU's
>>>>> output is dead, and the vm won't be accessible via SSH anymore, so it's
>>>>> very likely that the VM isn't booting up at all. Also, there are no error
>>>>> messages from QEMU on the console whatsoever which makes debugging it
>>>>> especially hard.
>>>>>
>>>>> This is how I start the vm with normal vga emulation:
>>>>> qemu-system-x86_64 -hda vm.ovl -boot c -enable-kvm -m 1024 -cpu
>>>>> host,kvm=off -smp cores=4,threads=2 -redir tcp:5022::22
>>>>> Everything runs fine in this case. To do the passthrough, I add this:
>>>>> -device vfio-pci,host=01:00.0,multifunction=on,x-vga=on -vga none
>>>>> This brings said problems with it. I also tried out multiple different
>>>>> combinations of -device's arguments or even adding a romfile for the GPU,
>>>>> but none of these steps changed anything at all.
>>>>>
>>>>> Obviously, I am using a BIOS installation and I'm well-aware with this
>>>>> bug: <https://bugzilla.kernel.org/show_bug.cgi?id=107561>
>>>>> https://bugzilla.kernel.org/show_bug.cgi?id=107561, but neither using
>>>>> less RAM (as you can see I am using 1GB now) nor switching to an older
>>>>> Kernel changed anything about the problem. I have tried Kernel 4.1.0 and
>>>>> 4.3.0.
>>>>>
>>>>> Host is Debian testing with QEMU 2.5.0.
>>>>> I tried both Debian and Windows 7 as a guest, but both are showing
>>>>> exactly the same behaviour.
>>>>> Mainboard is an ASUS Z87-PLUS. The 750ti is produced by ASUS aswell.
>>>>>
>>>>> Any idea how I could get passthrough running?
>>>>>
>>>>> _______________________________________________
>>>>> vfio-users mailing list
>>>>> vfio-users at redhat.com
>>>>> https://www.redhat.com/mailman/listinfo/vfio-users
>>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> vfio-users mailing list
>>>> vfio-users at redhat.com
>>>> https://www.redhat.com/mailman/listinfo/vfio-users
>>>>
>>>>
>>>
>>>
>> _______________________________________________
>> vfio-users mailing list
>> vfio-users at redhat.com
>> https://www.redhat.com/mailman/listinfo/vfio-users
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20160125/9a08bb3a/attachment.htm>