[vfio-users] amd gpu not being seen
Ryan Clawfish
chimmychainsaw at hotmail.com
Sat Oct 3 05:47:42 UTC 2015
Howdy!
I spent about 3 hours fiddling with the 280x trying to get the drivers to stop acting retarded. I plug in my 770 and add remove the hypervisor and it works instantly. I might give my friend his cars back to sell and keep the 770 after-all. The 770 appears in device manager just fine but i'm not sure how to go about actually using it. I can't seem to get rid of the emaulated "basic microsoft display adaptor".
<video>
<model type='cirrus' vram='16384' heads='1'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
</video>
I keep deleting the first line but it keeps coming back. The emulated display won't let me change the resolution, thought that might be because of the emulated display adapter. Please advice
<domain type='kvm'>
<name>generic-2</name>
<uuid>05b8ff63-7afc-4a17-9b72-70fc38029a0b</uuid>
<memory unit='KiB'>5144576</memory>
<currentMemory unit='KiB'>5144576</currentMemory>
<vcpu placement='static'>6</vcpu>
<os>
<type arch='x86_64' machine='pc-i440fx-2.4'>hvm</type>
<loader readonly='yes' type='pflash'>/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
<nvram>/var/lib/libvirt/qemu/nvram/generic-2_VARS.fd</nvram>
</os>
<features>
<acpi/>
<apic/>
<pae/>
<hyperv>
<relaxed state='off'/>
<vapic state='off'/>
<spinlocks state='off'/>
</hyperv>
<kvm>
<hidden state='on'/>
</kvm>
<vmport state='off'/>
</features>
<cpu mode='host-model'>
<model fallback='allow'/>
</cpu>
<clock offset='localtime'>
<timer name='hypervclock' present='no'/>
</clock>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<pm>
<suspend-to-mem enabled='no'/>
<suspend-to-disk enabled='no'/>
</pm>
<devices>
<emulator>/usr/bin/qemu-kvm</emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='raw'/>
<source file='/usr/share/edk2.git/aarch64/image_file'/>
<target dev='hda' bus='ide'/>
<boot order='2'/>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
<disk type='file' device='cdrom'>
<driver name='qemu' type='raw'/>
<source file='/home/ryan/kvm/win10.iso'/>
<target dev='hdb' bus='ide'/>
<readonly/>
<boot order='1'/>
<address type='drive' controller='0' bus='0' target='0' unit='1'/>
</disk>
<controller type='usb' index='0' model='ich9-ehci1'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x7'/>
</controller>
<controller type='usb' index='0' model='ich9-uhci1'>
<master startport='0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0' multifunction='on'/>
</controller>
<controller type='usb' index='0' model='ich9-uhci2'>
<master startport='2'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x1'/>
</controller>
<controller type='usb' index='0' model='ich9-uhci3'>
<master startport='4'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x2'/>
</controller>
<controller type='pci' index='0' model='pci-root'/>
<controller type='ide' index='0'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
</controller>
<controller type='virtio-serial' index='0'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
</controller>
<interface type='network'>
<mac address='52:54:00:42:fd:b1'/>
<source network='default'/>
<model type='rtl8139'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</interface>
<serial type='pty'>
<target port='0'/>
</serial>
<console type='pty'>
<target type='serial' port='0'/>
</console>
<channel type='spicevmc'>
<target type='virtio' name='com.redhat.spice.0'/>
<address type='virtio-serial' controller='0' bus='0' port='1'/>
</channel>
<input type='mouse' bus='ps2'/>
<input type='keyboard' bus='ps2'/>
<graphics type='spice' autoport='yes'>
<image compression='off'/>
</graphics>
<sound model='ich6'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
</sound>
<video>
<model type='cirrus' vram='16384' heads='1'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
</video>
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
</source>
<address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
</hostdev>
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x04' slot='0x00' function='0x1'/>
</source>
<address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
</hostdev>
<redirdev bus='usb' type='spicevmc'>
</redirdev>
<redirdev bus='usb' type='spicevmc'>
</redirdev>
<memballoon model='virtio'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
</memballoon>
</devices>
</domain>
Date: Fri, 2 Oct 2015 20:14:59 +0300
Subject: Re: [vfio-users] amd gpu not being seen
From: ihatethisfield at gmail.com
To: chimmychainsaw at hotmail.com
CC: vfio-users at redhat.com
My win10 VM goes black on boot from time to time. Just sayin'
Also, there is a neighbor thread about some kind of regression in recent kernels.
On Oct 2, 2015 8:08 PM, "Ryan Clawfish" <chimmychainsaw at hotmail.com> wrote:
I tried the solutions recommended to no avail. Switching pci slots and blacklisting the drivers seemed to have no effect on the horrible boot performance. I asked my friend if i could borrow my old 770 back to test it out in my arch machine and i had the same issues as before with the radeon card. I can't help but think the issue is either with my install or a bug with the latest 4.2.2.1 kernel as everything was working somewhat fine just a few days ago.
I decided to install fedora on a separate partition thinking if any os should work it's one made by redhat. And it did work! Sorta. Following alex's awesome vfio blog i got everything setup pretty much properly. Windows 10 boots normally with my 280x attached. My only issue now is trying to get windows 10 to stop crashing.Using seabios it crashes with the atikmdag error and with uefi it simply goes black.
Date: Wed, 30 Sep 2015 10:55:39 +0300
Subject: Re: [vfio-users] amd gpu not being seen
From: ihatethisfield at gmail.com
To: chimmychainsaw at hotmail.com
CC: vfio-users at redhat.com
Alright, i've studied your dmesg(fell asleep in one place), so:
You should blacklist the radeon driver. It seems like it's interfering not only with vfio, but somehow manages to screw up nvidia host driver.
And there is a high possibility that you should remove that wireless adapter and plug your third GPU there, because iommu grouping may be messed up.
Your host GPU is plugged in the first slot and should be the only one doing VGA. But for some reason it is not.
After blacklisting the radeon driver, you should reinstall the guest system.
On Sep 29, 2015 6:23 PM, "Ryan Clawfish" <chimmychainsaw at hotmail.com> wrote:
The output was huge and I thought only the vfio entries were relevant. My bad. I'm using vfio-pci because it's recommended on the arch wiki. I have not yet tried pci-stub.
I've added: options vfio-pci ids=1002:6798,1002:aaa0 to /etc/modprobe.d/vfio.conf
everything else is on http://pastebin.com/3DA3wUBd
Date: Tue, 29 Sep 2015 17:43:02 +0300
Subject: Re: [vfio-users] amd gpu not being seen
From: ihatethisfield at gmail.com
To: chimmychainsaw at hotmail.com
CC: vfio-users at redhat.com
Why did you grep...
Post the full one on pastebin or search for whatever using VGA or that GPUs. Do you use any kind of pre-binding(vfio-pci.ids=...) or pci-stub?
On Sep 29, 2015 5:36 PM, "Ryan Clawfish" <chimmychainsaw at hotmail.com> wrote:
Hotmail really needs to start accepting code tags, it would make things so much neater.
I'm using an fx 8350 with an asus m5a99 pro R2.0 motherboard.
lspci -nn
04:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc.
[AMD/ATI] Tahiti XT [Radeon HD 7970/8970 OEM / R9 280X] [1002:6798]
04:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Tahiti XT HDMI Audio [Radeon HD 7970 Series] [1002:aaa0]
[ryan at earth-archbox ~]$ dmesg | grep -i vfio
[ 1.066842] VFIO - User Level meta-driver version: 0.3
[ 1.079992] vfio_pci: add [1002:6798[ffff:ffff]] class 0x000000/00000000
[ 1.093349] vfio_pci: add [1002:aaa0[ffff:ffff]] class 0x000000/00000000
[ 108.858636] vfio-pci 0000:04:00.0: enabling device (0000 -> 0003)
[ 108.858820] vfio_ecap_init: 0000:04:00.0 hiding ecap 0x19 at 0x270
[ 108.858829] vfio_ecap_init: 0000:04:00.0 hiding ecap 0x1b at 0x2d0
[ 108.875285] vfio-pci 0000:04:00.1: enabling device (0000 -> 0002)
[ 1284.893125] vfio_ecap_init: 0000:04:00.0 hiding ecap 0x19 at 0x270
[ 1284.893133] vfio_ecap_init: 0000:04:00.0 hiding ecap 0x1b at 0x2d0
[ 1284.909667] vfio-pci 0000:04:00.1: enabling device (0400 -> 0402)
[ 1519.790793] vfio_ecap_init: 0000:04:00.0 hiding ecap 0x19 at 0x270
[ 1519.790805] vfio_ecap_init: 0000:04:00.0 hiding ecap 0x1b at 0x2d0
[ 1519.810519] vfio-pci 0000:04:00.1: enabling device (0400 -> 0402)
Hostdev
</video>
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
</source>
<address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
</hostdev>
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x04' slot='0x00' function='0x1'/>
</source>
<address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
</hostdev>
So you managed to get crossfire inside the emulation? I didn't think
it was possible but that would be beyond cool. Pain in the ass sure but
still cool as hell.
also: In my pre coffee daze i accidentally replied to ihatethisfield and not the mailing list. Sorry about that.
Date: Tue, 29 Sep 2015 07:46:23 +0300
Subject: Re: [vfio-users] amd gpu not being seen
From: ihatethisfield at gmail.com
To: chimmychainsaw at hotmail.com
CC: vfio-users at redhat.com
Your attached XML file lacks the most important part: <hostdev> entries with your GPUs.
Tell us about your CPU-MB combo and do lspci -nn and find all your needed devices in their respective iommu groups. And it would be good to see your dmesg, maybe you've forgot to blacklist some driver or something else tampers with VGA somehow.
Also, on my 2xHD7750 crossfire works flawlessly. They don't need a bridge to interconnect, but your cards might need it.
On Sep 29, 2015 4:26 AM, "Ryan Clawfish" <chimmychainsaw at hotmail.com> wrote:
I'm rather happy with 2x 280x's. Hopefully i can use linux virtualization for convinence gaming and reboot into a windows partition when i want insane crossfire graphics.
To be honest i'm a noob at this and i don't really know what the hell i'm doing. I'm mostly following the arch wiki guide https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF Because /dev/vfios/17 did not exist I decided to reinstall arch and restart the process from scratch, roughly documenting my steps. Here's what i did.
installed qemu and rpmextract
extracted the gerd hoffmons eufi and moved it to /usr/share
downloaded libvirt
enabled libvirt and start it
installed ebtables dnsmasq bridge-utils and bsd-netcat
added amd_iommu=on to /etc/default/grub, regenerated grub and rebuilt mkinitpcio
added modprobe vfio-pci
ran mkinitpcio
/etc/libvirt/qemu.conf added root permissions + /dev/vfio/17
created raw file for emulated hdd
installed virt-manager
sudo cp /usr/share/edk2.git/ovmf-x64/OVMF-pure-efi.fd /usr/share/ovmf/OVMF-win10.fd #pointless?
sudo ln -s /usr/share/ovmf/OVMF-win10.fd /usr/share/ovmf/OVMF-win10.fd #pointless
added uefi into in the os section of my xml file:
<loader readonly='yes' type='pflash'>/usr/share/edk2.git/ovmf-x64/OVMF_CODE-pure-efi.fd</loader>
<nvram template='/usr/share/edk2.git/ovmf-x64/OVMF_VARS-pure-efi.fd'/>
installed windows 10
installed radeon drivers
added target pci (gpu) from virt-manager gui
When i add the my gpu the uefi is delayed and booting windows 10 takes forever. There are lots of artifacts and it lags to the point of being completely unusable. I've tried adding my card via virt-manager gui and via the (depreciated?)<qemu:commandline>. Both have the same effect.
Here's my xml virt config file http://pastebin.com/sNxvXiQ6
Date: Mon, 28 Sep 2015 09:40:22 -0600
Subject: Re: [vfio-users] amd gpu not being seen
From: alex.l.williamson at gmail.com
To: chimmychainsaw at hotmail.com
CC: vfio-users at redhat.com
On Mon, Sep 28, 2015 at 9:12 AM, Ryan Clawfish <chimmychainsaw at hotmail.com> wrote:
My main issue is i keep getting an error saying "failed to get group 17" and "/dev/vfio/17 does not exist." If i ls -a my /dev , i see /dev/vfio does not exist. I created it using mkdir, chmod 666 it and added it in /etc/libvirt/qemu.conf but the error persists.
So all along, you've probably been trying to use legacy KVM device assignment, which led too the Code 43 with the GeForce card. Too bad, IMHO, the 770 was a better card. vfio has been the default for PCI hostdev devices since libvirt 1.1.3 and /dev/vfio/ should be present even before vfio is load since kernel v3.14. Are you even building vfio support into your kernel?
_______________________________________________
vfio-users mailing list
vfio-users at redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users
http://pastebin.com/HrCWcd6Shttp://pastebin.com/HrCWcd6Shttp://pastebin.com/HrCWcd6S
_______________________________________________
vfio-users mailing list
vfio-users at redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users
_______________________________________________
vfio-users mailing list
vfio-users at redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users
_______________________________________________
vfio-users mailing list
vfio-users at redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20151003/7fb03652/attachment.htm>
More information about the vfio-users
mailing list