[vfio-users] BSOD occuring with 1 game in VM

Fri May 13 00:34:57 UTC 2016

My 2c: on Win8.1 in Starcraft 2 (nvidia 970 GT, i7 4790k CPU host) I have
zero unknown msrs and have excellent sc2 performance. Several time I
switched to Win10 I got the msrs spam in logs, so I decided to revert back
to Win8.1 for now.

On Fri, May 13, 2016, 8:02 AM Quentin Deldycke <quentindeldycke at gmail.com>
wrote:

> Aw:
>
> You can have my xml here:
> https://github.com/qdel/scripts/blob/master/vfio/win10.xml
>
> Note that i have 6 cores allocated and on 7 there is also my emulator.
> This or leaving it alone change nothing. So i let it like this.
> My dpc latency is quite good actually.
>
> And as you can see, i virtualize "nothing", but only pass devices:
>
> - nvme ssd on pcie
> - sata controller
> - usb3 controller
> - gpu and i also use it's sound output for windows sound.
>
> Sadly, my sata controller cannot use msi, same as sound controller of gpu.
> But none seems to suffer of this. My sound is perfect and disk work
> perfectly (and in any cases, hots is on the ssd)
>
> Jan:
>
> I saw also some cases where my vm is faster than native. Weird stuff.
> For my case, it works nearly perfectly. I can even hard stress the host
> without any impact on the vm. (but using cgroups, i can still temporary
> make a make -j9 :P)
> This is typically limiting cpu execution as the game asks to reduce
> physics or stuff like this...
>
> Abdulla:
>
> Yep, sc2 make same mess, exactly same...
> Good thing is that Overwatch beta was not affected I had a steady 60fps on
> 4k with it. =)
> Diablo 2 is not affected, And Wow on basics area aren't also (no game time
> since years), same, much fps, and not a drop on gpu usage.
>
>
> Following, msr storm, mainly always same msr: 1c9 680 6c0! On intel it
> seems to be the "Performance Monitoring Unit". This snippet is on game
> menus. during party i can easly reach 1 000 000 suppression. (and it pop
> every 4-5 seconds)
>
> May 12 23:39:59 lindesk kernel: [10442.173587] kvm [12608]: vcpu5, guest
> rIP: 0xfffff80185f4c993 kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> May 12 23:40:04 lindesk kernel: [10447.174529] kvm_set_msr_common: 241580
> callbacks suppressed
> May 12 23:40:04 lindesk kernel: [10447.174537] kvm [12608]: vcpu3, guest
> rIP: 0xfffff80185f4c993 kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> May 12 23:40:04 lindesk kernel: [10447.174543] kvm_get_msr_common: 724760
> callbacks suppressed
> May 12 23:40:04 lindesk kernel: [10447.174545] kvm [12608]: vcpu3, guest
> rIP: 0xfffff80185f4c9fc ignored rdmsr: 0x1c9
> May 12 23:40:04 lindesk kernel: [10447.174547] kvm [12608]: vcpu3, guest
> rIP: 0xfffff80185f4ca0a ignored rdmsr: 0x680
> May 12 23:40:04 lindesk kernel: [10447.174549] kvm [12608]: vcpu3, guest
> rIP: 0xfffff80185f4ca21 ignored rdmsr: 0x6c0
> May 12 23:40:04 lindesk kernel: [10447.174556] kvm [12608]: vcpu3, guest
> rIP: 0xfffff80185f4c993 kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> May 12 23:40:04 lindesk kernel: [10447.174561] kvm [12608]: vcpu3, guest
> rIP: 0xfffff80185f4c9fc ignored rdmsr: 0x1c9
> May 12 23:40:04 lindesk kernel: [10447.174564] kvm [12608]: vcpu3, guest
> rIP: 0xfffff80185f4ca0a ignored rdmsr: 0x680
> May 12 23:40:04 lindesk kernel: [10447.174567] kvm [12608]: vcpu3, guest
> rIP: 0xfffff80185f4ca21 ignored rdmsr: 0x6c0
> May 12 23:40:04 lindesk kernel: [10447.174575] kvm [12608]: vcpu3, guest
> rIP: 0xfffff80185f4c993 kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> May 12 23:40:04 lindesk kernel: [10447.174592] kvm [12608]: vcpu3, guest
> rIP: 0xfffff80185f4c9fc ignored rdmsr: 0x1c9
> May 12 23:40:04 lindesk kernel: [10447.174594] kvm [12608]: vcpu3, guest
> rIP: 0xfffff80185f4ca0a ignored rdmsr: 0x680
> May 12 23:40:04 lindesk kernel: [10447.174596] kvm [12608]: vcpu3, guest
> rIP: 0xfffff80185f4ca21 ignored rdmsr: 0x6c0
> May 12 23:40:04 lindesk kernel: [10447.174603] kvm [12608]: vcpu3, guest
> rIP: 0xfffff80185f4c993 kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> May 12 23:40:04 lindesk kernel: [10447.174611] kvm [12608]: vcpu3, guest
> rIP: 0xfffff80185f4c9fc ignored rdmsr: 0x1c9
> May 12 23:40:04 lindesk kernel: [10447.174619] kvm [12608]: vcpu3, guest
> rIP: 0xfffff80185f4c993 kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> May 12 23:40:04 lindesk kernel: [10447.174642] kvm [12608]: vcpu3, guest
> rIP: 0xfffff80185f4c993 kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> May 12 23:40:04 lindesk kernel: [10447.174735] kvm [12608]: vcpu3, guest
> rIP: 0xfffff80185f4c993 kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> May 12 23:40:04 lindesk kernel: [10447.174772] kvm [12608]: vcpu3, guest
> rIP: 0xfffff80185f4c993 kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> May 12 23:40:04 lindesk kernel: [10447.174797] kvm [12608]: vcpu3, guest
> rIP: 0xfffff80185f4c993 kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> May 12 23:40:04 lindesk kernel: [10447.174815] kvm [12608]: vcpu3, guest
> rIP: 0xfffff80185f4c993 kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> May 12 23:40:09 lindesk kernel: [10452.175786] kvm_get_msr_common: 156326
> callbacks suppressed
> May 12 23:40:09 lindesk kernel: [10452.175795] kvm [12608]: vcpu2, guest
> rIP: 0xfffff80185f4c9fc ignored rdmsr: 0x1c9
> May 12 23:40:09 lindesk kernel: [10452.175798] kvm [12608]: vcpu2, guest
> rIP: 0xfffff80185f4ca0a ignored rdmsr: 0x680
> May 12 23:40:09 lindesk kernel: [10452.175802] kvm [12608]: vcpu2, guest
> rIP: 0xfffff80185f4ca21 ignored rdmsr: 0x6c0
> May 12 23:40:09 lindesk kernel: [10452.175810] kvm_set_msr_common: 52103
> callbacks suppressed
> May 12 23:40:09 lindesk kernel: [10452.175812] kvm [12608]: vcpu2, guest
> rIP: 0xfffff80185f4c993 kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> May 12 23:40:09 lindesk kernel: [10452.176286] kvm [12608]: vcpu2, guest
> rIP: 0xfffff80185f4c9fc ignored rdmsr: 0x1c9
> May 12 23:40:09 lindesk kernel: [10452.176292] kvm [12608]: vcpu2, guest
> rIP: 0xfffff80185f4ca0a ignored rdmsr: 0x680
> May 12 23:40:09 lindesk kernel: [10452.176295] kvm [12608]: vcpu2, guest
> rIP: 0xfffff80185f4ca21 ignored rdmsr: 0x6c0
> May 12 23:40:09 lindesk kernel: [10452.176304] kvm [12608]: vcpu2, guest
> rIP: 0xfffff80185f4c993 kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> May 12 23:40:09 lindesk kernel: [10452.176324] kvm [12608]: vcpu2, guest
> rIP: 0xfffff80185f4c9fc ignored rdmsr: 0x1c9
> May 12 23:40:09 lindesk kernel: [10452.176326] kvm [12608]: vcpu2, guest
> rIP: 0xfffff80185f4ca0a ignored rdmsr: 0x680
> May 12 23:40:09 lindesk kernel: [10452.176328] kvm [12608]: vcpu2, guest
> rIP: 0xfffff80185f4ca21 ignored rdmsr: 0x6c0
> May 12 23:40:09 lindesk kernel: [10452.176335] kvm [12608]: vcpu2, guest
> rIP: 0xfffff80185f4c993 kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> May 12 23:40:09 lindesk kernel: [10452.176785] kvm [12608]: vcpu2, guest
> rIP: 0xfffff80185f4c9fc ignored rdmsr: 0x1c9
> May 12 23:40:09 lindesk kernel: [10452.176799] kvm [12608]: vcpu2, guest
> rIP: 0xfffff80185f4c993 kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> May 12 23:40:09 lindesk kernel: [10452.177201] kvm [12608]: vcpu2, guest
> rIP: 0xfffff80185f4c993 kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> May 12 23:40:09 lindesk kernel: [10452.177291] kvm [12608]: vcpu2, guest
> rIP: 0xfffff80185f4c993 kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> May 12 23:40:09 lindesk kernel: [10452.177318] kvm [12608]: vcpu2, guest
> rIP: 0xfffff80185f4c993 kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> May 12 23:40:09 lindesk kernel: [10452.177530] kvm [12608]: vcpu2, guest
> rIP: 0xfffff80185f4c993 kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> May 12 23:40:09 lindesk kernel: [10452.177793] kvm [12608]: vcpu2, guest
> rIP: 0xfffff80185f4c993 kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
> May 12 23:40:09 lindesk kernel: [10452.178291] kvm [12608]: vcpu2, guest
> rIP: 0xfffff80185f4c993 kvm_set_msr_common: MSR_IA32_DEBUGCTLMSR 0x1, nop
>
>
>
> On 12 May 2016 11:12 pm, "Jan Wiele" <jan at wiele.org> wrote:
>
>> You are not alone. I have had the exact same symptoms. Wanted to build a
>> "gaming" machine with two seats for playing Heroes of the Storm.
>> Performance was bad. Very low fps especially in fights. No difference
>> between low and high graphic settings.
>>
>> Nevertheless, graphic benchmarks, like 3D mark, certified a very good GPU
>> performance: [1] VM, [2] Native (Yes, the VM performs  slightly better).
>> There, you can also see my system specs. I suppose this is some kind of CPU
>> scaling problem in HotS. :-/
>>
>> I'm very interested if you can actually find a solution :) Good luck!
>>
>> Regards,
>> Jan
>>
>> [1] http://www.3dmark.com/fs/7493163
>> [2] http://www.3dmark.com/fs/7492760
>>
>> Am 12.05.2016 um 22:09 schrieb Quentin Deldycke:
>>
>> I will try later with sc2.
>>
>> I do play in ultra, as changing from this to low makes no differences =)
>> The game works, no visual glitches. But it's performance is much more
>> lower than it should be...
>>
>> --
>> Deldycke Quentin
>>
>>
>> On 12 May 2016 at 22:04, Abdulla Bubshait <darkstego at gmail.com> wrote:
>>
>>> Honestly, with the msrs ignore I got the game to work fine. Running on
>>> ultra settings with no issue. This might not be an msrs issue. Try
>>> StarCraft 2, it uses the same engine and also produces msrs faults.
>>>
>>> On Thu, May 12, 2016, 15:52 Quentin Deldycke <quentindeldycke at gmail.com>
>>> wrote:
>>>
>>>> Same, does not work, even with the multiples binaries of the game
>>>> (launched threw battle.net)
>>>>
>>>> Just for people to know it makes the card have this kind of herratic
>>>> gpu usage during game... Which is quite a pain to play...
>>>>
>>>> [image: Inline images 1]
>>>>
>>>> Note that it became more quiet while entering menus after game end
>>>> (after the line with the hour...)
>>>>
>>>> --
>>>> Deldycke Quentin
>>>>
>>>>
>>>> On 11 May 2016 at 18:04, Abdulla Bubshait < <darkstego at gmail.com>
>>>> darkstego at gmail.com> wrote:
>>>>
>>>>> I have tried running it in Compatibility mode, but it didn't work.
>>>>>
>>>>> On Wed, May 11, 2016 at 10:04 AM Brett Peckinpaugh <
>>>>> <bp10 at erylflynn.com>bp10 at erylflynn.com> wrote:
>>>>>
>>>>>> Have you tried running the game in compatibility mode set to 8.1?
>>>>>>
>>>>>>
>>>>>> On May 11, 2016 5:26:19 AM PDT, Abdulla Bubshait <
>>>>>> <darkstego at gmail.com>darkstego at gmail.com> wrote:
>>>>>>>
>>>>>>> StarCraft and heroes are the two games that have this problem, they
>>>>>>> are both based on the same engine. Unfortunately I cannot test the cpu
>>>>>>> because if I were to change my cpu to anything but "host" my machine will
>>>>>>> not boot (I believe this is due to a problem with the AMD drivers).
>>>>>>>
>>>>>>> I believe the MSRs are called by Windows 10, since they are
>>>>>>> privileged registers. Likely for debugging purposes here.
>>>>>>>
>>>>>>> I would suggest trying something more related tou your  cpu
>>>>>>> ("Haswell" in my case) instead of "core2duo". I remember when I first tried
>>>>>>> to install Win10 under virt-manager I got a KVM error about unsupported cpu
>>>>>>> functions when I tried to clone the host cpu. Setting it to Haswell cleared
>>>>>>> those errors so this might have something to do with it. Unfortunately that
>>>>>>> didn't play nice with my video card so I ended up with a command line setup
>>>>>>> as virt-manager setup refused to load AMD drivers.
>>>>>>> Hope this helps.
>>>>>>>
>>>>>>> Abdulla
>>>>>>>
>>>>>>> On Wed, May 11, 2016, 04:03 Ivan Volosyuk <
>>>>>>> <ivan.volosyuk at gmail.com>ivan.volosyuk at gmail.com> wrote:
>>>>>>>
>>>>>>>> My 2c: I had the same issues with StarCraft crashing on Win10 due
>>>>>>>> to the unsupported msrs (or producing lots of logs in dmesg). On Win8.1 I
>>>>>>>> don't have this problem. My best guess is that nvidia drivers for Win10
>>>>>>>> started to use msrs unsupported by qemu. Can you change your CPU to
>>>>>>>> emulated core2duo and check if it actually works faster and doesn't produce
>>>>>>>> this logging spam?
>>>>>>>>
>>>>>>>> On Wed, May 11, 2016 at 4:50 PM Quentin Deldycke <
>>>>>>>> <quentindeldycke at gmail.com>quentindeldycke at gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hello,
>>>>>>>>>
>>>>>>>>> I also play quite much this game. Adding this option makes the
>>>>>>>>> game "Works".
>>>>>>>>>
>>>>>>>>> But for me, it is also the game with worst performance. As there
>>>>>>>>> is a storm of unsupported msr (not 1 or 2 but hundreds of thousands...)
>>>>>>>>>
>>>>>>>>> Do you have correct performance?  I go between 120 at begging to
>>>>>>>>> 15 during fights.
>>>>>>>>>
>>>>>>>>> Note that this is the only game making such mess with msr. Other
>>>>>>>>> blizzard games works perfectly...
>>>>>>>>> On 11 May 2016 1:04 am, "Abdulla Bubshait" < <darkstego at gmail.com>
>>>>>>>>> darkstego at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Just put that in and it solved the problem.
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>>
>>>>>>>>>> On Tue, May 10, 2016 at 6:54 PM Alex Williamson <
>>>>>>>>>> <alex.l.williamson at gmail.com>alex.l.williamson at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> On Tue, May 10, 2016 at 4:40 PM, Abdulla Bubshait <
>>>>>>>>>>> <darkstego at gmail.com>darkstego at gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I have a pretty stable VFIO setup running for a while, but I am
>>>>>>>>>>>> stuck with this
>>>>>>>>>>>> odd problem where 1 game (heroes of the storm) keeps giving me
>>>>>>>>>>>> a BSOD
>>>>>>>>>>>> whenever I try to run it in the VM under Windows 10.
>>>>>>>>>>>>
>>>>>>>>>>>>  All other games are running fine. If I install Windows 8 in
>>>>>>>>>>>> the VM the game runs fine.
>>>>>>>>>>>>  If I boot the machine into the Windows 10 HDD directly the
>>>>>>>>>>>> game runs fine.
>>>>>>>>>>>> This crash occurs with both Nvidia GTX 770 and AMD Fury X.
>>>>>>>>>>>> It only crashes when in VM and Windows 10.
>>>>>>>>>>>>
>>>>>>>>>>>> The BSOD is some form of exception. Examples that occur
>>>>>>>>>>>> SYSTEM_SERVICE_EXCEPTION
>>>>>>>>>>>> KMODE_EXCEPTION_NOT_HANDLED
>>>>>>>>>>>> SYSTEM_THREAD_EXCEPTION_NOT_HANDLED_M
>>>>>>>>>>>> The dump files seem to suggest a windows8 driver issue,
>>>>>>>>>>>> but I can't pinpoint any faulty driver.
>>>>>>>>>>>>
>>>>>>>>>>>> I am running netrunner (manjaro) kernel 4.4.9, qemu 2.5.1.
>>>>>>>>>>>>
>>>>>>>>>>>> My config is:
>>>>>>>>>>>> <http://pastebin.com/W6cPyMEB>http://pastebin.com/W6cPyMEB
>>>>>>>>>>>>
>>>>>>>>>>>> Sample BSOD dumps:
>>>>>>>>>>>> <http://pastebin.com/bgh2uEhf>http://pastebin.com/bgh2uEhf
>>>>>>>>>>>> <http://pastebin.com/SLPTVUwn>http://pastebin.com/SLPTVUwn
>>>>>>>>>>>> <http://pastebin.com/zdjTzKuV>http://pastebin.com/zdjTzKuV
>>>>>>>>>>>> <http://pastebin.com/8Lt5VfLg>http://pastebin.com/8Lt5VfLg
>>>>>>>>>>>>
>>>>>>>>>>>> Welcome any ideas to fix this problem. Thanks,
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Do you have the following set in a modprobe.d conf file?
>>>>>>>>>>>
>>>>>>>>>>> options kvm ignore_msrs=1
>>>>>>>>>>>
>>>>>>>>>>> Windows BSODs are often the result of calling an unsupported MSR
>>>>>>>>>>> and not handling the exception.  There's some risk to this option because
>>>>>>>>>>> zero isn't guaranteed to be a valid return for an unknown MSR, but it seems
>>>>>>>>>>> to solve a lot of problems.  YMMV.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> vfio-users mailing list
>>>>>>>>>> <vfio-users at redhat.com>vfio-users at redhat.com
>>>>>>>>>> <https://www.redhat.com/mailman/listinfo/vfio-users>
>>>>>>>>>> https://www.redhat.com/mailman/listinfo/vfio-users
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>> vfio-users mailing list
>>>>>>>>> <vfio-users at redhat.com>vfio-users at redhat.com
>>>>>>>>> <https://www.redhat.com/mailman/listinfo/vfio-users>
>>>>>>>>> https://www.redhat.com/mailman/listinfo/vfio-users
>>>>>>>>>
>>>>>>>> ------------------------------
>>>>>>>
>>>>>>> vfio-users mailing listvfio-users at redhat.comhttps://www.redhat.com/mailman/listinfo/vfio-users
>>>>>>>
>>>>>>> _______________________________________________
>> vfio-users mailing listvfio-users at redhat.comhttps://www.redhat.com/mailman/listinfo/vfio-users
>>
>>
>> _______________________________________________
>> vfio-users mailing list
>> vfio-users at redhat.com
>> https://www.redhat.com/mailman/listinfo/vfio-users
>>
>> _______________________________________________
> vfio-users mailing list
> vfio-users at redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20160513/6638470e/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 6922 bytes
Desc: not available
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20160513/6638470e/attachment.png>