[vfio-users] How to set PCI address in "options vfio-pci" ?

[Kyle Marek]

This should be simple (I did this on Ubuntu, first).

The softdep lines of the modprobe config order vfio-pci before the
drivers that usually bind to your device, so you don't need to unbind
anything; vfio-pci is bound, first.

Ubuntu's initramfs-tools copies your modprobe config
(/etc/modprobe.d/*.conf) into the initrds.

You can explicitly add modules to your initrds in
/etc/initramfs-tools/modules.

So:

 1. Adapt the mentioned modprobe config (maybe with Alex's install line
    modification) in /etc/modprobe.d/something.conf
 2. Place "vfio-pci" in /etc/initramfs-tools/modules
 3. `sudo update-initramfs -k all -u`


On 05/19/2018 12:23 AM, Eddie Yen wrote:
> Sorry about delay reply.
>
> Same reason as Kyle. We have 2 NICs which had same VID:PID. And want to
> passthrough one of them into VM.
> The script can use, but need to unbind device from NIC driver first as
> premise.
>
> echo $DEV > /sys/bus/pci/devices/$DEV/unbind
> echo $DEV > /sys/bus/pci/drivers/vfio-pci/new_id
> (Can't remember the second command path.)
>
>
> if using
>
> echo vfio-pci > /sys/bus/pci/devices/$DEV/driver_override
>
> lspci -k will shows NIC driver still occupied this NIC.
>
>
> I'ii try Kyle's method but need to think about how to transfer this method
> for Ubuntu (Debian)...
>
> 2018-05-18 1:43 GMT+08:00 Alex Williamson <alex.williamson at redhat.com>:
>
>> On Thu, 17 May 2018 13:12:07 -0400
>> Kyle Marek <psppsn96 at gmail.com> wrote:
>>
>>> On 05/17/2018 10:27 AM, Alex Williamson wrote:
>>>> On Thu, 17 May 2018 12:56:49 +0800
>>>> Eddie Yen <missile0407 at gmail.com> wrote:
>>>>
>>>>> Hi everyone,
>>>>>
>>>>> IME, you can set VID:PID by adding "options vfio-pci" into conf file
>> and
>>>>> put at modprobe.d
>>>>> So that VFIO can take over the device that had same VID:PID when boot.
>>>>>
>>>>> But how can I set this option by using PCI Address without using
>> override
>>>>> script?
>>>> There's no such module option, what's wrong with a script?  Part of the
>>>> reason we don't have such a module option is that PCI addresses are not
>>>> persistent, they can change based on devices added or removed, or even
>>>> motherboard BIOS updates.  Userspace has more flexibility for managing
>>>> this and is fully able to do so via initrd scripts and modprobe.d.
>>>> Thanks,
>>>>
>>>> Alex
>>> That's a good point.
>>>
>>> However, I had an environment where I needed to pass through one of two
>>> devices with the same PCI IDs. I accomplished it with an install command
>>> in my modprobe config.
>>>
>>> I can probably make this look at other factors before overriding the
>>> driver, but the following config might work well enough if the PCI IDs
>>> don't change:
>>>
>>> # For each PCI device listed in the for-loop, override their assigned
>> driver
>>> # to vfio-pci, for usage with PCI-passthrough.
>>> #
>>> # Please make sure to include vfio-pci in the initrd, or some other
>> modules
>>> # will likely claim the devices first. This can be done on systems that
>> use
>>> # initramfs-tools by adding "vfio-pci" to /etc/initramfs-tools/modules
>>> install vfio-pci for dev in 0000:03:00.0 0000:03:00.1; do echo vfio-pci
>>> "/sys/bus/pci/devices/$dev/driver_override"; done; /sbin/modprobe
>> --ignore-install vfio-pci $CMDLINE_OPTS
>>> # Load vfio-pci before some other drivers so they don't claim devices
>> first.
>>> # The pre-drm line should handle all graphics drivers.
>>> softdep snd_hda_intel pre: vfio-pci
>>> softdep drm pre: vfio-pci
>>>
>>>
>>> This worked well for me in dracut and initramfs-tools initrds with the
>>> vfio-pci module included.
>> Yes, this is a good demonstration of how userspace can easily
>> accomplish this through modprobe.d softdeps and custom install
>> options.  A more persistence focused version of the above might do the
>> same knowing that we're looking for all devices device downstream of
>> the root port at 0000:00:01.0, and would look something like:
>>
>> install vfio-pci for dev in $(find /sys/devices/pci0000:00/0000:00:01.0/
>> -maxdepth 1 -mindepth 1 -type d -name "????:??:??.?"); do echo vfio-pci >
>> $dev/driver_override; done; modprobe --ignore-install...
>>
>> or even
>>
>> for override in $(find /sys/devices/pci0000:00/0000:00:01.0/ -mindepth
>> 2 -name driver_override); do echo vfio-pci > $override; done...
>>
>> There are countless ways to do this, including ways that take advantage
>> of PCI slots information provided through ACPI if the system supports
>> it.  Thanks,
>>
>> Alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20180519/27c0659d/attachment.htm>