[vfio-users] Tracing PCI Device Behind PCI-Bridge

[John Stratoudakis]

Hi,

I have spent a lot of time researching this and trying to figure out how to
do this, so please forgive me if I did not find something already
documented somewhere.

So here is what I am trying to do:

I have a PXI device (National Instruments PXI-7953R), where PXI is an
extension of PCI.  This device is plugged in to a PXI-chassis (PXI-1033).
Then this PXI-chassis is connected to the host computer via a PCI card -
PCI-8361.

What I am trying to do is to trace the PCI-8361 device so that I can
reverse engineer the device (proprietary) drivers given to me by National
Instruments.

When I plug in the device (PCI-8361) and I run lspci, I get the following
output:

0c:01.0 PCI bridge: Pericom Semiconductor PI7C9X111SL PCIe-to-PCI
Reversible Bridge (rev 02) (prog-if 00 [Normal decode])
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Latency: 32, Cache Line Size: 64 bytes
Bus: primary=0c, secondary=0d, subordinate=0f, sec-latency=0
I/O behind bridge: 0000f000-00000fff
Memory behind bridge: f8000000-f81fffff
Prefetchable memory behind bridge: 00000000fff00000-00000000000fffff
Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort+ <SERR- <PERR-
BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B-
PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
Capabilities: [a8] Subsystem: National Instruments Device 8361
Kernel modules: shpchp

So it looks like it is using the shpchp kernel driver.  When I assign this
device to pci-stub during boot, and try to unbind it and bind it to
vfio-pci, it fails.  The device shows up in lspci with no loaded driver.

After I did some digging, I found out:
* One cannot bind a PCI-bridge to vfio-pci.
* One has to bind everything inside an iommu_group to vfio-pci

I also know that:
* Using mmiotrace will not trace all Memory-Mapped writes/reads that are
behind a PCI bridge

My questions are:
* How would someone reverse engineer PCI devices before IOMMU/VT-d hardware
existed?
* Can I pass this PCI-bridge the way it is (whatever that may mean) to a
qemu host and trace it along with all reads/writes using the qemu tracing
mechanisms?

Please correct me if I am wrong in any of my statements, I do not want
others to find this message and assume the wrong thing.

I also have a different PCI device - the NetFPGA 1G version, and I have
been able to use vfio-pci to bind it to a qemu host os (openSUSE) and I was
able to see and use that device from within qemu.

Thank you,

John
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/vfio-users/attachments/20151022/8758e186/attachment.htm>