Some bus drivers use rmans to suballocate resources to child devices.
When the driver for a child device requests a mapping for a
suballocated resource, the bus driver translates this into a mapping
request for a suitable subrange of the original resource the bus
driver allocated from its parent. This nested mapping request should
look like any other resource mapping request being made by the bus
device (i.e. as if the bus device had called bus_map_resource() or
bus_alloc_resource() with RF_ACTIVE).
I had slightly flubbed this last bit though since the direct use of
bus_generic_map/unmap_resource passed up the original child device
(second argument to the underlying kobj interface). While this is
currently harmless, it is not strictly correct as the resource being
mapped is owned by the bus device, not the child and can break for
other bus drivers in the future.
Instead, use bus_map/unmap_resource for the nested request where the
requesting device is now the bus device that owns the parent resource.
Reviewed by: imp
Fixes: 0e1246e33461 acpi: Cleanup handling of suballocated resources
Fixes: b377ff8110e3 pcib: Refine handling of resources allocated from bridge windows
Fixes: d79b6b8ec267 pci_host_generic: Don't rewrite resource start address for translation
Fixes: d714e73f7895 vmd: Use bus_generic_rman_* for PCI bus and memory resources
Differential Revision: https://reviews.freebsd.org/D45433
(cherry picked from commit 98056127ddfa36720bcf46edc09843c867784bcb)
While here, add custom bus_map/unmap_resource methods to request
mappings via the window memory resources allocated from the parent
bus.
Tested by: emaste
Differential Revision: https://reviews.freebsd.org/D43886
(cherry picked from commit d714e73f789515963a22fe64417bf3883cdb599c)
Using an explicit bus space tag and handle is deprecated.
Reviewed by: emaste
Differential Revision: https://reviews.freebsd.org/D43885
(cherry picked from commit 76c678688634e9e2ea5f6369fb979aabddbfe426)
By default all VMD devices remap children MSI/MSI-X interrupts into their
own. It creates additional isolation, but also complicates things due to
sharing, etc. Fortunately some VMD devices can bypass the remapping.
Add tunable to control it for remap testing or if something go wrong.
MFC after: 2 weeks
Summary:
This is based on the Linux kernel commit
922bfd001d1ac02111ebbe0524aaab6ca7925521.
Reviewed by: mav (src)
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D35996
- Re-implement pcib interface to use standard pci bus driver on top of
vmd(4) instead of custom one.
- Re-implement memory/bus resource allocation to properly handle even
complicated configurations.
- Re-implement interrupt handling to evenly distribute children's MSI/
MSI-X interrupts between available vmd(4) MSI-X vectors and setup them
to be handled by standard OS mechanisms with minimal overhead, except
sharing when unavoidable.
Successfully tested on Dell XPS 13 laptop with Core i7-1185G7 CPU (VMD
device ID 0x9a0b) and single NVMe SSD, dual-booting with Windows 10.
Successfully tested on Supermicro X11DPI-NT motherboard with Xeon(R)
Gold 6242R CPUs (VMD device ID 0x201d), simultaneously handling NVMe
SSD on one PCIe port and PLX bridge with 3 NVMe and 1 AHCI SSDs on
another. Handles SSD hot-plug (except Optane 905p for some reason,
which are not detected until manual bus rescan) and enabled IOMMU
(directly connected SSDs work, but ones connected to the PLX fail
without errors from IOMMU).
MFC after: 2 weeks
Sponsored by: iXsystems, Inc.
Differential revision: https://reviews.freebsd.org/D31762
This fixes a kernel panic when probing for vmd_bus on Intel TigerLake on
14-CURRENT. Apparently, vmd_bus is a type of PCI bus, but was registered
as a separate device class.
PR: 256915
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D31071
newer controller have a sparce bus space that can be figured
out by probing the HW. This gives the starting bus number.
When reading the PCI config. space behind the VMD controller,
the offset of the starting bus needs to be subtracted from
the bus being read.
Fixed a bug in which in which not all of the devices
directly attached to the VMD controller would be probed.
On my initial test HW, a switch was found at bus 0, slot 0
and function 0. All of the NVME drives were behind that
switch. Now scan for all slots and functions attached to
bus 0. If a something was found then run attach after the
scan. On detach also go through all slots and functions
on bus 0.
Tested with device ID's: 0x201d & 0x9a0b
Tested by: nc@
MFC after: 7 days
PR: 252253
starting at the max. domain, and then work down. Then existing FreeBSD
drivers will attach. Interrupt routing from the VMD MSI-X to the NVME
drive is not well known, so any interrupt is sent to all children that
register.
VROC used Intel meta data so graid(8) works with it. However, graid(8)
supports RAID 0,1,10 for read and write. I have some early code to
support writes with RAID 5. Note that RAID 5 can have life issues
with SSDs since it can cause write amplification from updating the parity
data.
Hot plug support needs a change to skip the following check to work:
if (pcib_request_feature(dev, PCI_FEATURE_HP) != 0) {
in sys/dev/pci/pci_pci.c.
Looked at by: imp, rpokala, bcr
Differential Revision: https://reviews.freebsd.org/D21383
