1
linux/drivers/pci/controller/vmd.c

1123 lines
29 KiB
C
Raw Normal View History

// SPDX-License-Identifier: GPL-2.0
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
/*
* Volume Management Device driver
* Copyright (c) 2015, Intel Corporation.
*/
#include <linux/device.h>
#include <linux/interrupt.h>
#include <linux/irq.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/msi.h>
#include <linux/pci.h>
#include <linux/pci-acpi.h>
PCI: Unify ECAM constants in native PCI Express drivers Add ECAM-related constants to provide a set of standard constants defining memory address shift values to the byte-level address that can be used to access the PCI Express Configuration Space, and then move native PCI Express controller drivers to use the newly introduced definitions retiring driver-specific ones. Refactor pci_ecam_map_bus() function to use newly added constants so that limits to the bus, device function and offset (now limited to 4K as per the specification) are in place to prevent the defective or malicious caller from supplying incorrect configuration offset and thus targeting the wrong device when accessing extended configuration space. This refactor also allows for the ".bus_shift" initialisers to be dropped when the user is not using a custom value as a default value will be used as per the PCI Express Specification. Thanks to Qian Cai <qcai@redhat.com>, Michael Walle <michael@walle.cc>, and Vladimir Oltean <olteanv@gmail.com> for reporting a pci_ecam_create() issue with .bus_shift and to Vladimir for proposing the fix. [bhelgaas: incorporate Vladimir's fix, update commit log] Suggested-by: Bjorn Helgaas <bhelgaas@google.com> Link: https://lore.kernel.org/r/20201129230743.3006978-2-kw@linux.com Tested-by: Michael Walle <michael@walle.cc> Signed-off-by: Krzysztof Wilczyński <kw@linux.com> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Jon Derrick <jonathan.derrick@intel.com> Reviewed-by: Bjorn Helgaas <bhelgaas@google.com>
2020-11-29 16:07:39 -07:00
#include <linux/pci-ecam.h>
#include <linux/srcu.h>
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
#include <linux/rculist.h>
#include <linux/rcupdate.h>
#include <asm/irqdomain.h>
#define VMD_CFGBAR 0
#define VMD_MEMBAR1 2
#define VMD_MEMBAR2 4
#define PCI_REG_VMCAP 0x40
#define BUS_RESTRICT_CAP(vmcap) (vmcap & 0x1)
#define PCI_REG_VMCONFIG 0x44
#define BUS_RESTRICT_CFG(vmcfg) ((vmcfg >> 8) & 0x3)
#define VMCONFIG_MSI_REMAP 0x2
#define PCI_REG_VMLOCK 0x70
#define MB2_SHADOW_EN(vmlock) (vmlock & 0x2)
#define MB2_SHADOW_OFFSET 0x2000
#define MB2_SHADOW_SIZE 16
enum vmd_features {
/*
* Device may contain registers which hint the physical location of the
* membars, in order to allow proper address translation during
* resource assignment to enable guest virtualization
*/
VMD_FEAT_HAS_MEMBAR_SHADOW = (1 << 0),
/*
* Device may provide root port configuration information which limits
* bus numbering
*/
VMD_FEAT_HAS_BUS_RESTRICTIONS = (1 << 1),
/*
* Device contains physical location shadow registers in
* vendor-specific capability space
*/
VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP = (1 << 2),
/*
* Device may use MSI-X vector 0 for software triggering and will not
* be used for MSI remapping
*/
VMD_FEAT_OFFSET_FIRST_VECTOR = (1 << 3),
/*
* Device can bypass remapping MSI-X transactions into its MSI-X table,
* avoiding the requirement of a VMD MSI domain for child device
* interrupt handling.
*/
VMD_FEAT_CAN_BYPASS_MSI_REMAP = (1 << 4),
2023-01-19 20:15:22 -07:00
/*
* Enable ASPM on the PCIE root ports and set the default LTR of the
* storage devices on platforms where these values are not configured by
* BIOS. This is needed for laptops, which require these settings for
* proper power management of the SoC.
*/
VMD_FEAT_BIOS_PM_QUIRK = (1 << 5),
};
2023-01-19 20:15:22 -07:00
#define VMD_BIOS_PM_QUIRK_LTR 0x1003 /* 3145728 ns */
#define VMD_FEATS_CLIENT (VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP | \
VMD_FEAT_HAS_BUS_RESTRICTIONS | \
2023-01-19 20:15:22 -07:00
VMD_FEAT_OFFSET_FIRST_VECTOR | \
VMD_FEAT_BIOS_PM_QUIRK)
static DEFINE_IDA(vmd_instance_ida);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
/*
* Lock for manipulating VMD IRQ lists.
*/
static DEFINE_RAW_SPINLOCK(list_lock);
/**
* struct vmd_irq - private data to map driver IRQ to the VMD shared vector
* @node: list item for parent traversal.
* @irq: back pointer to parent.
* @enabled: true if driver enabled IRQ
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
* @virq: the virtual IRQ value provided to the requesting driver.
*
* Every MSI/MSI-X IRQ requested for a device in a VMD domain will be mapped to
* a VMD IRQ using this structure.
*/
struct vmd_irq {
struct list_head node;
struct vmd_irq_list *irq;
bool enabled;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
unsigned int virq;
};
/**
* struct vmd_irq_list - list of driver requested IRQs mapping to a VMD vector
* @irq_list: the list of irq's the VMD one demuxes to.
* @srcu: SRCU struct for local synchronization.
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
* @count: number of child IRQs assigned to this vector; used to track
* sharing.
* @virq: The underlying VMD Linux interrupt number
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
*/
struct vmd_irq_list {
struct list_head irq_list;
struct srcu_struct srcu;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
unsigned int count;
unsigned int virq;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
};
struct vmd_dev {
struct pci_dev *dev;
spinlock_t cfg_lock;
void __iomem *cfgbar;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
int msix_count;
struct vmd_irq_list *irqs;
struct pci_sysdata sysdata;
struct resource resources[3];
struct irq_domain *irq_domain;
struct pci_bus *bus;
u8 busn_start;
u8 first_vec;
char *name;
int instance;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
};
static inline struct vmd_dev *vmd_from_bus(struct pci_bus *bus)
{
return container_of(bus->sysdata, struct vmd_dev, sysdata);
}
static inline unsigned int index_from_irqs(struct vmd_dev *vmd,
struct vmd_irq_list *irqs)
{
return irqs - vmd->irqs;
}
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
/*
* Drivers managing a device in a VMD domain allocate their own IRQs as before,
* but the MSI entry for the hardware it's driving will be programmed with a
* destination ID for the VMD MSI-X table. The VMD muxes interrupts in its
* domain into one of its own, and the VMD driver de-muxes these for the
* handlers sharing that VMD IRQ. The vmd irq_domain provides the operations
* and irq_chip to set this up.
*/
static void vmd_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
{
struct vmd_irq *vmdirq = data->chip_data;
struct vmd_irq_list *irq = vmdirq->irq;
struct vmd_dev *vmd = irq_data_get_irq_handler_data(data);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
memset(msg, 0, sizeof(*msg));
msg->address_hi = X86_MSI_BASE_ADDRESS_HIGH;
msg->arch_addr_lo.base_address = X86_MSI_BASE_ADDRESS_LOW;
msg->arch_addr_lo.destid_0_7 = index_from_irqs(vmd, irq);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
}
/*
* We rely on MSI_FLAG_USE_DEF_CHIP_OPS to set the IRQ mask/unmask ops.
*/
static void vmd_irq_enable(struct irq_data *data)
{
struct vmd_irq *vmdirq = data->chip_data;
x86/PCI: VMD: Use lock save/restore in interrupt enable path Enabling interrupts may result in an interrupt raised and serviced while VMD holds a lock, resulting in contention with the spin lock held while enabling interrupts. The solution is to disable preemption and save/restore the state during interrupt enable and disable. Fixes lockdep: ====================================================== [ INFO: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected ] 4.6.0-2016-06-16-lockdep+ #47 Tainted: G E ------------------------------------------------------ kworker/0:1/447 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: (list_lock){+.+...}, at: [<ffffffffa04eb8fc>] vmd_irq_enable+0x3c/0x70 [vmd] and this task is already holding: (&irq_desc_lock_class){-.-...}, at: [<ffffffff810e1ff6>] __setup_irq+0xa6/0x610 which would create a new lock dependency: (&irq_desc_lock_class){-.-...} -> (list_lock){+.+...} but this new dependency connects a HARDIRQ-irq-safe lock: (&irq_desc_lock_class){-.-...} ... which became HARDIRQ-irq-safe at: [<ffffffff810c9f21>] __lock_acquire+0x981/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffff810e36d4>] handle_level_irq+0x24/0x110 [<ffffffff8101f20a>] handle_irq+0x1a/0x30 [<ffffffff81675fc1>] do_IRQ+0x61/0x120 [<ffffffff8167404c>] ret_from_intr+0x0/0x20 [<ffffffff81672e30>] _raw_spin_unlock_irqrestore+0x40/0x60 [<ffffffff810e21ee>] __setup_irq+0x29e/0x610 [<ffffffff810e25a1>] setup_irq+0x41/0x90 [<ffffffff81f5777f>] setup_default_timer_irq+0x1e/0x20 [<ffffffff81f57798>] hpet_time_init+0x17/0x19 [<ffffffff81f5775a>] x86_late_time_init+0xa/0x11 [<ffffffff81f51e9b>] start_kernel+0x382/0x436 [<ffffffff81f51308>] x86_64_start_reservations+0x2a/0x2c [<ffffffff81f51445>] x86_64_start_kernel+0x13b/0x14a to a HARDIRQ-irq-unsafe lock: (list_lock){+.+...} ... which became HARDIRQ-irq-unsafe at: ... [<ffffffff810c9d8e>] __lock_acquire+0x7ee/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffffa04eba42>] vmd_msi_init+0x72/0x150 [vmd] [<ffffffff810e8597>] msi_domain_alloc+0xb7/0x140 [<ffffffff810e6b10>] irq_domain_alloc_irqs_recursive+0x40/0xa0 [<ffffffff810e6cea>] __irq_domain_alloc_irqs+0x14a/0x330 [<ffffffff810e8a8c>] msi_domain_alloc_irqs+0x8c/0x1d0 [<ffffffff813ca4e3>] pci_msi_setup_msi_irqs+0x43/0x70 [<ffffffff813cada1>] pci_enable_msi_range+0x131/0x280 [<ffffffff813bf5e0>] pcie_port_device_register+0x320/0x4e0 [<ffffffff813bf9a4>] pcie_portdrv_probe+0x34/0x60 [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff813b226b>] pci_device_probe+0xdb/0x130 [<ffffffff8149e3cc>] driver_probe_device+0x22c/0x440 [<ffffffff8149e774>] __device_attach_driver+0x94/0x110 [<ffffffff8149bfad>] bus_for_each_drv+0x5d/0x90 [<ffffffff8149e030>] __device_attach+0xc0/0x140 [<ffffffff8149e0c0>] device_attach+0x10/0x20 [<ffffffff813a77f7>] pci_bus_add_device+0x47/0x90 [<ffffffff813a7879>] pci_bus_add_devices+0x39/0x70 [<ffffffff813aaba7>] pci_rescan_bus+0x27/0x30 [<ffffffffa04ec1af>] vmd_probe+0x68f/0x76c [vmd] [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff81088064>] work_for_cpu_fn+0x14/0x20 [<ffffffff8108c244>] process_one_work+0x1f4/0x740 [<ffffffff8108c9c6>] worker_thread+0x236/0x4f0 [<ffffffff810935c2>] kthread+0xf2/0x110 [<ffffffff816738f2>] ret_from_fork+0x22/0x50 other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(list_lock); local_irq_disable(); lock(&irq_desc_lock_class); lock(list_lock); <Interrupt> lock(&irq_desc_lock_class); *** DEADLOCK *** Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-06-20 08:39:51 -07:00
unsigned long flags;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
x86/PCI: VMD: Use lock save/restore in interrupt enable path Enabling interrupts may result in an interrupt raised and serviced while VMD holds a lock, resulting in contention with the spin lock held while enabling interrupts. The solution is to disable preemption and save/restore the state during interrupt enable and disable. Fixes lockdep: ====================================================== [ INFO: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected ] 4.6.0-2016-06-16-lockdep+ #47 Tainted: G E ------------------------------------------------------ kworker/0:1/447 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: (list_lock){+.+...}, at: [<ffffffffa04eb8fc>] vmd_irq_enable+0x3c/0x70 [vmd] and this task is already holding: (&irq_desc_lock_class){-.-...}, at: [<ffffffff810e1ff6>] __setup_irq+0xa6/0x610 which would create a new lock dependency: (&irq_desc_lock_class){-.-...} -> (list_lock){+.+...} but this new dependency connects a HARDIRQ-irq-safe lock: (&irq_desc_lock_class){-.-...} ... which became HARDIRQ-irq-safe at: [<ffffffff810c9f21>] __lock_acquire+0x981/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffff810e36d4>] handle_level_irq+0x24/0x110 [<ffffffff8101f20a>] handle_irq+0x1a/0x30 [<ffffffff81675fc1>] do_IRQ+0x61/0x120 [<ffffffff8167404c>] ret_from_intr+0x0/0x20 [<ffffffff81672e30>] _raw_spin_unlock_irqrestore+0x40/0x60 [<ffffffff810e21ee>] __setup_irq+0x29e/0x610 [<ffffffff810e25a1>] setup_irq+0x41/0x90 [<ffffffff81f5777f>] setup_default_timer_irq+0x1e/0x20 [<ffffffff81f57798>] hpet_time_init+0x17/0x19 [<ffffffff81f5775a>] x86_late_time_init+0xa/0x11 [<ffffffff81f51e9b>] start_kernel+0x382/0x436 [<ffffffff81f51308>] x86_64_start_reservations+0x2a/0x2c [<ffffffff81f51445>] x86_64_start_kernel+0x13b/0x14a to a HARDIRQ-irq-unsafe lock: (list_lock){+.+...} ... which became HARDIRQ-irq-unsafe at: ... [<ffffffff810c9d8e>] __lock_acquire+0x7ee/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffffa04eba42>] vmd_msi_init+0x72/0x150 [vmd] [<ffffffff810e8597>] msi_domain_alloc+0xb7/0x140 [<ffffffff810e6b10>] irq_domain_alloc_irqs_recursive+0x40/0xa0 [<ffffffff810e6cea>] __irq_domain_alloc_irqs+0x14a/0x330 [<ffffffff810e8a8c>] msi_domain_alloc_irqs+0x8c/0x1d0 [<ffffffff813ca4e3>] pci_msi_setup_msi_irqs+0x43/0x70 [<ffffffff813cada1>] pci_enable_msi_range+0x131/0x280 [<ffffffff813bf5e0>] pcie_port_device_register+0x320/0x4e0 [<ffffffff813bf9a4>] pcie_portdrv_probe+0x34/0x60 [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff813b226b>] pci_device_probe+0xdb/0x130 [<ffffffff8149e3cc>] driver_probe_device+0x22c/0x440 [<ffffffff8149e774>] __device_attach_driver+0x94/0x110 [<ffffffff8149bfad>] bus_for_each_drv+0x5d/0x90 [<ffffffff8149e030>] __device_attach+0xc0/0x140 [<ffffffff8149e0c0>] device_attach+0x10/0x20 [<ffffffff813a77f7>] pci_bus_add_device+0x47/0x90 [<ffffffff813a7879>] pci_bus_add_devices+0x39/0x70 [<ffffffff813aaba7>] pci_rescan_bus+0x27/0x30 [<ffffffffa04ec1af>] vmd_probe+0x68f/0x76c [vmd] [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff81088064>] work_for_cpu_fn+0x14/0x20 [<ffffffff8108c244>] process_one_work+0x1f4/0x740 [<ffffffff8108c9c6>] worker_thread+0x236/0x4f0 [<ffffffff810935c2>] kthread+0xf2/0x110 [<ffffffff816738f2>] ret_from_fork+0x22/0x50 other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(list_lock); local_irq_disable(); lock(&irq_desc_lock_class); lock(list_lock); <Interrupt> lock(&irq_desc_lock_class); *** DEADLOCK *** Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-06-20 08:39:51 -07:00
raw_spin_lock_irqsave(&list_lock, flags);
WARN_ON(vmdirq->enabled);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
list_add_tail_rcu(&vmdirq->node, &vmdirq->irq->irq_list);
vmdirq->enabled = true;
x86/PCI: VMD: Use lock save/restore in interrupt enable path Enabling interrupts may result in an interrupt raised and serviced while VMD holds a lock, resulting in contention with the spin lock held while enabling interrupts. The solution is to disable preemption and save/restore the state during interrupt enable and disable. Fixes lockdep: ====================================================== [ INFO: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected ] 4.6.0-2016-06-16-lockdep+ #47 Tainted: G E ------------------------------------------------------ kworker/0:1/447 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: (list_lock){+.+...}, at: [<ffffffffa04eb8fc>] vmd_irq_enable+0x3c/0x70 [vmd] and this task is already holding: (&irq_desc_lock_class){-.-...}, at: [<ffffffff810e1ff6>] __setup_irq+0xa6/0x610 which would create a new lock dependency: (&irq_desc_lock_class){-.-...} -> (list_lock){+.+...} but this new dependency connects a HARDIRQ-irq-safe lock: (&irq_desc_lock_class){-.-...} ... which became HARDIRQ-irq-safe at: [<ffffffff810c9f21>] __lock_acquire+0x981/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffff810e36d4>] handle_level_irq+0x24/0x110 [<ffffffff8101f20a>] handle_irq+0x1a/0x30 [<ffffffff81675fc1>] do_IRQ+0x61/0x120 [<ffffffff8167404c>] ret_from_intr+0x0/0x20 [<ffffffff81672e30>] _raw_spin_unlock_irqrestore+0x40/0x60 [<ffffffff810e21ee>] __setup_irq+0x29e/0x610 [<ffffffff810e25a1>] setup_irq+0x41/0x90 [<ffffffff81f5777f>] setup_default_timer_irq+0x1e/0x20 [<ffffffff81f57798>] hpet_time_init+0x17/0x19 [<ffffffff81f5775a>] x86_late_time_init+0xa/0x11 [<ffffffff81f51e9b>] start_kernel+0x382/0x436 [<ffffffff81f51308>] x86_64_start_reservations+0x2a/0x2c [<ffffffff81f51445>] x86_64_start_kernel+0x13b/0x14a to a HARDIRQ-irq-unsafe lock: (list_lock){+.+...} ... which became HARDIRQ-irq-unsafe at: ... [<ffffffff810c9d8e>] __lock_acquire+0x7ee/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffffa04eba42>] vmd_msi_init+0x72/0x150 [vmd] [<ffffffff810e8597>] msi_domain_alloc+0xb7/0x140 [<ffffffff810e6b10>] irq_domain_alloc_irqs_recursive+0x40/0xa0 [<ffffffff810e6cea>] __irq_domain_alloc_irqs+0x14a/0x330 [<ffffffff810e8a8c>] msi_domain_alloc_irqs+0x8c/0x1d0 [<ffffffff813ca4e3>] pci_msi_setup_msi_irqs+0x43/0x70 [<ffffffff813cada1>] pci_enable_msi_range+0x131/0x280 [<ffffffff813bf5e0>] pcie_port_device_register+0x320/0x4e0 [<ffffffff813bf9a4>] pcie_portdrv_probe+0x34/0x60 [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff813b226b>] pci_device_probe+0xdb/0x130 [<ffffffff8149e3cc>] driver_probe_device+0x22c/0x440 [<ffffffff8149e774>] __device_attach_driver+0x94/0x110 [<ffffffff8149bfad>] bus_for_each_drv+0x5d/0x90 [<ffffffff8149e030>] __device_attach+0xc0/0x140 [<ffffffff8149e0c0>] device_attach+0x10/0x20 [<ffffffff813a77f7>] pci_bus_add_device+0x47/0x90 [<ffffffff813a7879>] pci_bus_add_devices+0x39/0x70 [<ffffffff813aaba7>] pci_rescan_bus+0x27/0x30 [<ffffffffa04ec1af>] vmd_probe+0x68f/0x76c [vmd] [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff81088064>] work_for_cpu_fn+0x14/0x20 [<ffffffff8108c244>] process_one_work+0x1f4/0x740 [<ffffffff8108c9c6>] worker_thread+0x236/0x4f0 [<ffffffff810935c2>] kthread+0xf2/0x110 [<ffffffff816738f2>] ret_from_fork+0x22/0x50 other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(list_lock); local_irq_disable(); lock(&irq_desc_lock_class); lock(list_lock); <Interrupt> lock(&irq_desc_lock_class); *** DEADLOCK *** Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-06-20 08:39:51 -07:00
raw_spin_unlock_irqrestore(&list_lock, flags);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
data->chip->irq_unmask(data);
}
static void vmd_irq_disable(struct irq_data *data)
{
struct vmd_irq *vmdirq = data->chip_data;
x86/PCI: VMD: Use lock save/restore in interrupt enable path Enabling interrupts may result in an interrupt raised and serviced while VMD holds a lock, resulting in contention with the spin lock held while enabling interrupts. The solution is to disable preemption and save/restore the state during interrupt enable and disable. Fixes lockdep: ====================================================== [ INFO: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected ] 4.6.0-2016-06-16-lockdep+ #47 Tainted: G E ------------------------------------------------------ kworker/0:1/447 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: (list_lock){+.+...}, at: [<ffffffffa04eb8fc>] vmd_irq_enable+0x3c/0x70 [vmd] and this task is already holding: (&irq_desc_lock_class){-.-...}, at: [<ffffffff810e1ff6>] __setup_irq+0xa6/0x610 which would create a new lock dependency: (&irq_desc_lock_class){-.-...} -> (list_lock){+.+...} but this new dependency connects a HARDIRQ-irq-safe lock: (&irq_desc_lock_class){-.-...} ... which became HARDIRQ-irq-safe at: [<ffffffff810c9f21>] __lock_acquire+0x981/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffff810e36d4>] handle_level_irq+0x24/0x110 [<ffffffff8101f20a>] handle_irq+0x1a/0x30 [<ffffffff81675fc1>] do_IRQ+0x61/0x120 [<ffffffff8167404c>] ret_from_intr+0x0/0x20 [<ffffffff81672e30>] _raw_spin_unlock_irqrestore+0x40/0x60 [<ffffffff810e21ee>] __setup_irq+0x29e/0x610 [<ffffffff810e25a1>] setup_irq+0x41/0x90 [<ffffffff81f5777f>] setup_default_timer_irq+0x1e/0x20 [<ffffffff81f57798>] hpet_time_init+0x17/0x19 [<ffffffff81f5775a>] x86_late_time_init+0xa/0x11 [<ffffffff81f51e9b>] start_kernel+0x382/0x436 [<ffffffff81f51308>] x86_64_start_reservations+0x2a/0x2c [<ffffffff81f51445>] x86_64_start_kernel+0x13b/0x14a to a HARDIRQ-irq-unsafe lock: (list_lock){+.+...} ... which became HARDIRQ-irq-unsafe at: ... [<ffffffff810c9d8e>] __lock_acquire+0x7ee/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffffa04eba42>] vmd_msi_init+0x72/0x150 [vmd] [<ffffffff810e8597>] msi_domain_alloc+0xb7/0x140 [<ffffffff810e6b10>] irq_domain_alloc_irqs_recursive+0x40/0xa0 [<ffffffff810e6cea>] __irq_domain_alloc_irqs+0x14a/0x330 [<ffffffff810e8a8c>] msi_domain_alloc_irqs+0x8c/0x1d0 [<ffffffff813ca4e3>] pci_msi_setup_msi_irqs+0x43/0x70 [<ffffffff813cada1>] pci_enable_msi_range+0x131/0x280 [<ffffffff813bf5e0>] pcie_port_device_register+0x320/0x4e0 [<ffffffff813bf9a4>] pcie_portdrv_probe+0x34/0x60 [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff813b226b>] pci_device_probe+0xdb/0x130 [<ffffffff8149e3cc>] driver_probe_device+0x22c/0x440 [<ffffffff8149e774>] __device_attach_driver+0x94/0x110 [<ffffffff8149bfad>] bus_for_each_drv+0x5d/0x90 [<ffffffff8149e030>] __device_attach+0xc0/0x140 [<ffffffff8149e0c0>] device_attach+0x10/0x20 [<ffffffff813a77f7>] pci_bus_add_device+0x47/0x90 [<ffffffff813a7879>] pci_bus_add_devices+0x39/0x70 [<ffffffff813aaba7>] pci_rescan_bus+0x27/0x30 [<ffffffffa04ec1af>] vmd_probe+0x68f/0x76c [vmd] [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff81088064>] work_for_cpu_fn+0x14/0x20 [<ffffffff8108c244>] process_one_work+0x1f4/0x740 [<ffffffff8108c9c6>] worker_thread+0x236/0x4f0 [<ffffffff810935c2>] kthread+0xf2/0x110 [<ffffffff816738f2>] ret_from_fork+0x22/0x50 other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(list_lock); local_irq_disable(); lock(&irq_desc_lock_class); lock(list_lock); <Interrupt> lock(&irq_desc_lock_class); *** DEADLOCK *** Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-06-20 08:39:51 -07:00
unsigned long flags;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
data->chip->irq_mask(data);
x86/PCI: VMD: Use lock save/restore in interrupt enable path Enabling interrupts may result in an interrupt raised and serviced while VMD holds a lock, resulting in contention with the spin lock held while enabling interrupts. The solution is to disable preemption and save/restore the state during interrupt enable and disable. Fixes lockdep: ====================================================== [ INFO: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected ] 4.6.0-2016-06-16-lockdep+ #47 Tainted: G E ------------------------------------------------------ kworker/0:1/447 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: (list_lock){+.+...}, at: [<ffffffffa04eb8fc>] vmd_irq_enable+0x3c/0x70 [vmd] and this task is already holding: (&irq_desc_lock_class){-.-...}, at: [<ffffffff810e1ff6>] __setup_irq+0xa6/0x610 which would create a new lock dependency: (&irq_desc_lock_class){-.-...} -> (list_lock){+.+...} but this new dependency connects a HARDIRQ-irq-safe lock: (&irq_desc_lock_class){-.-...} ... which became HARDIRQ-irq-safe at: [<ffffffff810c9f21>] __lock_acquire+0x981/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffff810e36d4>] handle_level_irq+0x24/0x110 [<ffffffff8101f20a>] handle_irq+0x1a/0x30 [<ffffffff81675fc1>] do_IRQ+0x61/0x120 [<ffffffff8167404c>] ret_from_intr+0x0/0x20 [<ffffffff81672e30>] _raw_spin_unlock_irqrestore+0x40/0x60 [<ffffffff810e21ee>] __setup_irq+0x29e/0x610 [<ffffffff810e25a1>] setup_irq+0x41/0x90 [<ffffffff81f5777f>] setup_default_timer_irq+0x1e/0x20 [<ffffffff81f57798>] hpet_time_init+0x17/0x19 [<ffffffff81f5775a>] x86_late_time_init+0xa/0x11 [<ffffffff81f51e9b>] start_kernel+0x382/0x436 [<ffffffff81f51308>] x86_64_start_reservations+0x2a/0x2c [<ffffffff81f51445>] x86_64_start_kernel+0x13b/0x14a to a HARDIRQ-irq-unsafe lock: (list_lock){+.+...} ... which became HARDIRQ-irq-unsafe at: ... [<ffffffff810c9d8e>] __lock_acquire+0x7ee/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffffa04eba42>] vmd_msi_init+0x72/0x150 [vmd] [<ffffffff810e8597>] msi_domain_alloc+0xb7/0x140 [<ffffffff810e6b10>] irq_domain_alloc_irqs_recursive+0x40/0xa0 [<ffffffff810e6cea>] __irq_domain_alloc_irqs+0x14a/0x330 [<ffffffff810e8a8c>] msi_domain_alloc_irqs+0x8c/0x1d0 [<ffffffff813ca4e3>] pci_msi_setup_msi_irqs+0x43/0x70 [<ffffffff813cada1>] pci_enable_msi_range+0x131/0x280 [<ffffffff813bf5e0>] pcie_port_device_register+0x320/0x4e0 [<ffffffff813bf9a4>] pcie_portdrv_probe+0x34/0x60 [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff813b226b>] pci_device_probe+0xdb/0x130 [<ffffffff8149e3cc>] driver_probe_device+0x22c/0x440 [<ffffffff8149e774>] __device_attach_driver+0x94/0x110 [<ffffffff8149bfad>] bus_for_each_drv+0x5d/0x90 [<ffffffff8149e030>] __device_attach+0xc0/0x140 [<ffffffff8149e0c0>] device_attach+0x10/0x20 [<ffffffff813a77f7>] pci_bus_add_device+0x47/0x90 [<ffffffff813a7879>] pci_bus_add_devices+0x39/0x70 [<ffffffff813aaba7>] pci_rescan_bus+0x27/0x30 [<ffffffffa04ec1af>] vmd_probe+0x68f/0x76c [vmd] [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff81088064>] work_for_cpu_fn+0x14/0x20 [<ffffffff8108c244>] process_one_work+0x1f4/0x740 [<ffffffff8108c9c6>] worker_thread+0x236/0x4f0 [<ffffffff810935c2>] kthread+0xf2/0x110 [<ffffffff816738f2>] ret_from_fork+0x22/0x50 other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(list_lock); local_irq_disable(); lock(&irq_desc_lock_class); lock(list_lock); <Interrupt> lock(&irq_desc_lock_class); *** DEADLOCK *** Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-06-20 08:39:51 -07:00
raw_spin_lock_irqsave(&list_lock, flags);
if (vmdirq->enabled) {
list_del_rcu(&vmdirq->node);
vmdirq->enabled = false;
}
x86/PCI: VMD: Use lock save/restore in interrupt enable path Enabling interrupts may result in an interrupt raised and serviced while VMD holds a lock, resulting in contention with the spin lock held while enabling interrupts. The solution is to disable preemption and save/restore the state during interrupt enable and disable. Fixes lockdep: ====================================================== [ INFO: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected ] 4.6.0-2016-06-16-lockdep+ #47 Tainted: G E ------------------------------------------------------ kworker/0:1/447 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: (list_lock){+.+...}, at: [<ffffffffa04eb8fc>] vmd_irq_enable+0x3c/0x70 [vmd] and this task is already holding: (&irq_desc_lock_class){-.-...}, at: [<ffffffff810e1ff6>] __setup_irq+0xa6/0x610 which would create a new lock dependency: (&irq_desc_lock_class){-.-...} -> (list_lock){+.+...} but this new dependency connects a HARDIRQ-irq-safe lock: (&irq_desc_lock_class){-.-...} ... which became HARDIRQ-irq-safe at: [<ffffffff810c9f21>] __lock_acquire+0x981/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffff810e36d4>] handle_level_irq+0x24/0x110 [<ffffffff8101f20a>] handle_irq+0x1a/0x30 [<ffffffff81675fc1>] do_IRQ+0x61/0x120 [<ffffffff8167404c>] ret_from_intr+0x0/0x20 [<ffffffff81672e30>] _raw_spin_unlock_irqrestore+0x40/0x60 [<ffffffff810e21ee>] __setup_irq+0x29e/0x610 [<ffffffff810e25a1>] setup_irq+0x41/0x90 [<ffffffff81f5777f>] setup_default_timer_irq+0x1e/0x20 [<ffffffff81f57798>] hpet_time_init+0x17/0x19 [<ffffffff81f5775a>] x86_late_time_init+0xa/0x11 [<ffffffff81f51e9b>] start_kernel+0x382/0x436 [<ffffffff81f51308>] x86_64_start_reservations+0x2a/0x2c [<ffffffff81f51445>] x86_64_start_kernel+0x13b/0x14a to a HARDIRQ-irq-unsafe lock: (list_lock){+.+...} ... which became HARDIRQ-irq-unsafe at: ... [<ffffffff810c9d8e>] __lock_acquire+0x7ee/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffffa04eba42>] vmd_msi_init+0x72/0x150 [vmd] [<ffffffff810e8597>] msi_domain_alloc+0xb7/0x140 [<ffffffff810e6b10>] irq_domain_alloc_irqs_recursive+0x40/0xa0 [<ffffffff810e6cea>] __irq_domain_alloc_irqs+0x14a/0x330 [<ffffffff810e8a8c>] msi_domain_alloc_irqs+0x8c/0x1d0 [<ffffffff813ca4e3>] pci_msi_setup_msi_irqs+0x43/0x70 [<ffffffff813cada1>] pci_enable_msi_range+0x131/0x280 [<ffffffff813bf5e0>] pcie_port_device_register+0x320/0x4e0 [<ffffffff813bf9a4>] pcie_portdrv_probe+0x34/0x60 [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff813b226b>] pci_device_probe+0xdb/0x130 [<ffffffff8149e3cc>] driver_probe_device+0x22c/0x440 [<ffffffff8149e774>] __device_attach_driver+0x94/0x110 [<ffffffff8149bfad>] bus_for_each_drv+0x5d/0x90 [<ffffffff8149e030>] __device_attach+0xc0/0x140 [<ffffffff8149e0c0>] device_attach+0x10/0x20 [<ffffffff813a77f7>] pci_bus_add_device+0x47/0x90 [<ffffffff813a7879>] pci_bus_add_devices+0x39/0x70 [<ffffffff813aaba7>] pci_rescan_bus+0x27/0x30 [<ffffffffa04ec1af>] vmd_probe+0x68f/0x76c [vmd] [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff81088064>] work_for_cpu_fn+0x14/0x20 [<ffffffff8108c244>] process_one_work+0x1f4/0x740 [<ffffffff8108c9c6>] worker_thread+0x236/0x4f0 [<ffffffff810935c2>] kthread+0xf2/0x110 [<ffffffff816738f2>] ret_from_fork+0x22/0x50 other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(list_lock); local_irq_disable(); lock(&irq_desc_lock_class); lock(list_lock); <Interrupt> lock(&irq_desc_lock_class); *** DEADLOCK *** Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-06-20 08:39:51 -07:00
raw_spin_unlock_irqrestore(&list_lock, flags);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
}
static struct irq_chip vmd_msi_controller = {
.name = "VMD-MSI",
.irq_enable = vmd_irq_enable,
.irq_disable = vmd_irq_disable,
.irq_compose_msi_msg = vmd_compose_msi_msg,
};
static irq_hw_number_t vmd_get_hwirq(struct msi_domain_info *info,
msi_alloc_info_t *arg)
{
return 0;
}
/*
* XXX: We can be even smarter selecting the best IRQ once we solve the
* affinity problem.
*/
static struct vmd_irq_list *vmd_next_irq(struct vmd_dev *vmd, struct msi_desc *desc)
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
{
x86/PCI: VMD: Use lock save/restore in interrupt enable path Enabling interrupts may result in an interrupt raised and serviced while VMD holds a lock, resulting in contention with the spin lock held while enabling interrupts. The solution is to disable preemption and save/restore the state during interrupt enable and disable. Fixes lockdep: ====================================================== [ INFO: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected ] 4.6.0-2016-06-16-lockdep+ #47 Tainted: G E ------------------------------------------------------ kworker/0:1/447 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: (list_lock){+.+...}, at: [<ffffffffa04eb8fc>] vmd_irq_enable+0x3c/0x70 [vmd] and this task is already holding: (&irq_desc_lock_class){-.-...}, at: [<ffffffff810e1ff6>] __setup_irq+0xa6/0x610 which would create a new lock dependency: (&irq_desc_lock_class){-.-...} -> (list_lock){+.+...} but this new dependency connects a HARDIRQ-irq-safe lock: (&irq_desc_lock_class){-.-...} ... which became HARDIRQ-irq-safe at: [<ffffffff810c9f21>] __lock_acquire+0x981/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffff810e36d4>] handle_level_irq+0x24/0x110 [<ffffffff8101f20a>] handle_irq+0x1a/0x30 [<ffffffff81675fc1>] do_IRQ+0x61/0x120 [<ffffffff8167404c>] ret_from_intr+0x0/0x20 [<ffffffff81672e30>] _raw_spin_unlock_irqrestore+0x40/0x60 [<ffffffff810e21ee>] __setup_irq+0x29e/0x610 [<ffffffff810e25a1>] setup_irq+0x41/0x90 [<ffffffff81f5777f>] setup_default_timer_irq+0x1e/0x20 [<ffffffff81f57798>] hpet_time_init+0x17/0x19 [<ffffffff81f5775a>] x86_late_time_init+0xa/0x11 [<ffffffff81f51e9b>] start_kernel+0x382/0x436 [<ffffffff81f51308>] x86_64_start_reservations+0x2a/0x2c [<ffffffff81f51445>] x86_64_start_kernel+0x13b/0x14a to a HARDIRQ-irq-unsafe lock: (list_lock){+.+...} ... which became HARDIRQ-irq-unsafe at: ... [<ffffffff810c9d8e>] __lock_acquire+0x7ee/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffffa04eba42>] vmd_msi_init+0x72/0x150 [vmd] [<ffffffff810e8597>] msi_domain_alloc+0xb7/0x140 [<ffffffff810e6b10>] irq_domain_alloc_irqs_recursive+0x40/0xa0 [<ffffffff810e6cea>] __irq_domain_alloc_irqs+0x14a/0x330 [<ffffffff810e8a8c>] msi_domain_alloc_irqs+0x8c/0x1d0 [<ffffffff813ca4e3>] pci_msi_setup_msi_irqs+0x43/0x70 [<ffffffff813cada1>] pci_enable_msi_range+0x131/0x280 [<ffffffff813bf5e0>] pcie_port_device_register+0x320/0x4e0 [<ffffffff813bf9a4>] pcie_portdrv_probe+0x34/0x60 [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff813b226b>] pci_device_probe+0xdb/0x130 [<ffffffff8149e3cc>] driver_probe_device+0x22c/0x440 [<ffffffff8149e774>] __device_attach_driver+0x94/0x110 [<ffffffff8149bfad>] bus_for_each_drv+0x5d/0x90 [<ffffffff8149e030>] __device_attach+0xc0/0x140 [<ffffffff8149e0c0>] device_attach+0x10/0x20 [<ffffffff813a77f7>] pci_bus_add_device+0x47/0x90 [<ffffffff813a7879>] pci_bus_add_devices+0x39/0x70 [<ffffffff813aaba7>] pci_rescan_bus+0x27/0x30 [<ffffffffa04ec1af>] vmd_probe+0x68f/0x76c [vmd] [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff81088064>] work_for_cpu_fn+0x14/0x20 [<ffffffff8108c244>] process_one_work+0x1f4/0x740 [<ffffffff8108c9c6>] worker_thread+0x236/0x4f0 [<ffffffff810935c2>] kthread+0xf2/0x110 [<ffffffff816738f2>] ret_from_fork+0x22/0x50 other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(list_lock); local_irq_disable(); lock(&irq_desc_lock_class); lock(list_lock); <Interrupt> lock(&irq_desc_lock_class); *** DEADLOCK *** Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-06-20 08:39:51 -07:00
unsigned long flags;
int i, best;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
if (vmd->msix_count == 1 + vmd->first_vec)
return &vmd->irqs[vmd->first_vec];
/*
* White list for fast-interrupt handlers. All others will share the
* "slow" interrupt vector.
*/
switch (msi_desc_to_pci_dev(desc)->class) {
case PCI_CLASS_STORAGE_EXPRESS:
break;
default:
return &vmd->irqs[vmd->first_vec];
}
x86/PCI: VMD: Use lock save/restore in interrupt enable path Enabling interrupts may result in an interrupt raised and serviced while VMD holds a lock, resulting in contention with the spin lock held while enabling interrupts. The solution is to disable preemption and save/restore the state during interrupt enable and disable. Fixes lockdep: ====================================================== [ INFO: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected ] 4.6.0-2016-06-16-lockdep+ #47 Tainted: G E ------------------------------------------------------ kworker/0:1/447 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: (list_lock){+.+...}, at: [<ffffffffa04eb8fc>] vmd_irq_enable+0x3c/0x70 [vmd] and this task is already holding: (&irq_desc_lock_class){-.-...}, at: [<ffffffff810e1ff6>] __setup_irq+0xa6/0x610 which would create a new lock dependency: (&irq_desc_lock_class){-.-...} -> (list_lock){+.+...} but this new dependency connects a HARDIRQ-irq-safe lock: (&irq_desc_lock_class){-.-...} ... which became HARDIRQ-irq-safe at: [<ffffffff810c9f21>] __lock_acquire+0x981/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffff810e36d4>] handle_level_irq+0x24/0x110 [<ffffffff8101f20a>] handle_irq+0x1a/0x30 [<ffffffff81675fc1>] do_IRQ+0x61/0x120 [<ffffffff8167404c>] ret_from_intr+0x0/0x20 [<ffffffff81672e30>] _raw_spin_unlock_irqrestore+0x40/0x60 [<ffffffff810e21ee>] __setup_irq+0x29e/0x610 [<ffffffff810e25a1>] setup_irq+0x41/0x90 [<ffffffff81f5777f>] setup_default_timer_irq+0x1e/0x20 [<ffffffff81f57798>] hpet_time_init+0x17/0x19 [<ffffffff81f5775a>] x86_late_time_init+0xa/0x11 [<ffffffff81f51e9b>] start_kernel+0x382/0x436 [<ffffffff81f51308>] x86_64_start_reservations+0x2a/0x2c [<ffffffff81f51445>] x86_64_start_kernel+0x13b/0x14a to a HARDIRQ-irq-unsafe lock: (list_lock){+.+...} ... which became HARDIRQ-irq-unsafe at: ... [<ffffffff810c9d8e>] __lock_acquire+0x7ee/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffffa04eba42>] vmd_msi_init+0x72/0x150 [vmd] [<ffffffff810e8597>] msi_domain_alloc+0xb7/0x140 [<ffffffff810e6b10>] irq_domain_alloc_irqs_recursive+0x40/0xa0 [<ffffffff810e6cea>] __irq_domain_alloc_irqs+0x14a/0x330 [<ffffffff810e8a8c>] msi_domain_alloc_irqs+0x8c/0x1d0 [<ffffffff813ca4e3>] pci_msi_setup_msi_irqs+0x43/0x70 [<ffffffff813cada1>] pci_enable_msi_range+0x131/0x280 [<ffffffff813bf5e0>] pcie_port_device_register+0x320/0x4e0 [<ffffffff813bf9a4>] pcie_portdrv_probe+0x34/0x60 [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff813b226b>] pci_device_probe+0xdb/0x130 [<ffffffff8149e3cc>] driver_probe_device+0x22c/0x440 [<ffffffff8149e774>] __device_attach_driver+0x94/0x110 [<ffffffff8149bfad>] bus_for_each_drv+0x5d/0x90 [<ffffffff8149e030>] __device_attach+0xc0/0x140 [<ffffffff8149e0c0>] device_attach+0x10/0x20 [<ffffffff813a77f7>] pci_bus_add_device+0x47/0x90 [<ffffffff813a7879>] pci_bus_add_devices+0x39/0x70 [<ffffffff813aaba7>] pci_rescan_bus+0x27/0x30 [<ffffffffa04ec1af>] vmd_probe+0x68f/0x76c [vmd] [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff81088064>] work_for_cpu_fn+0x14/0x20 [<ffffffff8108c244>] process_one_work+0x1f4/0x740 [<ffffffff8108c9c6>] worker_thread+0x236/0x4f0 [<ffffffff810935c2>] kthread+0xf2/0x110 [<ffffffff816738f2>] ret_from_fork+0x22/0x50 other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(list_lock); local_irq_disable(); lock(&irq_desc_lock_class); lock(list_lock); <Interrupt> lock(&irq_desc_lock_class); *** DEADLOCK *** Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-06-20 08:39:51 -07:00
raw_spin_lock_irqsave(&list_lock, flags);
best = vmd->first_vec + 1;
for (i = best; i < vmd->msix_count; i++)
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
if (vmd->irqs[i].count < vmd->irqs[best].count)
best = i;
vmd->irqs[best].count++;
x86/PCI: VMD: Use lock save/restore in interrupt enable path Enabling interrupts may result in an interrupt raised and serviced while VMD holds a lock, resulting in contention with the spin lock held while enabling interrupts. The solution is to disable preemption and save/restore the state during interrupt enable and disable. Fixes lockdep: ====================================================== [ INFO: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected ] 4.6.0-2016-06-16-lockdep+ #47 Tainted: G E ------------------------------------------------------ kworker/0:1/447 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: (list_lock){+.+...}, at: [<ffffffffa04eb8fc>] vmd_irq_enable+0x3c/0x70 [vmd] and this task is already holding: (&irq_desc_lock_class){-.-...}, at: [<ffffffff810e1ff6>] __setup_irq+0xa6/0x610 which would create a new lock dependency: (&irq_desc_lock_class){-.-...} -> (list_lock){+.+...} but this new dependency connects a HARDIRQ-irq-safe lock: (&irq_desc_lock_class){-.-...} ... which became HARDIRQ-irq-safe at: [<ffffffff810c9f21>] __lock_acquire+0x981/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffff810e36d4>] handle_level_irq+0x24/0x110 [<ffffffff8101f20a>] handle_irq+0x1a/0x30 [<ffffffff81675fc1>] do_IRQ+0x61/0x120 [<ffffffff8167404c>] ret_from_intr+0x0/0x20 [<ffffffff81672e30>] _raw_spin_unlock_irqrestore+0x40/0x60 [<ffffffff810e21ee>] __setup_irq+0x29e/0x610 [<ffffffff810e25a1>] setup_irq+0x41/0x90 [<ffffffff81f5777f>] setup_default_timer_irq+0x1e/0x20 [<ffffffff81f57798>] hpet_time_init+0x17/0x19 [<ffffffff81f5775a>] x86_late_time_init+0xa/0x11 [<ffffffff81f51e9b>] start_kernel+0x382/0x436 [<ffffffff81f51308>] x86_64_start_reservations+0x2a/0x2c [<ffffffff81f51445>] x86_64_start_kernel+0x13b/0x14a to a HARDIRQ-irq-unsafe lock: (list_lock){+.+...} ... which became HARDIRQ-irq-unsafe at: ... [<ffffffff810c9d8e>] __lock_acquire+0x7ee/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffffa04eba42>] vmd_msi_init+0x72/0x150 [vmd] [<ffffffff810e8597>] msi_domain_alloc+0xb7/0x140 [<ffffffff810e6b10>] irq_domain_alloc_irqs_recursive+0x40/0xa0 [<ffffffff810e6cea>] __irq_domain_alloc_irqs+0x14a/0x330 [<ffffffff810e8a8c>] msi_domain_alloc_irqs+0x8c/0x1d0 [<ffffffff813ca4e3>] pci_msi_setup_msi_irqs+0x43/0x70 [<ffffffff813cada1>] pci_enable_msi_range+0x131/0x280 [<ffffffff813bf5e0>] pcie_port_device_register+0x320/0x4e0 [<ffffffff813bf9a4>] pcie_portdrv_probe+0x34/0x60 [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff813b226b>] pci_device_probe+0xdb/0x130 [<ffffffff8149e3cc>] driver_probe_device+0x22c/0x440 [<ffffffff8149e774>] __device_attach_driver+0x94/0x110 [<ffffffff8149bfad>] bus_for_each_drv+0x5d/0x90 [<ffffffff8149e030>] __device_attach+0xc0/0x140 [<ffffffff8149e0c0>] device_attach+0x10/0x20 [<ffffffff813a77f7>] pci_bus_add_device+0x47/0x90 [<ffffffff813a7879>] pci_bus_add_devices+0x39/0x70 [<ffffffff813aaba7>] pci_rescan_bus+0x27/0x30 [<ffffffffa04ec1af>] vmd_probe+0x68f/0x76c [vmd] [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff81088064>] work_for_cpu_fn+0x14/0x20 [<ffffffff8108c244>] process_one_work+0x1f4/0x740 [<ffffffff8108c9c6>] worker_thread+0x236/0x4f0 [<ffffffff810935c2>] kthread+0xf2/0x110 [<ffffffff816738f2>] ret_from_fork+0x22/0x50 other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(list_lock); local_irq_disable(); lock(&irq_desc_lock_class); lock(list_lock); <Interrupt> lock(&irq_desc_lock_class); *** DEADLOCK *** Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-06-20 08:39:51 -07:00
raw_spin_unlock_irqrestore(&list_lock, flags);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
return &vmd->irqs[best];
}
static int vmd_msi_init(struct irq_domain *domain, struct msi_domain_info *info,
unsigned int virq, irq_hw_number_t hwirq,
msi_alloc_info_t *arg)
{
struct msi_desc *desc = arg->desc;
struct vmd_dev *vmd = vmd_from_bus(msi_desc_to_pci_dev(desc)->bus);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
struct vmd_irq *vmdirq = kzalloc(sizeof(*vmdirq), GFP_KERNEL);
if (!vmdirq)
return -ENOMEM;
INIT_LIST_HEAD(&vmdirq->node);
vmdirq->irq = vmd_next_irq(vmd, desc);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
vmdirq->virq = virq;
irq_domain_set_info(domain, virq, vmdirq->irq->virq, info->chip, vmdirq,
handle_untracked_irq, vmd, NULL);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
return 0;
}
static void vmd_msi_free(struct irq_domain *domain,
struct msi_domain_info *info, unsigned int virq)
{
struct vmd_irq *vmdirq = irq_get_chip_data(virq);
x86/PCI: VMD: Use lock save/restore in interrupt enable path Enabling interrupts may result in an interrupt raised and serviced while VMD holds a lock, resulting in contention with the spin lock held while enabling interrupts. The solution is to disable preemption and save/restore the state during interrupt enable and disable. Fixes lockdep: ====================================================== [ INFO: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected ] 4.6.0-2016-06-16-lockdep+ #47 Tainted: G E ------------------------------------------------------ kworker/0:1/447 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: (list_lock){+.+...}, at: [<ffffffffa04eb8fc>] vmd_irq_enable+0x3c/0x70 [vmd] and this task is already holding: (&irq_desc_lock_class){-.-...}, at: [<ffffffff810e1ff6>] __setup_irq+0xa6/0x610 which would create a new lock dependency: (&irq_desc_lock_class){-.-...} -> (list_lock){+.+...} but this new dependency connects a HARDIRQ-irq-safe lock: (&irq_desc_lock_class){-.-...} ... which became HARDIRQ-irq-safe at: [<ffffffff810c9f21>] __lock_acquire+0x981/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffff810e36d4>] handle_level_irq+0x24/0x110 [<ffffffff8101f20a>] handle_irq+0x1a/0x30 [<ffffffff81675fc1>] do_IRQ+0x61/0x120 [<ffffffff8167404c>] ret_from_intr+0x0/0x20 [<ffffffff81672e30>] _raw_spin_unlock_irqrestore+0x40/0x60 [<ffffffff810e21ee>] __setup_irq+0x29e/0x610 [<ffffffff810e25a1>] setup_irq+0x41/0x90 [<ffffffff81f5777f>] setup_default_timer_irq+0x1e/0x20 [<ffffffff81f57798>] hpet_time_init+0x17/0x19 [<ffffffff81f5775a>] x86_late_time_init+0xa/0x11 [<ffffffff81f51e9b>] start_kernel+0x382/0x436 [<ffffffff81f51308>] x86_64_start_reservations+0x2a/0x2c [<ffffffff81f51445>] x86_64_start_kernel+0x13b/0x14a to a HARDIRQ-irq-unsafe lock: (list_lock){+.+...} ... which became HARDIRQ-irq-unsafe at: ... [<ffffffff810c9d8e>] __lock_acquire+0x7ee/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffffa04eba42>] vmd_msi_init+0x72/0x150 [vmd] [<ffffffff810e8597>] msi_domain_alloc+0xb7/0x140 [<ffffffff810e6b10>] irq_domain_alloc_irqs_recursive+0x40/0xa0 [<ffffffff810e6cea>] __irq_domain_alloc_irqs+0x14a/0x330 [<ffffffff810e8a8c>] msi_domain_alloc_irqs+0x8c/0x1d0 [<ffffffff813ca4e3>] pci_msi_setup_msi_irqs+0x43/0x70 [<ffffffff813cada1>] pci_enable_msi_range+0x131/0x280 [<ffffffff813bf5e0>] pcie_port_device_register+0x320/0x4e0 [<ffffffff813bf9a4>] pcie_portdrv_probe+0x34/0x60 [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff813b226b>] pci_device_probe+0xdb/0x130 [<ffffffff8149e3cc>] driver_probe_device+0x22c/0x440 [<ffffffff8149e774>] __device_attach_driver+0x94/0x110 [<ffffffff8149bfad>] bus_for_each_drv+0x5d/0x90 [<ffffffff8149e030>] __device_attach+0xc0/0x140 [<ffffffff8149e0c0>] device_attach+0x10/0x20 [<ffffffff813a77f7>] pci_bus_add_device+0x47/0x90 [<ffffffff813a7879>] pci_bus_add_devices+0x39/0x70 [<ffffffff813aaba7>] pci_rescan_bus+0x27/0x30 [<ffffffffa04ec1af>] vmd_probe+0x68f/0x76c [vmd] [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff81088064>] work_for_cpu_fn+0x14/0x20 [<ffffffff8108c244>] process_one_work+0x1f4/0x740 [<ffffffff8108c9c6>] worker_thread+0x236/0x4f0 [<ffffffff810935c2>] kthread+0xf2/0x110 [<ffffffff816738f2>] ret_from_fork+0x22/0x50 other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(list_lock); local_irq_disable(); lock(&irq_desc_lock_class); lock(list_lock); <Interrupt> lock(&irq_desc_lock_class); *** DEADLOCK *** Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-06-20 08:39:51 -07:00
unsigned long flags;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
synchronize_srcu(&vmdirq->irq->srcu);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
/* XXX: Potential optimization to rebalance */
x86/PCI: VMD: Use lock save/restore in interrupt enable path Enabling interrupts may result in an interrupt raised and serviced while VMD holds a lock, resulting in contention with the spin lock held while enabling interrupts. The solution is to disable preemption and save/restore the state during interrupt enable and disable. Fixes lockdep: ====================================================== [ INFO: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected ] 4.6.0-2016-06-16-lockdep+ #47 Tainted: G E ------------------------------------------------------ kworker/0:1/447 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: (list_lock){+.+...}, at: [<ffffffffa04eb8fc>] vmd_irq_enable+0x3c/0x70 [vmd] and this task is already holding: (&irq_desc_lock_class){-.-...}, at: [<ffffffff810e1ff6>] __setup_irq+0xa6/0x610 which would create a new lock dependency: (&irq_desc_lock_class){-.-...} -> (list_lock){+.+...} but this new dependency connects a HARDIRQ-irq-safe lock: (&irq_desc_lock_class){-.-...} ... which became HARDIRQ-irq-safe at: [<ffffffff810c9f21>] __lock_acquire+0x981/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffff810e36d4>] handle_level_irq+0x24/0x110 [<ffffffff8101f20a>] handle_irq+0x1a/0x30 [<ffffffff81675fc1>] do_IRQ+0x61/0x120 [<ffffffff8167404c>] ret_from_intr+0x0/0x20 [<ffffffff81672e30>] _raw_spin_unlock_irqrestore+0x40/0x60 [<ffffffff810e21ee>] __setup_irq+0x29e/0x610 [<ffffffff810e25a1>] setup_irq+0x41/0x90 [<ffffffff81f5777f>] setup_default_timer_irq+0x1e/0x20 [<ffffffff81f57798>] hpet_time_init+0x17/0x19 [<ffffffff81f5775a>] x86_late_time_init+0xa/0x11 [<ffffffff81f51e9b>] start_kernel+0x382/0x436 [<ffffffff81f51308>] x86_64_start_reservations+0x2a/0x2c [<ffffffff81f51445>] x86_64_start_kernel+0x13b/0x14a to a HARDIRQ-irq-unsafe lock: (list_lock){+.+...} ... which became HARDIRQ-irq-unsafe at: ... [<ffffffff810c9d8e>] __lock_acquire+0x7ee/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffffa04eba42>] vmd_msi_init+0x72/0x150 [vmd] [<ffffffff810e8597>] msi_domain_alloc+0xb7/0x140 [<ffffffff810e6b10>] irq_domain_alloc_irqs_recursive+0x40/0xa0 [<ffffffff810e6cea>] __irq_domain_alloc_irqs+0x14a/0x330 [<ffffffff810e8a8c>] msi_domain_alloc_irqs+0x8c/0x1d0 [<ffffffff813ca4e3>] pci_msi_setup_msi_irqs+0x43/0x70 [<ffffffff813cada1>] pci_enable_msi_range+0x131/0x280 [<ffffffff813bf5e0>] pcie_port_device_register+0x320/0x4e0 [<ffffffff813bf9a4>] pcie_portdrv_probe+0x34/0x60 [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff813b226b>] pci_device_probe+0xdb/0x130 [<ffffffff8149e3cc>] driver_probe_device+0x22c/0x440 [<ffffffff8149e774>] __device_attach_driver+0x94/0x110 [<ffffffff8149bfad>] bus_for_each_drv+0x5d/0x90 [<ffffffff8149e030>] __device_attach+0xc0/0x140 [<ffffffff8149e0c0>] device_attach+0x10/0x20 [<ffffffff813a77f7>] pci_bus_add_device+0x47/0x90 [<ffffffff813a7879>] pci_bus_add_devices+0x39/0x70 [<ffffffff813aaba7>] pci_rescan_bus+0x27/0x30 [<ffffffffa04ec1af>] vmd_probe+0x68f/0x76c [vmd] [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff81088064>] work_for_cpu_fn+0x14/0x20 [<ffffffff8108c244>] process_one_work+0x1f4/0x740 [<ffffffff8108c9c6>] worker_thread+0x236/0x4f0 [<ffffffff810935c2>] kthread+0xf2/0x110 [<ffffffff816738f2>] ret_from_fork+0x22/0x50 other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(list_lock); local_irq_disable(); lock(&irq_desc_lock_class); lock(list_lock); <Interrupt> lock(&irq_desc_lock_class); *** DEADLOCK *** Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-06-20 08:39:51 -07:00
raw_spin_lock_irqsave(&list_lock, flags);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
vmdirq->irq->count--;
x86/PCI: VMD: Use lock save/restore in interrupt enable path Enabling interrupts may result in an interrupt raised and serviced while VMD holds a lock, resulting in contention with the spin lock held while enabling interrupts. The solution is to disable preemption and save/restore the state during interrupt enable and disable. Fixes lockdep: ====================================================== [ INFO: HARDIRQ-safe -> HARDIRQ-unsafe lock order detected ] 4.6.0-2016-06-16-lockdep+ #47 Tainted: G E ------------------------------------------------------ kworker/0:1/447 [HC0[0]:SC0[0]:HE0:SE1] is trying to acquire: (list_lock){+.+...}, at: [<ffffffffa04eb8fc>] vmd_irq_enable+0x3c/0x70 [vmd] and this task is already holding: (&irq_desc_lock_class){-.-...}, at: [<ffffffff810e1ff6>] __setup_irq+0xa6/0x610 which would create a new lock dependency: (&irq_desc_lock_class){-.-...} -> (list_lock){+.+...} but this new dependency connects a HARDIRQ-irq-safe lock: (&irq_desc_lock_class){-.-...} ... which became HARDIRQ-irq-safe at: [<ffffffff810c9f21>] __lock_acquire+0x981/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffff810e36d4>] handle_level_irq+0x24/0x110 [<ffffffff8101f20a>] handle_irq+0x1a/0x30 [<ffffffff81675fc1>] do_IRQ+0x61/0x120 [<ffffffff8167404c>] ret_from_intr+0x0/0x20 [<ffffffff81672e30>] _raw_spin_unlock_irqrestore+0x40/0x60 [<ffffffff810e21ee>] __setup_irq+0x29e/0x610 [<ffffffff810e25a1>] setup_irq+0x41/0x90 [<ffffffff81f5777f>] setup_default_timer_irq+0x1e/0x20 [<ffffffff81f57798>] hpet_time_init+0x17/0x19 [<ffffffff81f5775a>] x86_late_time_init+0xa/0x11 [<ffffffff81f51e9b>] start_kernel+0x382/0x436 [<ffffffff81f51308>] x86_64_start_reservations+0x2a/0x2c [<ffffffff81f51445>] x86_64_start_kernel+0x13b/0x14a to a HARDIRQ-irq-unsafe lock: (list_lock){+.+...} ... which became HARDIRQ-irq-unsafe at: ... [<ffffffff810c9d8e>] __lock_acquire+0x7ee/0xe00 [<ffffffff810cb039>] lock_acquire+0x119/0x220 [<ffffffff8167294d>] _raw_spin_lock+0x3d/0x80 [<ffffffffa04eba42>] vmd_msi_init+0x72/0x150 [vmd] [<ffffffff810e8597>] msi_domain_alloc+0xb7/0x140 [<ffffffff810e6b10>] irq_domain_alloc_irqs_recursive+0x40/0xa0 [<ffffffff810e6cea>] __irq_domain_alloc_irqs+0x14a/0x330 [<ffffffff810e8a8c>] msi_domain_alloc_irqs+0x8c/0x1d0 [<ffffffff813ca4e3>] pci_msi_setup_msi_irqs+0x43/0x70 [<ffffffff813cada1>] pci_enable_msi_range+0x131/0x280 [<ffffffff813bf5e0>] pcie_port_device_register+0x320/0x4e0 [<ffffffff813bf9a4>] pcie_portdrv_probe+0x34/0x60 [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff813b226b>] pci_device_probe+0xdb/0x130 [<ffffffff8149e3cc>] driver_probe_device+0x22c/0x440 [<ffffffff8149e774>] __device_attach_driver+0x94/0x110 [<ffffffff8149bfad>] bus_for_each_drv+0x5d/0x90 [<ffffffff8149e030>] __device_attach+0xc0/0x140 [<ffffffff8149e0c0>] device_attach+0x10/0x20 [<ffffffff813a77f7>] pci_bus_add_device+0x47/0x90 [<ffffffff813a7879>] pci_bus_add_devices+0x39/0x70 [<ffffffff813aaba7>] pci_rescan_bus+0x27/0x30 [<ffffffffa04ec1af>] vmd_probe+0x68f/0x76c [vmd] [<ffffffff813b0e85>] local_pci_probe+0x45/0xa0 [<ffffffff81088064>] work_for_cpu_fn+0x14/0x20 [<ffffffff8108c244>] process_one_work+0x1f4/0x740 [<ffffffff8108c9c6>] worker_thread+0x236/0x4f0 [<ffffffff810935c2>] kthread+0xf2/0x110 [<ffffffff816738f2>] ret_from_fork+0x22/0x50 other info that might help us debug this: Possible interrupt unsafe locking scenario: CPU0 CPU1 ---- ---- lock(list_lock); local_irq_disable(); lock(&irq_desc_lock_class); lock(list_lock); <Interrupt> lock(&irq_desc_lock_class); *** DEADLOCK *** Signed-off-by: Jon Derrick <jonathan.derrick@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Keith Busch <keith.busch@intel.com>
2016-06-20 08:39:51 -07:00
raw_spin_unlock_irqrestore(&list_lock, flags);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
kfree(vmdirq);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
}
static int vmd_msi_prepare(struct irq_domain *domain, struct device *dev,
int nvec, msi_alloc_info_t *arg)
{
struct pci_dev *pdev = to_pci_dev(dev);
struct vmd_dev *vmd = vmd_from_bus(pdev->bus);
if (nvec > vmd->msix_count)
return vmd->msix_count;
memset(arg, 0, sizeof(*arg));
return 0;
}
static void vmd_set_desc(msi_alloc_info_t *arg, struct msi_desc *desc)
{
arg->desc = desc;
}
static struct msi_domain_ops vmd_msi_domain_ops = {
.get_hwirq = vmd_get_hwirq,
.msi_init = vmd_msi_init,
.msi_free = vmd_msi_free,
.msi_prepare = vmd_msi_prepare,
.set_desc = vmd_set_desc,
};
static struct msi_domain_info vmd_msi_domain_info = {
.flags = MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS |
MSI_FLAG_NO_AFFINITY | MSI_FLAG_PCI_MSIX,
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
.ops = &vmd_msi_domain_ops,
.chip = &vmd_msi_controller,
};
static void vmd_set_msi_remapping(struct vmd_dev *vmd, bool enable)
{
u16 reg;
pci_read_config_word(vmd->dev, PCI_REG_VMCONFIG, &reg);
reg = enable ? (reg & ~VMCONFIG_MSI_REMAP) :
(reg | VMCONFIG_MSI_REMAP);
pci_write_config_word(vmd->dev, PCI_REG_VMCONFIG, reg);
}
static int vmd_create_irq_domain(struct vmd_dev *vmd)
{
struct fwnode_handle *fn;
fn = irq_domain_alloc_named_id_fwnode("VMD-MSI", vmd->sysdata.domain);
if (!fn)
return -ENODEV;
pci-v5.10-changes -----BEGIN PGP SIGNATURE----- iQJIBAABCgAyFiEEgMe7l+5h9hnxdsnuWYigwDrT+vwFAl+QUFkUHGJoZWxnYWFz QGdvb2dsZS5jb20ACgkQWYigwDrT+vw6SQ/9FHiAlHIa48/l5ZweqAuN3XnU8hoO sqMoJE8eqTkIYIT0aQdW6b1sDB0YE6b4UVxzg+UL/E0qYeJqgIUakig7QkyyF1qU aT5hq2ic+lk88G7AAxK3kgQGPk+JvP1EFIyOu6HBWzzDDzgLme1Iuh/5ulc2/lo+ E4biy0WOnI8vMfCieXGK4bSpc17Rn0+3N4cuVwZXBlntsvicE90VqeWBzqti1sk5 R6gkZuW+EIUNHHL7TLlkCeYZq6QNbXWzhfKCiaGW2wW4eJ4Ek1/ncQjyTbCFytKU 7OIYvrH20XO3L5GEfJ5fdbWErI1dRpoHO4NmhWljyBcVh44VYnM2ixhA7TuJ+TOk OtMbtoJAlP+QDlVdAW6rmRYmMPLFK/AQl5Aq7ftY22b2rYXqP20BobPy2MpDT71T sGC8z0ABl/ijo23g3I+3/2VzP/RzGhZJ0ZqagrXj8jHtg8SVy2fLcR5nr/dlrgFk TG83zML6ui1KViyx5nzElaEtw18aTqP61CNQxijQtNoYwKBTtRKNTrdRr4Qo7Hi6 6S+No3+4z8Kf8d90y0LkJQqr7JRkG6nI3AhXHO3rxXpXJOD2+QzlpwBZTQnASqq7 3kC1doUPmN97rFUYPQWWyOs6xSMcGbGIz8Uus3shH6yDtNxgpnIVoctH55hTEh6w nSY/4ssIfzJxZCE= =RCFo -----END PGP SIGNATURE----- Merge tag 'pci-v5.10-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci Pull PCI updates from Bjorn Helgaas: "Enumeration: - Print IRQ number used by PCIe Link Bandwidth Notification (Dongdong Liu) - Add schedule point in pci_read_config() to reduce max latency (Jiang Biao) - Add Kconfig options for MPS/MRRS strategy (Jim Quinlan) Resource management: - Fix pci_iounmap() memory leak when !CONFIG_GENERIC_IOMAP (Lorenzo Pieralisi) PCIe native device hotplug: - Reduce noisiness on hot removal (Lukas Wunner) Power management: - Revert "PCI/PM: Apply D2 delay as milliseconds, not microseconds" that was done on the basis of spec typo (Bjorn Helgaas) - Rename pci_dev.d3_delay to d3hot_delay to remove D3hot/D3cold ambiguity (Krzysztof Wilczyński) - Remove unused pcibios_pm_ops (Vaibhav Gupta) IOMMU: - Enable Translation Blocking for external devices to harden against DMA attacks (Rajat Jain) Error handling: - Add an ACPI APEI notifier chain for vendor CPER records to enable device-specific error handling (Shiju Jose) ASPM: - Remove struct aspm_register_info to simplify code (Saheed O. Bolarinwa) Amlogic Meson PCIe controller driver: - Build as module by default (Kevin Hilman) Ampere Altra PCIe controller driver: - Add MCFG quirk to work around non-standard ECAM implementation (Tuan Phan) Broadcom iProc PCIe controller driver: - Set affinity mask on MSI interrupts (Mark Tomlinson) Broadcom STB PCIe controller driver: - Make PCIE_BRCMSTB depend on ARCH_BRCMSTB (Jim Quinlan) - Add DT bindings for more Brcmstb chips (Jim Quinlan) - Add bcm7278 register info (Jim Quinlan) - Add bcm7278 PERST# support (Jim Quinlan) - Add suspend and resume pm_ops (Jim Quinlan) - Add control of rescal reset (Jim Quinlan) - Set additional internal memory DMA viewport sizes (Jim Quinlan) - Accommodate MSI for older chips (Jim Quinlan) - Set bus max burst size by chip type (Jim Quinlan) - Add support for bcm7211, bcm7216, bcm7445, bcm7278 (Jim Quinlan) Freescale i.MX6 PCIe controller driver: - Use dev_err_probe() to reduce redundant messages (Anson Huang) Freescale Layerscape PCIe controller driver: - Enforce 4K DMA buffer alignment in endpoint test (Hou Zhiqiang) - Add DT compatible strings for ls1088a, ls2088a (Xiaowei Bao) - Add endpoint support for ls1088a, ls2088a (Xiaowei Bao) - Add endpoint test support for lS1088a (Xiaowei Bao) - Add MSI-X support for ls1088a (Xiaowei Bao) HiSilicon HIP PCIe controller driver: - Handle HIP-specific errors via ACPI APEI (Yicong Yang) HiSilicon Kirin PCIe controller driver: - Return -EPROBE_DEFER if the GPIO isn't ready (Bean Huo) Intel VMD host bridge driver: - Factor out physical offset, bus offset, IRQ domain, IRQ allocation (Jon Derrick) - Use generic PCI PM correctly (Jon Derrick) Marvell Aardvark PCIe controller driver: - Fix compilation on s390 (Pali Rohár) - Implement driver 'remove' function and allow to build it as module (Pali Rohár) - Move PCIe reset card code to advk_pcie_train_link() (Pali Rohár) - Convert mvebu a3700 internal SMCC firmware return codes to errno (Pali Rohár) - Fix initialization with old Marvell's Arm Trusted Firmware (Pali Rohár) Microsoft Hyper-V host bridge driver: - Fix hibernation in case interrupts are not re-created (Dexuan Cui) NVIDIA Tegra PCIe controller driver: - Stop checking return value of debugfs_create() functions (Greg Kroah-Hartman) - Convert to use DEFINE_SEQ_ATTRIBUTE macro (Liu Shixin) Qualcomm PCIe controller driver: - Reset PCIe to work around Qsdk U-Boot issue (Ansuel Smith) Renesas R-Car PCIe controller driver: - Add DT documentation for r8a774a1, r8a774b1, r8a774e1 endpoints (Lad Prabhakar) - Add RZ/G2M, RZ/G2N, RZ/G2H IDs to endpoint test (Lad Prabhakar) - Add DT support for r8a7742 (Lad Prabhakar) Socionext UniPhier Pro5 controller driver: - Add DT descriptions of iATU register (host and endpoint) (Kunihiko Hayashi) Synopsys DesignWare PCIe controller driver: - Add link up check in dw_child_pcie_ops.map_bus() (racy, but seems unavoidable) (Hou Zhiqiang) - Fix endpoint Header Type check so multi-function devices work (Hou Zhiqiang) - Skip PCIE_MSI_INTR0* programming if MSI is disabled (Jisheng Zhang) - Stop leaking MSI page in suspend/resume (Jisheng Zhang) - Add common iATU register support instead of keystone-specific code (Kunihiko Hayashi) - Major config space access and other cleanups in dwc core and drivers that use it (al, exynos, histb, imx6, intel-gw, keystone, kirin, meson, qcom, tegra) (Rob Herring) - Add multiple PFs support for endpoint (Xiaowei Bao) - Add MSI-X doorbell mode in endpoint mode (Xiaowei Bao) Miscellaneous: - Use fallthrough pseudo-keyword (Gustavo A. R. Silva) - Fix "0 used as NULL pointer" warnings (Gustavo Pimentel) - Fix "cast truncates bits from constant value" warnings (Gustavo Pimentel) - Remove redundant zeroing for sg_init_table() (Julia Lawall) - Use scnprintf(), not snprintf(), in sysfs "show" functions (Krzysztof Wilczyński) - Remove unused assignments (Krzysztof Wilczyński) - Fix "0 used as NULL pointer" warning (Krzysztof Wilczyński) - Simplify bool comparisons (Krzysztof Wilczyński) - Use for_each_child_of_node() and for_each_node_by_name() (Qinglang Miao) - Simplify return expressions (Qinglang Miao)" * tag 'pci-v5.10-changes' of git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci: (147 commits) PCI: vmd: Update VMD PM to correctly use generic PCI PM PCI: vmd: Create IRQ allocation helper PCI: vmd: Create IRQ Domain configuration helper PCI: vmd: Create bus offset configuration helper PCI: vmd: Create physical offset helper PCI: v3-semi: Remove unneeded break PCI: dwc: Add link up check in dw_child_pcie_ops.map_bus() PCI/ASPM: Remove struct pcie_link_state.l1ss PCI/ASPM: Remove struct aspm_register_info.l1ss_cap PCI/ASPM: Pass L1SS Capabilities value, not struct aspm_register_info PCI/ASPM: Remove struct aspm_register_info.l1ss_ctl1 PCI/ASPM: Remove struct aspm_register_info.l1ss_ctl2 (unused) PCI/ASPM: Remove struct aspm_register_info.l1ss_cap_ptr PCI/ASPM: Remove struct aspm_register_info.latency_encoding PCI/ASPM: Remove struct aspm_register_info.enabled PCI/ASPM: Remove struct aspm_register_info.support PCI/ASPM: Use 'parent' and 'child' for readability PCI/ASPM: Move LTR path check to where it's used PCI/ASPM: Move pci_clear_and_set_dword() earlier PCI: dwc: Fix MSI page leakage in suspend/resume ...
2020-10-22 12:41:00 -07:00
vmd->irq_domain = pci_msi_create_irq_domain(fn, &vmd_msi_domain_info, NULL);
if (!vmd->irq_domain) {
irq_domain_free_fwnode(fn);
return -ENODEV;
}
return 0;
}
static void vmd_remove_irq_domain(struct vmd_dev *vmd)
{
/*
* Some production BIOS won't enable remapping between soft reboots.
* Ensure remapping is restored before unloading the driver.
*/
if (!vmd->msix_count)
vmd_set_msi_remapping(vmd, true);
if (vmd->irq_domain) {
struct fwnode_handle *fn = vmd->irq_domain->fwnode;
irq_domain_remove(vmd->irq_domain);
irq_domain_free_fwnode(fn);
}
}
static void __iomem *vmd_cfg_addr(struct vmd_dev *vmd, struct pci_bus *bus,
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
unsigned int devfn, int reg, int len)
{
PCI: Unify ECAM constants in native PCI Express drivers Add ECAM-related constants to provide a set of standard constants defining memory address shift values to the byte-level address that can be used to access the PCI Express Configuration Space, and then move native PCI Express controller drivers to use the newly introduced definitions retiring driver-specific ones. Refactor pci_ecam_map_bus() function to use newly added constants so that limits to the bus, device function and offset (now limited to 4K as per the specification) are in place to prevent the defective or malicious caller from supplying incorrect configuration offset and thus targeting the wrong device when accessing extended configuration space. This refactor also allows for the ".bus_shift" initialisers to be dropped when the user is not using a custom value as a default value will be used as per the PCI Express Specification. Thanks to Qian Cai <qcai@redhat.com>, Michael Walle <michael@walle.cc>, and Vladimir Oltean <olteanv@gmail.com> for reporting a pci_ecam_create() issue with .bus_shift and to Vladimir for proposing the fix. [bhelgaas: incorporate Vladimir's fix, update commit log] Suggested-by: Bjorn Helgaas <bhelgaas@google.com> Link: https://lore.kernel.org/r/20201129230743.3006978-2-kw@linux.com Tested-by: Michael Walle <michael@walle.cc> Signed-off-by: Krzysztof Wilczyński <kw@linux.com> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Jon Derrick <jonathan.derrick@intel.com> Reviewed-by: Bjorn Helgaas <bhelgaas@google.com>
2020-11-29 16:07:39 -07:00
unsigned int busnr_ecam = bus->number - vmd->busn_start;
u32 offset = PCIE_ECAM_OFFSET(busnr_ecam, devfn, reg);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
PCI: Unify ECAM constants in native PCI Express drivers Add ECAM-related constants to provide a set of standard constants defining memory address shift values to the byte-level address that can be used to access the PCI Express Configuration Space, and then move native PCI Express controller drivers to use the newly introduced definitions retiring driver-specific ones. Refactor pci_ecam_map_bus() function to use newly added constants so that limits to the bus, device function and offset (now limited to 4K as per the specification) are in place to prevent the defective or malicious caller from supplying incorrect configuration offset and thus targeting the wrong device when accessing extended configuration space. This refactor also allows for the ".bus_shift" initialisers to be dropped when the user is not using a custom value as a default value will be used as per the PCI Express Specification. Thanks to Qian Cai <qcai@redhat.com>, Michael Walle <michael@walle.cc>, and Vladimir Oltean <olteanv@gmail.com> for reporting a pci_ecam_create() issue with .bus_shift and to Vladimir for proposing the fix. [bhelgaas: incorporate Vladimir's fix, update commit log] Suggested-by: Bjorn Helgaas <bhelgaas@google.com> Link: https://lore.kernel.org/r/20201129230743.3006978-2-kw@linux.com Tested-by: Michael Walle <michael@walle.cc> Signed-off-by: Krzysztof Wilczyński <kw@linux.com> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Jon Derrick <jonathan.derrick@intel.com> Reviewed-by: Bjorn Helgaas <bhelgaas@google.com>
2020-11-29 16:07:39 -07:00
if (offset + len >= resource_size(&vmd->dev->resource[VMD_CFGBAR]))
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
return NULL;
PCI: Unify ECAM constants in native PCI Express drivers Add ECAM-related constants to provide a set of standard constants defining memory address shift values to the byte-level address that can be used to access the PCI Express Configuration Space, and then move native PCI Express controller drivers to use the newly introduced definitions retiring driver-specific ones. Refactor pci_ecam_map_bus() function to use newly added constants so that limits to the bus, device function and offset (now limited to 4K as per the specification) are in place to prevent the defective or malicious caller from supplying incorrect configuration offset and thus targeting the wrong device when accessing extended configuration space. This refactor also allows for the ".bus_shift" initialisers to be dropped when the user is not using a custom value as a default value will be used as per the PCI Express Specification. Thanks to Qian Cai <qcai@redhat.com>, Michael Walle <michael@walle.cc>, and Vladimir Oltean <olteanv@gmail.com> for reporting a pci_ecam_create() issue with .bus_shift and to Vladimir for proposing the fix. [bhelgaas: incorporate Vladimir's fix, update commit log] Suggested-by: Bjorn Helgaas <bhelgaas@google.com> Link: https://lore.kernel.org/r/20201129230743.3006978-2-kw@linux.com Tested-by: Michael Walle <michael@walle.cc> Signed-off-by: Krzysztof Wilczyński <kw@linux.com> Signed-off-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Jon Derrick <jonathan.derrick@intel.com> Reviewed-by: Bjorn Helgaas <bhelgaas@google.com>
2020-11-29 16:07:39 -07:00
return vmd->cfgbar + offset;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
}
/*
* CPU may deadlock if config space is not serialized on some versions of this
* hardware, so all config space access is done under a spinlock.
*/
static int vmd_pci_read(struct pci_bus *bus, unsigned int devfn, int reg,
int len, u32 *value)
{
struct vmd_dev *vmd = vmd_from_bus(bus);
void __iomem *addr = vmd_cfg_addr(vmd, bus, devfn, reg, len);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
unsigned long flags;
int ret = 0;
if (!addr)
return -EFAULT;
spin_lock_irqsave(&vmd->cfg_lock, flags);
switch (len) {
case 1:
*value = readb(addr);
break;
case 2:
*value = readw(addr);
break;
case 4:
*value = readl(addr);
break;
default:
ret = -EINVAL;
break;
}
spin_unlock_irqrestore(&vmd->cfg_lock, flags);
return ret;
}
/*
* VMD h/w converts non-posted config writes to posted memory writes. The
* read-back in this function forces the completion so it returns only after
* the config space was written, as expected.
*/
static int vmd_pci_write(struct pci_bus *bus, unsigned int devfn, int reg,
int len, u32 value)
{
struct vmd_dev *vmd = vmd_from_bus(bus);
void __iomem *addr = vmd_cfg_addr(vmd, bus, devfn, reg, len);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
unsigned long flags;
int ret = 0;
if (!addr)
return -EFAULT;
spin_lock_irqsave(&vmd->cfg_lock, flags);
switch (len) {
case 1:
writeb(value, addr);
readb(addr);
break;
case 2:
writew(value, addr);
readw(addr);
break;
case 4:
writel(value, addr);
readl(addr);
break;
default:
ret = -EINVAL;
break;
}
spin_unlock_irqrestore(&vmd->cfg_lock, flags);
return ret;
}
static struct pci_ops vmd_ops = {
.read = vmd_pci_read,
.write = vmd_pci_write,
};
#ifdef CONFIG_ACPI
static struct acpi_device *vmd_acpi_find_companion(struct pci_dev *pci_dev)
{
struct pci_host_bridge *bridge;
u32 busnr, addr;
if (pci_dev->bus->ops != &vmd_ops)
return NULL;
bridge = pci_find_host_bridge(pci_dev->bus);
busnr = pci_dev->bus->number - bridge->bus->number;
/*
* The address computation below is only applicable to relative bus
* numbers below 32.
*/
if (busnr > 31)
return NULL;
addr = (busnr << 24) | ((u32)pci_dev->devfn << 16) | 0x8000FFFFU;
dev_dbg(&pci_dev->dev, "Looking for ACPI companion (address 0x%x)\n",
addr);
return acpi_find_child_device(ACPI_COMPANION(bridge->dev.parent), addr,
false);
}
static bool hook_installed;
static void vmd_acpi_begin(void)
{
if (pci_acpi_set_companion_lookup_hook(vmd_acpi_find_companion))
return;
hook_installed = true;
}
static void vmd_acpi_end(void)
{
if (!hook_installed)
return;
pci_acpi_clear_companion_lookup_hook();
hook_installed = false;
}
#else
static inline void vmd_acpi_begin(void) { }
static inline void vmd_acpi_end(void) { }
#endif /* CONFIG_ACPI */
static void vmd_domain_reset(struct vmd_dev *vmd)
{
u16 bus, max_buses = resource_size(&vmd->resources[0]);
u8 dev, functions, fn, hdr_type;
char __iomem *base;
for (bus = 0; bus < max_buses; bus++) {
for (dev = 0; dev < 32; dev++) {
base = vmd->cfgbar + PCIE_ECAM_OFFSET(bus,
PCI_DEVFN(dev, 0), 0);
hdr_type = readb(base + PCI_HEADER_TYPE);
functions = (hdr_type & PCI_HEADER_TYPE_MFD) ? 8 : 1;
for (fn = 0; fn < functions; fn++) {
base = vmd->cfgbar + PCIE_ECAM_OFFSET(bus,
PCI_DEVFN(dev, fn), 0);
hdr_type = readb(base + PCI_HEADER_TYPE) &
PCI_HEADER_TYPE_MASK;
if (hdr_type != PCI_HEADER_TYPE_BRIDGE ||
(readw(base + PCI_CLASS_DEVICE) !=
PCI_CLASS_BRIDGE_PCI))
continue;
PCI: vmd: Disable bridge window for domain reset During domain reset process vmd_domain_reset() clears PCI configuration space of VMD root ports. But certain platform has observed following errors and failed to boot. ... DMAR: VT-d detected Invalidation Queue Error: Reason f DMAR: VT-d detected Invalidation Time-out Error: SID ffff DMAR: VT-d detected Invalidation Completion Error: SID ffff DMAR: QI HEAD: UNKNOWN qw0 = 0x0, qw1 = 0x0 DMAR: QI PRIOR: UNKNOWN qw0 = 0x0, qw1 = 0x0 DMAR: Invalidation Time-out Error (ITE) cleared The root cause is that memset_io() clears prefetchable memory base/limit registers and prefetchable base/limit 32 bits registers sequentially. This seems to be enabling prefetchable memory if the device disabled prefetchable memory originally. Here is an example (before memset_io()): PCI configuration space for 10000:00:00.0: 86 80 30 20 06 00 10 00 04 00 04 06 00 00 01 00 00 00 00 00 00 00 00 00 00 01 01 00 00 00 00 20 00 00 00 00 01 00 01 00 ff ff ff ff 75 05 00 00 ... So, prefetchable memory is ffffffff00000000-575000fffff, which is disabled. When memset_io() clears prefetchable base 32 bits register, the prefetchable memory becomes 0000000000000000-575000fffff, which is enabled and incorrect. Here is the quote from section 7.5.1.3.9 of PCI Express Base 6.0 spec: The Prefetchable Memory Limit register must be programmed to a smaller value than the Prefetchable Memory Base register if there is no prefetchable memory on the secondary side of the bridge. This is believed to be the reason for the failure and in addition the sequence of operation in vmd_domain_reset() is not following the PCIe specs. Disable the bridge window by executing a sequence of operations borrowed from pci_disable_bridge_window() and pci_setup_bridge_io(), that comply with the PCI specifications. Link: https://lore.kernel.org/r/20230810215029.1177379-1-nirmal.patel@linux.intel.com Signed-off-by: Nirmal Patel <nirmal.patel@linux.intel.com> Signed-off-by: Lorenzo Pieralisi <lpieralisi@kernel.org>
2023-08-10 14:50:29 -07:00
/*
* Temporarily disable the I/O range before updating
* PCI_IO_BASE.
*/
writel(0x0000ffff, base + PCI_IO_BASE_UPPER16);
/* Update lower 16 bits of I/O base/limit */
writew(0x00f0, base + PCI_IO_BASE);
/* Update upper 16 bits of I/O base/limit */
writel(0, base + PCI_IO_BASE_UPPER16);
/* MMIO Base/Limit */
writel(0x0000fff0, base + PCI_MEMORY_BASE);
/* Prefetchable MMIO Base/Limit */
writel(0, base + PCI_PREF_LIMIT_UPPER32);
writel(0x0000fff0, base + PCI_PREF_MEMORY_BASE);
writel(0xffffffff, base + PCI_PREF_BASE_UPPER32);
}
}
}
}
static void vmd_attach_resources(struct vmd_dev *vmd)
{
vmd->dev->resource[VMD_MEMBAR1].child = &vmd->resources[1];
vmd->dev->resource[VMD_MEMBAR2].child = &vmd->resources[2];
}
static void vmd_detach_resources(struct vmd_dev *vmd)
{
vmd->dev->resource[VMD_MEMBAR1].child = NULL;
vmd->dev->resource[VMD_MEMBAR2].child = NULL;
}
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
/*
* VMD domains start at 0x10000 to not clash with ACPI _SEG domains.
* Per ACPI r6.0, sec 6.5.6, _SEG returns an integer, of which the lower
* 16 bits are the PCI Segment Group (domain) number. Other bits are
* currently reserved.
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
*/
static int vmd_find_free_domain(void)
{
int domain = 0xffff;
struct pci_bus *bus = NULL;
while ((bus = pci_find_next_bus(bus)) != NULL)
domain = max_t(int, domain, pci_domain_nr(bus));
return domain + 1;
}
static int vmd_get_phys_offsets(struct vmd_dev *vmd, bool native_hint,
resource_size_t *offset1,
resource_size_t *offset2)
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
{
struct pci_dev *dev = vmd->dev;
u64 phys1, phys2;
if (native_hint) {
u32 vmlock;
int ret;
ret = pci_read_config_dword(dev, PCI_REG_VMLOCK, &vmlock);
if (ret || PCI_POSSIBLE_ERROR(vmlock))
return -ENODEV;
if (MB2_SHADOW_EN(vmlock)) {
void __iomem *membar2;
membar2 = pci_iomap(dev, VMD_MEMBAR2, 0);
if (!membar2)
return -ENOMEM;
phys1 = readq(membar2 + MB2_SHADOW_OFFSET);
phys2 = readq(membar2 + MB2_SHADOW_OFFSET + 8);
pci_iounmap(dev, membar2);
} else
return 0;
} else {
/* Hypervisor-Emulated Vendor-Specific Capability */
int pos = pci_find_capability(dev, PCI_CAP_ID_VNDR);
u32 reg, regu;
pci_read_config_dword(dev, pos + 4, &reg);
/* "SHDW" */
if (pos && reg == 0x53484457) {
pci_read_config_dword(dev, pos + 8, &reg);
pci_read_config_dword(dev, pos + 12, &regu);
phys1 = (u64) regu << 32 | reg;
pci_read_config_dword(dev, pos + 16, &reg);
pci_read_config_dword(dev, pos + 20, &regu);
phys2 = (u64) regu << 32 | reg;
} else
return 0;
}
*offset1 = dev->resource[VMD_MEMBAR1].start -
(phys1 & PCI_BASE_ADDRESS_MEM_MASK);
*offset2 = dev->resource[VMD_MEMBAR2].start -
(phys2 & PCI_BASE_ADDRESS_MEM_MASK);
return 0;
}
static int vmd_get_bus_number_start(struct vmd_dev *vmd)
{
struct pci_dev *dev = vmd->dev;
u16 reg;
pci_read_config_word(dev, PCI_REG_VMCAP, &reg);
if (BUS_RESTRICT_CAP(reg)) {
pci_read_config_word(dev, PCI_REG_VMCONFIG, &reg);
switch (BUS_RESTRICT_CFG(reg)) {
case 0:
vmd->busn_start = 0;
break;
case 1:
vmd->busn_start = 128;
break;
case 2:
vmd->busn_start = 224;
break;
default:
pci_err(dev, "Unknown Bus Offset Setting (%d)\n",
BUS_RESTRICT_CFG(reg));
return -ENODEV;
}
}
return 0;
}
static irqreturn_t vmd_irq(int irq, void *data)
{
struct vmd_irq_list *irqs = data;
struct vmd_irq *vmdirq;
int idx;
idx = srcu_read_lock(&irqs->srcu);
list_for_each_entry_rcu(vmdirq, &irqs->irq_list, node)
generic_handle_irq(vmdirq->virq);
srcu_read_unlock(&irqs->srcu, idx);
return IRQ_HANDLED;
}
static int vmd_alloc_irqs(struct vmd_dev *vmd)
{
struct pci_dev *dev = vmd->dev;
int i, err;
vmd->msix_count = pci_msix_vec_count(dev);
if (vmd->msix_count < 0)
return -ENODEV;
vmd->msix_count = pci_alloc_irq_vectors(dev, vmd->first_vec + 1,
vmd->msix_count, PCI_IRQ_MSIX);
if (vmd->msix_count < 0)
return vmd->msix_count;
vmd->irqs = devm_kcalloc(&dev->dev, vmd->msix_count, sizeof(*vmd->irqs),
GFP_KERNEL);
if (!vmd->irqs)
return -ENOMEM;
for (i = 0; i < vmd->msix_count; i++) {
err = init_srcu_struct(&vmd->irqs[i].srcu);
if (err)
return err;
INIT_LIST_HEAD(&vmd->irqs[i].irq_list);
vmd->irqs[i].virq = pci_irq_vector(dev, i);
err = devm_request_irq(&dev->dev, vmd->irqs[i].virq,
vmd_irq, IRQF_NO_THREAD,
vmd->name, &vmd->irqs[i]);
if (err)
return err;
}
return 0;
}
/*
* Since VMD is an aperture to regular PCIe root ports, only allow it to
* control features that the OS is allowed to control on the physical PCI bus.
*/
static void vmd_copy_host_bridge_flags(struct pci_host_bridge *root_bridge,
struct pci_host_bridge *vmd_bridge)
{
vmd_bridge->native_pcie_hotplug = root_bridge->native_pcie_hotplug;
vmd_bridge->native_shpc_hotplug = root_bridge->native_shpc_hotplug;
vmd_bridge->native_aer = root_bridge->native_aer;
vmd_bridge->native_pme = root_bridge->native_pme;
vmd_bridge->native_ltr = root_bridge->native_ltr;
vmd_bridge->native_dpc = root_bridge->native_dpc;
}
2023-01-19 20:15:22 -07:00
/*
* Enable ASPM and LTR settings on devices that aren't configured by BIOS.
*/
static int vmd_pm_enable_quirk(struct pci_dev *pdev, void *userdata)
{
unsigned long features = *(unsigned long *)userdata;
u16 ltr = VMD_BIOS_PM_QUIRK_LTR;
u32 ltr_reg;
int pos;
if (!(features & VMD_FEAT_BIOS_PM_QUIRK))
return 0;
pci_enable_link_state_locked(pdev, PCIE_LINK_STATE_ALL);
2023-01-19 20:15:22 -07:00
pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_LTR);
if (!pos)
return 0;
/*
* Skip if the max snoop LTR is non-zero, indicating BIOS has set it
* so the LTR quirk is not needed.
*/
pci_read_config_dword(pdev, pos + PCI_LTR_MAX_SNOOP_LAT, &ltr_reg);
if (!!(ltr_reg & (PCI_LTR_VALUE_MASK | PCI_LTR_SCALE_MASK)))
return 0;
/*
* Set the default values to the maximum required by the platform to
* allow the deepest power management savings. Write as a DWORD where
* the lower word is the max snoop latency and the upper word is the
* max non-snoop latency.
*/
ltr_reg = (ltr << 16) | ltr;
pci_write_config_dword(pdev, pos + PCI_LTR_MAX_SNOOP_LAT, ltr_reg);
pci_info(pdev, "VMD: Default LTR value set by driver\n");
return 0;
}
static int vmd_enable_domain(struct vmd_dev *vmd, unsigned long features)
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
{
struct pci_sysdata *sd = &vmd->sysdata;
struct resource *res;
u32 upper_bits;
unsigned long flags;
LIST_HEAD(resources);
resource_size_t offset[2] = {0};
resource_size_t membar2_offset = 0x2000;
struct pci_bus *child;
struct pci_dev *dev;
int ret;
/*
* Shadow registers may exist in certain VMD device ids which allow
* guests to correctly assign host physical addresses to the root ports
* and child devices. These registers will either return the host value
* or 0, depending on an enable bit in the VMD device.
*/
if (features & VMD_FEAT_HAS_MEMBAR_SHADOW) {
membar2_offset = MB2_SHADOW_OFFSET + MB2_SHADOW_SIZE;
ret = vmd_get_phys_offsets(vmd, true, &offset[0], &offset[1]);
if (ret)
return ret;
} else if (features & VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP) {
ret = vmd_get_phys_offsets(vmd, false, &offset[0], &offset[1]);
if (ret)
return ret;
}
/*
* Certain VMD devices may have a root port configuration option which
* limits the bus range to between 0-127, 128-255, or 224-255
*/
if (features & VMD_FEAT_HAS_BUS_RESTRICTIONS) {
ret = vmd_get_bus_number_start(vmd);
if (ret)
return ret;
}
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
res = &vmd->dev->resource[VMD_CFGBAR];
vmd->resources[0] = (struct resource) {
.name = "VMD CFGBAR",
.start = vmd->busn_start,
.end = vmd->busn_start + (resource_size(res) >> 20) - 1,
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
.flags = IORESOURCE_BUS | IORESOURCE_PCI_FIXED,
};
/*
* If the window is below 4GB, clear IORESOURCE_MEM_64 so we can
* put 32-bit resources in the window.
*
* There's no hardware reason why a 64-bit window *couldn't*
* contain a 32-bit resource, but pbus_size_mem() computes the
* bridge window size assuming a 64-bit window will contain no
* 32-bit resources. __pci_assign_resource() enforces that
* artificial restriction to make sure everything will fit.
*
* The only way we could use a 64-bit non-prefetchable MEMBAR is
* if its address is <4GB so that we can convert it to a 32-bit
* resource. To be visible to the host OS, all VMD endpoints must
* be initially configured by platform BIOS, which includes setting
* up these resources. We can assume the device is configured
* according to the platform needs.
*/
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
res = &vmd->dev->resource[VMD_MEMBAR1];
upper_bits = upper_32_bits(res->end);
flags = res->flags & ~IORESOURCE_SIZEALIGN;
if (!upper_bits)
flags &= ~IORESOURCE_MEM_64;
vmd->resources[1] = (struct resource) {
.name = "VMD MEMBAR1",
.start = res->start,
.end = res->end,
.flags = flags,
.parent = res,
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
};
res = &vmd->dev->resource[VMD_MEMBAR2];
upper_bits = upper_32_bits(res->end);
flags = res->flags & ~IORESOURCE_SIZEALIGN;
if (!upper_bits)
flags &= ~IORESOURCE_MEM_64;
vmd->resources[2] = (struct resource) {
.name = "VMD MEMBAR2",
.start = res->start + membar2_offset,
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
.end = res->end,
.flags = flags,
.parent = res,
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
};
sd->vmd_dev = vmd->dev;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
sd->domain = vmd_find_free_domain();
if (sd->domain < 0)
return sd->domain;
sd->node = pcibus_to_node(vmd->dev->bus);
/*
* Currently MSI remapping must be enabled in guest passthrough mode
* due to some missing interrupt remapping plumbing. This is probably
* acceptable because the guest is usually CPU-limited and MSI
* remapping doesn't become a performance bottleneck.
*/
if (!(features & VMD_FEAT_CAN_BYPASS_MSI_REMAP) ||
offset[0] || offset[1]) {
ret = vmd_alloc_irqs(vmd);
if (ret)
return ret;
vmd_set_msi_remapping(vmd, true);
ret = vmd_create_irq_domain(vmd);
if (ret)
return ret;
/*
* Override the IRQ domain bus token so the domain can be
* distinguished from a regular PCI/MSI domain.
*/
irq_domain_update_bus_token(vmd->irq_domain, DOMAIN_BUS_VMD_MSI);
} else {
vmd_set_msi_remapping(vmd, false);
}
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
pci_add_resource(&resources, &vmd->resources[0]);
pci_add_resource_offset(&resources, &vmd->resources[1], offset[0]);
pci_add_resource_offset(&resources, &vmd->resources[2], offset[1]);
vmd->bus = pci_create_root_bus(&vmd->dev->dev, vmd->busn_start,
&vmd_ops, sd, &resources);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
if (!vmd->bus) {
pci_free_resource_list(&resources);
vmd_remove_irq_domain(vmd);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
return -ENODEV;
}
vmd_copy_host_bridge_flags(pci_find_host_bridge(vmd->dev->bus),
to_pci_host_bridge(vmd->bus->bridge));
vmd_attach_resources(vmd);
if (vmd->irq_domain)
dev_set_msi_domain(&vmd->bus->dev, vmd->irq_domain);
else
dev_set_msi_domain(&vmd->bus->dev,
dev_get_msi_domain(&vmd->dev->dev));
PCI: vmd: Create domain symlink before pci_bus_add_devices() The vmd driver creates a "domain" symlink in sysfs for each VMD bridge. Previously this symlink was created after pci_bus_add_devices() added devices below the VMD bridge and emitted udev events to announce them to userspace. This led to a race between userspace consumers of the udev events and the kernel creation of the symlink. One such consumer is mdadm, which assembles block devices into a RAID array, and for devices below a VMD bridge, mdadm depends on the "domain" symlink. If mdadm loses the race, it may be unable to assemble a RAID array, which may cause a boot failure or other issues, with complaints like this: (udev-worker)[2149]: nvme1n1: '/sbin/mdadm -I /dev/nvme1n1'(err) 'mdadm: Unable to get real path for '/sys/bus/pci/drivers/vmd/0000:c7:00.5/domain/device'' (udev-worker)[2149]: nvme1n1: '/sbin/mdadm -I /dev/nvme1n1'(err) 'mdadm: /dev/nvme1n1 is not attached to Intel(R) RAID controller.' (udev-worker)[2149]: nvme1n1: '/sbin/mdadm -I /dev/nvme1n1'(err) 'mdadm: No OROM/EFI properties for /dev/nvme1n1' (udev-worker)[2149]: nvme1n1: '/sbin/mdadm -I /dev/nvme1n1'(err) 'mdadm: no RAID superblock on /dev/nvme1n1.' (udev-worker)[2149]: nvme1n1: Process '/sbin/mdadm -I /dev/nvme1n1' failed with exit code 1. This symptom prevents the OS from booting successfully. After a NVMe disk is probed/added by the nvme driver, udevd invokes mdadm to detect if there is a mdraid associated with this NVMe disk, and mdadm determines if a NVMe device is connected to a particular VMD domain by checking the "domain" symlink. For example: Thread A Thread B Thread mdadm vmd_enable_domain pci_bus_add_devices __driver_probe_device ... work_on_cpu schedule_work_on : wakeup Thread B nvme_probe : wakeup scan_work to scan nvme disk and add nvme disk then wakeup udevd : udevd executes mdadm command flush_work main : wait for nvme_probe done ... __driver_probe_device find_driver_devices : probe next nvme device : 1) Detect domain symlink ... 2) Find domain symlink ... from vmd sysfs ... 3) Domain symlink not ... created yet; failed sysfs_create_link : create domain symlink Create the VMD "domain" symlink before invoking pci_bus_add_devices() to avoid this race. Suggested-by: Adrian Huang <ahuang12@lenovo.com> Link: https://lore.kernel.org/linux-pci/20240605124844.24293-1-sjiwei@163.com Signed-off-by: Jiwei Sun <sunjw10@lenovo.com> Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org> [bhelgaas: commit log] Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Nirmal Patel <nirmal.patel@linux.intel.com>
2024-06-05 05:48:44 -07:00
WARN(sysfs_create_link(&vmd->dev->dev.kobj, &vmd->bus->dev.kobj,
"domain"), "Can't create symlink to domain\n");
vmd_acpi_begin();
pci_scan_child_bus(vmd->bus);
vmd_domain_reset(vmd);
/* When Intel VMD is enabled, the OS does not discover the Root Ports
* owned by Intel VMD within the MMCFG space. pci_reset_bus() applies
* a reset to the parent of the PCI device supplied as argument. This
* is why we pass a child device, so the reset can be triggered at
* the Intel bridge level and propagated to all the children in the
* hierarchy.
*/
list_for_each_entry(child, &vmd->bus->children, node) {
if (!list_empty(&child->devices)) {
dev = list_first_entry(&child->devices,
struct pci_dev, bus_list);
ret = pci_reset_bus(dev);
if (ret)
pci_warn(dev, "can't reset device: %d\n", ret);
break;
}
}
pci_assign_unassigned_bus_resources(vmd->bus);
2023-01-19 20:15:22 -07:00
pci_walk_bus(vmd->bus, vmd_pm_enable_quirk, &features);
/*
* VMD root buses are virtual and don't return true on pci_is_pcie()
* and will fail pcie_bus_configure_settings() early. It can instead be
* run on each of the real root ports.
*/
list_for_each_entry(child, &vmd->bus->children, node)
pcie_bus_configure_settings(child);
pci_bus_add_devices(vmd->bus);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
vmd_acpi_end();
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
return 0;
}
static int vmd_probe(struct pci_dev *dev, const struct pci_device_id *id)
{
unsigned long features = (unsigned long) id->driver_data;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
struct vmd_dev *vmd;
int err;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
if (resource_size(&dev->resource[VMD_CFGBAR]) < (1 << 20))
return -ENOMEM;
vmd = devm_kzalloc(&dev->dev, sizeof(*vmd), GFP_KERNEL);
if (!vmd)
return -ENOMEM;
vmd->dev = dev;
vmd->instance = ida_alloc(&vmd_instance_ida, GFP_KERNEL);
if (vmd->instance < 0)
return vmd->instance;
vmd->name = devm_kasprintf(&dev->dev, GFP_KERNEL, "vmd%d",
vmd->instance);
if (!vmd->name) {
err = -ENOMEM;
goto out_release_instance;
}
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
err = pcim_enable_device(dev);
if (err < 0)
goto out_release_instance;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
vmd->cfgbar = pcim_iomap(dev, VMD_CFGBAR, 0);
if (!vmd->cfgbar) {
err = -ENOMEM;
goto out_release_instance;
}
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
pci_set_master(dev);
if (dma_set_mask_and_coherent(&dev->dev, DMA_BIT_MASK(64)) &&
dma_set_mask_and_coherent(&dev->dev, DMA_BIT_MASK(32))) {
err = -ENODEV;
goto out_release_instance;
}
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
if (features & VMD_FEAT_OFFSET_FIRST_VECTOR)
vmd->first_vec = 1;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
spin_lock_init(&vmd->cfg_lock);
pci_set_drvdata(dev, vmd);
err = vmd_enable_domain(vmd, features);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
if (err)
goto out_release_instance;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
dev_info(&vmd->dev->dev, "Bound to PCI domain %04x\n",
vmd->sysdata.domain);
return 0;
out_release_instance:
ida_free(&vmd_instance_ida, vmd->instance);
return err;
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
}
static void vmd_cleanup_srcu(struct vmd_dev *vmd)
{
int i;
for (i = 0; i < vmd->msix_count; i++)
cleanup_srcu_struct(&vmd->irqs[i].srcu);
}
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
static void vmd_remove(struct pci_dev *dev)
{
struct vmd_dev *vmd = pci_get_drvdata(dev);
pci_stop_root_bus(vmd->bus);
PCI: vmd: Create domain symlink before pci_bus_add_devices() The vmd driver creates a "domain" symlink in sysfs for each VMD bridge. Previously this symlink was created after pci_bus_add_devices() added devices below the VMD bridge and emitted udev events to announce them to userspace. This led to a race between userspace consumers of the udev events and the kernel creation of the symlink. One such consumer is mdadm, which assembles block devices into a RAID array, and for devices below a VMD bridge, mdadm depends on the "domain" symlink. If mdadm loses the race, it may be unable to assemble a RAID array, which may cause a boot failure or other issues, with complaints like this: (udev-worker)[2149]: nvme1n1: '/sbin/mdadm -I /dev/nvme1n1'(err) 'mdadm: Unable to get real path for '/sys/bus/pci/drivers/vmd/0000:c7:00.5/domain/device'' (udev-worker)[2149]: nvme1n1: '/sbin/mdadm -I /dev/nvme1n1'(err) 'mdadm: /dev/nvme1n1 is not attached to Intel(R) RAID controller.' (udev-worker)[2149]: nvme1n1: '/sbin/mdadm -I /dev/nvme1n1'(err) 'mdadm: No OROM/EFI properties for /dev/nvme1n1' (udev-worker)[2149]: nvme1n1: '/sbin/mdadm -I /dev/nvme1n1'(err) 'mdadm: no RAID superblock on /dev/nvme1n1.' (udev-worker)[2149]: nvme1n1: Process '/sbin/mdadm -I /dev/nvme1n1' failed with exit code 1. This symptom prevents the OS from booting successfully. After a NVMe disk is probed/added by the nvme driver, udevd invokes mdadm to detect if there is a mdraid associated with this NVMe disk, and mdadm determines if a NVMe device is connected to a particular VMD domain by checking the "domain" symlink. For example: Thread A Thread B Thread mdadm vmd_enable_domain pci_bus_add_devices __driver_probe_device ... work_on_cpu schedule_work_on : wakeup Thread B nvme_probe : wakeup scan_work to scan nvme disk and add nvme disk then wakeup udevd : udevd executes mdadm command flush_work main : wait for nvme_probe done ... __driver_probe_device find_driver_devices : probe next nvme device : 1) Detect domain symlink ... 2) Find domain symlink ... from vmd sysfs ... 3) Domain symlink not ... created yet; failed sysfs_create_link : create domain symlink Create the VMD "domain" symlink before invoking pci_bus_add_devices() to avoid this race. Suggested-by: Adrian Huang <ahuang12@lenovo.com> Link: https://lore.kernel.org/linux-pci/20240605124844.24293-1-sjiwei@163.com Signed-off-by: Jiwei Sun <sunjw10@lenovo.com> Signed-off-by: Krzysztof Wilczyński <kwilczynski@kernel.org> [bhelgaas: commit log] Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Reviewed-by: Nirmal Patel <nirmal.patel@linux.intel.com>
2024-06-05 05:48:44 -07:00
sysfs_remove_link(&vmd->dev->dev.kobj, "domain");
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
pci_remove_root_bus(vmd->bus);
vmd_cleanup_srcu(vmd);
vmd_detach_resources(vmd);
vmd_remove_irq_domain(vmd);
ida_free(&vmd_instance_ida, vmd->instance);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
}
static void vmd_shutdown(struct pci_dev *dev)
{
struct vmd_dev *vmd = pci_get_drvdata(dev);
vmd_remove_irq_domain(vmd);
}
#ifdef CONFIG_PM_SLEEP
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
static int vmd_suspend(struct device *dev)
{
struct pci_dev *pdev = to_pci_dev(dev);
struct vmd_dev *vmd = pci_get_drvdata(pdev);
int i;
for (i = 0; i < vmd->msix_count; i++)
devm_free_irq(dev, vmd->irqs[i].virq, &vmd->irqs[i]);
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
return 0;
}
static int vmd_resume(struct device *dev)
{
struct pci_dev *pdev = to_pci_dev(dev);
struct vmd_dev *vmd = pci_get_drvdata(pdev);
int err, i;
vmd_set_msi_remapping(vmd, !!vmd->irq_domain);
for (i = 0; i < vmd->msix_count; i++) {
err = devm_request_irq(dev, vmd->irqs[i].virq,
vmd_irq, IRQF_NO_THREAD,
vmd->name, &vmd->irqs[i]);
if (err)
return err;
}
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
return 0;
}
#endif
static SIMPLE_DEV_PM_OPS(vmd_dev_pm_ops, vmd_suspend, vmd_resume);
static const struct pci_device_id vmd_ids[] = {
{PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_VMD_201D),
.driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW_VSCAP,},
{PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_VMD_28C0),
.driver_data = VMD_FEAT_HAS_MEMBAR_SHADOW |
VMD_FEAT_HAS_BUS_RESTRICTIONS |
VMD_FEAT_CAN_BYPASS_MSI_REMAP,},
{PCI_VDEVICE(INTEL, 0x467f),
.driver_data = VMD_FEATS_CLIENT,},
{PCI_VDEVICE(INTEL, 0x4c3d),
.driver_data = VMD_FEATS_CLIENT,},
{PCI_VDEVICE(INTEL, 0xa77f),
.driver_data = VMD_FEATS_CLIENT,},
{PCI_VDEVICE(INTEL, 0x7d0b),
.driver_data = VMD_FEATS_CLIENT,},
{PCI_VDEVICE(INTEL, 0xad0b),
.driver_data = VMD_FEATS_CLIENT,},
{PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_VMD_9A0B),
.driver_data = VMD_FEATS_CLIENT,},
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
{0,}
};
MODULE_DEVICE_TABLE(pci, vmd_ids);
static struct pci_driver vmd_drv = {
.name = "vmd",
.id_table = vmd_ids,
.probe = vmd_probe,
.remove = vmd_remove,
.shutdown = vmd_shutdown,
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
.driver = {
.pm = &vmd_dev_pm_ops,
},
};
module_pci_driver(vmd_drv);
MODULE_AUTHOR("Intel Corporation");
MODULE_DESCRIPTION("Volume Management Device driver");
x86/PCI: Add driver for Intel Volume Management Device (VMD) The Intel Volume Management Device (VMD) is a Root Complex Integrated Endpoint that acts as a host bridge to a secondary PCIe domain. BIOS can reassign one or more Root Ports to appear within a VMD domain instead of the primary domain. The immediate benefit is that additional PCIe domains allow more than 256 buses in a system by letting bus numbers be reused across different domains. VMD domains do not define ACPI _SEG, so to avoid domain clashing with host bridges defining this segment, VMD domains start at 0x10000, which is greater than the highest possible 16-bit ACPI defined _SEG. This driver enumerates and enables the domain using the root bus configuration interface provided by the PCI subsystem. The driver provides configuration space accessor functions (pci_ops), bus and memory resources, an MSI IRQ domain with irq_chip implementation, and DMA operations necessary to use devices through the VMD endpoint's interface. VMD routes I/O as follows: 1) Configuration Space: BAR 0 ("CFGBAR") of VMD provides the base address and size for configuration space register access to VMD-owned root ports. It works similarly to MMCONFIG for extended configuration space. Bus numbering is independent and does not conflict with the primary domain. 2) MMIO Space: BARs 2 and 4 ("MEMBAR1" and "MEMBAR2") of VMD provide the base address, size, and type for MMIO register access. These addresses are not translated by VMD hardware; they are simply reservations to be distributed to root ports' memory base/limit registers and subdivided among devices downstream. 3) DMA: To interact appropriately with an IOMMU, the source ID DMA read and write requests are translated to the bus-device-function of the VMD endpoint. Otherwise, DMA operates normally without VMD-specific address translation. 4) Interrupts: Part of VMD's BAR 4 is reserved for VMD's MSI-X Table and PBA. MSIs from VMD domain devices and ports are remapped to appear as if they were issued using one of VMD's MSI-X table entries. Each MSI and MSI-X address of VMD-owned devices and ports has a special format where the address refers to specific entries in the VMD's MSI-X table. As with DMA, the interrupt source ID is translated to VMD's bus-device-function. The driver provides its own MSI and MSI-X configuration functions specific to how MSI messages are used within the VMD domain, and provides an irq_chip for independent IRQ allocation to relay interrupts from VMD's interrupt handler to the appropriate device driver's handler. 5) Errors: PCIe error message are intercepted by the root ports normally (e.g., AER), except with VMD, system errors (i.e., firmware first) are disabled by default. AER and hotplug interrupts are translated in the same way as endpoint interrupts. 6) VMD does not support INTx interrupts or IO ports. Devices or drivers requiring these features should either not be placed below VMD-owned root ports, or VMD should be disabled by BIOS for such endpoints. [bhelgaas: add VMD BAR #defines, factor out vmd_cfg_addr(), rework VMD resource setup, whitespace, changelog] Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Bjorn Helgaas <bhelgaas@google.com> Acked-by: Thomas Gleixner <tglx@linutronix.de> (IRQ-related parts)
2016-01-12 13:18:10 -07:00
MODULE_LICENSE("GPL v2");
MODULE_VERSION("0.6");