Use this field to save the pasid/ssid bits that a device is able to
support with its IOMMU hardware. It is a generic attribute of a device
and lifting it into the per-device dev_iommu struct makes it possible
to allocate a PASID for device without calls into the IOMMU drivers.
Any iommu driver which suports PASID related features should set this
field before features are enabled on the devices.
For initialization of this field in the VT-d driver, the
info->pasid_supported is only set for PCI devices. So the status is
that non-PCI SVA hasn't been supported yet. Setting this field only for
PCI devices has no functional change.
Signed-off-by: Lu Baolu <[email protected]>
Reviewed-by: Jean-Philippe Brucker <[email protected]>
---
include/linux/iommu.h | 1 +
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 2 ++
drivers/iommu/intel/iommu.c | 5 ++++-
3 files changed, 7 insertions(+), 1 deletion(-)
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index 5e1afe169549..b8ffaf2cb1d0 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -373,6 +373,7 @@ struct dev_iommu {
struct iommu_fwspec *fwspec;
struct iommu_device *iommu_dev;
void *priv;
+ unsigned int pasid_bits;
};
int iommu_device_register(struct iommu_device *iommu,
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 627a3ed5ee8f..afc63fce6107 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2681,6 +2681,8 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev)
smmu->features & ARM_SMMU_FEAT_STALL_FORCE)
master->stall_enabled = true;
+ dev->iommu->pasid_bits = master->ssid_bits;
+
return &smmu->iommu;
err_free_master:
diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 2990f80c5e08..99643f897f26 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -4624,8 +4624,11 @@ static struct iommu_device *intel_iommu_probe_device(struct device *dev)
if (pasid_supported(iommu)) {
int features = pci_pasid_features(pdev);
- if (features >= 0)
+ if (features >= 0) {
info->pasid_supported = features | 1;
+ dev->iommu->pasid_bits =
+ fls(pci_max_pasids(pdev)) - 1;
+ }
}
if (info->ats_supported && ecap_prs(iommu->ecap) &&
--
2.25.1
On Tue, May 10, 2022 at 02:17:28PM +0800, Lu Baolu wrote:
> int iommu_device_register(struct iommu_device *iommu,
> diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
> index 627a3ed5ee8f..afc63fce6107 100644
> +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
> @@ -2681,6 +2681,8 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev)
> smmu->features & ARM_SMMU_FEAT_STALL_FORCE)
> master->stall_enabled = true;
>
> + dev->iommu->pasid_bits = master->ssid_bits;
> return &smmu->iommu;
>
> err_free_master:
> diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
> index 2990f80c5e08..99643f897f26 100644
> +++ b/drivers/iommu/intel/iommu.c
> @@ -4624,8 +4624,11 @@ static struct iommu_device *intel_iommu_probe_device(struct device *dev)
> if (pasid_supported(iommu)) {
> int features = pci_pasid_features(pdev);
>
> - if (features >= 0)
> + if (features >= 0) {
> info->pasid_supported = features | 1;
> + dev->iommu->pasid_bits =
> + fls(pci_max_pasids(pdev)) - 1;
> + }
It is not very nice that both the iommu drivers have to duplicate the
code to read the pasid capability out of the PCI device.
IMHO it would make more sense for the iommu layer to report the
capability of its own HW block only, and for the core code to figure
out the master's limitation using a bus-specific approach.
It is also unfortunate that the enable/disable pasid is inside the
iommu driver as well - ideally the PCI driver itself would do this
when it knows it wants to use PASIDs.
The ordering interaction with ATS makes this look quite annoying
though. :(
I'm also not convinced individual IOMMU drivers should be forcing ATS
on, there are performance and functional implications here. Using ATS
or not is possibly best left as an administrator policy controlled by
the core code. Again we seem to have some mess.
Jason
On 2022/5/10 22:34, Jason Gunthorpe wrote:
> On Tue, May 10, 2022 at 02:17:28PM +0800, Lu Baolu wrote:
>
>> int iommu_device_register(struct iommu_device *iommu,
>> diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
>> index 627a3ed5ee8f..afc63fce6107 100644
>> +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
>> @@ -2681,6 +2681,8 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev)
>> smmu->features & ARM_SMMU_FEAT_STALL_FORCE)
>> master->stall_enabled = true;
>>
>> + dev->iommu->pasid_bits = master->ssid_bits;
>> return &smmu->iommu;
>>
>> err_free_master:
>> diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
>> index 2990f80c5e08..99643f897f26 100644
>> +++ b/drivers/iommu/intel/iommu.c
>> @@ -4624,8 +4624,11 @@ static struct iommu_device *intel_iommu_probe_device(struct device *dev)
>> if (pasid_supported(iommu)) {
>> int features = pci_pasid_features(pdev);
>>
>> - if (features >= 0)
>> + if (features >= 0) {
>> info->pasid_supported = features | 1;
>> + dev->iommu->pasid_bits =
>> + fls(pci_max_pasids(pdev)) - 1;
>> + }
>
> It is not very nice that both the iommu drivers have to duplicate the
> code to read the pasid capability out of the PCI device.
>
> IMHO it would make more sense for the iommu layer to report the
> capability of its own HW block only, and for the core code to figure
> out the master's limitation using a bus-specific approach.
Fair enough. The iommu hardware capability could be reported in
/**
* struct iommu_device - IOMMU core representation of one IOMMU hardware
* instance
* @list: Used by the iommu-core to keep a list of registered iommus
* @ops: iommu-ops for talking to this iommu
* @dev: struct device for sysfs handling
*/
struct iommu_device {
struct list_head list;
const struct iommu_ops *ops;
struct fwnode_handle *fwnode;
struct device *dev;
};
I haven't checked ARM code yet, but it works for x86 as far as I can
see.
>
> It is also unfortunate that the enable/disable pasid is inside the
> iommu driver as well - ideally the PCI driver itself would do this
> when it knows it wants to use PASIDs.
>
> The ordering interaction with ATS makes this look quite annoying
> though. :(
>
> I'm also not convinced individual IOMMU drivers should be forcing ATS
> on, there are performance and functional implications here. Using ATS
> or not is possibly best left as an administrator policy controlled by
> the core code. Again we seem to have some mess.
Agreed with you. This has already been in my task list. I will start to
solve it after the iommufd tasks.
Best regards,
baolu
On Wed, May 11, 2022 at 10:25:48AM +0800, Baolu Lu wrote:
> On 2022/5/10 22:34, Jason Gunthorpe wrote:
> > On Tue, May 10, 2022 at 02:17:28PM +0800, Lu Baolu wrote:
> >
> > > int iommu_device_register(struct iommu_device *iommu,
> > > diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
> > > index 627a3ed5ee8f..afc63fce6107 100644
> > > +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
> > > @@ -2681,6 +2681,8 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev)
> > > smmu->features & ARM_SMMU_FEAT_STALL_FORCE)
> > > master->stall_enabled = true;
> > > + dev->iommu->pasid_bits = master->ssid_bits;
> > > return &smmu->iommu;
> > > err_free_master:
> > > diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
> > > index 2990f80c5e08..99643f897f26 100644
> > > +++ b/drivers/iommu/intel/iommu.c
> > > @@ -4624,8 +4624,11 @@ static struct iommu_device *intel_iommu_probe_device(struct device *dev)
> > > if (pasid_supported(iommu)) {
> > > int features = pci_pasid_features(pdev);
> > > - if (features >= 0)
> > > + if (features >= 0) {
> > > info->pasid_supported = features | 1;
> > > + dev->iommu->pasid_bits =
> > > + fls(pci_max_pasids(pdev)) - 1;
> > > + }
> >
> > It is not very nice that both the iommu drivers have to duplicate the
> > code to read the pasid capability out of the PCI device.
> >
> > IMHO it would make more sense for the iommu layer to report the
> > capability of its own HW block only, and for the core code to figure
> > out the master's limitation using a bus-specific approach.
>
> Fair enough. The iommu hardware capability could be reported in
>
> /**
> * struct iommu_device - IOMMU core representation of one IOMMU hardware
> * instance
> * @list: Used by the iommu-core to keep a list of registered iommus
> * @ops: iommu-ops for talking to this iommu
> * @dev: struct device for sysfs handling
> */
> struct iommu_device {
> struct list_head list;
> const struct iommu_ops *ops;
> struct fwnode_handle *fwnode;
> struct device *dev;
> };
>
> I haven't checked ARM code yet, but it works for x86 as far as I can
> see.
Arm also supports non-PCI PASID by reading a firmware property:
device_property_read_u32(dev, "pasid-num-bits", &master->ssid_bits);
should be the only difference
Thanks,
Jean
>
> >
> > It is also unfortunate that the enable/disable pasid is inside the
> > iommu driver as well - ideally the PCI driver itself would do this
> > when it knows it wants to use PASIDs.
> >
> > The ordering interaction with ATS makes this look quite annoying
> > though. :(
> >
> > I'm also not convinced individual IOMMU drivers should be forcing ATS
> > on, there are performance and functional implications here. Using ATS
> > or not is possibly best left as an administrator policy controlled by
> > the core code. Again we seem to have some mess.
>
> Agreed with you. This has already been in my task list. I will start to
> solve it after the iommufd tasks.
>
> Best regards,
> baolu
On Wed, May 11, 2022 at 09:00:50AM +0100, Jean-Philippe Brucker wrote:
> > /**
> > * struct iommu_device - IOMMU core representation of one IOMMU hardware
> > * instance
> > * @list: Used by the iommu-core to keep a list of registered iommus
> > * @ops: iommu-ops for talking to this iommu
> > * @dev: struct device for sysfs handling
> > */
> > struct iommu_device {
> > struct list_head list;
> > const struct iommu_ops *ops;
> > struct fwnode_handle *fwnode;
> > struct device *dev;
> > };
> >
> > I haven't checked ARM code yet, but it works for x86 as far as I can
> > see.
>
> Arm also supports non-PCI PASID by reading a firmware property:
>
> device_property_read_u32(dev, "pasid-num-bits", &master->ssid_bits);
>
> should be the only difference
That is not "ARM" that is generic DT/ACPI for platform devices and
should be handled by the core code in the same place it does PCI
discovery.
Jason