Hi,
here is a patch-set to rename iommu_param to dev_iommu and
establish it as a struct for generic per-device iommu-data.
Also move the iommu_fwspec pointer from struct device into
dev_iommu to have less iommu-related pointers in struct
device.
The bigger part of this patch-set moves the iommu_priv
pointer from struct iommu_fwspec to dev_iommu, making is
usable for iommu-drivers which do not use fwspecs.
The changes for that were mostly straightforward, except for
the arm-smmu (_not_ arm-smmu-v3) and the qcom iommu driver.
Unfortunatly I don't have the hardware for those, so any
testing of these drivers is greatly appreciated.
I boot-tested these changes on arm-smmu-v3 enabled KVM VM.
Please review.
Regards,
Joerg
Joerg Roedel (14):
ACPI/IORT: Remove direct access of dev->iommu_fwspec
drm/msm/mdp5: Remove direct access of dev->iommu_fwspec
iommu/tegra-gart: Remove direct access of dev->iommu_fwspec
iommu/arm-smmu: Fix uninitilized variable warning
iommu: Rename struct iommu_param to dev_iommu
iommu: Move iommu_fwspec to struct dev_iommu
iommu: Introduce accessors for iommu private data
iommu/arm-smmu-v3: Use accessor functions for iommu private data
iommu/arm-smmu: Use accessor functions for iommu private data
iommu/renesas: Use accessor functions for iommu private data
iommu/mediatek: Use accessor functions for iommu private data
iommu/qcom: Use accessor functions for iommu private data
iommu/virtio: Use accessor functions for iommu private data
iommu: Move fwspec->iommu_priv to struct dev_iommu
drivers/acpi/arm64/iort.c | 3 +-
drivers/gpu/drm/msm/disp/mdp5/mdp5_kms.c | 2 +-
drivers/iommu/arm-smmu-v3.c | 10 ++--
drivers/iommu/arm-smmu.c | 58 +++++++++++-----------
drivers/iommu/iommu.c | 31 ++++++------
drivers/iommu/ipmmu-vmsa.c | 7 +--
drivers/iommu/mtk_iommu.c | 13 +++--
drivers/iommu/mtk_iommu_v1.c | 14 +++---
drivers/iommu/qcom_iommu.c | 61 ++++++++++++++----------
drivers/iommu/tegra-gart.c | 2 +-
drivers/iommu/virtio-iommu.c | 11 ++---
include/linux/device.h | 7 ++-
include/linux/iommu.h | 29 ++++++++---
13 files changed, 137 insertions(+), 111 deletions(-)
--
2.17.1
From: Joerg Roedel <[email protected]>
Use the accessor functions instead of directly dereferencing
dev->iommu_fwspec.
Signed-off-by: Joerg Roedel <[email protected]>
---
drivers/iommu/tegra-gart.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/iommu/tegra-gart.c b/drivers/iommu/tegra-gart.c
index 3fb7ba72507d..db6559e8336f 100644
--- a/drivers/iommu/tegra-gart.c
+++ b/drivers/iommu/tegra-gart.c
@@ -247,7 +247,7 @@ static int gart_iommu_add_device(struct device *dev)
{
struct iommu_group *group;
- if (!dev->iommu_fwspec)
+ if (!dev_iommu_fwspec_get(dev))
return -ENODEV;
group = iommu_group_get_for_dev(dev);
--
2.17.1
Hi Joerg,
On Fri, Feb 28, 2020 at 04:08:06PM +0100, Joerg Roedel wrote:
> here is a patch-set to rename iommu_param to dev_iommu and
> establish it as a struct for generic per-device iommu-data.
> Also move the iommu_fwspec pointer from struct device into
> dev_iommu to have less iommu-related pointers in struct
> device.
>
> The bigger part of this patch-set moves the iommu_priv
> pointer from struct iommu_fwspec to dev_iommu, making is
> usable for iommu-drivers which do not use fwspecs.
>
> The changes for that were mostly straightforward, except for
> the arm-smmu (_not_ arm-smmu-v3) and the qcom iommu driver.
> Unfortunatly I don't have the hardware for those, so any
> testing of these drivers is greatly appreciated.
I haven't had a chance to review this properly yet, but I did take it
for a spin on my Seattle board with MMU-400 (arm-smmu) and it seems to
work the same as before, so:
Tested-by: Will Deacon <[email protected]> # arm-smmu
I'll try to review the patches soon.
Cheers,
Will
Hi Will,
On Tue, Mar 03, 2020 at 07:16:25PM +0000, Will Deacon wrote:
> I haven't had a chance to review this properly yet, but I did take it
> for a spin on my Seattle board with MMU-400 (arm-smmu) and it seems to
> work the same as before, so:
>
> Tested-by: Will Deacon <[email protected]> # arm-smmu
>
> I'll try to review the patches soon.
Thanks for testing it! I will send out a new version probably beginning
of next week (I am travelling this week) to fix the kbuild issue and
anything you might find.
Thanks,
Joerg
Hi Joerg,
On 2020/2/28 23:08, Joerg Roedel wrote:
> Hi,
>
> here is a patch-set to rename iommu_param to dev_iommu and
> establish it as a struct for generic per-device iommu-data.
> Also move the iommu_fwspec pointer from struct device into
> dev_iommu to have less iommu-related pointers in struct
> device.
>
> The bigger part of this patch-set moves the iommu_priv
> pointer from struct iommu_fwspec to dev_iommu, making is
> usable for iommu-drivers which do not use fwspecs.
>
> The changes for that were mostly straightforward, except for
> the arm-smmu (_not_ arm-smmu-v3) and the qcom iommu driver.
> Unfortunatly I don't have the hardware for those, so any
> testing of these drivers is greatly appreciated.
I tested this patch set on Kunpeng 920 ARM64 server which
using smmu-v3 with ACPI booting, but triggered a NULL
pointer dereference and panic at boot:
[ 14.832752] Unable to handle kernel NULL pointer dereference at virtual address 0000000000000010
[ 14.851425] Mem abort info:
[ 14.858940] ESR = 0x96000004
[ 14.866519] EC = 0x25: DABT (current EL), IL = 32 bits
[ 14.876580] SET = 0, FnV = 0
[ 14.884412] EA = 0, S1PTW = 0
[ 14.892275] Data abort info:
[ 14.899802] ISV = 0, ISS = 0x00000004
[ 14.908141] CM = 0, WnR = 0
[ 14.915401] [0000000000000010] user address but active_mm is swapper
[ 14.926367] Internal error: Oops: 96000004 [#1] SMP
[ 14.935992] Modules linked in:
[ 14.943724] CPU: 36 PID: 1 Comm: swapper/0 Not tainted 5.6.0-rc2+ #4
[ 14.955020] Hardware name: Huawei TaiShan 2280 V2/BC82AMDC, BIOS 0.81 07/10/2019
[ 14.972008] pstate: 80c00009 (Nzcv daif +PAN +UAO)
[ 14.981774] pc : iort_iommu_configure+0xdc/0x230
[ 14.991481] lr : iort_iommu_configure+0xcc/0x230
[ 15.001075] sp : ffff800011b3bad0
[ 15.009338] x29: ffff800011b3bad0 x28: ffff8000110a8968
[ 15.019483] x27: ffff800011540000 x26: ffff8000110004c8
[ 15.029390] x25: 0000000000000006 x24: 0000000000000000
[ 15.039336] x23: 0000000000000000 x22: 0000000000000000
[ 15.049270] x21: ffff8000113f9000 x20: ffff00002f5b0414
[ 15.059038] x19: ffff002fdc8f90b0 x18: ffffffffffffffff
[ 15.068590] x17: 0000000000000008 x16: 0000000000000005
[ 15.078182] x15: ffff8000113f9948 x14: ffff2027d993e91c
[ 15.087824] x13: ffff2027d993e16d x12: 0000000000000000
[ 15.097440] x11: 0101010101010101 x10: 00000000ffffff76
[ 15.106995] x9 : 0000000000000000 x8 : ffff2027d9a79b80
[ 15.116629] x7 : 0000000000000000 x6 : 000000000000003f
[ 15.126252] x5 : 0000000000000001 x4 : 0000000000000000
[ 15.135781] x3 : 0000000000000600 x2 : 0000000000000040
[ 15.145221] x1 : 0000000000000004 x0 : 0000000000000001
[ 15.154472] Call trace:
[ 15.160674] iort_iommu_configure+0xdc/0x230
[ 15.168752] acpi_dma_configure+0x88/0xb8
[ 15.176461] pci_dma_configure+0xc0/0xe0
[ 15.183935] really_probe+0xbc/0x498
[ 15.190853] driver_probe_device+0x12c/0x148
[ 15.198426] device_driver_attach+0x74/0x98
[ 15.205860] __driver_attach+0xc4/0x178
[ 15.213045] bus_for_each_dev+0x84/0xd8
[ 15.220185] driver_attach+0x30/0x40
[ 15.227051] bus_add_driver+0x170/0x258
[ 15.234241] driver_register+0x64/0x118
[ 15.241432] __pci_register_driver+0x58/0x68
[ 15.249202] hibmc_pci_driver_init+0x28/0x30
[ 15.256966] do_one_initcall+0x54/0x250
[ 15.264301] kernel_init_freeable+0x24c/0x2e0
[ 15.272164] kernel_init+0x18/0x110
[ 15.279068] ret_from_fork+0x10/0x18
[ 15.286033] Code: 2a0003f6 35000840 b9401a80 360005a0 (b94012e0)
[ 15.295791] ---[ end trace 881fe61747538fd0 ]---
[ 15.304039] Kernel panic - not syncing: Fatal exception
I don't have a time slot to do the detail investigation, but seems
that we don't have the iommu_fwspec properly initialized with ACPI
booting after applying this patch set.
Thanks
Hanjun
On Fri, Mar 06, 2020 at 04:39:37PM +0800, Hanjun Guo wrote:
> Hi Joerg,
>
> On 2020/2/28 23:08, Joerg Roedel wrote:
> > Hi,
> >
> > here is a patch-set to rename iommu_param to dev_iommu and
> > establish it as a struct for generic per-device iommu-data.
> > Also move the iommu_fwspec pointer from struct device into
> > dev_iommu to have less iommu-related pointers in struct
> > device.
> >
> > The bigger part of this patch-set moves the iommu_priv
> > pointer from struct iommu_fwspec to dev_iommu, making is
> > usable for iommu-drivers which do not use fwspecs.
> >
> > The changes for that were mostly straightforward, except for
> > the arm-smmu (_not_ arm-smmu-v3) and the qcom iommu driver.
> > Unfortunatly I don't have the hardware for those, so any
> > testing of these drivers is greatly appreciated.
>
> I tested this patch set on Kunpeng 920 ARM64 server which
> using smmu-v3 with ACPI booting, but triggered a NULL
> pointer dereference and panic at boot:
I think that's because patch 01/14 move the fwspec access too early. In
err = pci_for_each_dma_alias(to_pci_dev(dev),
iort_pci_iommu_init, &info);
if (!err && iort_pci_rc_supports_ats(node))
dev->iommu_fwspec->flags |= IOMMU_FWSPEC_PCI_RC_ATS;
the iommu_fwspec is only valid if iort_pci_iommu_init() initialized it
successfully, if err == 0. The following might fix it:
diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
index 0e981d7f3c7d..7d04424189df 100644
--- a/drivers/acpi/arm64/iort.c
+++ b/drivers/acpi/arm64/iort.c
@@ -1015,7 +1015,7 @@ const struct iommu_ops *iort_iommu_configure(struct device *dev)
return ops;
if (dev_is_pci(dev)) {
- struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
+ struct iommu_fwspec *fwspec;
struct pci_bus *bus = to_pci_dev(dev)->bus;
struct iort_pci_alias_info info = { .dev = dev };
@@ -1028,7 +1028,8 @@ const struct iommu_ops *iort_iommu_configure(struct device *dev)
err = pci_for_each_dma_alias(to_pci_dev(dev),
iort_pci_iommu_init, &info);
- if (!err && iort_pci_rc_supports_ats(node))
+ fwspec = dev_iommu_fwspec_get(dev);
+ if (fwspec && iort_pci_rc_supports_ats(node))
fwspec->flags |= IOMMU_FWSPEC_PCI_RC_ATS;
} else {
int i = 0;
Note that this use of iommu_fwspec will be removed by the ATS cleanup
series [1], but this change should work as a temporary fix.
Thanks,
Jean
[1] https://lore.kernel.org/linux-iommu/[email protected]/
On 2020/3/6 18:09, Jean-Philippe Brucker wrote:
> On Fri, Mar 06, 2020 at 04:39:37PM +0800, Hanjun Guo wrote:
>> Hi Joerg,
>>
>> On 2020/2/28 23:08, Joerg Roedel wrote:
>>> Hi,
>>>
>>> here is a patch-set to rename iommu_param to dev_iommu and
>>> establish it as a struct for generic per-device iommu-data.
>>> Also move the iommu_fwspec pointer from struct device into
>>> dev_iommu to have less iommu-related pointers in struct
>>> device.
>>>
>>> The bigger part of this patch-set moves the iommu_priv
>>> pointer from struct iommu_fwspec to dev_iommu, making is
>>> usable for iommu-drivers which do not use fwspecs.
>>>
>>> The changes for that were mostly straightforward, except for
>>> the arm-smmu (_not_ arm-smmu-v3) and the qcom iommu driver.
>>> Unfortunatly I don't have the hardware for those, so any
>>> testing of these drivers is greatly appreciated.
>>
>> I tested this patch set on Kunpeng 920 ARM64 server which
>> using smmu-v3 with ACPI booting, but triggered a NULL
>> pointer dereference and panic at boot:
>
> I think that's because patch 01/14 move the fwspec access too early. In
>
> err = pci_for_each_dma_alias(to_pci_dev(dev),
> iort_pci_iommu_init, &info);
>
> if (!err && iort_pci_rc_supports_ats(node))
> dev->iommu_fwspec->flags |= IOMMU_FWSPEC_PCI_RC_ATS;
>
> the iommu_fwspec is only valid if iort_pci_iommu_init() initialized it
> successfully, if err == 0. The following might fix it:
Good catch :)
>
> diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
> index 0e981d7f3c7d..7d04424189df 100644
> --- a/drivers/acpi/arm64/iort.c
> +++ b/drivers/acpi/arm64/iort.c
> @@ -1015,7 +1015,7 @@ const struct iommu_ops *iort_iommu_configure(struct device *dev)
> return ops;
>
> if (dev_is_pci(dev)) {
> - struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
> + struct iommu_fwspec *fwspec;
> struct pci_bus *bus = to_pci_dev(dev)->bus;
> struct iort_pci_alias_info info = { .dev = dev };
>
> @@ -1028,7 +1028,8 @@ const struct iommu_ops *iort_iommu_configure(struct device *dev)
> err = pci_for_each_dma_alias(to_pci_dev(dev),
> iort_pci_iommu_init, &info);
>
> - if (!err && iort_pci_rc_supports_ats(node))
> + fwspec = dev_iommu_fwspec_get(dev);
> + if (fwspec && iort_pci_rc_supports_ats(node))
> fwspec->flags |= IOMMU_FWSPEC_PCI_RC_ATS;
> } else {
> int i = 0;
And the panic disappeared. Joerg, please feel free to add my Tested-by
for smmu-v3 and IORT ACPI patches with above changes.
>
>
> Note that this use of iommu_fwspec will be removed by the ATS cleanup
> series [1], but this change should work as a temporary fix.
Yes, as your patch set will set the ats_supported flag in the
host bridge level, not per device, nice cleanup.
Thanks
Hanjun
Hi Jean-Philippe,
On Fri, Mar 06, 2020 at 11:09:55AM +0100, Jean-Philippe Brucker wrote:
> I think that's because patch 01/14 move the fwspec access too early. In
>
> err = pci_for_each_dma_alias(to_pci_dev(dev),
> iort_pci_iommu_init, &info);
>
> if (!err && iort_pci_rc_supports_ats(node))
> dev->iommu_fwspec->flags |= IOMMU_FWSPEC_PCI_RC_ATS;
>
> the iommu_fwspec is only valid if iort_pci_iommu_init() initialized it
> successfully, if err == 0. The following might fix it:
>
> diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
> index 0e981d7f3c7d..7d04424189df 100644
> --- a/drivers/acpi/arm64/iort.c
> +++ b/drivers/acpi/arm64/iort.c
> @@ -1015,7 +1015,7 @@ const struct iommu_ops *iort_iommu_configure(struct device *dev)
> return ops;
>
> if (dev_is_pci(dev)) {
> - struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
> + struct iommu_fwspec *fwspec;
> struct pci_bus *bus = to_pci_dev(dev)->bus;
> struct iort_pci_alias_info info = { .dev = dev };
>
> @@ -1028,7 +1028,8 @@ const struct iommu_ops *iort_iommu_configure(struct device *dev)
> err = pci_for_each_dma_alias(to_pci_dev(dev),
> iort_pci_iommu_init, &info);
>
> - if (!err && iort_pci_rc_supports_ats(node))
> + fwspec = dev_iommu_fwspec_get(dev);
> + if (fwspec && iort_pci_rc_supports_ats(node))
> fwspec->flags |= IOMMU_FWSPEC_PCI_RC_ATS;
> } else {
> int i = 0;
Thanks a lot for the fix! I added it to patch 1/14.
Regards,
Joerg