The existing code puts all devices from a root PE to the same IOMMU group.
However it is a possible situation when subordinate buses belong to
separate PEs, in this case devices from these subordinate buses
should be added to lower level PE rather to the root PE.
This limits pnv_ioda_setup_bus_dma() invocation to only PEs which own
all subordinate buses.
Suggested-by: Gavin Shan <[email protected]>
Signed-off-by: Alexey Kardashevskiy <[email protected]>
---
This would be nice to have together with the DDW patchset.
This does not fix anything DDW patchset did, it fixes IOMMU
groups management which is essential for the whole feature to work.
---
arch/powerpc/platforms/powernv/pci-ioda.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
index a0b00c1..3b1dc79 100644
--- a/arch/powerpc/platforms/powernv/pci-ioda.c
+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
@@ -1702,7 +1702,7 @@ static void pnv_ioda_setup_bus_dma(struct pnv_ioda_pe *pe,
set_iommu_table_base(&dev->dev, pe->table_group.tables[0]);
iommu_add_device(&dev->dev);
- if (dev->subordinate)
+ if ((pe->flags & PNV_IODA_PE_BUS_ALL) && dev->subordinate)
pnv_ioda_setup_bus_dma(pe, dev->subordinate);
}
}
--
2.4.0.rc3.8.gfb3e7d5
On Fri, Jun 12, 2015 at 04:19:17PM +1000, Alexey Kardashevskiy wrote:
>The existing code puts all devices from a root PE to the same IOMMU group.
>However it is a possible situation when subordinate buses belong to
>separate PEs, in this case devices from these subordinate buses
>should be added to lower level PE rather to the root PE.
>
>This limits pnv_ioda_setup_bus_dma() invocation to only PEs which own
>all subordinate buses.
>
>Suggested-by: Gavin Shan <[email protected]>
>Signed-off-by: Alexey Kardashevskiy <[email protected]>
Alexey, I think it's good candidate for stable.
Thanks,
Gavin
>---
>
>This would be nice to have together with the DDW patchset.
>This does not fix anything DDW patchset did, it fixes IOMMU
>groups management which is essential for the whole feature to work.
>
>---
> arch/powerpc/platforms/powernv/pci-ioda.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
>diff --git a/arch/powerpc/platforms/powernv/pci-ioda.c b/arch/powerpc/platforms/powernv/pci-ioda.c
>index a0b00c1..3b1dc79 100644
>--- a/arch/powerpc/platforms/powernv/pci-ioda.c
>+++ b/arch/powerpc/platforms/powernv/pci-ioda.c
>@@ -1702,7 +1702,7 @@ static void pnv_ioda_setup_bus_dma(struct pnv_ioda_pe *pe,
> set_iommu_table_base(&dev->dev, pe->table_group.tables[0]);
> iommu_add_device(&dev->dev);
>
>- if (dev->subordinate)
>+ if ((pe->flags & PNV_IODA_PE_BUS_ALL) && dev->subordinate)
> pnv_ioda_setup_bus_dma(pe, dev->subordinate);
> }
> }
>--
>2.4.0.rc3.8.gfb3e7d5
>
On 12.06.2015 [16:47:03 +1000], Gavin Shan wrote:
> On Fri, Jun 12, 2015 at 04:19:17PM +1000, Alexey Kardashevskiy wrote:
> >The existing code puts all devices from a root PE to the same IOMMU group.
> >However it is a possible situation when subordinate buses belong to
> >separate PEs, in this case devices from these subordinate buses
> >should be added to lower level PE rather to the root PE.
> >
> >This limits pnv_ioda_setup_bus_dma() invocation to only PEs which own
> >all subordinate buses.
> >
> >Suggested-by: Gavin Shan <[email protected]>
> >Signed-off-by: Alexey Kardashevskiy <[email protected]>
>
> Alexey, I think it's good candidate for stable.
Why? This commit message doesn't imply there is a serious bug just a
"possible" (does that mean theoretical?) situtation and the following:
> >This would be nice to have together with the DDW patchset.
> >This does not fix anything DDW patchset did, it fixes IOMMU
> >groups management which is essential for the whole feature to work.
implies it is both related and independent of DDW?
-Nish
On 06/13/2015 01:06 AM, Nishanth Aravamudan wrote:
> On 12.06.2015 [16:47:03 +1000], Gavin Shan wrote:
>> On Fri, Jun 12, 2015 at 04:19:17PM +1000, Alexey Kardashevskiy wrote:
>>> The existing code puts all devices from a root PE to the same IOMMU group.
>>> However it is a possible situation when subordinate buses belong to
>>> separate PEs, in this case devices from these subordinate buses
>>> should be added to lower level PE rather to the root PE.
>>>
>>> This limits pnv_ioda_setup_bus_dma() invocation to only PEs which own
>>> all subordinate buses.
>>>
>>> Suggested-by: Gavin Shan <[email protected]>
>>> Signed-off-by: Alexey Kardashevskiy <[email protected]>
>>
>> Alexey, I think it's good candidate for stable.
>
> Why? This commit message doesn't imply there is a serious bug just a
> "possible" (does that mean theoretical?) situtation and the following:
>
>>> This would be nice to have together with the DDW patchset.
>>> This does not fix anything DDW patchset did, it fixes IOMMU
>>> groups management which is essential for the whole feature to work.
>
> implies it is both related and independent of DDW?
DDW is for fast devices and I just got my hands on a "firestone" machine
(p8 + nvidia tesla = very fast machine) where this bug appeared first time.
--
Alexey
On Fri, Jun 12, 2015 at 08:06:43AM -0700, Nishanth Aravamudan wrote:
>On 12.06.2015 [16:47:03 +1000], Gavin Shan wrote:
>> On Fri, Jun 12, 2015 at 04:19:17PM +1000, Alexey Kardashevskiy wrote:
>> >The existing code puts all devices from a root PE to the same IOMMU group.
>> >However it is a possible situation when subordinate buses belong to
>> >separate PEs, in this case devices from these subordinate buses
>> >should be added to lower level PE rather to the root PE.
>> >
>> >This limits pnv_ioda_setup_bus_dma() invocation to only PEs which own
>> >all subordinate buses.
>> >
>> >Suggested-by: Gavin Shan <[email protected]>
>> >Signed-off-by: Alexey Kardashevskiy <[email protected]>
>>
>> Alexey, I think it's good candidate for stable.
>
>Why? This commit message doesn't imply there is a serious bug just a
>"possible" (does that mean theoretical?) situtation and the following:
>
The problem depends on the PCI topology. Below topology, existing on
firestone machine, is one of the cases that wrong IOMMU group is used
for devices behind the upstream port of the PCIe switch: The PE for
Bus#2 has same IOMMU group as that one of the PE for "Root Bus" wrongly
here.
|
+---------------+ (Root Bus)
| |
[ Root Port ] [ System Peripheral ]
+
| (Bus#1)
+
[ Up Port ]
+
|
+------------+-----------+ (Bus#2)
| |
[ Dn port] [ Dn Port]
>> >This would be nice to have together with the DDW patchset.
>> >This does not fix anything DDW patchset did, it fixes IOMMU
>> >groups management which is essential for the whole feature to work.
>
>implies it is both related and independent of DDW?
>
The problem is independent of DDW.
Thanks,
Gavin
>-Nish
On Fri, 2015-12-06 at 06:19:17 UTC, Alexey Kardashevskiy wrote:
> The existing code puts all devices from a root PE to the same IOMMU group.
> However it is a possible situation when subordinate buses belong to
> separate PEs, in this case devices from these subordinate buses
> should be added to lower level PE rather to the root PE.
>
> This limits pnv_ioda_setup_bus_dma() invocation to only PEs which own
> all subordinate buses.
>
> Suggested-by: Gavin Shan <[email protected]>
> Signed-off-by: Alexey Kardashevskiy <[email protected]>
Alexey/Gavin, can one of you write a better changelog for this. Explaining
what exactly is broken without it, how it fixes the problem, and why that is
the right fix.
cheers