Hi all,
I am working on VFIO DMA dirty pages tracking based on ARM SMMU HTTU,
and have done a lot of testing.In the test, I found a problem that
greatly affects
performance of VFIO DMA dirty pages tracking.
According to the current arm-smmu-v3 driver, multiple VFIO pass-through
device comes from different SMMUs will be assigned to different
arm_smmu_domain. It will create page table for each arm_smmu_domain,
even though these page tables are exactly the same. Bacause dirty pages
tracking needs to traverse the page table, multiple page tables will make
performance worse.
I learned the ARM SMMUv3 spec and had some exchanges with my colleagues
who work on SMMU hardware. I did not find the restriction that multiple
SMMUs cannot share the same page table. We migth be able to do this like
x86 IOMMU. If I have missed something, please point it out.
Looking forward to your suggestions.????
Thanks,
Kunkun Jiang
On 2021-09-07 08:41, Kunkun Jiang wrote:
> Hi all,
>
> I am working on VFIO DMA dirty pages tracking based on ARM SMMU HTTU,
> and have done a lot of testing.In the test, I found a problem that
> greatly affects
> performance of VFIO DMA dirty pages tracking.
>
> According to the current arm-smmu-v3 driver, multiple VFIO pass-through
> device comes from different SMMUs will be assigned to different
> arm_smmu_domain. It will create page table for each arm_smmu_domain,
> even though these page tables are exactly the same. Bacause dirty pages
> tracking needs to traverse the page table, multiple page tables will make
> performance worse.
>
> I learned the ARM SMMUv3 spec and had some exchanges with my colleagues
> who work on SMMU hardware. I did not find the restriction that multiple
> SMMUs cannot share the same page table. We migth be able to do this like
> x86 IOMMU. If I have missed something, please point it out.
Sure, it's not impossible, there are just a lot of fiddly
considerations, mostly around how to handle SMMU instances with
different capabilities. We haven't had a strong need to support this
case so far, so we've simply been avoiding all that complexity.
Robin.
> Looking forward to your suggestions.????
>
> Thanks,
> Kunkun Jiang
>
On 2021/9/7 17:43, Robin Murphy wrote:
> On 2021-09-07 08:41, Kunkun Jiang wrote:
>> Hi all,
>>
>> I am working on VFIO DMA dirty pages tracking based on ARM SMMU HTTU,
>> and have done a lot of testing.In the test, I found a problem that
>> greatly affects
>> performance of VFIO DMA dirty pages tracking.
>>
>> According to the current arm-smmu-v3 driver, multiple VFIO pass-through
>> device comes from different SMMUs will be assigned to different
>> arm_smmu_domain. It will create page table for each arm_smmu_domain,
>> even though these page tables are exactly the same. Bacause dirty pages
>> tracking needs to traverse the page table, multiple page tables will
>> make
>> performance worse.
>>
>> I learned the ARM SMMUv3 spec and had some exchanges with my colleagues
>> who work on SMMU hardware. I did not find the restriction that multiple
>> SMMUs cannot share the same page table. We migth be able to do this like
>> x86 IOMMU. If I have missed something, please point it out.
>
> Sure, it's not impossible, there are just a lot of fiddly
> considerations, mostly around how to handle SMMU instances with
> different capabilities. We haven't had a strong need to support this
> case so far, so we've simply been avoiding all that complexity.
Yes, there's a lot to consider here. I'll try to analyze it carefully.
We can discuss
some of the difficulties in the future. From the perspective of
improving the
performance of VFIO DMA dirty pages tracking, it is worth supporting
this feature.
Thanks,
Kunkun Jiang
>
> Robin.
>
>> Looking forward to your suggestions.????
>>
>> Thanks,
>> Kunkun Jiang
>>
> .