Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <bfc2eb22-54af-dab8-72e3-a142470590c3@intel.com>
Date:   Thu, 10 Aug 2023 09:31:41 +0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Firefox/102.0 Thunderbird/102.13.0
Subject: Re: [PATCH 0/5] Share sva domains with all devices bound to a mm
Content-Language: en-US
To:     "Tian, Kevin" <kevin.tian@intel.com>,
        Jason Gunthorpe <jgg@ziepe.ca>,
        "Lu Baolu" <baolu.lu@linux.intel.com>,
        Michael Shavit <mshavit@google.com>
CC:     "iommu@lists.linux.dev" <iommu@lists.linux.dev>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
References: <20230808074944.7825-1-tina.zhang@intel.com>
 <BN9PR11MB527638E7361714F8AA5FD4448C12A@BN9PR11MB5276.namprd11.prod.outlook.com>
From:   Tina Zhang <tina.zhang@intel.com>
In-Reply-To: <BN9PR11MB527638E7361714F8AA5FD4448C12A@BN9PR11MB5276.namprd11.prod.outlook.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Ni8zbG5icUZ1YjZJR3RWck40VUFtbmpwZzgyOTdVMkxyQzJOUFNIWDdpRFI3?=
 =?utf-8?B?Mndkbm50Z3VSZkR6amZFZFJqUkJzYUVMdlhKUEl0bHB3RHlxMnI4bEF5R0tu?=
 =?utf-8?B?Mm1HdFpNZDludWptUVNJa0p1a3NaNmxWdXZWUm1BTHI0a1U3M0xoaEs2MHRi?=
 =?utf-8?B?M0Vvc01HNFlvbHI2d1BKSnVnbjgzTnM4TEJIWTlBbEdjcUxpQUZIMUp6QWFv?=
 =?utf-8?B?em5kdnZCLzlXSUdnb3VLVGdNQWRHbG9XSFpDVkxwN3JtWUlIblpqbVVaRm5M?=
 =?utf-8?B?OFhTelNiNUl5aEZOTkIyUW1meW1xSXJrWThzb1BkRGNaM2hEODVKbkZOZHN5?=
 =?utf-8?B?eVREUHozMXdmc3c1RjY5VXY3R0p5MGxDSVVBeWlGc1NTcWlCQXNVM28yanNF?=
 =?utf-8?B?dVE5Q1BacGZIa2VETy9iYUptSU5DSENkbUZaY2tESjZKRGJJNitncW4vclRY?=
 =?utf-8?B?OHF6VU1waUd6dGt0Y0ZsUzUycnN0VkFBTFRIVndGVTZURjNObXdBTCtXOVRE?=
 =?utf-8?B?TDEvQjl6TlhVR3RoVUt0WXR6VFFMTVBPUnlYamNFbis2dnl1d3RtRW4rYXEv?=
 =?utf-8?B?b1Qxb1ZPWUxUZlBJQmFkbTdSSUVCV1NhcDU5TFRBL2plallDelJqRE0wS1dy?=
 =?utf-8?B?V0FtajlYTCtyTTErUExObjhKdittUUN3OEtXT2lPUnBFTmRGY1B3alZHMWY0?=
 =?utf-8?B?RTM1cTJpNWtlaVpvSHBRZCttQWVncEZhekNlWXh5T0R5YjRsZkZFVFNvSURM?=
 =?utf-8?B?YmhqZm9ZTDhhVVRZUi9mR3Y4K2NjcmdGSWZpMEFVZnVZQ3MrcmZJWDRrWXY2?=
 =?utf-8?B?K3dHWXZCUForVlY4NFQ0amlKUjNEVEE2TzMzeElKRmdRaWJuanQ4SnRzQzk0?=
 =?utf-8?B?anAvRjA4SHJhQkYrb2hoQUVNUXZjWm4zMlo4MTlaRHJJZHZaT2o0akxqUDE1?=
 =?utf-8?B?NzNUM09NczRLNGVncVluTkVrMzZ6YTNzTkZ3UGU1Y1dtaDBQeGZ3QlA5Wlpx?=
 =?utf-8?B?QUc2WU1JSVgzZkFvbDRyMmI4Rm1qS2I1SDc5a05wdmpHcnpCdDA2Sm9pcFdx?=
 =?utf-8?B?Y05VcVJtN0U3MG5hMWZidGtIZC9HQ2tLbVAxL0loOGdRVkRRUGh1L2pNb2Zh?=
 =?utf-8?B?S0dlNmhFb2Zkb0gwLzZ1VzU5NnZXb3o1eEdXajZxUFlkZlhCZ1pMYUdrR2cr?=
 =?utf-8?B?QytVWUVNaEhvcnhBaXJWWlJ1ZmlTaC9IVnRucTRTOWdIcVprUmsxMUF3a21P?=
 =?utf-8?B?QUxDcEJzVlVpeTZJczdhTHJYN2RVZ0lRS1RiM2hFcWhQV0IxekxJd2tmR3p4?=
 =?utf-8?B?THBrNE83T09MMnNGT1V3TzhBK0ZSR1dXd1NuQlM4ZU43YnB6V3VHazlIT0tW?=
 =?utf-8?B?bVVBZkc3aUxRSGExdml0UDg4Uk45MENSL3llVkhibU5tQSt1WWprS2YvSENm?=
 =?utf-8?B?WVRqaUVXdmNPc0tlS0doTmtHbjgzeTVLUEtFdko2OUZrd2JlcmVJZjF1NDdL?=
 =?utf-8?B?MmhtWkZ0ajhlcGJYb2lFQjJOeGN5OExtQ3hSemExbENBbnl6QWplQkNyQkJH?=
 =?utf-8?B?OCtLYXJUNlZjT3RLOFdKR2Q4V1E4MHc2eTE3SUtZUGcvcUtIanZTeVlsRmdv?=
 =?utf-8?B?N29Sa2RvbmlyL2RCcWQxV25DVWYzcGttVnpiUG53TFpodEYwNHFUZU9VUWpQ?=
 =?utf-8?B?SWpOQ3VVeHhMNEhlWjQ2YjJjRjdoVUdic2E4NGY1cWk2U3AvZkJvUC9zRHhQ?=
 =?utf-8?B?bEU0RXNzcjNBZHVnZTBpQUtEQW1iQ2ExM0NaVk16RUZkN3BRTTlIZUt3R3pm?=
 =?utf-8?B?UHZ5MjFyWkZqbitrVmh4QnVWVDIxVzhwTitYQWt4dS9TRldwUTRzRXJ5ZzIr?=
 =?utf-8?B?emZ4K1VDUDhFTHdiTmQ3cWRKWmlvZG43WERKRWdjeWJpM2xuZFVEV01KdDlM?=
 =?utf-8?B?TTF3ZHdVdUlYTFhiR0h4WWRud3BCOHViWkdPemRBbWFmNnIzaHFzbmVDSnlS?=
 =?utf-8?B?dTJPQkJESWRtSXFjUFBtOVZJRExCU2UrODAwTTVJZ28yanJCTGErYk0zOTN3?=
 =?utf-8?B?aW9SQTlMdytqR1k3QmhvV0JHWGVmVFZwMUJ0WWhLV2t2OW5uUSt2S1R0eTcr?=
 =?utf-8?Q?r1G92uLXPAgX0Zha+jpuRE4Lc?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 49ba532a-58b0-48cf-f817-08db99419302
X-MS-Exchange-CrossTenant-AuthSource: MW5PR11MB5881.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Aug 2023 01:31:52.4445
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: EAqpPkY2Ft/2RnhliLKMUPZ1iNSvLgeWz6Sp6qug4zOL1CV84IlPOkKhRMIJNtbI/B3vgstbJrEgqvz9SdGv3Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR11MB4769
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-5.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,
        RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,
        SPF_NONE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi,

On 8/9/23 17:41, Tian, Kevin wrote:
>> From: Zhang, Tina <tina.zhang@intel.com>
>> Sent: Tuesday, August 8, 2023 3:50 PM
>>
>> A sva domain's lifetime begins with binding a device to a mm and ends
>> by releasing all the bound devices from that sva domain. Technically,
>> there could be more than one sva domain identified by the mm PASID for
>> the use of bound devices issuing DMA transactions.
> 
> Could you elaborate it with some concrete examples which motivate
> this change?
The motivation is to remove the superfluous IOTLB invalidation in 
current VT-d driver.

Currently, in VT-d driver, due to lacking shared sva domain info, in 
intel_flush_svm_range(), both iotlb and dev-tlb invalidation operations 
are performed per-device. However, difference devices could be behind 
one IOMMU (e.g., four devices are behind one IOMMU) and invoking iotlb 
per-device gives us more iotlb invalidation than necessary (4 iotlb 
invalidation instead of 1). This issue may give more performance impact 
when in a virtual machine guest, as currently we have one virtual VT-d 
for in front of those virtual devices.


This patch fixes this issue by attaching shared sva domain information 
to mm, so that it can be utilized in the mm_notifier_ops callbacks.

Regards,
-Tina

> 
>>
>> To support mm PASID 1:n with sva domains, each mm needs to keep both a
>> reference list of allocated sva domains and the corresponding PASID.
>> However, currently, mm struct only has one pasid field for sva usage,
>> which is used to keep the info of an assigned PASID. That pasid field
>> cannot provide sufficient info to build up the 1:n mapping between PASID
>> and sva domains.
>>
>> This patch-set fills the gap by adding an mm_iommu field[1], whose type is
>> mm_iommu_data struct, to replace the old pasid field. The introduced
>> mm_iommu_data struct keeps info of both a reference list of sva domains
>> and an assigned PASID.
>>
>>
>> [1]: https://lore.kernel.org/linux-iommu/ZIBxPd1%2FJCAle6yP@nvidia.com/
>>
>>
>> The RFC version of this patch-set is here:
>> https://lore.kernel.org/linux-iommu/20230707013441.365583-1-
>> tina.zhang@intel.com/
>>
>> Tina Zhang (5):
>>    iommu: Add mm_get_pasid() helper function
>>    iommu: Call helper function to get assigned pasid value
>>    mm: Add structure to keep sva information
>>    iommu: Support mm PASID 1:n with sva domains
>>    mm: Deprecate pasid field
>>
>>   arch/x86/kernel/traps.c                       |  2 +-
>>   .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c   | 12 ++---
>>   drivers/iommu/intel/svm.c                     |  8 +--
>>   drivers/iommu/iommu-sva.c                     | 50 ++++++++++++-------
>>   include/linux/iommu.h                         | 19 +++++--
>>   include/linux/mm_types.h                      |  3 +-
>>   kernel/fork.c                                 |  1 -
>>   mm/init-mm.c                                  |  3 --
>>   8 files changed, 58 insertions(+), 40 deletions(-)
>>
>> --
>> 2.17.1
>