Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Tue, 30 May 2023 14:08:34 -0300
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     "Tian, Kevin" <kevin.tian@intel.com>
Cc:     Jacob Pan <jacob.jun.pan@linux.intel.com>,
        LKML <linux-kernel@vger.kernel.org>,
        "iommu@lists.linux.dev" <iommu@lists.linux.dev>,
        Lu Baolu <baolu.lu@linux.intel.com>,
        Joerg Roedel <joro@8bytes.org>,
        Robin Murphy <robin.murphy@arm.com>,
        Jean-Philippe Brucker <jean-philippe@linaro.com>,
        "dmaengine@vger.kernel.org" <dmaengine@vger.kernel.org>,
        "vkoul@kernel.org" <vkoul@kernel.org>,
        Will Deacon <will@kernel.org>,
        David Woodhouse <dwmw2@infradead.org>,
        "Raj, Ashok" <ashok.raj@intel.com>,
        "Liu, Yi L" <yi.l.liu@intel.com>,
        "Yu, Fenghua" <fenghua.yu@intel.com>,
        "Jiang, Dave" <dave.jiang@intel.com>,
        "Luck, Tony" <tony.luck@intel.com>,
        "Zanussi, Tom" <tom.zanussi@intel.com>,
        "Ranganathan, Narayan" <narayan.ranganathan@intel.com>
Subject: Re: [PATCH v7 3/4] iommu/vt-d: Add set_dev_pasid callback for dma
 domain
Message-ID: <ZHYtkioNR2YfC18C@nvidia.com>
References: <20230523173451.2932113-1-jacob.jun.pan@linux.intel.com>
 <20230523173451.2932113-4-jacob.jun.pan@linux.intel.com>
 <BN9PR11MB52760139A4844C8DF0EE1BE98C469@BN9PR11MB5276.namprd11.prod.outlook.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <BN9PR11MB52760139A4844C8DF0EE1BE98C469@BN9PR11MB5276.namprd11.prod.outlook.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?29OcSqrffcCtpINESYyUeWZL9pkQZa/vb60L2ypZ6a7BtW2YU+tfCjr6438v?=
 =?us-ascii?Q?1Mqbi6BmZJ+HuWgF9JjFowZ0p6O9+PKSP4EEdysMDjuzbKNx/Ce8jQ+BGLjq?=
 =?us-ascii?Q?Bu1m4oROXb6TSo/YwSGgSjuEjVk5t5WVpCehIprHXya9SBo/PRytxxJgqtdG?=
 =?us-ascii?Q?NaqtGad+X1KO2hAC5Kr1l+YAXnr/UotawaW38pGO8Nh6rWK7AWOR6mDuGbgz?=
 =?us-ascii?Q?sGSCEz5TZFTcWBOUeEPnwr0UfRsB9BYThIQu/d/CuPhnTD0dcCn5JUEZ01Vt?=
 =?us-ascii?Q?Ho1mND144F2VY5qq9Q1HF5dAKFXg3wkADtr6TcJTxUajQ5lXUAfwvp91Qoah?=
 =?us-ascii?Q?Db3NmDWkO4pTaRcF68YFVNsjZjyLloeXygs+8b9pwscLnCY5UQRMgTVhvrp8?=
 =?us-ascii?Q?pYIKukYWe4Gx6jHXKeilLqNTNHZIfykgEWwyokH0QpfPch2yZvs0mh7Re0ro?=
 =?us-ascii?Q?N704BMjt3ETo+5uQFqMtV5+a9TFwIrDw3i5L9x714RBDhlYh5Jx/zwXM8rkD?=
 =?us-ascii?Q?jpf7JZGnVQbBeluNLTADLY/oAmJFjnZwr2o5gdjAxIbreN2ZJLMEcHdf98Bs?=
 =?us-ascii?Q?W8F2NeOZnyyCFMpa9s3BZYXyrWjHVCW3slDM7n5ldo0SS44TjCK9Bua0CRKs?=
 =?us-ascii?Q?xT437M7vi/PuLF2b7EKIXDUt/txatZY3G6nCDk4UDNqWEpPVLNkeWU1lLNNE?=
 =?us-ascii?Q?dQ18U9CEywgD/1WaDPzKGFHwGelC42vCHt0Q62/WlELjloLB4WsI/Zyttx/L?=
 =?us-ascii?Q?Ebovbxck7EZe4vHKhb2/Bdh93gGZ5Qmc05sNW/bCdaTQfu/DWK9pBKBYcRU7?=
 =?us-ascii?Q?O5U1DccFCFMWOMyuYZ+FM8BBYTX+rEPjNjEy7D++SsBbEHcMt28e1sTCJncM?=
 =?us-ascii?Q?aFlDFg97zZlvtDHn1+e4CyHeSYXArRN6POkd6fR/1SWE9gp4cQljI4AP1VZO?=
 =?us-ascii?Q?2ppvnSPn2jmS3AuWUsHM5ZN0ZtacedlTePps78KuILhbG946NQvSsz3OpqQ7?=
 =?us-ascii?Q?VBvFhY3Xj2p0a8jQR+XtTETdkGGBY+2GqY3DCr1xK4f+fpYHAsEGmg7bIwx8?=
 =?us-ascii?Q?r6uiaCGVFNN+uR2Ew1PH6d+mcrB33nqm6fiIhFGV6t3osjTEsGms7N3d/8Zl?=
 =?us-ascii?Q?USCxXK8JeEO1vFwTzNdJhgxj5mDULsooEyyf/dwLwHgNtu2N/rQhPU5K0DH+?=
 =?us-ascii?Q?o/F7r7sQAJudRyFbqwKzVUCDLUFyDUZTqMBRdxKiCM2fRl2Jw7U9jFaey6+j?=
 =?us-ascii?Q?aSgKRGFcxlZ3te6NtsK9Vlu7BEJZKqCeB7LfdgaFum1GrCG1dNIpq5Yil+oH?=
 =?us-ascii?Q?haUerbHaYdEAcIIJhF8eCY9DABkw6L0s92Nz8cYZO5+8BQMoe8j2dLY7AGKP?=
 =?us-ascii?Q?ci+R3p/P9XqyMRciyKEXN9Irgmb07dl8gKKt7IIWm5inPZAc35TCEtCNADYP?=
 =?us-ascii?Q?364FdBIS9FF1Mn2BABA20tr7zhDu8nsvWgGIFPS8qi1OMDQUR6rHSFUl0oKP?=
 =?us-ascii?Q?8b1YnGVXbDZvSgRBj6Sew77UP8lJKAZ1d/ew3KrCqiGiq8og1NBTpcTds47U?=
 =?us-ascii?Q?ZAhxkmweCbAEoEQ9z+nKu3pHzG+eL5ddp1hpevMv?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d95cecad-4f16-4bf4-38c9-08db61308141
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 May 2023 17:08:35.9689
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: rd6WOy694REP123c78GQdXCuNO3BMob1ae0pERCKxFpc10G8TnYNAY6JGrnv4uJK
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB8054
X-Spam-Status: No, score=-1.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,
        T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=no autolearn_force=no
        version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

> > +	/*
> > +	 * The SVA implementation needs to stop mm notification, drain the
> > +	 * pending page fault requests before tearing down the pasid entry.
> > +	 * The VT-d spec (section 6.2.3.1) also recommends that software
> > +	 * could use a reserved domain id for all first-only and pass-through
> > +	 * translations. Hence there's no need to call
> > domain_detach_iommu()
> > +	 * in the sva domain case.
> > +	 */
> 
> It's probably clearer to say:

Is this what is going on ??

> /*
>  * SVA domain requires special treatment before tearing down the pasid
>  * entry:
>  *   1) pasid is stored in mm instead of in dev_pasid;

Why? The mm pasid should not be used by any driver code, the PASID the
SVA is hooked to does NOT have to be the mm PASID.

>  *   2) all SVA domains share a reserved domain id per recommendation
>  *      from VT-d spec (section 6.2.3.1) so domain_detach_iommu() is
>  *      not required;

The DID should be managed and allocated for the S2 pointer and the
flushing logic should work genericly by tracking the S2's being used
and flushing their DIDs when all the S2s fall out of use. The special
identity S2 just gets a static DID that never falls out of use.

Jason