Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   "Zhang, Tina" <tina.zhang@intel.com>
To:     Jason Gunthorpe <jgg@ziepe.ca>
CC:     "Tian, Kevin" <kevin.tian@intel.com>,
        Joerg Roedel <joro@8bytes.org>,
        "Will Deacon" <will@kernel.org>,
        Lu Baolu <baolu.lu@linux.intel.com>,
        "Michael Shavit" <mshavit@google.com>,
        "iommu@lists.linux.dev" <iommu@lists.linux.dev>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: RE: [RFC PATCH 5/6] iommu: Support mm PASID 1:1 with sva domain
Thread-Topic: [RFC PATCH 5/6] iommu: Support mm PASID 1:1 with sva domain
Thread-Index: AQHZsHNM7tdDs3iVJ0y7IfdNR+l0+a+zRwSAgAB1NCA=
Date:   Tue, 11 Jul 2023 01:42:03 +0000
Message-ID: <MW5PR11MB588170AB944EED07823993528931A@MW5PR11MB5881.namprd11.prod.outlook.com>
References: <20230707013441.365583-1-tina.zhang@intel.com>
 <20230707013441.365583-6-tina.zhang@intel.com> <ZKw/xS7wOoRvNfnH@ziepe.ca>
In-Reply-To: <ZKw/xS7wOoRvNfnH@ziepe.ca>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?7T+ClMLKcOsFZtmQLPuvXwDQ955jj6Z3bF7Jll0JZlwkmekMpzYiuDQVqU3q?=
 =?us-ascii?Q?CqLuxK+oEVRaCMgeX1W8964lOPJTpYXTWQaPDWpqElTr+YMenpwFRnn31b1w?=
 =?us-ascii?Q?mt7+zDFOYqhOxWuosSAyv60rMreyYYfrTOg32dvjtK+RSeMm/r/diRJxls21?=
 =?us-ascii?Q?tm37VXItkbfSH+4cdU4q4n1Lako/wjwhMRij0ATQ+1+b4egueWPxQ/NhxsJs?=
 =?us-ascii?Q?3ZAD/2v55QgY0TIuQNmE6dZzmVbYOVMebWhw8Nx6WS2Zm1aDwk08FCdTfcdk?=
 =?us-ascii?Q?XZeUTn/uTe3dQXWn+m+Hsmy/JqF252b0ZgPpQTmhChfb8DLF+NhoOcHVk6i/?=
 =?us-ascii?Q?T796InmiAk8MGrXd1WVVSJMQM8PcLC2xm5fcCZRHU5FGb5GTeV9Mci9M5/4J?=
 =?us-ascii?Q?tdqISQJqQk2B4dGIf7GEpFknFCPEJMKIVHcA5thTpq92zlRQOpxJr6+AMEbN?=
 =?us-ascii?Q?Tumhn6fI7dThAqpo66HhcjpmazZG9i6ODOgSlnw77KFB8oTZgALpp+rJU6qB?=
 =?us-ascii?Q?sY1xtF48DMXb0c3ZPR/8AWy4SekpOcd+DWgmlShdb31vdyYCAnSDFCcVxQXS?=
 =?us-ascii?Q?KBB+ctbk1/QjIkeLuwpYG0mze4vFFeUkatsAUYIt3AEdYalPQY18JpfVmmTX?=
 =?us-ascii?Q?dr59HKpPpFptEvxO5If6e91ma/bMTUvwJMZOb9Up1c1UJlaU5rjebEkR5Cxw?=
 =?us-ascii?Q?p6SAdh27EctkQDtpTnX29VRTAfrp5pyWIbz0LWBDKoyUqn//LUGIlYHgTosf?=
 =?us-ascii?Q?SICH6Id0eDUVfb13kP0uj7n0SqrwSGvIqSoZn56iylx8cRZM36396vhcj9t0?=
 =?us-ascii?Q?cU2GkEb9s74aWp06Ei5NF0NVWYpZk7pawcqrQmiQH37O8F7QdJFLjcXwW7Yh?=
 =?us-ascii?Q?InB4WyfgqkmR9oDlgDonyU/EppcbEfopb+MG+xbPUxfXqSWXo4pbXoWFYdru?=
 =?us-ascii?Q?ndqqlte5HlOCBR1Dx/cRk7nNqQ2kI27eUkM8iKbBvctrdlKVjhP7dJCpwWSf?=
 =?us-ascii?Q?gnD8RZ+rOJZyYHdN1t4oUtIU4ThWY4nE6Og7YhYXHowgRhqKsvPC8jXbWfZB?=
 =?us-ascii?Q?h9X2Z9hb23lJJiiZgoYz0XooBCXUaJnia5IuyTWJ/VJJrOzrXZbSQVoYGHBH?=
 =?us-ascii?Q?ua/9AkpDIcpEbvIQEyHcLCdOw5qP3wrc+2InQfvfL+xIhQ5CwbEOOUG0gh1L?=
 =?us-ascii?Q?lsYjpJftul6zmLDUDXvp3+AWVyCZAf7uwVe29bEzEeguMxMn7c8UloHU5ji8?=
 =?us-ascii?Q?DTkIgLhc47ZNlG5+B7XrgD5z1gzZV/vGd7Z01hKBJBYfWg44t8AmpQka45OI?=
 =?us-ascii?Q?g7bKegsKINzpPAAcVbOHLnfJT014ydX8RcYzSAIl3SUcXMNHUdLpui0LW/AO?=
 =?us-ascii?Q?DAggY0ZSb1Ct7lqEZILvOCASZ8ugeQ+BTmXQLACQgxsFaQJZNC0FNOK8qRJf?=
 =?us-ascii?Q?CqrL/pn3YGFIcUO49nLb6mN0YpuRPbXyLs1Za4miUw0a7UOftKQsIyoU6MJe?=
 =?us-ascii?Q?Ts00vB9hXGEVuxanI8VoQNIL9XnMDfxYGFlKAqk1Ey7ulWBA6q42iluntYId?=
 =?us-ascii?Q?xG7T45sitz6VZQG7htlMrykd0wEvj+O+SF+HF2rc?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MW5PR11MB5881.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6ed0f3eb-8fcd-4ff3-22a9-08db81b006fe
X-MS-Exchange-CrossTenant-originalarrivaltime: 11 Jul 2023 01:42:03.4105
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: hUtfz2bQIMpAi1hkFGWCGa0BZIEuORgtW6loJsxehFcWFX4IbaADBnnhoe1p/U4Qc0QPs2srr+lrnShmIi1C0g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR11MB5705
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


> -----Original Message-----
> From: Jason Gunthorpe <jgg@ziepe.ca>
> Sent: Tuesday, July 11, 2023 1:29 AM
> To: Zhang, Tina <tina.zhang@intel.com>
> Cc: Tian, Kevin <kevin.tian@intel.com>; Joerg Roedel <joro@8bytes.org>; W=
ill
> Deacon <will@kernel.org>; Lu Baolu <baolu.lu@linux.intel.com>; Michael
> Shavit <mshavit@google.com>; iommu@lists.linux.dev; linux-
> kernel@vger.kernel.org
> Subject: Re: [RFC PATCH 5/6] iommu: Support mm PASID 1:1 with sva domain
>=20
> On Fri, Jul 07, 2023 at 09:34:40AM +0800, Tina Zhang wrote:
> > Each mm bound to devices gets a PASID and a corresponding sva domain
> > allocated in iommu_sva_bind_device(), which are referenced by iommu_mm
> > field of the mm. And that PASID and sva domain get released in
> > iommu_sva_
> > unbind_device() when no devices are binding to that mm. As a result,
> > during the life cycle, sva domain has 1:1 with mm PASID.
> >
> > Since the required info of PASID and sva domain are kept in struct
> > iommu_mm_data of a mm, use mm->iommu_mm field instead of the old
> pasid
> > field in mm struct.
>=20
> This is not technically right, the domains need to be a list, and we need=
 to
> search the list. Almost always the list will have 0 or 1 entries in it.
OK. Then, the mapping between sva_domain and pasid is n:1. IIUC, the reason=
 why we want this n:1 is because sva_domain is a type of iommu_domain and t=
herefore may have some hardware IOMMU specific configurations that cannot b=
e shared between different IOMMUs.

>=20
> > @@ -88,31 +98,41 @@ struct iommu_sva *iommu_sva_bind_device(struct
> device *dev, struct mm_struct *mm
> >  		goto out_unlock;
> >  	}
> >
> > -	if (domain) {
> > -		domain->users++;
> > -		goto out;
> > +	if (unlikely(domain)) {
> > +		/* Re-attach the device to the same domain? */
> > +		if (domain =3D=3D sva_domain) {
> > +			goto out;
> > +		} else {
> > +			/* Didn't get detached from the previous domain? */
> > +			ret =3D -EBUSY;
> > +			goto out_unlock;
> > +		}
> >  	}
>=20
> And if we do all of this we should just get rid of the horrible
> iommu_get_domain_for_dev_pasid() entirely.
I think it depends on whether we could get rid of iommu_group->pasid_array,=
 right? In the current implementation, it seems iommu_group->pasid_array is=
 expected to identify the usages that need PASID-granular DMA address trans=
lation (i.e., not only for sva usage).

>=20
> > diff --git a/include/linux/iommu.h b/include/linux/iommu.h index
> > 20135912584ba..1511ded7bc910 100644
> > --- a/include/linux/iommu.h
> > +++ b/include/linux/iommu.h
> > @@ -1175,20 +1175,20 @@ static inline bool
> > tegra_dev_iommu_get_stream_id(struct device *dev, u32 *stream  #ifdef
> > CONFIG_IOMMU_SVA  static inline void mm_pasid_init(struct mm_struct
> > *mm)  {
> > -	mm->pasid =3D IOMMU_PASID_INVALID;
> > +	mm->iommu_mm =3D &default_iommu_mm;
> >  }
> >  static inline bool mm_valid_pasid(struct mm_struct *mm)  {
> > -	return mm->pasid !=3D IOMMU_PASID_INVALID;
> > +	return mm->iommu_mm->pasid !=3D IOMMU_PASID_INVALID;
> >  }
>=20
> And these can now just test if the iommu_mmu->sva_domain is NULL
Right. Baolu also shared a similar suggestion. Thanks.

Regards,
-Tina

>=20
> Jaon