Date:   Thu, 9 Feb 2023 20:01:00 -0400
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Nicolin Chen <nicolinc@nvidia.com>
Cc:     "Tian, Kevin" <kevin.tian@intel.com>,
        "joro@8bytes.org" <joro@8bytes.org>,
        "will@kernel.org" <will@kernel.org>,
        "robin.murphy@arm.com" <robin.murphy@arm.com>,
        "alex.williamson@redhat.com" <alex.williamson@redhat.com>,
        "shuah@kernel.org" <shuah@kernel.org>,
        "Liu, Yi L" <yi.l.liu@intel.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "iommu@lists.linux.dev" <iommu@lists.linux.dev>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "linux-kselftest@vger.kernel.org" <linux-kselftest@vger.kernel.org>,
        "baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>
Subject: Re: [PATCH v2 08/10] iommufd/device: Use iommu_group_replace_domain()
Message-ID: <Y+WJPH3pbYLIs2iv@nvidia.com>
References: <cover.1675802050.git.nicolinc@nvidia.com>
 <4653f009c3dacae8ebf3a4865aaa944aa9c7cc7e.1675802050.git.nicolinc@nvidia.com>
 <BN9PR11MB5276C1807B710CAD3E5820D78CD99@BN9PR11MB5276.namprd11.prod.outlook.com>
 <Y+Vh479cDD7LX2x/@Asurada-Nvidia>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Y+Vh479cDD7LX2x/@Asurada-Nvidia>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?OR1occMZ9BTqFKd2JbUsoUuEALpoaGRDGwCzOD9fooq6BsaSNReRzLuG4U+s?=
 =?us-ascii?Q?8SccrwIp7wv98iGY3TIn005DB2QD2H1eCbONw5qoXJnevMgr3OvcXwcKyEel?=
 =?us-ascii?Q?I+mUmrCreTF7eAvNq8ARVBbfOKgwSCNRiEavAFdRrPF7fs4E+bkzGqFOB5OA?=
 =?us-ascii?Q?ppobOMZzqLBquSVPLT3V3WHrAi461vY7MXW/tG7xqhhtT+s1NhfPq0QxALdQ?=
 =?us-ascii?Q?O9y6APhpvoGDA39iqie7S/SOhla0OY7irBlAqXpmoNLLQK9POGQ/aoiJoDTS?=
 =?us-ascii?Q?VO7qBFRsrtaUvzpwEpwMwSKz1KEA+tDTjMykN8Ex/Rv7E7iQELP4AkZX3HDw?=
 =?us-ascii?Q?H6VGF0gXlQNeUkB2JchD9jo/ueRfT8oWtt7coSXLohOUPHL/EhuK/2qE3WZB?=
 =?us-ascii?Q?/JAF84+8LMi7+fJYZY4Rd+nMcI6/EQavqaEDDfn+qu4VVBkJRzNOIN4tXEYM?=
 =?us-ascii?Q?2kcv5JCZH1ZJNGBGuEabaNJ2XTNLK7jr0fcYAoWzKA2dbQwaTaom+A7uT1NS?=
 =?us-ascii?Q?kwWZe3cLm6nKWJVbiNB2LloaS6eQGrpl+1iFDvrTzDd1wd4QG76kXK7c+2xy?=
 =?us-ascii?Q?TkITFEGuGmR0SUnD5LkmOJjTZuKFMF313Py01stEYSfOAy5kYXUsWAdKQHto?=
 =?us-ascii?Q?0Rl0+mIEL0WtYxx2RORpUaUrvei/hhS7L7kQc9He0zNWkfg0LFMlfQJgjU4B?=
 =?us-ascii?Q?DJRlHu9nPB7EXtigkDjizuYKOxXFNy1Z3gAK6FZtj2O8hJ+Wj0RY5QSyYU99?=
 =?us-ascii?Q?/whaaPoc820Emu8ZrcyIXfd3cpxkoO+cAx7EeIfErnoBBrn7s8PG/gCJukMj?=
 =?us-ascii?Q?mgMOMMjgRmUXxIEM8t3vCDnP/JmpqWPfvyON2E3QhOsaIk5KsjA6bb/AkVlS?=
 =?us-ascii?Q?nz9sp7dV2nNI7J1Rq13ADespEaNKjeMwPOHBzsbqaWHMtWDEKUw2cBCPPdHN?=
 =?us-ascii?Q?gppcXwiwJat1nmfOj7lkO6uI+n/2L+cf1PlamnCKjW6w7i01uUR+ddkaQkvM?=
 =?us-ascii?Q?7M+sNakjQcoKQqLfb7odNJ/hhC3fEMXlecZi74FvCyns1r4jfegFDpAK8ihj?=
 =?us-ascii?Q?vTRH/Vn346Hq+5wBapzP8gHokYR8SbbQr8hia+cYyst7lUMOlH7X5ZXVvhLW?=
 =?us-ascii?Q?JlWrA/K7LpXjYrCKolk0Z3Eq6jUCNVxLwNrK4StL2I9RBQ5u6FtGKEKdr4h/?=
 =?us-ascii?Q?cnM9XOenjy0N2U4P/tCLK2fmGyyVdaWHPVJYQavKa2U65y+aXA6deQosZmgO?=
 =?us-ascii?Q?zcc/yOGhEFIZ7ah9NZK+kiGgF8T8NHuKdZV+QVv522xVK1GEy+EoeXuf7xdo?=
 =?us-ascii?Q?9ZQKNYqFfdItStK2SGAUBpbN54kChfiof9KEa3tYuXS5aYhjYgsjrIaN0PDa?=
 =?us-ascii?Q?LpRM/oV9cBXVHPohM6b1B2/v1gPhAhOWujt7Sf8sDBj9rMtVIJP6K9WuKFPE?=
 =?us-ascii?Q?kJ1Cwgs1rDoEsPtC4Y/qZvbSMEEy/BEJFGtLsRfVu70uxCtzNNucXx28rN1A?=
 =?us-ascii?Q?qZOEtdCjqVW7RIs48y4mmdqTjUTED0GWecfYuzAAueoUqyED0FbRNWDqZkcH?=
 =?us-ascii?Q?W6Ri/nXz3ZXTndEyX6LeJxqVlLMS8GdIeamFh2/t?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7c66e62a-139d-499b-c3a2-08db0af9e506
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Feb 2023 00:01:00.9867
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: KSrIT5fqbrsckflnsTJ+mptT0fS7vB4PwP2lbZ4D33Q/rot3RFQMUmlJx5Pe4FOr
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6466
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thu, Feb 09, 2023 at 01:13:07PM -0800, Nicolin Chen wrote:
> On Thu, Feb 09, 2023 at 04:00:52AM +0000, Tian, Kevin wrote:
>  
> > > From: Nicolin Chen <nicolinc@nvidia.com>
> > > Sent: Wednesday, February 8, 2023 5:18 AM
> > >
> > > iommu_group_replace_domain() is introduced to support use cases where
> > > an
> > > iommu_group can be attached to a new domain without getting detached
> > > from
> > > the old one. This replacement feature will be useful, for cases such as:
> > > 1) vPASID mode, when a guest wants to replace a single pasid (PASID=0)
> > >    table with a larger table (PASID=N)
> > > 2) Nesting mode, when switching the attaching device from an S2 domain
> > >    to an S1 domain, or when switching between relevant S1 domains.
> > > as it allows these cases to switch seamlessly without a DMA disruption.
> > >
> > > So, call iommu_group_replace_domain() in the iommufd_device_do_attach().
> > > And add a __iommmufd_device_detach helper to allow the replace routine
> > > to
> > > do a partial detach on the current hwpt that's being replaced. Though the
> > > updated locking logic is overcomplicated, it will be eased, once those
> > > iopt_table_add/remove_ioas and list_add/del calls are moved to hwpt's
> > > allocation/destroy() functions in the coming nesting series, as that'll
> > > depend on a new ->domain_alloc_user op in the iommu core.
> > 
> > then why not moving those changes into this series to make it simple?
> 
> The simplification depends on the new ->domain_alloc_user op and
> its implementation in SMMU driver, which would be introduced by
> the nesting series of VT-d and SMMU respectively.

Since we are not fixing all the drivers at this point, this argument
doesn't hold water.

It is as I said in the other email, there should be no changes to the
normal attach/replace path when adding manual HWPT creation - once
those are removed there should be minimal connection between these two
series.

Jason