Date:   Wed, 15 Feb 2023 08:51:18 -0400
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Nicolin Chen <nicolinc@nvidia.com>
Cc:     "Tian, Kevin" <kevin.tian@intel.com>,
        "joro@8bytes.org" <joro@8bytes.org>,
        "will@kernel.org" <will@kernel.org>,
        "robin.murphy@arm.com" <robin.murphy@arm.com>,
        "alex.williamson@redhat.com" <alex.williamson@redhat.com>,
        "shuah@kernel.org" <shuah@kernel.org>,
        "Liu, Yi L" <yi.l.liu@intel.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "iommu@lists.linux.dev" <iommu@lists.linux.dev>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "linux-kselftest@vger.kernel.org" <linux-kselftest@vger.kernel.org>,
        "baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>
Subject: Re: [PATCH v2 08/10] iommufd/device: Use iommu_group_replace_domain()
Message-ID: <Y+zVRrt7mGkqtBBo@nvidia.com>
References: <BN9PR11MB5276268D3ED0360913A05C368CDE9@BN9PR11MB5276.namprd11.prod.outlook.com>
 <Y+bc8OrWfw3Fq57n@Asurada-Nvidia>
 <BL1PR11MB5271180A1DE8C739455AD06D8CDD9@BL1PR11MB5271.namprd11.prod.outlook.com>
 <Y+nrTsIz2UHH/1lD@Asurada-Nvidia>
 <Y+pN/vtO8I/oiMOJ@nvidia.com>
 <Y+toXAUa2Cbwn7I0@Asurada-Nvidia>
 <BN9PR11MB5276CBCF780BD7C61154BF8B8CA39@BN9PR11MB5276.namprd11.prod.outlook.com>
 <Y+w8TJ2jfDATftbr@Asurada-Nvidia>
 <BN9PR11MB5276FA23E61977389D0C1A198CA39@BN9PR11MB5276.namprd11.prod.outlook.com>
 <Y+yGfXYlumqOU5xj@Asurada-Nvidia>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Y+yGfXYlumqOU5xj@Asurada-Nvidia>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?WREYtMbsyFq5JB/WMANd3OqAs+Mb1yKcowxuCBe0a0uaBS19bMuxllDT4FyE?=
 =?us-ascii?Q?TsKhKf0jhc/N7ZocWdkCFANpKurdOYb9ZtBmXC4Weqg/smnuf6V54HtkN84w?=
 =?us-ascii?Q?MDGduRwDwSJK96/LPR2HvgGDAIpnztl1dGhmZkZQvp0WeKYlp0PIOVPJo5dq?=
 =?us-ascii?Q?SnDywM/GCdoDsZ1mC4eJyz+h3NZvVQOl0k8f5u+pxA3M156D59jp/1uiE7tQ?=
 =?us-ascii?Q?sPu8DfprdXWh6TQVDNqKXNn5ec3iawfdbswJqbLWXQ8Domuc8xvTEnJRD1/m?=
 =?us-ascii?Q?GoY6qJ7Mb71scuY0gYTHG8oHVzJbihhTMVMteHaT8e2wRi3KXjZdMJahUot0?=
 =?us-ascii?Q?foEvF6oYkAFDS8XXi6CmGR5pTEyDjp0mSRWvvHfk8ZQpKtm2nQKXI7z7y0JD?=
 =?us-ascii?Q?LrV4ol5jo1+npdTN1aQl2l3yRI+A0oP8FiljcLHVoZSieBh0SPzqHwF73AQI?=
 =?us-ascii?Q?l+rPRGT36aogTn9NxqZZM+mEIMhM4nZE38HN5bQdaUEusYfqenA5TrYyMjb0?=
 =?us-ascii?Q?CG5/Cre8QxSio4No8qXpoDf6DFgcp28YuIeX3mp/dm3XztanSlRPZ5xnvE/5?=
 =?us-ascii?Q?Vv2Y2JP3N1eMkSh1Yp7mL3LcJWvJCKbn6QCeka2Y8f2W9aPzCFzTrmytC9Pi?=
 =?us-ascii?Q?vSTfATjat0AE//wymfjHgtVEF/WRwoliB9l95jy2fZxgLscaXFqYiBVqH3yt?=
 =?us-ascii?Q?eQsGJXAHHcBgtfHmy9SYW44mcm/YogSASKliZCUqOBtcqZ+kwdBnOSy95nVT?=
 =?us-ascii?Q?GECSpSQNKiJgsOZWJ1SjvQFyvAio0QzOxFRU9BJfMqSRGT0eUStVCmTjxmrW?=
 =?us-ascii?Q?zwVWXUkUfKNfYCtEqEzCrDnj+bATpx84UdqQUI7TYSIi8bFrxSgPRFouzeyK?=
 =?us-ascii?Q?NuQdHONYfPOgsJDRbkVq0ni71eQHZ/DvA51KLaQdGl3uqHXcaOriarBu4lO0?=
 =?us-ascii?Q?QqNzc4wMel2GdsxdkLCV2VF1BmabkmXBnxJSRTRNqzOmYt0x4jvjVPgP1Zk2?=
 =?us-ascii?Q?pASf1uHd4b34uSslPj5bO3hOukRDn+QYA6x7eamUb2AX1NRM39DUu1UTimvW?=
 =?us-ascii?Q?sclB/5yE8ttUoddkiy/jjEG1zosBxAayukBJ55VdKD+kuXARhLILOfN4BLZ0?=
 =?us-ascii?Q?GAGY3mHd4TSykwtx/JI4iFolllsq2NuV+IQrEZzDc+UIrC3qra6rYjPvjlCJ?=
 =?us-ascii?Q?99xi9GRAxiCFK7ZfK03CUKCKNy/sMYIQYUWKb2NTeHzQYlAPB06Sr3TpbGlv?=
 =?us-ascii?Q?t6D0qYXhEiJLWThyQ4FJu87TVwntRtPmaumQfFznqY30kSFpp6gGj33r0M35?=
 =?us-ascii?Q?7Z/avL6rKonaTxcu3r8Ur9RcBhhLtOVG3U1r+rSh+oEl8KZoS5wYouzD3a9a?=
 =?us-ascii?Q?BiLC1QSVZehEu971tP7YFXJY5hCe8SNu5C14uKgcUpdOYkNqQGAfUkU0RGQr?=
 =?us-ascii?Q?aYFq+kb13FYNTbpKz4eGsMkDFhbvpvSZqik4p1DFiWqrV52Cq2XZykn/evPe?=
 =?us-ascii?Q?KU1UQODQm0tnTrjbZyEkpHa/q94vzdHROIHGFiVW0IhefvxDY8NZoHLi+rbg?=
 =?us-ascii?Q?Mah4cm4gDqpZRxvm5hg1OW4UpfqHIbIdIesAes10?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bd37a8b6-ddac-4cd7-dc4c-08db0f535551
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Feb 2023 12:51:19.2593
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: +bPZYDPtFbP6fCW7x38u9P20qc6Ofruc/qmGXTRq23F5RXL++VLquvXbV3+3u2Wm
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV2PR12MB5989
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Tue, Feb 14, 2023 at 11:15:09PM -0800, Nicolin Chen wrote:

> But things will be out of control, if user space continues mapping
> something onto domain1's iopt for idev2, even after it is attached
> covertly to domain2's iopt by the replace routine. I wonder how
> kernel should handle this and keep the consistency between IOMMUFD
> objects and iommu_group.

I've been looking at this, the reason the locking is such a PITA is
because we are still trying to use the device list short cut.

We need to have a iommu group object instead then everything will work
smoothly.

Jason