Date:   Fri, 10 Mar 2023 08:55:17 -0400
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Nicolin Chen <nicolinc@nvidia.com>
Cc:     Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>,
        Jean-Philippe Brucker <jean-philippe@linaro.org>,
        "robin.murphy@arm.com" <robin.murphy@arm.com>,
        "will@kernel.org" <will@kernel.org>,
        "eric.auger@redhat.com" <eric.auger@redhat.com>,
        "kevin.tian@intel.com" <kevin.tian@intel.com>,
        "baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>,
        "joro@8bytes.org" <joro@8bytes.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "iommu@lists.linux.dev" <iommu@lists.linux.dev>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "yi.l.liu@intel.com" <yi.l.liu@intel.com>
Subject: Re: [PATCH v1 02/14] iommufd: Add nesting related data structures
 for ARM SMMUv3
Message-ID: <ZAsotUK6d4L2ULUq@nvidia.com>
References: <cover.1678348754.git.nicolinc@nvidia.com>
 <364cfbe5b228ab178093db2de13fa3accf7a6120.1678348754.git.nicolinc@nvidia.com>
 <20230309134217.GA1673607@myrica>
 <213a05ef06944a55a148995053b187f8@huawei.com>
 <ZAn94BXkjiJKQ66O@nvidia.com>
 <39979642ba8042ba9d4de651fecfeffb@huawei.com>
 <ZAoCf9+kpMof6R+4@nvidia.com>
 <4c1f57adbe0c4d48849d585508595939@huawei.com>
 <ZAq/n4PNeow6wftN@Asurada-Nvidia>
 <ZArB0n0J3GW75WNz@Asurada-Nvidia>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ZArB0n0J3GW75WNz@Asurada-Nvidia>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?rhlWMmrtIl/rFr/AqvP6V7vDjiDSiVL9zOms8ErkEmalTZAWDPog4U5Lyb8G?=
 =?us-ascii?Q?IdVrZJTmFrOWRgcRGQR0/luoFGNvdJNrgJRjmHdeq83ljrFJW30atrWRMlfZ?=
 =?us-ascii?Q?EhKtDYqdbJjIA3oVgsMiP14F0jINkng/OBzGwsJ/wraKmrEiY/0+nHAovOCi?=
 =?us-ascii?Q?xExL/o6ynsJWJstkj6wFcZF+QCFYIA2wSDlPunklcmTZgNpe1O0QH3LoRHSK?=
 =?us-ascii?Q?S4K8RWbdRg01wTo6evjzm467q8OzMSMHe/rkrb/R2JJAak7MtCyiWLBI50Bx?=
 =?us-ascii?Q?VjoDME+OcnxxFuTqhYqJ/ZoBCjazcvzhWhS1a88ZF7tc57uDQpu1HZH2UwS5?=
 =?us-ascii?Q?H70W466sFj9WtNg+CAPsMd26pgfLqwvD36catBYKQ8nzy1xWYQXOi+BuXc0J?=
 =?us-ascii?Q?vrVmgl0CSQtd/OOh9S1mzqwQcJ1YSTVcT/ChsfwlnzdB/IvdPMEoXUBxJOAJ?=
 =?us-ascii?Q?8NrTg/9m7P5sH8Pyan4kriMPMihc8CiX6CgsEp2wskBuLeNQDRRYEQ7hgcSL?=
 =?us-ascii?Q?lBcMwR1cJaq3FhR+PpImhl9ZVClble/ibHeVwYvgtns0H5/X8W6k2ce63M2J?=
 =?us-ascii?Q?7wOBq/hloCemk1PSqeEgJy7EZ0hxkIwxdWlZzoSWn2n5qb3Q7D9BcVi4+In+?=
 =?us-ascii?Q?GBw68vL0KC28cs7P+PEQDdad/wlVN7PdIXa3J/QUFTxhS7eWfcpCj9Xb/zfB?=
 =?us-ascii?Q?Hc/4vUjqLraChLMXf2KmdSvzbLBT2Cfi4/NBK37P7LN/RRj6N5gLLAaXtfso?=
 =?us-ascii?Q?UvVVInh+ZUVP+G/BqvTtE9ZP1LbaqGvTs9XFOjYCQZmi4bTzERjtxpAZ/ld1?=
 =?us-ascii?Q?iOSyuGrGwtZRWKPmG06IGi/bRF+J9waAzVRC67QRe7PAaT5Eq39L8RtsMF5R?=
 =?us-ascii?Q?vX2z3hjwnq7idz1xcKjj0WFgvQGgCz4L6OmrygMymEA7IO/6flX+SwByihJJ?=
 =?us-ascii?Q?AqntEHI43ZVqkOxyKPsdSQ+GecR0gK7JipOwSkJG5oQRedCWHtjxX/AITCb+?=
 =?us-ascii?Q?8ezsDTBS7Q9fnn7UOEvl4llYP2H4LF/c8DnNnXEFv9Qz4wiJ44tnffz8FVcJ?=
 =?us-ascii?Q?ZWm8EVQK28FUX1aNRhPzBasqNg4DMi4rfjz+iQDnzYUGBspJzRMiVRih3mig?=
 =?us-ascii?Q?jHp1zKLtqJVLTRneFVMa7jCiXqqjAoNezc7TiaatOpIT0MvuaJSApP8EEJn6?=
 =?us-ascii?Q?BXevMZcYObiMMcKRoMSTa+RA72J8O+iH7gcuiHvksq1xKXLtRFlT8JRCcQFn?=
 =?us-ascii?Q?cz6B0HrzvTKfGYFWUEQA/Bj2SLkoCQ/aTl40WepXx3Grp8d2PX70AcokvZXB?=
 =?us-ascii?Q?0Dz3NRc5MzDsQVemqmzWfYuU9YlTjOUK+F+dniWa74eVocCEGZhmnYxVKtzC?=
 =?us-ascii?Q?TVaup5AHOE1gQBBQpKCs5tECa/qmUIjgZrZidHuneyt2nyTOR0msq71my26y?=
 =?us-ascii?Q?V8mivF9qgivDltt1MTR5wTq9uHCxDHxPLxLw+J6uli1dkgP3DXHLSDwr9BX1?=
 =?us-ascii?Q?aksjb5hxuXTIFmA/Sj/SQfM5s4IBfP+WD/DYz0nKZJHoAyKAphwmvwb5SN2n?=
 =?us-ascii?Q?M8d4iJyw4sO8kzFNf69i+aJ08DDE2nsmSFQGATPF?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2fbc8e6f-8b47-4373-7fcb-08db2166b50c
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Mar 2023 12:55:21.2495
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: nHuUmPd9ofZKTqLgKrs4G0GHxv/OXRyCVA3DwbOAucu3Cl1aAEdEtng7wStpkuYT
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4941
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thu, Mar 09, 2023 at 09:36:18PM -0800, Nicolin Chen wrote:
> > Yea, a vmid is only allocated in an S2 domain allocation. So,
> > a guest allocating only S1 domains always sets VMID=0. Yet, I
> > think that the hypervisor or some where in host kernel should
> > replace the VMID=0 with a unified VMID.
> 
> Ah, I just recall a conversation with Jason that a VM should only
> have one S2 domain. In that case, the VMID is already unified?

Not requried per-say, but yes, most likely qemu would run that way.

But you can't just re-use the VMID however you like. AFAIK the VMID is
the cache tag for the S2 IOPTEs, so every VMID must refer to the same
S2 translation.

You can't mix different S2's with the same VMID.

Thus you are stuck with the single S2 model in qemu if you want to use
a userspace CMDQ.

I suppose that suggests that if KVM supplies the VMID then it is
assigned to a singular S2 iommu_domain also.

Jason