Date:   Fri, 10 Mar 2023 11:56:36 -0400
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
Cc:     Eric Auger <eric.auger@redhat.com>,
        Nicolin Chen <nicolinc@nvidia.com>,
        "robin.murphy@arm.com" <robin.murphy@arm.com>,
        "will@kernel.org" <will@kernel.org>,
        "kevin.tian@intel.com" <kevin.tian@intel.com>,
        "baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>,
        "joro@8bytes.org" <joro@8bytes.org>,
        "jean-philippe@linaro.org" <jean-philippe@linaro.org>,
        "linux-arm-kernel@lists.infradead.org" 
        <linux-arm-kernel@lists.infradead.org>,
        "iommu@lists.linux.dev" <iommu@lists.linux.dev>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v1 01/14] iommu: Add iommu_get_unmanaged_domain helper
Message-ID: <ZAtTNK1NbMJ4iyE0@nvidia.com>
References: <cover.1678348754.git.nicolinc@nvidia.com>
 <9b1077601cace998533129327f5e7ad946752d29.1678348754.git.nicolinc@nvidia.com>
 <eddde530-cd37-f493-7f0f-c97905f0de64@redhat.com>
 <ZAtNrFAEHvmM4FOW@nvidia.com>
 <f61ca5693d8845eba0a2139324e52d98@huawei.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <f61ca5693d8845eba0a2139324e52d98@huawei.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?beIUcbfoO11TAq09WIg1WjftKaGvrBvUjjxmSekohRR7gn/DlV6/B067AMNC?=
 =?us-ascii?Q?Mzzaai41+YjrUVbf4uNeWPXdmWBtOqq5+2Yh3493TKmaVch/Z1HB8u6r+oC6?=
 =?us-ascii?Q?9I8IJ+SQxZsqaoIEdF3pQMcA6NuUti5awT9GEgrXjqJWiPHcxinyqAISot44?=
 =?us-ascii?Q?Dy5HkdxDwYVNMfG6lCepeHEm0fYmbs3zrrEBlYAEQ6Ic9rq3SP6OMuymbuIv?=
 =?us-ascii?Q?4306jifNmkFKxP90tZW/BWunnYZHGggvDbM4ny6MNdytMIR4a/1rc1yKZPn5?=
 =?us-ascii?Q?SlpArROAYUTm7qHRNTtDoMxMf7fXtq4CT1wY32ApjeVLliXbx8Mkh3Sv/pjJ?=
 =?us-ascii?Q?StiKRMTSi4epZhQPbh52JG+kvly29uPpC0YyHQ3FACQA44JmNcD0JT+fYZsP?=
 =?us-ascii?Q?UtFKkaVgYX0O2NyBH8EQNL2uMHiUCvra/OMIEmppeMSwsRtQFbFINMoHTPaj?=
 =?us-ascii?Q?i6oM21WaxTTMfCpnIDiy5aVIQzNQo4wp+8DP6bZq3u+fN8iVMGdwZHC08+/c?=
 =?us-ascii?Q?sjIP6ZQdcuYjMCL/cojrBzuTkZ1FxBg2UpUR6Q31PqnlGNRZ3cP2YlxUgWOj?=
 =?us-ascii?Q?3kSfGacWDFqkptii5Qs98LmiAjAa7mPpxz//pRx4x/+stmaohdO7/re7MJ11?=
 =?us-ascii?Q?0QxKcDzkYei+7vC00AmNYnbOpGtbmTqeqJIx+v7YcKk+nltULeT8oN4wPXos?=
 =?us-ascii?Q?pjMYpX6h/3n9UQwFROM9tCh2tXMoR5O9sHR5q23FHONNoWppxQOChHaOCMCc?=
 =?us-ascii?Q?MY7NzjZU3+48MPBkYhT3h08vyRaXQtNVwFXjFUn6KFreNIOKnk/Q/aCxoyWV?=
 =?us-ascii?Q?AIWl7DoFn64K7etzEJznGlpUC7d5cLkBKtS98gehOTwmqtPM/mUF6XzuE040?=
 =?us-ascii?Q?GiM+fLM7E61vEQNy8ks/YuhtYeFuRELbFWn8wLA4xL4yTh3oycnzSMY+Z9A+?=
 =?us-ascii?Q?AT1srk+9IBfCrYuPQmk68dG+avB3S2e8k/5BpeqUQJ8vctue6YtiAPexdIpR?=
 =?us-ascii?Q?m8mFv51q4eameJ48oBov43up498pmqQQx8OMkDlV6FLH0jE8IGABPdaGRC3E?=
 =?us-ascii?Q?F4jbPqIHb4j9C5VbquvxvMdXPkiqUAGDRQi6ViHq7ToaAnqC3ysW/SLIY58U?=
 =?us-ascii?Q?+BbprzCMmRwIS9a80DMgjFd9eLUh3QKG0L4JZCFqucz7LLp8ECyz+ayhPyQp?=
 =?us-ascii?Q?nBQTE27uB5hHINU0/RljrVFXp9CG4DmQwUjT+9toovxTbD71T9VEcXo5dC+c?=
 =?us-ascii?Q?mNwVDjm/rcBFyxWFzcwNZESVkctXpUPKUGhkN9gf4aEogN6MkMmj2wdT9psD?=
 =?us-ascii?Q?ZDze4n/ffx+ZRExLdBY8uOEc36APcQ0fQWw/gnyQjRFgORGxs8/95lxW1vq/?=
 =?us-ascii?Q?1S7jMe++TpRCRqJ1pFqquGpXObHtoZTIbCihWRuRn+KAAs7FWyoCeLkRrWl2?=
 =?us-ascii?Q?QrI+whjE0DVzMXRPL2OPZbeS1BgBWxVQSKByNHLya49l+4GDtp/uGZ4iQll3?=
 =?us-ascii?Q?InHobWW0x/WL1FYodoFb9R94sbbje+VQ4t+ZrQh2TVPMSpHAy2+SaJKS+eZL?=
 =?us-ascii?Q?rN0PLiOhYoamZUxom2AnMV9gMqrNPhW5VGQnTSja?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9f66cb4d-1787-4d51-e2a0-08db218008cc
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Mar 2023 15:56:39.1210
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: MemlDcusNIKxptbOoGoFsOLyReuH8F8ZWGtbi/DKyfO3ZdAR7NShAoviqy7ypDZz
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4422
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Fri, Mar 10, 2023 at 03:44:02PM +0000, Shameerali Kolothum Thodi wrote:
> 
> 
> > -----Original Message-----
> > From: Jason Gunthorpe [mailto:jgg@nvidia.com]
> > Sent: 10 March 2023 15:33
> > To: Eric Auger <eric.auger@redhat.com>
> > Cc: Nicolin Chen <nicolinc@nvidia.com>; robin.murphy@arm.com;
> > will@kernel.org; kevin.tian@intel.com; baolu.lu@linux.intel.com;
> > joro@8bytes.org; Shameerali Kolothum Thodi
> > <shameerali.kolothum.thodi@huawei.com>; jean-philippe@linaro.org;
> > linux-arm-kernel@lists.infradead.org; iommu@lists.linux.dev;
> > linux-kernel@vger.kernel.org
> > Subject: Re: [PATCH v1 01/14] iommu: Add iommu_get_unmanaged_domain
> > helper
> > 
> > On Fri, Mar 10, 2023 at 11:14:59AM +0100, Eric Auger wrote:
> > > Hi Nicolin,
> > >
> > > On 3/9/23 11:53, Nicolin Chen wrote:
> > > > The nature of ITS virtualization on ARM is done via hypercalls, so
> > > > kernel handles all IOVA mappings for the MSI doorbell in
> > > > iommu_dma_prepare_msi() and iommu_dma_compose_msi_msg(). The
> > current
> > > > virtualization solution with a 2-stage nested translation setup is
> > > > to do 1:1 IOVA mappings at stage-1
> > > Note that if we still intend to use that trick there is a known issue
> > > at kernel side that needs to be fixed.
> > >
> > > ARM DEN 0049E.b IORT specification mandates that when RMRs are
> > > present, the OS must preserve PCIe configuration performed by the boot
> > > FW.
> > 
> > This limitation doesn't seem necessary for this MSI stuff?
> > 
> > What is it for?
> 
> That is to make sure the Stream Ids specified in RMR are still valid and is not being
> reassigned by OS. The kernel checks for this(iort_rmr_has_dev()),
> https://lore.kernel.org/linux-arm-kernel/20220420164836.1181-5-shameerali.kolothum.thodi@huawei.com/

So "boot configration" is more like "don't change the RIDs"? Ie don't
enable SRIOV?

Jason