Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   "Tian, Kevin" <kevin.tian@intel.com>
To:     Jason Gunthorpe <jgg@nvidia.com>,
        Robin Murphy <robin.murphy@arm.com>,
        Alex Williamson <alex.williamson@redhat.com>
CC:     Baolu Lu <baolu.lu@linux.intel.com>,
        Alexander Duyck <alexander.duyck@gmail.com>,
        LKML <linux-kernel@vger.kernel.org>,
        linux-pci <linux-pci@vger.kernel.org>,
        "iommu@lists.linux.dev" <iommu@lists.linux.dev>
Subject: RE: Question about reserved_regions w/ Intel IOMMU
Thread-Topic: Question about reserved_regions w/ Intel IOMMU
Thread-Index: AQHZmZR0JpzuQIuRD0G85gaP7++jm6+AOJSAgADP/QCAB+LmgIAEOrcw
Date:   Fri, 16 Jun 2023 08:39:46 +0000
Message-ID: <BN9PR11MB52765C24405D2475CF3CBEBE8C58A@BN9PR11MB5276.namprd11.prod.outlook.com>
References: <CAKgT0UezciLjHacOx372+v8MZkDf22D5Thn82n-07xxKy_0FTQ@mail.gmail.com>
 <CAKgT0UfMeVOz6AOqSvVvzpsedGDiXCNQrjM+4KDv7qJJ1orpsw@mail.gmail.com>
 <a1cff65b-b390-3872-25b5-dd6bbfb3524c@linux.intel.com>
 <b24a6c7b-27fc-41c0-5c82-15696b4a7dc1@arm.com> <ZIiRK2Dzl2/9Jqle@ziepe.ca>
In-Reply-To: <ZIiRK2Dzl2/9Jqle@ziepe.ca>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?yFpPsARGunojYpUy5CXUGTNvVTnCLkBXfWkrZfC0I66UZfqbLCwBIMQCrmBu?=
 =?us-ascii?Q?+ySf/xPLkmkFhQI6iXjy/NXg5dEpmG44W2svcM5cXWZw1ZNJG2KemPOcm56m?=
 =?us-ascii?Q?idX9ro30x+aisT3jw7QH57p+yp57S8uW58nFn6+Vr26edWVowWCQ3KCgN79/?=
 =?us-ascii?Q?kjZ94QohNgQzthx9LhRjZzBUa3FI0Q1d3z/SNZC/+FG0jrNIOpl49KfTohX7?=
 =?us-ascii?Q?Ie6DnOuiVKgK5COBSa3awxlv50M5baGQOvs5KcxWuv68GOUFSnRryifakX2Q?=
 =?us-ascii?Q?jikTAFjvGN+mFJR9yXezw5oGtlD1HLXywXmNpqDg/OeWZ4N27UDnshI1GT73?=
 =?us-ascii?Q?IQo4BqXXcYM8Iy45TKlKdciRruP/lRUTCgyAd0xcvVzQZ6WIfT1H4JpDDOHF?=
 =?us-ascii?Q?RpyG5peN1SBD5cEp23lyw+glKZdT82+VSJNFx1aU54+qq2tTX6JImdi5jzvN?=
 =?us-ascii?Q?zCctedp1ZgaX+nZLXF3eXWWZ0C4PVPm33zQEsK1Vy+PV+ncpiaAhLeMst7xa?=
 =?us-ascii?Q?GFFYQNh7dTYvu+C5EZr/sjSknmO7rVBlZ3FGWnRz95rDheC5MNioS+dEd8SY?=
 =?us-ascii?Q?fTu2AZHynMZoI3KV3wJvn3O5vlOdC+QV1vQE+hTCBtQypuzMp6dmmuFhCMlY?=
 =?us-ascii?Q?Qtz6FDa9ESDGmC9chJF4Dy7smKXwCYWEV3d/mUqx7yp2IexKEowrgiDLTLUt?=
 =?us-ascii?Q?Jrr4dcEOzAtfhh5g29/FpK6yGcqrWCh0DvaMRrF9K9Lnjm7qbpT+KiNDuk4U?=
 =?us-ascii?Q?+EbJyLqawbKvIT7f3wl+t7PzcoCo/dYGr8Djrxr6WC4l9XhK56odf+nzzjjT?=
 =?us-ascii?Q?tWQIqbvTOY8CdM5eSBUH+uK0ciqvO8WqDE40/H3t9AF74jeZNkb3aPlHXvi6?=
 =?us-ascii?Q?RlohaR30GX2M2KwHqTuZmgjR0kP8vrzxy558nY21DVzDdqhFNpP/dZMBCAkU?=
 =?us-ascii?Q?9wQ32PBRRNgivAmuFIMzL2RpI2h9UqUEICnuaFyUbzajU3TCu2YljRk81EkW?=
 =?us-ascii?Q?Uvt1Mt/8JkIRECpfFW/bOfxS+43BJ066vsL7JIaIInV4PXwZ4lsdwVXfrmIT?=
 =?us-ascii?Q?a64g1B66eRcZsY1RXA07ARvPDEbytmz2etLahDKbjvm89yINkoZJPfiYJ5f9?=
 =?us-ascii?Q?XsuKi5aQYX/NW0kNnm0wxngPFqN1ETODkePTSmwuxD/mz8B3I+YztgLIU7cp?=
 =?us-ascii?Q?5evxXx7h8sKtrFspSBkh2jk/YhWWz9Oem/zwIJodSmYhM5t99fsz6HZ8TQZ0?=
 =?us-ascii?Q?tfp5IVCuz/s10GzIfR+FzP3L84LZrWYOnkQjNtDkqKxWqJYrDBbCqumL0S2U?=
 =?us-ascii?Q?zz0vaQBfCaut78SLUFJuY6YMhlBZUUjrYLdYgm9MNA7bV6+KNwVAZw3k1x8D?=
 =?us-ascii?Q?RY2N+jTKzu3M9VjMc+fGHVRW+kRedmz2/PdbDpWOCTpkGGFQijJwbE2994+e?=
 =?us-ascii?Q?vw7ptS5/mWDMy9Q016Q9kTFWc0JkvD2alUXRMMD1+jRIIOJ0eFkgGTeQ+Cra?=
 =?us-ascii?Q?tIh5A9txqr7GaP58k/wvOG2e/UfBeqgn7+DepnCSRCnh/3ELXJe/7WjJ3C6+?=
 =?us-ascii?Q?/8XVd1rN6okOHCB9SkbXRwd9+FRTX0CH+Gxvzcx6?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5276.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9c77e599-48c8-4aa6-f534-08db6e453d8b
X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Jun 2023 08:39:46.7104
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 8AEgxb+BHw07ZI4TDf7nnm46DZV02gSYyHds1cQuaMf+npumahV1JdHRVZVpJpWGDzqXMOUIhEe03QV+sCzu+g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR11MB6714
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        SPF_HELO_PASS,SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

+Alex

> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Tuesday, June 13, 2023 11:54 PM
>=20
> On Thu, Jun 08, 2023 at 04:28:24PM +0100, Robin Murphy wrote:
>=20
> > > The iova_reserve_pci_windows() you've seen is for kernel DMA interfac=
es
> > > which is not related to peer-to-peer accesses.
> >
> > Right, in general the IOMMU driver cannot be held responsible for
> whatever
> > might happen upstream of the IOMMU input.
>=20
> The driver yes, but..
>=20
> > The DMA layer carves PCI windows out of its IOVA space
> > unconditionally because we know that they *might* be problematic,
> > and we don't have any specific constraints on our IOVA layout so
> > it's no big deal to just sacrifice some space for simplicity.
>=20
> This is a problem for everything using UNMANAGED domains. If the iommu
> API user picks an IOVA it should be able to expect it to work. If the
> intereconnect fails to allow it to work then this has to be discovered
> otherwise UNAMANGED domains are not usable at all.
>=20
> Eg vfio and iommufd are also in trouble on these configurations.
>=20

If those PCI windows are problematic e.g. due to ACS they belong to
a single iommu group. If a vfio user opens all the devices in that group
then it can discover and reserve those windows in its IOVA space. The
problem is that the user may not open all the devices then currently
there is no way for it to know the windows on those unopened devices.

Curious why nobody complains about this gap before this thread...