Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Tue, 13 Jun 2023 12:54:19 -0300
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Robin Murphy <robin.murphy@arm.com>
Cc:     Baolu Lu <baolu.lu@linux.intel.com>,
        Alexander Duyck <alexander.duyck@gmail.com>,
        LKML <linux-kernel@vger.kernel.org>,
        linux-pci <linux-pci@vger.kernel.org>, iommu@lists.linux.dev
Subject: Re: Question about reserved_regions w/ Intel IOMMU
Message-ID: <ZIiRK2Dzl2/9Jqle@ziepe.ca>
References: <CAKgT0UezciLjHacOx372+v8MZkDf22D5Thn82n-07xxKy_0FTQ@mail.gmail.com>
 <CAKgT0UfMeVOz6AOqSvVvzpsedGDiXCNQrjM+4KDv7qJJ1orpsw@mail.gmail.com>
 <a1cff65b-b390-3872-25b5-dd6bbfb3524c@linux.intel.com>
 <b24a6c7b-27fc-41c0-5c82-15696b4a7dc1@arm.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <b24a6c7b-27fc-41c0-5c82-15696b4a7dc1@arm.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?xFyyQBKNLSsV59It5gzIYK+VS7ISJoLc5nRVdboTFTXIL1OyqdcPDvmPUFzV?=
 =?us-ascii?Q?MuqDWkrc8hDd3h9IvN0xr4D0VzwIcpOXFmFd3kG3oDoIsQB0eCU0pDTZ2fTO?=
 =?us-ascii?Q?zqqgP/KL30Tc6HzJpdoWGzx7ls2hKgPtxvq1P/qZPrfSNFu+8/wJDHeYvuAc?=
 =?us-ascii?Q?IE3CAbcLHc88tCtfh3cyJlTLOG9OiUWDnFpIQ9TNVngMfuA86t5HH5jTE5J4?=
 =?us-ascii?Q?iLaej/CQzzLcCcCrP7fLbHOdoBMJSJkH+Mh1ArHYVjYMtKS5vMrpCdLnB5Ce?=
 =?us-ascii?Q?u/4SsW0JMvcGpEpX1Kl7l4BStMXccp2wqZK3SE8F7mF2y8JrIIhfOmgvGtpt?=
 =?us-ascii?Q?efeA8yH4gFLf9VDQqXwGCI2iDiZ5cuQOgJf2MU0mrtINZ7r9TGglDbPy4ZP5?=
 =?us-ascii?Q?c+n6GNIMFSmG/wz0rzJMXbIEb5MTx2QY5sokhloIfw7ireuyQHu5xZVixK0b?=
 =?us-ascii?Q?9KWi8XidSZ+jz1oYeMU/V8gPspiwX183IanV4Q8kkIBnKg0YzSMk3yg3blzk?=
 =?us-ascii?Q?qLqP6/ZsB7UkBkyGtZnUCQMrta0ZUgq2qTWpvgWhFTfvTiYq7gGW3ojB1J3m?=
 =?us-ascii?Q?2Qp2Mx20+JdGcHO/uRvGAiBdxKvWMvXdCg2NWK72Sf4JpH2RW/kRyl4IFS/6?=
 =?us-ascii?Q?keBhJZYis8y/ig4SMYYEMpqZfFu63DmBoMnpwXHoU4sO1QeOqeISc5ZE4qcb?=
 =?us-ascii?Q?sUKMFPe4zusTo2Y3TlDUCRLzyKjJl7+OPuxVWI9YK4/AxUh83uF1uIzPqgYf?=
 =?us-ascii?Q?gqsPNIyinUrfgBbbQ0PZC2DrMxkGL6RKcbu6hJs1Ig3xoLvHy9y/Iu2o2Rm4?=
 =?us-ascii?Q?WMA9KcumHUkMrNlFIbiCEKnS1VEARGJsKPJeU8baHTTuZZSmzOG1l8nNZDvE?=
 =?us-ascii?Q?EHcNvbSBeUgGtFGOLRKW7Ug8MDml07btgRwWgbct0IyF86onLpQ6OlI+Rcw/?=
 =?us-ascii?Q?rzk9iOEFeUyNG62ACvL8UR638521adWHegm4WesLJAWdDZ2e+4aTDu/DelwT?=
 =?us-ascii?Q?yTfCNd/5m2H6nf/gwUp6wRP0/DdhasoGM54/01qTdbgqGefZinAqSkczBYut?=
 =?us-ascii?Q?U0hax9VBWtFoTZu9wcqphT/ktjnH28pVGdcBVxHz0PgzPw1j5/YKJGuaPzp2?=
 =?us-ascii?Q?JFjQO04LysyRSlCMoCnCfAsdZP7B5G6MXIWe3Pje9Nf3o3t8/lad1jqr+SmR?=
 =?us-ascii?Q?3OXQdIPIzqfKpWDCdmEwamW3ttuzwsCXMgu54aTdgrxkFeP651xaNo4ihPjS?=
 =?us-ascii?Q?FYbqsCBV9Ux2TyAMy2scWracfqcAVQQYi455U6sPRTEj4hQT/T29+MtQvXV+?=
 =?us-ascii?Q?s51qGWLEHOyLtcBwia/3GnLAXLaD0tQ6BFGgFwEM8MzJRVZsfCU0BMXytk+U?=
 =?us-ascii?Q?tO9m/gbMzPxBKukcdYa7MvaeD1lG8Bp9IEl0AkjwdOpgpK8mMI8b6mQVoIAu?=
 =?us-ascii?Q?9n4KacMj9W3TmmD6Y1d82t9xYSpzjnbdqXmIBn35VH2e5yRLumokfH8HnfHH?=
 =?us-ascii?Q?CW/Ca03+lQlofXmcyihf5btYjpStPaCWufmh+wwBUj+xiB5FX+mwOswtNN+q?=
 =?us-ascii?Q?e5xbZ3AMaRstiKpwn1aahUwVcd5/E7XVjW/CfRJ6?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8eba0d27-9f4e-4afc-550c-08db6c2673b0
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jun 2023 15:54:21.3393
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 9pHomoj0VDrqQOJS0OSkr86yUjOt/WAIUGQfgLNxllq18IrIVfrGfHJQQDqHrRnE
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6628
X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,
        T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=no autolearn_force=no
        version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thu, Jun 08, 2023 at 04:28:24PM +0100, Robin Murphy wrote:

> > The iova_reserve_pci_windows() you've seen is for kernel DMA interfaces
> > which is not related to peer-to-peer accesses.
> 
> Right, in general the IOMMU driver cannot be held responsible for whatever
> might happen upstream of the IOMMU input. 

The driver yes, but..

> The DMA layer carves PCI windows out of its IOVA space
> unconditionally because we know that they *might* be problematic,
> and we don't have any specific constraints on our IOVA layout so
> it's no big deal to just sacrifice some space for simplicity.

This is a problem for everything using UNMANAGED domains. If the iommu
API user picks an IOVA it should be able to expect it to work. If the
intereconnect fails to allow it to work then this has to be discovered
otherwise UNAMANGED domains are not usable at all.

Eg vfio and iommufd are also in trouble on these configurations.

We shouldn't expect every iommu user to fix this entirely on their
own.

> We don't want to have to go digging any further into bus-specific
> code to reason about whether the right ACS capabilities are present
> and enabled everywhere to prevent direct P2P or not. Other use-cases
> may have different requirements, though, so it's up to them what
> they want to do.

I agree the dma-iommu stuff doesn't have to be as precise as other
places might want (but also wouldn't be harmed by being more precise)

But I can't think of any place that can just ignore this and still be
correct..

So, I think it make sense that the iommu driver not be involved, but
IMHO the core code should have APIs to report IOVA that doesn't work
and every user of UNMANAGED domains needs to check it.

IOW it should probably come out of the existing reserved regions
interface.

Jason