Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <a1cff65b-b390-3872-25b5-dd6bbfb3524c@linux.intel.com>
Date:   Thu, 8 Jun 2023 11:03:59 +0800
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.11.0
Cc:     baolu.lu@linux.intel.com
Subject: Re: Question about reserved_regions w/ Intel IOMMU
Content-Language: en-US
To:     Alexander Duyck <alexander.duyck@gmail.com>,
        LKML <linux-kernel@vger.kernel.org>,
        linux-pci <linux-pci@vger.kernel.org>, iommu@lists.linux.dev
References: <CAKgT0UezciLjHacOx372+v8MZkDf22D5Thn82n-07xxKy_0FTQ@mail.gmail.com>
 <CAKgT0UfMeVOz6AOqSvVvzpsedGDiXCNQrjM+4KDv7qJJ1orpsw@mail.gmail.com>
From:   Baolu Lu <baolu.lu@linux.intel.com>
In-Reply-To: <CAKgT0UfMeVOz6AOqSvVvzpsedGDiXCNQrjM+4KDv7qJJ1orpsw@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk

On 6/8/23 7:03 AM, Alexander Duyck wrote:
> On Wed, Jun 7, 2023 at 3:40 PM Alexander Duyck
> <alexander.duyck@gmail.com> wrote:
>>
>> I am running into a DMA issue that appears to be a conflict between
>> ACS and IOMMU. As per the documentation I can find, the IOMMU is
>> supposed to create reserved regions for MSI and the memory window
>> behind the root port. However looking at reserved_regions I am not
>> seeing that. I only see the reservation for the MSI.
>>
>> So for example with an enabled NIC and iommu enabled w/o passthru I am seeing:
>> # cat /sys/bus/pci/devices/0000\:83\:00.0/iommu_group/reserved_regions
>> 0x00000000fee00000 0x00000000feefffff msi
>>
>> Shouldn't there also be a memory window for the region behind the root
>> port to prevent any possible peer-to-peer access?
> 
> Since the iommu portion of the email bounced I figured I would fix
> that and provide some additional info.
> 
> I added some instrumentation to the kernel to dump the resources found
> in iova_reserve_pci_windows. From what I can tell it is finding the
> correct resources for the Memory and Prefetchable regions behind the
> root port. It seems to be calling reserve_iova which is successfully
> allocating an iova to reserve the region.
> 
> However still no luck on why it isn't showing up in reserved_regions.

Perhaps I can ask the opposite question, why it should show up in
reserve_regions? Why does the iommu subsystem block any possible peer-
to-peer DMA access? Isn't that a decision of the device driver.

The iova_reserve_pci_windows() you've seen is for kernel DMA interfaces
which is not related to peer-to-peer accesses.

Best regards,
baolu