Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <d2e591c1-eb43-377b-d396-8335f77acef6@arm.com>
Date:   Tue, 30 May 2023 13:14:41 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:102.0) Gecko/20100101
 Thunderbird/102.11.0
Subject: Re: [PATCH 2/2] arm64: Notify on pte permission upgrades
Content-Language: en-GB
To:     Jason Gunthorpe <jgg@nvidia.com>,
        Alistair Popple <apopple@nvidia.com>
Cc:     Andrew Morton <akpm@linux-foundation.org>, will@kernel.org,
        catalin.marinas@arm.com, linux-mm@kvack.org,
        linux-kernel@vger.kernel.org, nicolinc@nvidia.com,
        linux-arm-kernel@lists.infradead.org, kvm@vger.kernel.org,
        John Hubbard <jhubbard@nvidia.com>, zhi.wang.linux@gmail.com,
        Sean Christopherson <seanjc@google.com>
References: <3cece716fc09724793aa832e755abfc9d70a8bb3.1684892404.git-series.apopple@nvidia.com>
 <5d8e1f752051173d2d1b5c3e14b54eb3506ed3ef.1684892404.git-series.apopple@nvidia.com>
 <ZHKaBQt8623s9+VK@nvidia.com> <87pm6ii6qi.fsf@nvidia.com>
 <ZHXj/6Bjraxqk4YR@nvidia.com>
From:   Robin Murphy <robin.murphy@arm.com>
In-Reply-To: <ZHXj/6Bjraxqk4YR@nvidia.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 2023-05-30 12:54, Jason Gunthorpe wrote:
> On Tue, May 30, 2023 at 06:05:41PM +1000, Alistair Popple wrote:
>>
>>>> As no notification is sent and the SMMU does not snoop TLB invalidates
>>>> it will continue to return read-only entries to a device even though
>>>> the CPU page table contains a writable entry. This leads to a
>>>> continually faulting device and no way of handling the fault.
>>>
>>> Doesn't the fault generate a PRI/etc? If we get a PRI maybe we should
>>> just have the iommu driver push an iotlb invalidation command before
>>> it acks it? PRI is already really slow so I'm not sure a pipelined
>>> invalidation is going to be a problem? Does the SMMU architecture
>>> permit negative caching which would suggest we need it anyhow?
>>
>> Yes, SMMU architecture (which matches the ARM architecture in regards to
>> TLB maintenance requirements) permits negative caching of some mapping
>> attributes including the read-only attribute. Hence without the flushing
>> we fault continuously.
> 
> Sounds like a straight up SMMU bug, invalidate the cache after
> resolving the PRI event.

No, if the IOPF handler calls back into the mm layer to resolve the 
fault, and the mm layer issues an invalidation in the process of that 
which isn't propagated back to the SMMU (as it would be if BTM were in 
use), logically that's the mm layer's failing. The SMMU driver shouldn't 
have to issue extra mostly-redundant invalidations just because 
different CPU architectures have different idiosyncracies around caching 
of permissions.

Thanks,
Robin.