Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates
 216.228.112.34 as permitted sender) receiver=protection.outlook.com;
 client-ip=216.228.112.34; helo=mail.nvidia.com;
Subject: Re: [RFC 1/2] vfio/pci: keep the prefetchable attribute of a BAR
 region in VMA
To:     Alex Williamson <alex.williamson@redhat.com>
CC:     Marc Zyngier <maz@kernel.org>, Will Deacon <will@kernel.org>,
        "Catalin Marinas" <catalin.marinas@arm.com>,
        Christoffer Dall <christoffer.dall@arm.com>,
        <linux-arm-kernel@lists.infradead.org>,
        <kvmarm@lists.cs.columbia.edu>, <linux-kernel@vger.kernel.org>,
        <kvm@vger.kernel.org>, Vikram Sethi <vsethi@nvidia.com>,
        Jason Sequeira <jsequeira@nvidia.com>
References: <20210429162906.32742-1-sdonthineni@nvidia.com>
 <20210429162906.32742-2-sdonthineni@nvidia.com>
 <20210429122840.4f98f78e@redhat.com>
 <470360a7-0242-9ae5-816f-13608f957bf6@nvidia.com>
 <20210429134659.321a5c3c@redhat.com>
From:   Shanker R Donthineni <sdonthineni@nvidia.com>
Message-ID: <e3d7fda8-5263-211c-3686-f699765ab715@nvidia.com>
Date:   Fri, 30 Apr 2021 06:25:08 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <20210429134659.321a5c3c@redhat.com>
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 8bit
Content-Language: en-US
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Apr 2021 11:25:20.8272
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 6c4336d8-031b-43ee-6d92-08d90bcaa3d2
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.34];Helo=[mail.nvidia.com]
X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT051.eop-nam11.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB3776
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Alex

On 4/29/21 2:46 PM, Alex Williamson wrote:
> If an alignment fault is fixed by configuring a WC mapping, doesn't
> that suggest that the driver performed an unaligned access itself and
> is relying on write combining by the processor to correct that error?
> That's wrong.  Fix the driver or please offer another explanation of
> how the WC mapping resolves this.  I suspect you could enable tracing
> in QEMU, disable MMIO mmaps on the vfio-pci device and find the invalid
> access.
>
>> We've two concerns here:
>>    - Performance impacts for pass-through devices.
>>    - The definition of ioremap_wc() function doesn't match the host
>> kernel on ARM64
> Performance I can understand, but I think you're also using it to mask
> a driver bug which should be resolved first.  Thank

We’ve already instrumented the driver code and found the code path for the unaligned
accesses. We’ll fix this issue if it’s not following WC semantics.

Fixing the performance concern will be under KVM stage-2 page-table control. We're
looking for a guidance/solution for updating stage-2  PTE based on PCI-BAR attribute.