Received-SPF: pass (google.com: domain of linux-kernel+bounces-18348-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Date: Fri, 5 Jan 2024 20:42:31 +0000
From: Oliver Upton <oliver.upton@linux.dev>
To: Catalin Marinas <catalin.marinas@arm.com>
Cc: Lorenzo Pieralisi <lpieralisi@kernel.org>,
	Jason Gunthorpe <jgg@nvidia.com>, ankita@nvidia.com, maz@kernel.org,
	suzuki.poulose@arm.com, yuzenghui@huawei.com, will@kernel.org,
	alex.williamson@redhat.com, kevin.tian@intel.com,
	yi.l.liu@intel.com, ardb@kernel.org, akpm@linux-foundation.org,
	gshan@redhat.com, linux-mm@kvack.org, aniketa@nvidia.com,
	cjia@nvidia.com, kwankhede@nvidia.com, targupta@nvidia.com,
	vsethi@nvidia.com, acurrid@nvidia.com, apopple@nvidia.com,
	jhubbard@nvidia.com, danw@nvidia.com, mochs@nvidia.com,
	kvmarm@lists.linux.dev, kvm@vger.kernel.org,
	linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org,
	james.morse@arm.com
Subject: Re: [PATCH v3 2/2] kvm: arm64: set io memory s2 pte as normalnc for
 vfio pci devices
Message-ID: <ZZhpt8vdlnOP7i82@linux.dev>
References: <20231208164709.23101-1-ankita@nvidia.com>
 <20231208164709.23101-3-ankita@nvidia.com>
 <ZXicemDzXm8NShs1@arm.com>
 <20231212181156.GO3014157@nvidia.com>
 <ZXoOieQN7rBiLL4A@linux.dev>
 <ZXsjv+svp44YjMmh@lpieralisi>
 <ZXszoQ48pZ7FnQNV@linux.dev>
 <ZYQ7VjApH1v1QwTW@arm.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ZYQ7VjApH1v1QwTW@arm.com>

On Thu, Dec 21, 2023 at 01:19:18PM +0000, Catalin Marinas wrote:

[...]

> > Apologies, I didn't mean to question what's going on here from the
> > hardware POV. My concern was more from the kernel + user interfaces POV,
> > this all seems to work (specifically for PCI) by maintaining an
> > intentional mismatch between the VFIO stage-1 and KVM stage-2 mappings.
> 
> If you stare at it long enough, the mismatch starts to look fine ;).
> Even if you have the VFIO stage 1 Normal NC, KVM stage 2 Normal NC, you
> can still have the guest setting stage 1 to Device and introduce an
> architectural mismatch. These aliases have some bad reputation but the
> behaviour is constrained architecturally.
> 
> IMHO we should move on from this attribute mismatch since we can't fully
> solve it anyway and focus instead on what the device, system can
> tolerate, who's responsible for deciding which MMIO ranges can be mapped
> as Normal NC.

Fair enough :) The other slightly unsavory part is that we're baking
the mapping policy into KVM. I'd prefer it if this policy were kept in
userspace somehow, but there's no actual usecase for userspace selecting
memory attributes at this point.

> If we really want to avoid any aliases (though I think we are spending
> too many cycles on something that's not a real issue), the only way is
> to have fd-based mappings in KVM so that there's no VMM alias. After
> that we need to choose between (2) and (3) since the VMM may no longer
> be able to probe the device and figure out which ranges need what
> attributes.

These are the sorts of things I was more worried about. I completely
agree that the patches are fine for relaxing the 'simple' PCIe use
cases, I just don't want to establish the precedent that the kernel/KVM
will be on the hook to work out more complex use cases that may require
the composition of various mappings.

But I'm happy to table that discussion until the usecase arises :)

-- 
Thanks,
Oliver