Received-SPF: pass (google.com: domain of linux-kernel+bounces-37508-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Reply-To: Sean Christopherson <seanjc@google.com>
Date: Wed, 24 Jan 2024 10:24:44 -0800
Precedence: bulk
Mime-Version: 1.0
Message-ID: <20240124182444.2598714-1-seanjc@google.com>
Subject: [ANNOUNCE] PUCK Notes - 2024.01.24 - Memtypes for non-coherent DMA
From: Sean Christopherson <seanjc@google.com>
To: Sean Christopherson <seanjc@google.com>
Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, 
	Jason Gunthorpe <jgg@nvidia.com>, Yan Zhao <yan.y.zhao@intel.com>, Kevin Tian <kevin.tian@intel.com>
Content-Type: text/plain; charset="UTF-8"

Recording and slides:

  https://drive.google.com/corp/drive/folders/18QbkitOXcZyYXpT558wXf9Hs-rQs8mhX?resourcekey=0-qOuxyhLUBUGlCwHrzPqAkQ

Key Takeways:

 - Intel CPU<->CPU accesses are coherent for guest WC/UC accesses, so KVM can
   honor guest PAT for all VMs without putting the host or guest at risk.  I.e.
   KVM x86 doesn't need new uAPI, we can simply delete the IPAT code.

 - Intel CPUs need an SFENCE after VM-Exit, but there's already an smp_mb()
   buried in srcu_read_lock(), and KVM uses SRCU to protect memslots, i.e. an
   SFENCE is guaranteed before KVM (or userspace) will access guest memory after
   VM-Exit.  TODO: add and use smp_mb__after_srcu_read_lock() to pair with
   smp_mb__after_srcu_read_unlock() and document the need for a barrier on Intel.

 - IOMMU (via VFIO/IOMMUFD) mappings need cache flush operations on map() and
   unmap() to prevent the guest from using non-coherent DMA to read stale data
   on x86 (and likely other architectures).

 - ARM's architecture doesn't guarantee coherency for mismatched memtypes, so
   KVM still needs to figure out a solution for ARM, and possibly RISC-V as
   well.  But for CPU<->CPU access, KVM guarantees host safety, just not
   functional correctness for the guest, i.e. finding a solution can likely be
   deferred until a use case comes along.

 - CPU<->Device coherency on ARM is messy and needs further discussion.
 
 - GPU drivers flush caches when mapping and unmapping buffers, so the existing
   virtio GPU use case is ok (though ideally it would be ported to use IOMMUFD's
   mediated device support).

 - Virtio GPU guest drivers are responsible for using CLFLUSH{OPT} instead of
   WBVIND (which is intercept and ignored by KVM).

 - KVM x86's support for virtualizing MTRRs on Intel CPUs can also be dropped
   (it was effectively a workaround for ignoring guest PAT).