Received-SPF: pass (google.com: domain of linux-kernel+bounces-18310-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Date: Fri, 5 Jan 2024 15:55:51 -0400
From: Jason Gunthorpe <jgg@nvidia.com>
To: Yan Zhao <yan.y.zhao@intel.com>
Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
	dri-devel@lists.freedesktop.org, pbonzini@redhat.com,
	seanjc@google.com, olvaffe@gmail.com, kevin.tian@intel.com,
	zhiyuan.lv@intel.com, zhenyu.z.wang@intel.com, yongwei.ma@intel.com,
	vkuznets@redhat.com, wanpengli@tencent.com, jmattson@google.com,
	joro@8bytes.org, gurchetansingh@chromium.org, kraxel@redhat.com,
	zzyiwei@google.com, ankita@nvidia.com, alex.williamson@redhat.com,
	maz@kernel.org, oliver.upton@linux.dev, james.morse@arm.com,
	suzuki.poulose@arm.com, yuzenghui@huawei.com
Subject: Re: [PATCH 0/4] KVM: Honor guest memory types for virtio GPU devices
Message-ID: <20240105195551.GE50406@nvidia.com>
References: <20240105091237.24577-1-yan.y.zhao@intel.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240105091237.24577-1-yan.y.zhao@intel.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?U4Dl9fv/fpldBexb1HfQQXXj5a4XwftUQYeP9GUGbMZz2ZxgAEY00Le/h1eE?=
 =?us-ascii?Q?3wHtnCITPnpaXMKUPZ7Fo1PGtrkiv4uu7Mv0SfLhr+GrsPrZ//j1WDGuK+dp?=
 =?us-ascii?Q?rbgrOenYS2K4SWCW0aw0CWuhMqJQr3i5msC64a10IHtVv94u4Z3gKjgfCYly?=
 =?us-ascii?Q?V7IQL+IuJG0iNRweMhmaAGTpJaE0i29QkyQjLbPpsOQg554E86A/Gf/kFbM0?=
 =?us-ascii?Q?cJUyb4RV5HGZEqkb7SE34WQhPEApMHnHsn1jZ22VOcqi627YFpaWyLRtF/7Q?=
 =?us-ascii?Q?Vg6t4kbCDkD3122jnABW1/YAFiti075vNKBkLy/5LKkM3NUA/3706Zx54Xqu?=
 =?us-ascii?Q?3NyDrnDoIHhACE3Zfa+fqDb2M4NKAzRU6X+YvQOZTaYc2cP/6Q3OpCaJNOoS?=
 =?us-ascii?Q?/8hltgxKWTkoXmn6wRSUeXkZ66pfq/RPGzvZw1L28lOfiI4x8ULZBVQ41x0V?=
 =?us-ascii?Q?v4QyX2+USFBtDGDg0NR9q+nrZLoO4u2mCz0yDNMBDAKE3iTLP42+mKUBN0sP?=
 =?us-ascii?Q?CrapybSTkbrD9aHs2CBjhdW312sct7+c5U5DK1jWh/y6RU0bcKLywWOCuj7o?=
 =?us-ascii?Q?XPKqRlV7awiZ2PM6cL5/cY9fdpKd+C3Xb2LyDOt7l1Iv1nq2ybUjeGp6dO+F?=
 =?us-ascii?Q?N1MPxFiLYlzk7KBcfrjRaTnbXdFb/qEzQfDIx+40HomeuzsMpKVZRo5MDJqD?=
 =?us-ascii?Q?V9Atx8jod0UDRlldqI8TTtxn8QBponUBE7vevrlNrcROoEe1s1sziuDwoCTJ?=
 =?us-ascii?Q?Z+vw5YYKgE/5rK00n3p+GBJDCxCf2gkJUmTxz8UKQJ48S7uGKRBsrC2twX8+?=
 =?us-ascii?Q?lD+y1Q/UB4l0DJXdp30celSAaHDbl8Ofz2J6jlDr1ahQXbth1Pm2FpFIy6xS?=
 =?us-ascii?Q?LfWTysAlTjh9nRfR/d8SK1vxXkmEEVhNyiwkRSnm9ipBV83RkUIhr5CaYNp9?=
 =?us-ascii?Q?NArB7VrYgBhd/ekJf0Bz4pYx0CLYPwWYDcDH5/zY0XBfuuTNJXg1En8b7upG?=
 =?us-ascii?Q?SP906ws6Ogmwr/EkLZaCyz8p0pEYM6EVI/SjiY49gFDPu0a/HiK2p2i2vodj?=
 =?us-ascii?Q?SBqh+akFgcAanOYW4RTgu4jurhVN+9X0FM8/P4s89qUpbpPl66pLWLQye6KB?=
 =?us-ascii?Q?Yk41Rg6iA9emK+bPNQfo6/OUxWLmx5pbITQCnHH37WUF8X48eTHGKV+PviJa?=
 =?us-ascii?Q?mC7wgzVXYVVbjMMqwDjp3eAvB+NftRG6GG9Q6agX7PYQsY8ErOr9x2F8ln5y?=
 =?us-ascii?Q?d0eC3zCds2gW3fOaPOXGVyVKgW7/9z5vVRXDkENs7+ZL5zF1p3cfEZPr4Cch?=
 =?us-ascii?Q?mw02GEOTjMeeJYRTfxLuYQpOaOoj4aAZNpJ+5rn+PsvaAc2ZDbUp95ygzffZ?=
 =?us-ascii?Q?BJcIU0UBvp7isKBvvufThRUy77uqntIaAsxWLb9npsFpXJWW4GWlrvo0XeFA?=
 =?us-ascii?Q?WxiDPRlCcyhWVqiXEori/AoydVwOEm4NfVMvNR7cuqJ4SNA4SnpGbdiTDF4O?=
 =?us-ascii?Q?Y75vCBDOMPweoYdm2Xh8m+T+L1uL4H/0khL4xzZ+xRMUoerBJla+bCC1jL6X?=
 =?us-ascii?Q?fKkGMxNxPtZTmurW/5Y=3D?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d53bf452-9a85-402f-35f6-08dc0e2852f0
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Jan 2024 19:55:53.4043
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: mG3Ywe2XwtfNoG5mGRY3ZPwX9NvLkUB+5Jc3qF2m3OgrEy1phvYuklxQqkO2kvn2
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB6240

On Fri, Jan 05, 2024 at 05:12:37PM +0800, Yan Zhao wrote:
> This series allow user space to notify KVM of noncoherent DMA status so as
> to let KVM honor guest memory types in specified memory slot ranges.
> 
> Motivation
> ===
> A virtio GPU device may want to configure GPU hardware to work in
> noncoherent mode, i.e. some of its DMAs do not snoop CPU caches.

Does this mean some DMA reads do not snoop the caches or does it
include DMA writes not synchronizing the caches too?

> This is generally for performance consideration.
> In certain platform, GFX performance can improve 20+% with DMAs going to
> noncoherent path.
> 
> This noncoherent DMA mode works in below sequence:
> 1. Host backend driver programs hardware not to snoop memory of target
>    DMA buffer.
> 2. Host backend driver indicates guest frontend driver to program guest PAT
>    to WC for target DMA buffer.
> 3. Guest frontend driver writes to the DMA buffer without clflush stuffs.
> 4. Hardware does noncoherent DMA to the target buffer.
> 
> In this noncoherent DMA mode, both guest and hardware regard a DMA buffer
> as not cached. So, if KVM forces the effective memory type of this DMA
> buffer to be WB, hardware DMA may read incorrect data and cause misc
> failures.

I don't know all the details, but a big concern would be that the
caches remain fully coherent with the underlying memory at any point
where kvm decides to revoke the page from the VM.

If you allow an incoherence of cache != physical then it opens a
security attack where the observed content of memory can change when
it should not.

ARM64 has issues like this and due to that ARM has to have explict,
expensive, cache flushing at certain points.

Jason