Received-SPF: pass (google.com: domain of linux-kernel+bounces-27393-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Date: Tue, 16 Jan 2024 08:54:26 -0400
From: Jason Gunthorpe <jgg@nvidia.com>
To: "Tian, Kevin" <kevin.tian@intel.com>
Cc: "Zhao, Yan Y" <yan.y.zhao@intel.com>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"dri-devel@lists.freedesktop.org" <dri-devel@lists.freedesktop.org>,
	"pbonzini@redhat.com" <pbonzini@redhat.com>,
	"seanjc@google.com" <seanjc@google.com>,
	"olvaffe@gmail.com" <olvaffe@gmail.com>,
	"Lv, Zhiyuan" <zhiyuan.lv@intel.com>,
	"Wang, Zhenyu Z" <zhenyu.z.wang@intel.com>,
	"Ma, Yongwei" <yongwei.ma@intel.com>,
	"vkuznets@redhat.com" <vkuznets@redhat.com>,
	"wanpengli@tencent.com" <wanpengli@tencent.com>,
	"jmattson@google.com" <jmattson@google.com>,
	"joro@8bytes.org" <joro@8bytes.org>,
	"gurchetansingh@chromium.org" <gurchetansingh@chromium.org>,
	"kraxel@redhat.com" <kraxel@redhat.com>,
	"zzyiwei@google.com" <zzyiwei@google.com>,
	"ankita@nvidia.com" <ankita@nvidia.com>,
	"alex.williamson@redhat.com" <alex.williamson@redhat.com>,
	"maz@kernel.org" <maz@kernel.org>,
	"oliver.upton@linux.dev" <oliver.upton@linux.dev>,
	"james.morse@arm.com" <james.morse@arm.com>,
	"suzuki.poulose@arm.com" <suzuki.poulose@arm.com>,
	"yuzenghui@huawei.com" <yuzenghui@huawei.com>
Subject: Re: [PATCH 0/4] KVM: Honor guest memory types for virtio GPU devices
Message-ID: <20240116125426.GA734935@nvidia.com>
References: <20240105091237.24577-1-yan.y.zhao@intel.com>
 <20240105195551.GE50406@nvidia.com>
 <ZZuQEQAVX28v7p9Z@yzhao56-desk.sh.intel.com>
 <20240108140250.GJ50406@nvidia.com>
 <ZZyG9n0qZEr6dLlZ@yzhao56-desk.sh.intel.com>
 <20240109002220.GA439767@nvidia.com>
 <ZZyrS4RiHvktDZXb@yzhao56-desk.sh.intel.com>
 <20240115163050.GI734935@nvidia.com>
 <BN9PR11MB5276B5603D9D777F01D64EDA8C732@BN9PR11MB5276.namprd11.prod.outlook.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <BN9PR11MB5276B5603D9D777F01D64EDA8C732@BN9PR11MB5276.namprd11.prod.outlook.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?2T/llDK5cbu2v2aCrEFdwGImspbEDve0nKNaSUYRGLf96VEF6hCYdaewZu6d?=
 =?us-ascii?Q?tHIW5I6wuTCDH6CJGM6WhwSefBoj2QFK7usFQ7tvgcYtPXJK1gZrIomtczRI?=
 =?us-ascii?Q?vx41IOOnYSeeCA1+FmIKSpRp9aJDlZlN20TDGP+uQaPuAEfs1pgJBqD4iWAo?=
 =?us-ascii?Q?6pGvUK11hH+KSRJW9bNabc8ltZWfjge2suQeCOYH7GwGGy5i8lrBe2SZMo7o?=
 =?us-ascii?Q?xVfChFcs7LfQYPwciZZzkwqNMALpd/pACFryRbF6kHXmsAzXaIOCJ7SSnEsx?=
 =?us-ascii?Q?J3Il6MqpJeOYI7H7lizf0s2ne8U/Ph0qwG5tgozQow0qKy8hIahEZPPt0RsU?=
 =?us-ascii?Q?4edqcQIrh+rs++v54ptLhRvcgcfsX0YqAd6Ba6yXFgxVLk2bVbxNJUR7CvLK?=
 =?us-ascii?Q?hS0OQkiudrc6S4APRL05GuV2G0z+nw/9pQCs+a0zzz8ezL/7XquWL29mma5k?=
 =?us-ascii?Q?dd8ytyPO5q2g6QYIgK+Y//XfV9Dbi9b91rLhgZQkPSAqKbpoyGsgeVpVIwG3?=
 =?us-ascii?Q?ODpbb0hvRRjyjKeWNVWgj3deY+XMC6Yi2Hb+oQkRpwkfD5mH1GV9jb9R5ZLc?=
 =?us-ascii?Q?FfmEY3ZZmPU1c9hSCISHzY6CnpLAziBy/5MJIU1tF0WaP8bOGMiTZTiVdBT0?=
 =?us-ascii?Q?DUOQ5TaBR1S6qugVQjYguQcjxqXZ3bwbBk0d8IIfp8GMYfgdBIpP9H3mXjWH?=
 =?us-ascii?Q?JDPblWeKTXRAST2aNStnq8BQjU6Tcvvc8sD/0nou/KTO7oU7Ln+WMn/34M6f?=
 =?us-ascii?Q?8fhcvrqozDIXoL9tuyNlpjx6AJF4jXZ0FWjXhw4Si1aeYi4uK8RHjf+kGVlW?=
 =?us-ascii?Q?lPMA2EpTBF2+LiJqKX4kDKdWC7KtK6a7s3ZdB/taTcCwUfnZ4rkA+DxhXF/Q?=
 =?us-ascii?Q?kufC6fL79TQTcwIJAAKfKHTIERvPzaWMNmJxjsNzF8O1OWOe0NNiIhHa6y4p?=
 =?us-ascii?Q?d0SO78Cy5FGHsWoi13oiJUCGdyul9dci1Zg79uQfzK/55WRUwF/efAEb+Wwr?=
 =?us-ascii?Q?DvBKZmzOkG02uu/fHRsxUKbu/WVjFGRjwkjU+FOuf3Md/eDA3b4Kxogwm2CP?=
 =?us-ascii?Q?Tn2jZJZzocKvqcHDvej2bWiD1v9c9mnnRwnB6SJfjUc1PbSvkXLqRri+xi8n?=
 =?us-ascii?Q?l3pIP/GeA2cGaQr0vet14ZD0IdHeLGcQg2y1ERKWNElNojG5BZFFtPR0/oI0?=
 =?us-ascii?Q?uW9a++xiYp6ReLUodghGcUzcUkTNIhdr7VtHd9XeaCwWBw66b4vAwu7cjwYP?=
 =?us-ascii?Q?UtnEsKRf/NiQFjmw255kd15lJ6hUS1VJAsPbwhEb1V/rIzlrndhGvbHMN3uX?=
 =?us-ascii?Q?DuXucPw6ivRjZA9ByGg9RAEC+9KCxyTf09dFq++5bPdnHtA4XeUvF5c7USRX?=
 =?us-ascii?Q?S4BWeYplrSLLg07AokJqDL+s7gGmti41Rc1w2pV+olaz4o7iaYJZFfM2Rns8?=
 =?us-ascii?Q?EuyLLn0MgfjI5hnA+p6BQYMYkblUxLXXyP+8aNJKmxiz7qIYcTkOIZsJv3zF?=
 =?us-ascii?Q?eBxOy8rb+en6Prr7iBivbyNjWEX942sOQiFb9KRwn4hJxdJyh6+IZIKoB5kj?=
 =?us-ascii?Q?IS063ptCFOvUMc6icZQkH4zXaE6lcB4oRGWjGU41?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3ef99777-6fbc-48df-4520-08dc169245f7
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jan 2024 12:54:27.5880
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: OXPuni8jv8zzAXck6BBCSjSTMp2bmqH+0WaucUfzuZivNTb7c8/TkOlDGO3X4El3
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB7153

On Tue, Jan 16, 2024 at 04:05:08AM +0000, Tian, Kevin wrote:
> > From: Tian, Kevin
> > Sent: Tuesday, January 16, 2024 8:46 AM
> > 
> > > From: Jason Gunthorpe <jgg@nvidia.com>
> > > Sent: Tuesday, January 16, 2024 12:31 AM
> > >
> > > On Tue, Jan 09, 2024 at 10:11:23AM +0800, Yan Zhao wrote:
> > >
> > > > > Well, for instance, when you install pages into the KVM the hypervisor
> > > > > will have taken kernel memory, then zero'd it with cachable writes,
> > > > > however the VM can read it incoherently with DMA and access the
> > > > > pre-zero'd data since the zero'd writes potentially hasn't left the
> > > > > cache. That is an information leakage exploit.
> > > >
> > > > This makes sense.
> > > > How about KVM doing cache flush before installing/revoking the
> > > > page if guest memory type is honored?
> > >
> > > I think if you are going to allow the guest to bypass the cache in any
> > > way then KVM should fully flush the cache before allowing the guest to
> > > access memory and it should fully flush the cache after removing
> > > memory from the guest.
> > 
> > For GPU passthrough can we rely on the fact that the entire guest memory
> > is pinned so the only occurrence of removing memory is when killing the
> > guest then the pages will be zero-ed by mm before next use? then we
> > just need to flush the cache before the 1st guest run to avoid information
> > leak.
> 
> Just checked your past comments. If there is no guarantee that the removed
> pages will be zero-ed before next use then yes cache has to be flushed
> after the page is removed from the guest. :/

Next use may include things like swap to disk or live migrate the VM.

So it isn't quite so simple in the general case.

> > > Noting that fully removing the memory now includes VFIO too, which is
> > > going to be very hard to co-ordinate between KVM and VFIO.
> 
> Probably we could just handle cache flush in IOMMUFD or VFIO type1
> map/unmap which is the gate of allowing/denying non-coherent DMAs
> to specific pages.

Maybe, and on live migrate dma stop..

Jason