Received-SPF: pass (google.com: domain of linux-kernel+bounces-20292-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Date: Tue, 9 Jan 2024 10:11:23 +0800
From: Yan Zhao <yan.y.zhao@intel.com>
To: Jason Gunthorpe <jgg@nvidia.com>
CC: <kvm@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
	<dri-devel@lists.freedesktop.org>, <pbonzini@redhat.com>,
	<seanjc@google.com>, <olvaffe@gmail.com>, <kevin.tian@intel.com>,
	<zhiyuan.lv@intel.com>, <zhenyu.z.wang@intel.com>, <yongwei.ma@intel.com>,
	<vkuznets@redhat.com>, <wanpengli@tencent.com>, <jmattson@google.com>,
	<joro@8bytes.org>, <gurchetansingh@chromium.org>, <kraxel@redhat.com>,
	<zzyiwei@google.com>, <ankita@nvidia.com>, <alex.williamson@redhat.com>,
	<maz@kernel.org>, <oliver.upton@linux.dev>, <james.morse@arm.com>,
	<suzuki.poulose@arm.com>, <yuzenghui@huawei.com>
Subject: Re: [PATCH 0/4] KVM: Honor guest memory types for virtio GPU devices
Message-ID: <ZZyrS4RiHvktDZXb@yzhao56-desk.sh.intel.com>
Reply-To: Yan Zhao <yan.y.zhao@intel.com>
References: <20240105091237.24577-1-yan.y.zhao@intel.com>
 <20240105195551.GE50406@nvidia.com>
 <ZZuQEQAVX28v7p9Z@yzhao56-desk.sh.intel.com>
 <20240108140250.GJ50406@nvidia.com>
 <ZZyG9n0qZEr6dLlZ@yzhao56-desk.sh.intel.com>
 <20240109002220.GA439767@nvidia.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20240109002220.GA439767@nvidia.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?4Pioqf2uu4CepzEf4O7slo2VONoKY/9xQ7lA41L9WJXuKqwKHp0u4UnVLkI6?=
 =?us-ascii?Q?aLqP180irJhaNjdCI9v4vyJbREu0frF2Dv3WHRXBMTJh1+WJVO5FReFgbvpu?=
 =?us-ascii?Q?Q9i0FoqCKRud517snv+7Tqrh91+tWgJtZ3pVcW9KWphysf/LbZIhFxTyLEy+?=
 =?us-ascii?Q?pwLvWUSwo47N5Ml2a+fjFO2CvQhPsOe0zJGIe5x0Z7ykrH2fXbl2qX49+HA3?=
 =?us-ascii?Q?DMxmWqckZoLXdpi2C10uTAPkaLRRgJQXc+rvWiqUoqNHe+ueye5SrkCyQUS/?=
 =?us-ascii?Q?ss/DWbGOhFaDGigqQAZVEvQsQZd7Tp1zvX9VwK2tIsDsrILhYFYCC4z/f3pn?=
 =?us-ascii?Q?0AYr8adfKRlxNnSgxxlorsf1TLKQ+fclQCH3gz1KMn30Bzk9p7d40E7AEFaQ?=
 =?us-ascii?Q?qWHYJmOmO7GXJcwMeoRuJ4lHaToQwHdiI43UCJViQKbXe1qLJh0f1ob8IRAy?=
 =?us-ascii?Q?/ivG/YcD7vWqg1IxeHnzWy8rivcdUo4ngTy9s2Tkmy6/0ldrxyehvc+ancoP?=
 =?us-ascii?Q?vXeQ9ubB9/okJsKLg4ecNDUtQpN4QmhIpV9qKvK6pjFwLC5FlkyOOhrRb1Sm?=
 =?us-ascii?Q?KOuSXqB4oxH0rMii5Absya6B166OCCGrBlukwcbs4YgfsUgXjZVGP4EUO4Cy?=
 =?us-ascii?Q?RyX6F9Mtv2iTOVNCcDdKDzvjcm7XWDl8K3ZVg2QVYg6gijTcKrUmJP6Nvd6Q?=
 =?us-ascii?Q?weHUtAhnip1SrOtQhbv/CEsZzRQgEJOdmHHQdB/C85mj+NGZ0rTCIdxU2beK?=
 =?us-ascii?Q?im5vevGNFTibLA+EoBsuN3GNVGg8+TuMcIiryEo9wRSCuJrKCkrst8u76iP5?=
 =?us-ascii?Q?/Su4hAX8Ta9SJ/caVk0+kwodhXkBBoDIz1Z8N6IPOkcyAH4cDiFIZHxN6yM9?=
 =?us-ascii?Q?/B4Z91FeaHsOem2pttr0DcTQm8G8VNRE5yEoJpaGrSWWz0DeGaXnhmMaVffo?=
 =?us-ascii?Q?ecJy3WS/5KcSVnaelKRT499U5N1+uznDKXyC3sMWgto7flmPINF4O2OeGHCy?=
 =?us-ascii?Q?eBLDPViVDn3u+lI7bo0KgcUS1iDmY30IHyMKJrAxPSkUb/4y9v6rFpWaYhE0?=
 =?us-ascii?Q?5VrSfIUUkOpxkM6bTrBHTGrgJjiRPsKksmLKDM2nd2PaOXQhSUtxmZasmFPX?=
 =?us-ascii?Q?+sy3L5ZViQC642juFuzvWCFh5YBZhqWnwdtSAvv/qW5DdlkbD8k+mnekbuDJ?=
 =?us-ascii?Q?ahn8oqQF9trpsDtCYWiJV4T7e/Zd/tqC8UP4AoyF+wPVp/vwqS1wHa2tEHme?=
 =?us-ascii?Q?+15mC2IA3Kb90xx1WmWvW8FSUt+t7gVYBZjDyd7CMMWmZfb1N9JAlraksGeI?=
 =?us-ascii?Q?Di5yvTd93rablVhGAPKkb2nXyBcgMn8Ds1v1j+68w7RkyPFWKPJ11t2d4PWM?=
 =?us-ascii?Q?yLdVO9jZ2liLiLLMNzrFsWpXSZ5vxjl6Zl8338wD9OCI6IJndziRDCZ7x2Wr?=
 =?us-ascii?Q?GJvcMywBkA2mtA9gDxshV8+RWoA/QQ3DH/cjEVQs5wPobfGHIr74AK7+l/4O?=
 =?us-ascii?Q?UCjXEpdMrp9SWnb+2OOPVES93cvampvtmXAogdv4D7827AF/zuJ+n42GSkGp?=
 =?us-ascii?Q?jnW1+/eVQF3nLa5rBOnxjTj/M4kj192z2/x/duhK?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 0b1662fe-0b7c-4a60-0cd5-08dc10bc6299
X-MS-Exchange-CrossTenant-AuthSource: DS7PR11MB5966.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Jan 2024 02:40:47.5358
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: nFOd29LTDWGmxYZnqpFmlRl36FHBVDVXwSaZTiN0CpwENhfGA5qHHc2FptP97UkzN2YeTUDL4m+2gjB6WbJ2Gg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW6PR11MB8366
X-OriginatorOrg: intel.com

On Mon, Jan 08, 2024 at 08:22:20PM -0400, Jason Gunthorpe wrote:
> On Tue, Jan 09, 2024 at 07:36:22AM +0800, Yan Zhao wrote:
> > On Mon, Jan 08, 2024 at 10:02:50AM -0400, Jason Gunthorpe wrote:
> > > On Mon, Jan 08, 2024 at 02:02:57PM +0800, Yan Zhao wrote:
> > > > On Fri, Jan 05, 2024 at 03:55:51PM -0400, Jason Gunthorpe wrote:
> > > > > On Fri, Jan 05, 2024 at 05:12:37PM +0800, Yan Zhao wrote:
> > > > > > This series allow user space to notify KVM of noncoherent DMA status so as
> > > > > > to let KVM honor guest memory types in specified memory slot ranges.
> > > > > > 
> > > > > > Motivation
> > > > > > ===
> > > > > > A virtio GPU device may want to configure GPU hardware to work in
> > > > > > noncoherent mode, i.e. some of its DMAs do not snoop CPU caches.
> > > > > 
> > > > > Does this mean some DMA reads do not snoop the caches or does it
> > > > > include DMA writes not synchronizing the caches too?
> > > > Both DMA reads and writes are not snooped.
> > > 
> > > Oh that sounds really dangerous.
> > >
> > But the IOMMU for Intel GPU does not do force-snoop, no matter KVM
> > honors guest memory type or not.
> 
> Yes, I know. Sounds dangerous!
> 
> > > Not just migration. Any point where KVM revokes the page from the
> > > VM. Ie just tearing down the VM still has to make the cache coherent
> > > with physical or there may be problems.
> > Not sure what's the mentioned problem during KVM revoking.
> > In host,
> > - If the memory type is WB, as the case in intel GPU passthrough,
> >   the mismatch can only happen when guest memory type is UC/WC/WT/WP, all
> >   stronger than WB.
> >   So, even after KVM revoking the page, the host will not get delayed
> >   data from cache.
> > - If the memory type is WC, as the case in virtio GPU, after KVM revoking
> >   the page, the page is still hold in the virtio host side.
> >   Even though a incooperative guest can cause wrong data in the page,
> >   the guest can achieve the purpose in a more straight-forward way, i.e.
> >   writing a wrong data directly to the page.
> >   So, I don't see the problem in this case too.
> 
> You can't let cache incoherent memory leak back into the hypervisor
> for other uses or who knows what can happen. In many cases something
> will zero the page and you can probably reliably argue that will make
> the cache coherent, but there are still all sorts of cases where pages
> are write protected and then used in the hypervisor context. Eg page
> out or something where the incoherence is a big problem.
> 
> eg RAID parity and mirror calculations become at-rist of
> malfunction. Storage CRCs stop working reliably, etc, etc.
> 
> It is certainly a big enough problem that a generic KVM switch to
> allow incoherence should be trated with alot of skepticism. You can't
> argue that the only use of the generic switch will be with GPUs that
> exclude all the troublesome cases!
>
You are right. It's more safe with only one view of memory.
But even something will zero the page, if it happens before returning
the page to host, looks the impact is constrained in VM scope? e.g.
for the write protected page, hypervisor cannot rely on the page content
is correct or expected.

For virtio GPU's use case, do you think a better way for KVM is to pull
the memory type from host page table in the specified memslot?

But for noncoherent DMA device passthrough, we can't pull host memory
type, because we rely on guest device driver to do cache flush
properly, and if the guest device driver thinks a memory is uncached
while it's effectively cached, the device cannot work properly.

> > > > In this case, will this security attack impact other guests?
> > > 
> > > It impacts the hypervisor potentially. It depends..
> > Could you elaborate more on how it will impact hypervisor?
> > We can try to fix it if it's really a case.
> 
> Well, for instance, when you install pages into the KVM the hypervisor
> will have taken kernel memory, then zero'd it with cachable writes,
> however the VM can read it incoherently with DMA and access the
> pre-zero'd data since the zero'd writes potentially hasn't left the
> cache. That is an information leakage exploit.
This makes sense.
How about KVM doing cache flush before installing/revoking the
page if guest memory type is honored?

> Who knows what else you can get up to if you are creative. The whole
> security model assumes there is only one view of memory, not two.
>