Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:2 as permitted sender) client-ip=2620:137:e000::3:2;
Date:   Thu, 12 Oct 2023 12:44:39 -0300
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Will Deacon <will@kernel.org>
Cc:     Catalin Marinas <catalin.marinas@arm.com>,
        Lorenzo Pieralisi <lpieralisi@kernel.org>, ankita@nvidia.com,
        maz@kernel.org, oliver.upton@linux.dev, aniketa@nvidia.com,
        cjia@nvidia.com, kwankhede@nvidia.com, targupta@nvidia.com,
        vsethi@nvidia.com, acurrid@nvidia.com, apopple@nvidia.com,
        jhubbard@nvidia.com, danw@nvidia.com,
        linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev,
        linux-kernel@vger.kernel.org
Subject: Re: [PATCH v1 2/2] KVM: arm64: allow the VM to select DEVICE_* and
 NORMAL_NC for IO memory
Message-ID: <20231012154439.GM3952@nvidia.com>
References: <ZP8q71+YXoU6O9uh@lpieralisi>
 <ZP9MQdRYmlawNsbC@nvidia.com>
 <ZQHUifAfJ+lZikAn@lpieralisi>
 <ZQIFfqgR5zcidRR3@nvidia.com>
 <ZRKW6uDR/+eXYMzl@lpieralisi>
 <ZRLiDf204zCpO6Mv@arm.com>
 <ZR6IZwcFNw55asW0@lpieralisi>
 <20231012123541.GB11824@willie-the-truck>
 <ZSf6Ue09IO6QMBs1@arm.com>
 <20231012144807.GA12374@willie-the-truck>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20231012144807.GA12374@willie-the-truck>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?sBO4Y3HF4VQ/+B6d0CcrjisNxyQa+IIbHHgOx2gAQKoULJHr3gnH8hfLjWt/?=
 =?us-ascii?Q?nr3Inh3ugzAOxS3FDZlkZrihIXeNIsDJ1fv+c+kP86zj+RaXH0z0JchjxmOp?=
 =?us-ascii?Q?ZeFLazcsNEaZaCS1LwxwB3FmyjOjYjI00jboVKma0GltfomMYnvHg6mvkYhk?=
 =?us-ascii?Q?O59LHtJC1u8zq9WLVGZhLs8oiEHimvf/6Au72tqDv2DyrornEWutWrHPmi5P?=
 =?us-ascii?Q?4zjdolwTMnZllbQsItTzJf4cBIYm4LIxHdxMmMIqOP5WxcSjFcHidmfkcwEz?=
 =?us-ascii?Q?WM0jZjRXGBrlLieYu3/6hFvXanhsLLabr2BsW8X7c1mUrC01/JpnaAl2PHsW?=
 =?us-ascii?Q?eyEU+uHFeb6wxHKZPIdPI51Yye9h98eeygl4TRLjgWSZ1ekuqWKE/y+a/ZII?=
 =?us-ascii?Q?d+odcCIUT8etoVTsVPlJb3SKoPAJISfThhAOGc/e2hHcLZBSJy0qi1U+JBYI?=
 =?us-ascii?Q?lcXmlewNb9PoI6wut+SrFoX91SjR82U5wslkSsQsZHlBTcOdAfKz6P12fJdw?=
 =?us-ascii?Q?UuiwptAldsGOJQUiw8ihndz6jEOI72thGtGU8lmRyn+b/kM49QCbYHy3ehZ/?=
 =?us-ascii?Q?8xMaSXUDvsu4w0mvWe5M5XD9TsdSSRPDbYbonCBmL4EVi/ZEOF1VEY1r35pN?=
 =?us-ascii?Q?5OKlDFIuixJSN+Wm2Xx7i5SuTg4RiyJ7n+AV2Qf5+XCDBbqNNJK8ceHEttqJ?=
 =?us-ascii?Q?NaqY/n0JnJWIfth1qsaAWJpBpG7IrGmj59C5lvCGGr4eh0ssE6EWeX8LCGgh?=
 =?us-ascii?Q?fS7CUa3Lu3Yfk8ERpong+ZxiY3Xr/QFWt+6vfsNrbXvxiePoXDusrADIxY28?=
 =?us-ascii?Q?SQSM53pkvmlInstvFplIp2gnX/0cW033vCrsESipPgKu9kwu2DM0SxQPtKJz?=
 =?us-ascii?Q?s9wo3F1C5cOvfB8LFY/rOcYUAw7FY+r8e7U0nPrLg97T14iTylF9evVrDbTy?=
 =?us-ascii?Q?2phGfP/gIJ2iVooqX6pOujFfYnARgQp9SZTAPl8S26GH47Xmlou7ripqmw6N?=
 =?us-ascii?Q?jSwn7s1gM6qG1uXedpoI1D57xTJikLJSVLYViZh80apeyIKn1uHKon1wFCJj?=
 =?us-ascii?Q?QnYm9l+YKTyNHRHKUAm/IO5dX5CUc1Kkwk1GBFKY+z+9J92KqKThnCej1jFL?=
 =?us-ascii?Q?Bp5r9dwEjzqbCNf2aIyzoV9+KfFK+UuDG5SL056iFlchAFGHMTdXtNoYdr9Z?=
 =?us-ascii?Q?cmcBxDvdYvSWRofFNTK/8z2aSAjKvFRDOfguFGx0DqnXLlgoZRnwTVM4PWBz?=
 =?us-ascii?Q?wZ2HVVxt0q5RowFoxF3pM2JAoRXS/f0Q8akh+5GYLNliOE5M3GhrzNWAzDyS?=
 =?us-ascii?Q?HR4a6G/xPtfSURHLyiT2W5VQQSXGLHq4Ogg5iuphrbKWFFUuYrlcy8mgZw6K?=
 =?us-ascii?Q?Nh0EfeoWO/fsgNWs5W1gALNWym0S11Ycthcl1eaebe2aQ0vAVK44Ek1GxUj8?=
 =?us-ascii?Q?1aLys2el5t0dGMa/ZS4LaE6Ieg4Oci7jHUUAz2bGvv42X8fh8cxPJDgsQ5aw?=
 =?us-ascii?Q?huRO7UBjieSvzgjwsfd3IZ//BDPndZZywboNuKzppqQ8rvzL8ZJ3mwoxWRGa?=
 =?us-ascii?Q?F6vPdiV8de8w8aljqtVBbSaZJtNTQRYSgJAwo75Y?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 91b6a8ba-45b3-4b22-e5ed-08dbcb3a25a5
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Oct 2023 15:44:40.5063
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: GXRr415bNy4O0USnq63oii2zKkOsjYnxQqgYzjYNxmBYLLzGUxZmzITF311Q36Fl
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB6142
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Thu, 12 Oct 2023 08:44:57 -0700 (PDT)

On Thu, Oct 12, 2023 at 03:48:08PM +0100, Will Deacon wrote:

> I guess my wider point is that I'm not convinced that non-cacheable is
> actually much better and I think we're going way off the deep end looking
> at what particular implementations do and trying to justify to ourselves
> that non-cacheable is safe, even though it's still a normal memory type
> at the end of the day.

When we went over this with ARM it became fairly clear there wasn't an
official statement that Device-* is safe from uncontained
failures. For instance, looking at the actual IP, our architects
pointed out that ARM IP already provides ways for Device-* to trigger
uncontained failures today.

We then mutually concluded that KVM safe implementations must already
be preventing uncontained failures for Device-* at the system level
and that same prevention will carry over to NormalNC as well.

IMHO, this seems to be a gap where ARM has not fully defined when
uncontained failures are allowed and left that as an implementation
choice.

In other words, KVM safety around uncontained failure is not a
property that can be reasoned about from the ARM architecture alone.

> The current wording talks about use-cases (I get this) and error containment
> (it's a property of the system) but doesn't talk at all about why Normal-NC
> is the right result.

Given that Device-* and NormalNC are equally implementation defined
with regards to uncontained failures, NormalNC allows more VM
functionality.

Further, we have a broad agreement that this use case is important,
and that NormalNC is the correct way to adress it.

I think you are right to ask for more formality from ARM team but also
we shouldn't hold up fixing real functional bugs in real shipping
server ARM products.

Jason