Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) client-ip=23.128.96.36;
Date:   Tue, 5 Dec 2023 09:05:17 -0400
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Catalin Marinas <catalin.marinas@arm.com>
Cc:     Marc Zyngier <maz@kernel.org>, ankita@nvidia.com,
        Shameerali Kolothum Thodi 
        <shameerali.kolothum.thodi@huawei.com>, oliver.upton@linux.dev,
        suzuki.poulose@arm.com, yuzenghui@huawei.com, will@kernel.org,
        ardb@kernel.org, akpm@linux-foundation.org, gshan@redhat.com,
        aniketa@nvidia.com, cjia@nvidia.com, kwankhede@nvidia.com,
        targupta@nvidia.com, vsethi@nvidia.com, acurrid@nvidia.com,
        apopple@nvidia.com, jhubbard@nvidia.com, danw@nvidia.com,
        mochs@nvidia.com, kvmarm@lists.linux.dev, kvm@vger.kernel.org,
        lpieralisi@kernel.org, linux-kernel@vger.kernel.org,
        linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH v2 1/1] KVM: arm64: allow the VM to select DEVICE_* and
 NORMAL_NC for IO memory
Message-ID: <20231205130517.GD2692119@nvidia.com>
References: <20231205033015.10044-1-ankita@nvidia.com>
 <86fs0hatt3.wl-maz@kernel.org>
 <ZW8MP2tDt4_9ROBz@arm.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ZW8MP2tDt4_9ROBz@arm.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?e7bNus6lxkkGg3qK6NnYfg57Pjw1hptAHOjGhq2kimQN+zPDNlnl8r62eSzw?=
 =?us-ascii?Q?8dxnhVSFCIJOYCzWxMQxcjZTmRxX3KWiM/VsnmtgEqixhKvCYCJQrtMXmVR6?=
 =?us-ascii?Q?EsrH+dio0QTiXwp1LE8ko982r28kHY8PFIrYix6kex0HSOF2MkgwOGNfxnNE?=
 =?us-ascii?Q?X+XqIg7V/bHFebVJt5QZ0Kkr7un1QtiE2uoYUDrkZhGUAtv+U68yrifUlpkS?=
 =?us-ascii?Q?fj/OdnMxDdB4U2zHETRjBIwx20AuroL8lHVTyD1fJNsLWomWV5kSKjZgSg8o?=
 =?us-ascii?Q?Wh5CLQhedgsxGgqWUkuriDaGS54G18L3F7HPo4IoIWe1C9lJoGz0YqJnTL/L?=
 =?us-ascii?Q?xOIFrNvTRloquThfLA1Y2wRSCUMAPYjBuCFT7HVtW8V9YHzvvbKWp/0NR8KA?=
 =?us-ascii?Q?ol+Kmv33spEwRy5YKcN7sE0XXA1SwT4m+rtWf8ncN198T+k8UTX/cK/8c04E?=
 =?us-ascii?Q?Thx7hkn2A67P1JakDlSBF4XmXMyay3scINrlReTrPx5GqrkT8TQgH/pC5rko?=
 =?us-ascii?Q?xFI+s1cjFskjrmH334vnOFyoS/XahGrxeQO5PgPhj3vRDmLv7rLZGFI7q/wO?=
 =?us-ascii?Q?UNsvHkxqY55VM2i161aM7YtJ4FP6Lv8KY/xCQX1JqsRHmZkk1lpNy8TAt4gI?=
 =?us-ascii?Q?jd8spf6UK2fWUMUQmUhyXbYEuaeqiQeWM+UC/1BQMz1sAOegA9LsZ4VmPO7d?=
 =?us-ascii?Q?yqmyKqbHUD5mGRDRrFdQUhTT0SVQuUeBHrOO3eany3i/+imkFZjb38vkBdnP?=
 =?us-ascii?Q?vENZ5ufbjTPbfWnH1DTYLA7p8Ma9wsNAw1y7POr76zzM8hdeP3WKjTkOg7sS?=
 =?us-ascii?Q?MAZ8N6DNBbRXpywl+B+Ltslglw8BWIaXfXnc6xU2xTCn8xzZPp+fasAGtX+z?=
 =?us-ascii?Q?w0FyUUM37UvFhYCA2TDQhRhLqE4AZ69b/3dAZfVqdNpcjIs2ZBv++8ybs57p?=
 =?us-ascii?Q?mYl5a5tdD2ADHTJGfn35+iLRGpp8K9qd3r5o06UUXeUJRwGWKzoFGB3TzZ4K?=
 =?us-ascii?Q?WBbHqYpar531t50sCkpB8cXHUCBNBR/sVg2VRVHHazJ0tJyBJa1iWgMe7K5Z?=
 =?us-ascii?Q?pmB6+6bxoZBYjJCOzcEf+E9aL3krqXaQ5UvErVYI+0uCLuwbV8953VqRNzP7?=
 =?us-ascii?Q?yTLAoiUotR94mWsWv7CahIC9UZD/O1eEYFA8vedFmWl6yRpZ+LFTvmYgOoBj?=
 =?us-ascii?Q?9UbrTahn1S2RlVf4ZQYNAfOzq2gg4g1PFzdFAnBSoI+xQW85Mkfk2rZfxAXF?=
 =?us-ascii?Q?wLaxY2VaHM+BA6xcZNeRbil3pkpnOsJPTMzl2WN1W8QTdgn2Txgi8rNeeDFQ?=
 =?us-ascii?Q?vUh9w7Ds3BAzs7TlVoZVuc4ffENm5Ry0St7DtCxUGdTYN0oP4t6pr454ytbp?=
 =?us-ascii?Q?Ohi+mo04pBGRr9P7bhFnbudB5nBAtMJK8Ck02oMTnNzwLorBIhsBwidTuxSe?=
 =?us-ascii?Q?4aI5qrhhWBnPTUqaxfZ389ui3mosiKW+y5hz10TiLZHMEe++t1E+7sNTgZ27?=
 =?us-ascii?Q?3ntiVWcnya1tRX7KoViY2ubEd+KDLc9AiaTmOJzAk30NNoCJzYghiRO8jEtx?=
 =?us-ascii?Q?PYFUZhy7xgo7gKUKcMZNQ0zYoOElsEU9QVVfNUBd?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bb5669e1-140f-49c5-c9d2-08dbf592d4a1
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Dec 2023 13:05:18.6264
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: gWCuyrMlPwTvPFym0+leD+qRh3BN+nTC1KGGWzyP6pcaazBh6tFBufM55PQIBoEn
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6738
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE
	autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Tue, 05 Dec 2023 05:05:40 -0800 (PST)

On Tue, Dec 05, 2023 at 11:40:47AM +0000, Catalin Marinas wrote:
> > - Will had unanswered questions in another part of the thread:
> > 
> >   https://lore.kernel.org/all/20231013092954.GB13524@willie-the-truck/
> > 
> >   Can someone please help concluding it?
> 
> Is this about reclaiming the device? I think we concluded that we can't
> generalise this beyond PCIe, though not sure there was any formal
> statement to that thread. The other point Will had was around stating
> in the commit message why we only relax this to Normal NC. I haven't
> checked the commit message yet, it needs careful reading ;).

Not quite, we said reclaiming is VFIO's problem and if VFIO can't
reliably reclaim a device it shouldn't create it in the first place.

Again, I think alot of this is trying to take VFIO problems into KVM.

VFIO devices should not exist if they pose a harm to the system. If
VFIO decided to create the devices anyhow (eg admin override or
something) then it is not KVM's job to do any further enforcement.

Remember, the feedback we got from the CPU architects was that even
DEVICE_* will experience an uncontained failure if the device tiggers
an error response in shipping ARM IP.

The reason PCIe is safe is because the PCI bridge does not generate
errors in the first place!

Thus, the way a platform device can actually be safe is if it too
never generates errors in the first place! Obviously this approach
works just as well with NORMAL_NC.

If a platform device does generate errors then we shouldn't expect
containment at all, and the memory type has no bearing on the
safety. The correct answer is to block these platform devices from
VFIO/KVM/etc because they can trigger uncontained failures.

If you have learned something different then please share it..

IOW, what is the practical scenario where DEVICE_* has contained
errors but NORMAL_NC does not?

Jason