Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) client-ip=23.128.96.35;
Date:   Tue, 12 Dec 2023 14:11:56 -0400
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Catalin Marinas <catalin.marinas@arm.com>
Cc:     ankita@nvidia.com, maz@kernel.org, oliver.upton@linux.dev,
        suzuki.poulose@arm.com, yuzenghui@huawei.com, will@kernel.org,
        alex.williamson@redhat.com, kevin.tian@intel.com,
        yi.l.liu@intel.com, ardb@kernel.org, akpm@linux-foundation.org,
        gshan@redhat.com, linux-mm@kvack.org, lpieralisi@kernel.org,
        aniketa@nvidia.com, cjia@nvidia.com, kwankhede@nvidia.com,
        targupta@nvidia.com, vsethi@nvidia.com, acurrid@nvidia.com,
        apopple@nvidia.com, jhubbard@nvidia.com, danw@nvidia.com,
        mochs@nvidia.com, kvmarm@lists.linux.dev, kvm@vger.kernel.org,
        linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH v3 2/2] kvm: arm64: set io memory s2 pte as normalnc for
 vfio pci devices
Message-ID: <20231212181156.GO3014157@nvidia.com>
References: <20231208164709.23101-1-ankita@nvidia.com>
 <20231208164709.23101-3-ankita@nvidia.com>
 <ZXicemDzXm8NShs1@arm.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ZXicemDzXm8NShs1@arm.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?lY+ljsP3PhtpT/EpVX1ZKQab0bzV4kfOgaErxMjpaqM+y+mB//tsww8slkYC?=
 =?us-ascii?Q?/YryN+gOdI4qJ7siLtRknWGYLBba8O5EfmSW+zEXd0Qc1L3keG3JiJXFmSfK?=
 =?us-ascii?Q?xxeOSIfR0bEGZJfFF8TCdlG4vYPMFgBBLSBcGGlSSOPI11P0mqc6VS0kPDdv?=
 =?us-ascii?Q?U3TPQQG0YO0xmbT+tHUlS3cg2VgMgWASkF0ksdJ/7At4Mffx6/hHsU74bFs2?=
 =?us-ascii?Q?d0tT+59qQOLLGYiGU5w8JwGOiTpRRUPAFsVqKoPNCDIOkYSJWyhjdwqTKeiT?=
 =?us-ascii?Q?t20eIVYfJQbAnKrfkjb8XLw4mi1kS6JDg8JmdPkZ4y1OyAlmPAYRHH9kHJ6b?=
 =?us-ascii?Q?NRkKy2A1gBhd1oY8JjB+MU7m7HQ7ywH5l5BGozVGWMsDiZH0IJmo+Sk6igMD?=
 =?us-ascii?Q?sQPkubIT8ENgDK8sJJ9fM8N4h9/QYVA2WyVy5KfmAFbqlz8+/uEtvYHYydzp?=
 =?us-ascii?Q?/nM1n4zeNw+0hy0X6cEITbgA5hZzVjlhbM4kzSrHdu5mejTjWdGp9Osy5X0L?=
 =?us-ascii?Q?HfB9mWoWd+RDxLQQdIWM4SvQyjO9GlBBga1lNADdN+U/XWgx6gAwc+6ezZdF?=
 =?us-ascii?Q?pSsTovZF5KlmVicqmG5EYlKT8XE76cWjJFxTBXMcFlnZm05UP+CzOavFs3KW?=
 =?us-ascii?Q?2gOwFlunuzn5gZk4/7ejOEEOzkTsfHLH4AEiYLARwyotXNVXh2GKkPdOxKSL?=
 =?us-ascii?Q?g1cCKurXjfYly46m8V4HXFmErGes6EclMCHuiC8CyxiYyronW0Ok+GJngJoC?=
 =?us-ascii?Q?XyYPjiMlL+Ys1x38QDGDlyOoI2v12Im7AoMqv366aIULCZQYvJsz/qsS+k54?=
 =?us-ascii?Q?1HYvhcL37vCFWAsSJRQqtPSbJ3TW+f/sp0ZbB7IUOYmyFDnEQxkarl1tDf7S?=
 =?us-ascii?Q?bvauImFSOpA6sQ+vEVr4ZkIDrGYNb+SALYylcSnXdcSb5QVi9jI4p+h7EyZt?=
 =?us-ascii?Q?DU1X1WKLj61Z8U+AkL12v9/tmC9smeXWdCJr01Lkwzhxqa228AM9v/9nlqbg?=
 =?us-ascii?Q?i1ZdcoS7JRygPZ922SAEvLnflK2/jAO32oI8ECd08hxJIzdRYLhCGkKqOSXC?=
 =?us-ascii?Q?bCN/iFqBAXIeuQE6fo1yPX3xEI1BX9yfO5L/j+iR2XKTHOzb4XlgKsGZw8Gm?=
 =?us-ascii?Q?UGFwH8Ndx7y20DtPvXKYvKEHz2pZxQQSi+kkbpTDN43BDgqtkAU+3Kp/nrhW?=
 =?us-ascii?Q?p7X6JePiqJkhw06Szr76Cgjjvn7i7Sx5pXAFMohSMRWjHFQAtGVYsOVqrkdK?=
 =?us-ascii?Q?eIFMrofFXPFyT95w0ZN0wLUKHn1VDKd5UK82Gr5Piv47IrHggJuYWiEx4f/w?=
 =?us-ascii?Q?Z7fhfcde/+9hJgT533DEjyxDH1kN+3hcT1RhAAsvpvaslcoIT5ki6BMyKWmS?=
 =?us-ascii?Q?CAZP8V+RAKcQQItELXI36j3lWYb5XF426amMOZ2OJIZ/zxIrRUQb9apAbwEU?=
 =?us-ascii?Q?d5fY19ZfMD4kCRTPZMNnSlRGijD8nt5GJAGahG8H3fshgkXA2zyCWMfc+aGB?=
 =?us-ascii?Q?pATNqK3BS8EkrtBXeNxRwqCbXhrv/mcNlgGGLsCTQz0esE738B1MiVdfO0Z/?=
 =?us-ascii?Q?ZcRZC0gT/laouWX/surdnG5ECZRLdH9/o8EOvSsM?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d3fafca0-d646-4b38-167e-08dbfb3dd430
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Dec 2023 18:11:57.5689
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 9bXSN1XWQUcSUdDFg3DI3BMjBwwQ9lTSMbnpSNQJbgBDOo8vCF3oA2FmClbwkLnJ
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB8428
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE
	autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Tue, 12 Dec 2023 10:12:12 -0800 (PST)

On Tue, Dec 12, 2023 at 05:46:34PM +0000, Catalin Marinas wrote:
> > diff --git a/include/linux/mm.h b/include/linux/mm.h
> > index a422cc123a2d..8d3c4820c492 100644
> > --- a/include/linux/mm.h
> > +++ b/include/linux/mm.h
> > @@ -391,6 +391,13 @@ extern unsigned int kobjsize(const void *objp);
> >  # define VM_UFFD_MINOR		VM_NONE
> >  #endif /* CONFIG_HAVE_ARCH_USERFAULTFD_MINOR */
> >  
> > +#ifdef CONFIG_64BIT
> > +#define VM_VFIO_ALLOW_WC_BIT	39	/* Convey KVM to map S2 NORMAL_NC */
> 
> This comment shouldn't be in the core header file. It knows nothing
> about S2 and Normal-NC, that's arm64 terminology. You can mention
> something like VFIO can use this flag hint that write-combining is
> allowed.

Let's write a comment down here to address both remarks:

 This flag is used to connect VFIO to arch specific KVM code. It
 indicates that the memory under this VMA is safe for use with any
 non-cachable memory type inside KVM. Some VFIO devices, on some
 platforms, are thought to be unsafe and can cause machine crashes if
 KVM does not lock down the memory type.

> should know the implications. There's also an expectation that the
> actual driver (KVM guests) or maybe later DPDK can choose the safe
> non-cacheable or write-combine (Linux terminology) attributes for the
> BAR.

DPDK won't rely on this interface

Thanks,
Jason