Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Sun, 16 Jul 2023 19:30:23 -0300
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Catalin Marinas <catalin.marinas@arm.com>
Cc:     Benjamin Herrenschmidt <benh@kernel.crashing.org>,
        Marc Zyngier <maz@kernel.org>, ankita@nvidia.com,
        alex.williamson@redhat.com, naoya.horiguchi@nec.com,
        oliver.upton@linux.dev, aniketa@nvidia.com, cjia@nvidia.com,
        kwankhede@nvidia.com, targupta@nvidia.com, vsethi@nvidia.com,
        acurrid@nvidia.com, apopple@nvidia.com, jhubbard@nvidia.com,
        danw@nvidia.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
        linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org,
        Lorenzo Pieralisi <lpieralisi@kernel.org>,
        Clint Sbisa <csbisa@amazon.com>, osamaabb@amazon.com
Subject: Re: [PATCH v3 1/6] kvm: determine memory type from VMA
Message-ID: <ZLRvf1M3gk4jjPp0@nvidia.com>
References: <20230405180134.16932-1-ankita@nvidia.com>
 <20230405180134.16932-2-ankita@nvidia.com>
 <86r0spl18x.wl-maz@kernel.org>
 <ZDarrZmLWlA+BHQG@nvidia.com>
 <ZHcxHbCb439I1Uk2@arm.com>
 <67a7374a72053107661ecc2b2f36fdb3ff6cc6ae.camel@kernel.crashing.org>
 <ZLQIDkFysVJ8kzkQ@arm.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ZLQIDkFysVJ8kzkQ@arm.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?dOmWtTcJ7dWn4naN4pDr0YpY4LZKyiuyKq+4I/gSp/VgN/LcVsw8uy47K/lC?=
 =?us-ascii?Q?KWgMgz1AOdHRFnPg9geFQyNV+6YWS3A/pPUBCAcdsh1sreed/z9nqnxdHdgR?=
 =?us-ascii?Q?8IRS9oklcCFZjAJdM1LE1I9PZs3WFsqfiASzqVWoBkN6dMk3odD/i7EvGqXR?=
 =?us-ascii?Q?LSTZdYJ2ncLU7FMi86Nh3436ElelBWP2HUv0+VGxajieRS3P1Ntvy4OMo4e/?=
 =?us-ascii?Q?lQmiQRalVXZx+jxVRUdS2Y59xpMvjUBvgK3Df5EhAjhiPEREutTzHIcVEc+I?=
 =?us-ascii?Q?h6GEPeVOHG1ayQVySnTA9COA03EohdgSIt105nO5YIY4XD2XAkREfZiVZXWJ?=
 =?us-ascii?Q?48/J7zuOYKXGn003tosbKbWHj+WRbJh46jeykM8k0WJSt0JEP4fQM+cosDWz?=
 =?us-ascii?Q?9e8lHkFtR/2zQLeHvA/2mCm0G0zAMIuOJsHxJJjDRQrgI93STYhMax66JgKi?=
 =?us-ascii?Q?6X4D604Shlt0Z++ogTXs3G1hE1XBrZfyFSKrB0yexHdTH2sFXy3fRzXsJIoE?=
 =?us-ascii?Q?bKxx2E/33HTbPw2oicyiKNbHTT+goJxuOJpNsbh7fn7sWxwtU5/F+uGATVWC?=
 =?us-ascii?Q?Kk4mHyZ7l9ZtaAASa0ZeeeIHnrYCFQP7DbCjkGYuuoPw+DGSyC3v6ZnwgtWA?=
 =?us-ascii?Q?yctkqOsEhKM2JHj2NRjDytuqymrr2FZiHlhNa0NNP3/7yC8L0bWrGP+pyY2F?=
 =?us-ascii?Q?1JXU6toF2zilulm3nGQYFjF1XFCenGAoy0JVyW0FH8jGeyIyQJoCZJgJ13lV?=
 =?us-ascii?Q?pn7Xy0+TEAUJhL5mhqLhPReh5N+SSZjzzuxF3drZhAlR1ov9UZ0mXb79Z3wK?=
 =?us-ascii?Q?/CWdeHsdu4CsA7iGvcpyB+VOsc3Y1rPhI2dJJVVJ+d58MVQY3Z/AMw7w1W8y?=
 =?us-ascii?Q?4YplSjMuWqpuHxBWIXRfoa/dCBl7qAl8DGMFAc9qL0+/lhwzNH6RDJC7HOwa?=
 =?us-ascii?Q?/E1aRCCgXxAWtKToVmchakf0azaweAv8CldcVJ5i2eeJ2yskEizK0HlwFRtK?=
 =?us-ascii?Q?/6v9e3JNA6OgudnJJjjVmgbdvf/Ohzey2zFZsRf/clCrFPBOO1leInxCPfSj?=
 =?us-ascii?Q?3f2L3AA+Dd+9UHqnZeA3GidFVmTqabpDFy8yG3pnIqtgXupAujcfqyQqlpui?=
 =?us-ascii?Q?LUtv9aSIIT0u/SFuhUk139MpMoji7oOIZyAj119GsQPUO17XlMC/Qn+TemJE?=
 =?us-ascii?Q?dwaGhL/akLRWXCdE2vo3V3jQ+XeYr83/TH0WkgoVuphAEOqSkjY0AKww29l1?=
 =?us-ascii?Q?z0fqZDHQKz54W11YzGBACa4MVhUMTgS1Z8JZtXxJMgvVWcb09U9JRTa9LBX7?=
 =?us-ascii?Q?qE7sHHTIdj1QOFMzS5DUnKEMwQ8PTeImDVK8Gx5d9O0XR+wxxRHpxmcSRbID?=
 =?us-ascii?Q?ocw+/Iy1h/gW/XJySszNOeK8xM6KKjq3DZUdTJNlfrAMWXp3tE7xkuIWmXOt?=
 =?us-ascii?Q?/gGY+vQSMrq/WMtmxyNcb32ixsMg63sHkQc8PjWcscO2kAIhI77wxicNGg9y?=
 =?us-ascii?Q?Sa8H8iT1BZd2YJduvVMazcYMq89O0F80soMiDe+HvDj+5I8BDIjIoQmoIqQ/?=
 =?us-ascii?Q?l5hpgQmHxznb57ifWRc7t/MdfxD8fyR7DjEbf0uJ?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b720841f-411c-4c5a-afeb-08db864c4011
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jul 2023 22:30:25.6646
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: fd4X/IlFgBMMJZYuaRei3VfUzQApRSea22BJk+eSgfT6wEdxjKkn1fUK3hfGa3c4
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB7508
X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Sun, Jul 16, 2023 at 08:09:02AM -0700, Catalin Marinas wrote:

> In terms of security for arm64 at least, Device vs Normal NC (or nc vs
> wc in Linux terminology) doesn't make much difference with the former
> occasionally being worse. The kernel would probably trust the DPDK code
> if it allows direct device access.

RDMA and DRM already allow device drivers to map WC to userspace on
demand, we expect the platform to support this.

> > So the userspace component needs to be responsible for selecting the
> > mapping, the same way using the PCI sysfs resource files today allows
> > to do that by selecting the _wc variant.
> 
> I guess the sysfs interface is just trying to work around the VFIO
> limitations.

I think just nobody has ever asked for VFIO WC support. The main
non-VM user is DPDK and none of the NIC drivers have wanted this (DPDK
applications areis more of throughput than latency focused typically)

> > This is particularly suited for the case (which used to exist, I don't
> > know if it still does) where the buffer that wants write combining
> > reside in the same BAR as registers that otherwise don't.
> 
> IIUC that's still the case for some devices (I think Jason mentioned
> some Mellanox cards).

Right, VFIO will have to allow it page-by-page

> I think this interface would help KVM when we'll need a cacheable
> mapping. For WC, we are ok without any VFIO changes.

Yes, it may be interesting to map cachable CXL memory as NORMAL_NC
into userspace for similar reasons.

Jason