Received-SPF: pass (google.com: domain of linux-kernel+bounces-203100-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
From: Martin Oliveira <martin.oliveira@eideticom.com>
To: linux-rdma@vger.kernel.org,
	linux-kernel@vger.kernel.org,
	linux-pci@vger.kernel.org,
	linux-mm@kvack.org
Cc: Jason Gunthorpe <jgg@ziepe.ca>,
	Leon Romanovsky <leon@kernel.org>,
	Bjorn Helgaas <bhelgaas@google.com>,
	Logan Gunthorpe <logang@deltatee.com>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Tejun Heo <tj@kernel.org>,
	"Rafael J. Wysocki" <rafael@kernel.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Mike Marciniszyn <mike.marciniszyn@intel.com>,
	Michael Guralnik <michaelgur@nvidia.com>,
	Martin Oliveira <martin.oliveira@eideticom.com>,
	Dan Williams <dan.j.williams@intel.com>,
	Ard Biesheuvel <ardb@kernel.org>,
	Valentine Sinitsyn <valesini@yandex-team.ru>,
	Lukas Wunner <lukas@wunner.de>
Subject: [PATCH 0/6] Enable P2PDMA in Userspace RDMA
Date: Wed,  5 Jun 2024 13:29:28 -0600
Message-Id: <20240605192934.742369-1-martin.oliveira@eideticom.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?NfAlDP+QdePrynhmfTr9zTOJJlcnAZ2KfM0R11/QIrLHR/aruq0k2bj1BSan?=
 =?us-ascii?Q?PxtGGmWKE4J5CdtOxVpyJ9aWZsMZ0d5elaeQqVHM8b3ZB8ETGM2u18veDyR/?=
 =?us-ascii?Q?DFPfEO07JpPtoNbFcfp7CNOqW93jUeXIg1c11EOqoDN70H/jhhllboRmy+to?=
 =?us-ascii?Q?w1HCvjZCqP0NkdRFv9kS+i3dHrHSOvQF1tf/tLUeC6wKG4GrqiYkXmCguN0W?=
 =?us-ascii?Q?88k07D0DBhpPR4B6bnpOFOxp4lNvFVMZuiCV/VDKLkJwRr609nkdMXS9sGdE?=
 =?us-ascii?Q?isr5TtjRaftC+Y1EVtK7r8ntaD3sL/W/HA0Nuk4V8Yict4n0nQDGa2gfCFNe?=
 =?us-ascii?Q?V/VUsa/ZrtHolrGaU/qyqBpEyMPUP0wewN2p0N+so/RwfE585s5cj+RM/mB1?=
 =?us-ascii?Q?wkv+OTacuX+zpZNePoGs2J6ZAGg8mFp6nbNxJF1LWjGVRPHTxsqJlTak1tqG?=
 =?us-ascii?Q?giSQwDHLHXje1RPc0cfKMbyY5Tk/33Mq7xLIXgkxrsa6gs+Gbx25GQkqxfAs?=
 =?us-ascii?Q?w7mEZKqSZKlQqX9lKT8uj/kuJbv2jDf3ajRUAMKxBbYnCnWttrnXJJJ3QpOO?=
 =?us-ascii?Q?8s0vuAPPdByCGOP9PBBfM5zm6I/C9l4kc702PVboBQ/doLosq0DbgT7pU1E2?=
 =?us-ascii?Q?Hi9tgfOSGfYug5yBhb41uWIgdRdfp+h/LSvqSjiU4M7q2JG+5IgMfrdhw6wa?=
 =?us-ascii?Q?FDbJLbmrGKw3ZTmrAryR6ajMtCEbndo8b6CyfObPDqQe5OSiaIJP6QutXLtf?=
 =?us-ascii?Q?aFVrq/j9ZvK/ZTFgjjKUbTkfH/xfMJ4TljW4/e4eJHr6walzPYJzWHWHVtN/?=
 =?us-ascii?Q?R+oVy2pm4YGA2maSgpQBVWIop3aLc2RyevURvGFjLkZ1ul6vuGphdvasaw/N?=
 =?us-ascii?Q?t8cZzPXgIZDAaMpvlBLEBsLytM7SGxKPHx2xpr2J4Yb9j20G7f04yimc7t9M?=
 =?us-ascii?Q?AKya56vZ7DfM5W1gwLUBQQ7ymngOPotdeBKGLK/QaUI7cenn5hrowfV81H74?=
 =?us-ascii?Q?Q//uXuQShN5oQQBdv+ht1PAoKDJqvMqVDl1Dnen8IJVDC+X7JYbLNXLlVy4B?=
 =?us-ascii?Q?iFRX+d8jXJk7MzQUSr1W84Xb2jxlNTYmRQ9yzNFW4bVb8UkiWSXso0ifUOpw?=
 =?us-ascii?Q?BAEWcd8Peqr3SdxF18v9aqdYU1gXIggCncF36vWvm7JBwlkjYaSuuCmQaxRv?=
 =?us-ascii?Q?ycctx7B0JXIIax41XFNyTysyg6yM/wVEfXadreutMgsxCLwM2OiElQ9rVS2G?=
 =?us-ascii?Q?+CSSU6Tf0/XyUqUbwyem/QLYqNP7Mbo7V3EeHberICuUwwdKLZ73PHFDXlLq?=
 =?us-ascii?Q?SDH6HOF/Jg+3xxuyXHqTUSfRaNszTd8azY52XEjCqLCU9VsBSeQSq05m6p4F?=
 =?us-ascii?Q?+dbPfjZqC0a42t35p4cixXpxSec2z8iMBlE8h81LejxLuwCQ5rg+UUiA6AgZ?=
 =?us-ascii?Q?YcWmz7o7tzzML5cgFaPDVyeaxmQazV3rUUabjFBhjaggy1Y0RRET2v1VDh57?=
 =?us-ascii?Q?4IAUVkqxUS9KGXekPe7zNwwnzWwtCHZUCfud/5daWkP44G8JvFUDTPlmIufp?=
 =?us-ascii?Q?g//7ci7rim3+X/MBTp7nN3mly6IaL9o15wpZkPzwiMSONVDhsnhhFDX51V+y?=
 =?us-ascii?Q?5A=3D=3D?=
X-OriginatorOrg: eideticom.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 48b86261-45e5-4e55-5546-08dc8595e6f6
X-MS-Exchange-CrossTenant-AuthSource: MW3PR19MB4250.namprd19.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Jun 2024 19:30:05.2381
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3490cd4b-0360-4377-abb1-15f8c5af8fc2
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 1k3er67CFE4N2MmJR94CeHeDErKUJM5IfRvkbhinqgElgDFvNRcukJxSztK4wRJG07Mu6BWjSEAxC6iWGdmQJJPnh8aq2Q8syY717iOOUKs=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR19MB7093

This patch series enables P2PDMA memory to be used in userspace RDMA
transfers. With this series, P2PDMA memory mmaped into userspace (ie.
only NVMe CMBs, at the moment) can then be used with ibv_reg_mr() (or
similar) interfaces. This can be tested by passing a sysfs p2pmem
allocator to the --mmap flag of the perftest tools.

This requires addressing three issues:

* Stop exporting the P2PDMA VMAs with page_mkwrite which is incompatible
with FOLL_LONGTERM

* Fix folio_fast_pin_allowed() path to take into account ZONE_DEVICE pages.

* Remove the restriction on FOLL_LONGTREM with FOLL_PCI_P2PDMA which was
initially put in place due to excessive caution with assuming P2PDMA
would have similar problems to fsdax with unmap_mapping_range(). Seeing
P2PDMA only uses unmap_mapping_range() on device unbind and immediately
waits for all page reference counts to go to zero after calling it, it
is actually believed to be safe from reuse and user access faults. See
[1] for more discussion.

This was tested using a Mellanox ConnectX-6 SmartNIC (MT28908 Family),
using the mlx5_core driver, as well as an NVMe CMB.

Thanks,
Martin

[1]: https://lore.kernel.org/linux-mm/87cypuvh2i.fsf@nvdebian.thelocal/T/

Martin Oliveira (6):
  kernfs: create vm_operations_struct without page_mkwrite()
  sysfs: add mmap_allocates parameter to struct bin_attribute
  PCI/P2PDMA: create VMA without page_mkwrite() operator
  mm/gup: handle ZONE_DEVICE pages in folio_fast_pin_allowed()
  mm/gup: allow FOLL_LONGTERM & FOLL_PCI_P2PDMA
  RDMA/umem: add support for P2P RDMA

 drivers/infiniband/core/umem.c |  3 +++
 drivers/pci/p2pdma.c           |  1 +
 fs/kernfs/file.c               | 15 ++++++++++++++-
 fs/sysfs/file.c                | 25 +++++++++++++++++++------
 include/linux/kernfs.h         |  7 +++++++
 include/linux/sysfs.h          |  1 +
 mm/gup.c                       |  9 ++++-----
 7 files changed, 49 insertions(+), 12 deletions(-)


base-commit: c3f38fa61af77b49866b006939479069cd451173
-- 
2.34.1