Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 16 Jan 2023 14:23:56 -0400
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Daisuke Matsuda <matsuda-daisuke@fujitsu.com>
Cc:     linux-rdma@vger.kernel.org, leonro@nvidia.com,
        zyjzyj2000@gmail.com, nvdimm@lists.linux.dev,
        linux-kernel@vger.kernel.org, rpearsonhpe@gmail.com,
        yangx.jy@fujitsu.com, lizhijian@fujitsu.com, y-goto@fujitsu.com
Subject: Re: [PATCH for-next v3 4/7] RDMA/rxe: Add page invalidation support
Message-ID: <Y8WWPAHvZsXpwc0M@nvidia.com>
References: <cover.1671772917.git.matsuda-daisuke@fujitsu.com>
 <97b0362e256dd3eb022d81c30941edd8fb0caba9.1671772917.git.matsuda-daisuke@fujitsu.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <97b0362e256dd3eb022d81c30941edd8fb0caba9.1671772917.git.matsuda-daisuke@fujitsu.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?txICi5OcQtfnK6iYvCTaw8xl1r28dgQapaDApWjH3GRY5QFoyYJYVgDhzzi0?=
 =?us-ascii?Q?LIgBUiQBu3lEzzL9lOiTLxlhO8tW1SxlKFYeBl6B8qQN+Jh4j/l+KU+jUJNa?=
 =?us-ascii?Q?VQo92/KJnQWHfQxc2480eMWRiWz+Wq097pYNjsq+0Fq4NmCjd3OVfBEPTHsC?=
 =?us-ascii?Q?DBhs0A+hDjPCqPR7NTwfARLkeKkVjF4kQI6rqRUTV1l/dsKD89DZxJbw7oZz?=
 =?us-ascii?Q?7zDmktQaS2p/cG51YOilH8JF9V9oV9hHaWg6oEyLpNTe2mYB5VA+REWaHEEb?=
 =?us-ascii?Q?B5DJw+fHu4yT3z7X9yds4nNB8/Mrq8rexBM9EjlUlSq3w8vBTnZC76UFuxfR?=
 =?us-ascii?Q?MgYaSRQaOkEwkbA3m92Kw8PJU4tjgXU2W3rUby3JMkqbEIZR1h3zmaXIErgf?=
 =?us-ascii?Q?KrocUP1nzEOtWJOF5I3fGEEodCTVJirCLogEeQBxuAJS2lpI36fgj3++B0dN?=
 =?us-ascii?Q?STKUwVmgsaapGCAUnF/t5vEVGV4nsy/CZTrcocQapaKt1OxbN9NL+6F+MobR?=
 =?us-ascii?Q?z+LbVJVHV3+vQZLJ8d6DuGTR87SLkg4g66aHymx1GKhbsYDpU40KGhL0ovyy?=
 =?us-ascii?Q?E8ROhV0ih24kQthrgVFXjiWet5Aa4MSDqnHxmqDSpY9Nnk1jSENZFByqTXSS?=
 =?us-ascii?Q?6ER3t9kbos8S2vMCZBDCKqa1B4LXUlcy0RTs6h5ZrXI33oLuAxl5in5v91J9?=
 =?us-ascii?Q?oXl/9Pt6nBS4yD/XLEyD2ZGILMB1kRCZZpGWiriDH7belRQsNzetlw1hkt/7?=
 =?us-ascii?Q?3/NAT2QvN5NTI9O0I26r40EbX5QHiZhmGxmzd3ZBQji14uChREQxrwjZ8Qi1?=
 =?us-ascii?Q?32RL1bUuHsqVMrt2sxx8eTENPVyeb/ld6pNOuO07JEDetHTHgW4//iMyJ10Y?=
 =?us-ascii?Q?qEeZYIIwOYysHodibrcGg+YBCpMglDCJQWAhIb3D1CKSzJEjYLX+oG0oGvXz?=
 =?us-ascii?Q?hFJc1+bf/9d5WGZ6MWenkmyVVRUoGiMDqfex2ddDBEJU3YHcjkbmZB7lSGVd?=
 =?us-ascii?Q?utwja/ThfRzZvIpu6nnB3B50SYnmUxFdqr0eYqWK/sf93tLqQiNwOv5aXXsQ?=
 =?us-ascii?Q?LxhRbCDkFReevf50048HrfbKd2U2aZ+Fgc12fgD3AURILviWhKmPQrxDEAbF?=
 =?us-ascii?Q?mLRw7QheiqNaUGByp6EYl2Ny5ZpMa8AyAHCdU3mKl9t+Fz8kSOje96iZ57v6?=
 =?us-ascii?Q?QgDmrJtCpNjuuq8hJq8QK3H5OP475QPP/EpSA+1jMh3jsDUQxfH9SqZQebHd?=
 =?us-ascii?Q?bekw07alPiEeVcT/1lUs8X8qckikuKiJNMeeXKI4984+W1johMpxHLlJWBlX?=
 =?us-ascii?Q?eq60pY2LbYFkC/qIbgj8OGEf8W4vX2LXKiVpnluv16ZlnG7M78WCU9j+CbG+?=
 =?us-ascii?Q?LZDR3+Z5+/plw/pE0d8cDWMY44RMdpHEMyHBqyaC2ETyxi00sU0iz0VrDgaq?=
 =?us-ascii?Q?leNjBXqAvdQxMtxwbCM1WVKHx7TrPv5/s8q2mybBQiOHdF5oVU+RtXU7KLAJ?=
 =?us-ascii?Q?zYRZeBWJQCikiPCJ9aQkpbLiQEX0z3zdiVyIKLnSJ3J3FrFHxi1J6L/4Yt2z?=
 =?us-ascii?Q?Ju1D6MTrtjyZVgouCyfQR6YZbB6WJa/1bXghft9I?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6e989f49-0c24-4e5c-35c0-08daf7eed4e4
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jan 2023 18:23:57.4224
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: +qZJd4+rBM/F1JYOn2gVUt1V+PbWrEFcXN3jV2bkHuCGp0D4cqL/b3stzJncyYra
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7123
X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE
        autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Fri, Dec 23, 2022 at 03:51:55PM +0900, Daisuke Matsuda wrote:

> +static bool rxe_ib_invalidate_range(struct mmu_interval_notifier *mni,
> +				    const struct mmu_notifier_range *range,
> +				    unsigned long cur_seq)
> +{
> +	struct ib_umem_odp *umem_odp =
> +		container_of(mni, struct ib_umem_odp, notifier);
> +	unsigned long start;
> +	unsigned long end;
> +
> +	if (!mmu_notifier_range_blockable(range))
> +		return false;
> +
> +	mutex_lock(&umem_odp->umem_mutex);
> +	mmu_interval_set_seq(mni, cur_seq);
> +
> +	start = max_t(u64, ib_umem_start(umem_odp), range->start);
> +	end = min_t(u64, ib_umem_end(umem_odp), range->end);
> +
> +	ib_umem_odp_unmap_dma_pages(umem_odp, start, end);

After bob's xarray conversion this can be done alot faster, it just an
xa_for_each_range and make the xarray items non-present

non-present is probably just a null struct page in the xarray.

Jason