Date:   Mon, 30 Jan 2023 09:20:41 -0400
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Tejun Heo <tj@kernel.org>,
        "Daniel P. Berrange" <berrange@redhat.com>,
        Alex Williamson <alex.williamson@redhat.com>
Cc:     Alistair Popple <apopple@nvidia.com>, linux-mm@kvack.org,
        cgroups@vger.kernel.org, linux-kernel@vger.kernel.org,
        jhubbard@nvidia.com, tjmercier@google.com, hannes@cmpxchg.org,
        surenb@google.com, mkoutny@suse.com, daniel@ffwll.ch,
        Zefan Li <lizefan.x@bytedance.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        "libvir-list@redhat.com" <libvir-list@redhat.com>,
        Laine Stump <laine@redhat.com>
Subject: Re: [RFC PATCH 14/19] mm: Introduce a cgroup for pinned memory
Message-ID: <Y9fEKfdlL+hCiWpz@nvidia.com>
References: <cover.f52b9eb2792bccb8a9ecd6bc95055705cfe2ae03.1674538665.git-series.apopple@nvidia.com>
 <183372b80aac73e640d9f5ac3c742d505fc6c1f2.1674538665.git-series.apopple@nvidia.com>
 <Y9RFs+90TyzVMs83@slm.duckdns.org>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Y9RFs+90TyzVMs83@slm.duckdns.org>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?0B6AcIKgj37S/hSkjtTjqQ/ObqsxPAOt0kzUfCyxtdHwiOeI859IiwLM8i5v?=
 =?us-ascii?Q?P4UyeN5lmiJiQjYuXh21JeLcbZI0H9kSNEyUXM3mO6XYObapupSKxzl8gRfQ?=
 =?us-ascii?Q?6/5fbUw0gcOxMFHPdWWmttu4MEVFjnw98yNLVFYfTaIDYMi+UN97On6+nhea?=
 =?us-ascii?Q?nNP3eKDkcV/6Zsz6mNiViS2ZNHqJln5M3sbHCFGOk9uMlW5l0wUlw9tffsMQ?=
 =?us-ascii?Q?2+IpQw3WlPRhOKl2GIkajicQT69UeIXP3GfWlZcz23v61iXG9Ga2F2FO8OW2?=
 =?us-ascii?Q?GQme0zezTn+A1J28A6dajidc8ENB/jkYaDTSwBr62J5X2LVaKFpnZ1Q6lpYH?=
 =?us-ascii?Q?79tKDGZdtjV4GDBlb0mVCzUoY2/q96WJbMQUACjWgx9wf19VIikf3DVw7opw?=
 =?us-ascii?Q?iY/uudCbUxl2G9qlPuVzDn9Ohq7d15dKrRFEtQei2FBX1qN/5HXV/uHcaI9n?=
 =?us-ascii?Q?w5mw62lUWlWf9DM505R3wVa17sKtzSzOMkgbUcVenK0zT5mREme8crS+LB6r?=
 =?us-ascii?Q?CghAFzDiRLAOMeTl2NoLloBGKEroxOPIbY/h1hF3KW5R1vpjIXPIJH6xHAk+?=
 =?us-ascii?Q?FZm5J5yhKPPY50ZaefVpoPfhUL7l5/VrgMu+4CbEQhJ+SOwlFU5jrpPjvy/7?=
 =?us-ascii?Q?m6o0e4SqvugC2chARi/PwP8vyygMP8ynBdUqKlySzDggdFLCk1BmCKCYjWQE?=
 =?us-ascii?Q?gsZo3l4ElkRX1CEwBYFpjTzTQuFlEPp/ONyAMf8bIb9SyGz+tZ6eCacB5ZH8?=
 =?us-ascii?Q?UobXEXHirPgldeMl6DQ49JANedNPXgpLwsk0ZinPFzVDxpn2SVmvzT27DOGr?=
 =?us-ascii?Q?gQQZsHmMP5sNvsj/KL6jOQfR8mZFBHbCOw4myS3WY1PJ07iqx0G/EjnZBqpg?=
 =?us-ascii?Q?Htqjr4fWyrPTndED1Gb2umMK4sezr+7sBWx5gj/pTNUocaPgnptp62TrAYu4?=
 =?us-ascii?Q?2el3qDWUT2rvK5/tJAYEU981DClSLdsVotqbLqZh52PG+QvS9lavjsgjkJn+?=
 =?us-ascii?Q?NpBUuPw7gQwt8YhPMYZN6FSebZETffR7AH66xjI23a3dJY7Kt26f7m3IM99m?=
 =?us-ascii?Q?UIIsADCPcEG6TFoeG4j+AkHICsIEj7hSuP0mPlEJqmRJWceKu8vJwT6qb5Dl?=
 =?us-ascii?Q?RqMMDcqVrUyjOVK4MmyGzqBLa2MhSlw/Z1K41yIyHRAgIDxaJhmsU9sZ4F/F?=
 =?us-ascii?Q?3qNoQixC6Sp9b4Tp7F+QXR3VZLwgIq+N6KD57nXh45FLsA+ApkAk3SuIZ4Ui?=
 =?us-ascii?Q?oLV3GuKmdgz5k1LcFbcgwvBNoucVWPm3LxanmSBshjRDJT5ddydz7dp0pYp2?=
 =?us-ascii?Q?fYG2xUp8NuqocHGh1mwlySntB7gHQZ85y0NbAo7PIUikVMZHEdav+gfVs7gH?=
 =?us-ascii?Q?AbOFCYOmIJLQm4vAv/9aOue/r1lGzOq9sZf9gNak7YGyAMWTlRPdd7QUm24q?=
 =?us-ascii?Q?X8nZY7saRVZn/PFTbACTWxciE78ZrWwSA16ALFqutxlGykvxeWMvpKx2enCZ?=
 =?us-ascii?Q?KmcMd8+d5rqr9yrhlABd5olA6QyH7/z0MG6uKWd1rqmsW4Lookl7TePHxejb?=
 =?us-ascii?Q?JAxqPGb8/pDLqbp/iWnLomd2JWQrXMub9EMLD2Hp?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5bc8978f-06c7-43d8-1347-08db02c4c98b
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jan 2023 13:20:42.3112
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: qAPUbRrKUzMTHz3zHcwuLTVaw5U2wShSkTyvQZOlJ1aJFdeeHzSMdPxmoN7gHwNK
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB7500
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Fri, Jan 27, 2023 at 11:44:19AM -1000, Tejun Heo wrote:
> On Tue, Jan 24, 2023 at 04:42:43PM +1100, Alistair Popple wrote:
> > If too much memory in a system is pinned or locked it can lead to
> > problems such as performance degredation or in the worst case
> > out-of-memory errors as such memory cannot be moved or paged out.
> > 
> > In order to prevent users without CAP_IPC_LOCK from causing these
> > issues the amount of memory that can be pinned is typically limited by
> > RLIMIT_MEMLOCK. However this is inflexible as limits can't be shared
> > between tasks and the enforcement of these limits is inconsistent
> > between in-kernel users of pinned memory such as mlock() and device
> > drivers which may also pin pages with pin_user_pages().
> > 
> > To allow for a single limit to be set introduce a cgroup controller
> > which can be used to limit the number of pages being pinned by all
> > tasks in the cgroup.
> 
> The use case makes some sense to me but I wonder whether this'd fit a lot
> better in memcg rather than being its own controller.

As long as the pinned limitation has its own bucket it is probably
fine? The underlying memory allocations should have already been
charged to the memcg - so we don't want to double account.

Alex and Daniel were looking at this from the qemu/libvirt
perspective, perhaps they have some insight what they would like to
see?

Thanks,
Jason