Date:   Thu, 23 Feb 2023 14:10:56 -0400
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Yosry Ahmed <yosryahmed@google.com>
Cc:     "T.J. Mercier" <tjmercier@google.com>,
        Alistair Popple <apopple@nvidia.com>,
        Tejun Heo <tj@kernel.org>, Michal Hocko <mhocko@suse.com>,
        linux-mm@kvack.org, cgroups@vger.kernel.org,
        linux-kernel@vger.kernel.org, jhubbard@nvidia.com,
        hannes@cmpxchg.org, surenb@google.com, mkoutny@suse.com,
        daniel@ffwll.ch, "Daniel P . Berrange" <berrange@redhat.com>,
        Alex Williamson <alex.williamson@redhat.com>,
        Zefan Li <lizefan.x@bytedance.com>,
        Andrew Morton <akpm@linux-foundation.org>
Subject: Re: [PATCH 14/19] mm: Introduce a cgroup for pinned memory
Message-ID: <Y/esMBOyTaJnv5CW@nvidia.com>
References: <Y/Ua6VcNe/DFh7X4@nvidia.com>
 <Y/UfS8TDIXhUlJ/I@slm.duckdns.org>
 <Y/UiQmuVwh2eqrfA@nvidia.com>
 <87o7pmnd0p.fsf@nvidia.com>
 <Y/YRJNwwvqp7nKKt@nvidia.com>
 <87k009nvnr.fsf@nvidia.com>
 <Y/bHNO7A8T3QQ5T+@nvidia.com>
 <CABdmKX18MY19bnsxN5W38Z9zmoaZx+S4+zzN_5XCYDBruwPrLg@mail.gmail.com>
 <Y/eiLBo88pgr2IUm@nvidia.com>
 <CAJD7tkadBRP22qP63-SjKSch1im9sHLoMzc6c2h10+ggbuxqMg@mail.gmail.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAJD7tkadBRP22qP63-SjKSch1im9sHLoMzc6c2h10+ggbuxqMg@mail.gmail.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?F8BS+5wpS1p7jnfRKneLY7201Qn0jCflI5BRLafA7R1jyCtLP/DwgmaO+4of?=
 =?us-ascii?Q?iJVBtfXNyrNWzCASXNcEC2mz303KWS2lf/4qvWPSJPcnH7CbsD0A02IbIvej?=
 =?us-ascii?Q?eRZSLT2WouusZsO8CTEmYdGseGKMoiiZIytFPonMntwKlkmL74m061aQNei4?=
 =?us-ascii?Q?uq9uC5Fuiwa8ZwxB7kBy2YGVoN62+legFVV7t9Nw5rPBNOHNbAgEfdAOsSWf?=
 =?us-ascii?Q?wV5K0MHHtR5Y96eYsnUZRNw9vZO5gjQcvyyhxP2OdduZ7RZfVm6DDkBvWxJQ?=
 =?us-ascii?Q?PEgLjIOxyTCvKNZMlAPN5r4yg8evkmLNBunLd/qTDb+j9d3KC4MpzwgY9wx/?=
 =?us-ascii?Q?b8TwOQTY35gwk81oKnk9I6tFseatcuIwIdDvwtDUKkVKWPD4zg4/QR/j4E4J?=
 =?us-ascii?Q?6GkjFkFH9NzaZPIQR8GBSYdm/mQ6gbqdrXBnnrWkCCbgYe5Kf2NSTFtoWfuy?=
 =?us-ascii?Q?igrQLf3j7nqgDhlTUAZ678gpLLLYdNLmqHvXLMhn2YBfuNtk+bTBuR/khe3m?=
 =?us-ascii?Q?SdlDdRrCh68sZ8+J94BDINI+QhRsGXswgy0ZS1Tvip+zOwJpuS4c4GgxyhHW?=
 =?us-ascii?Q?I4v4nDaHdf9R+WPJuzsAffMhDtD+kWVSyv8PD17Gg5JYC7XEiXJCI93FKVGU?=
 =?us-ascii?Q?f0DUy67Ldmgm8+ipQ1LV6GYBNV/m2Ym5u4sTzl1wSTLhEpOQw+WI3M/Y+ZAG?=
 =?us-ascii?Q?ytUc+iisnjNMcEoIkB5LuL5CWBn0zu0TafcpjD/1vRF74J7rj3WwH6RS73gC?=
 =?us-ascii?Q?sV/NqMKfnWVh5NEi9yCiBzsIC0O3EN/6W9MqPLdc0e5Zyoqq0IH67jXYjt4d?=
 =?us-ascii?Q?5M76wKL9cQFU+FB0DT2LvCx1oBigelsbxHGy0Jy93E7gOqir3qlpSRlEdB5v?=
 =?us-ascii?Q?92R0TpdRE7Tg9J7NwihAO+uXZhAlN5Am6JCr6IhQkJtzH+HwIiH1jI101xkn?=
 =?us-ascii?Q?0mX7JyuzMhPFbLPOlKrpFJtdZek10qaUNg1n9+Sff5iZOi4ggKe6ObqGOfN5?=
 =?us-ascii?Q?liGEunca9gPMyn8n/aQ0+oe01t/LqI2c8trdHdlq9/Sx+YyljV9yVaxvJ+25?=
 =?us-ascii?Q?2jUc2iirkCVxBx4Zr1pYzzOUlAsSerjpht/1UmTYXamjNjO0eAbuSRkwaBfP?=
 =?us-ascii?Q?Gpejpw8WdF2JN9Fap4ZBasP94mH0hmcSRCTx5xaem6BGNjKFF/toVtUP9l65?=
 =?us-ascii?Q?jgIrc3rNpNOFRON6ev/6uvRkMw6zM4wZGuOWoksbPpjvSTzi1LIorDhhL3gO?=
 =?us-ascii?Q?Pd0vSZoegIten3RPVwoNkHZQntvHW0ETKTUuuAjSQmidu6S/17hxypZ+Sw/V?=
 =?us-ascii?Q?2WREv2vOnkhQNi44wVwXazKFa2XCXNuVpsCJm6fOsVcssWBCz9fi8dfYw6L8?=
 =?us-ascii?Q?zHdGfBRutnfwwf1Rong4rnCVT96/NT4SSZHupTQ97sdfk+LoacznZxhp6e8R?=
 =?us-ascii?Q?zdgnWzNiT8+KlushFbbwPy+AZaexu7Llez2YSb2fu8MUVn4iximanhZFO+KK?=
 =?us-ascii?Q?5/v58uqDf3CV1MU49VAIFMtmHNz0r1yEzsrNGC+TnJdtzrkFJZ008nNH4dWa?=
 =?us-ascii?Q?cG84/tyIA/cMpW3CvuoMhrRmCcnsUmkkcenGzql3?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7caabec5-fefe-4683-076a-08db15c94f7a
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Feb 2023 18:10:57.0451
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: q+2jp3Gg951800U1wXfEWiGl1DOjJwHfpQjlqbPLzwokoW2uGiLY/xP6aQ/1sbPC
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4355
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thu, Feb 23, 2023 at 10:03:50AM -0800, Yosry Ahmed wrote:
> On Thu, Feb 23, 2023 at 9:28 AM Jason Gunthorpe <jgg@nvidia.com> wrote:
> >
> > On Thu, Feb 23, 2023 at 09:18:23AM -0800, T.J. Mercier wrote:
> >
> > > > Solving that problem means figuring out when every cgroup stops using
> > > > the memory - pinning or not. That seems to be very costly.
> > > >
> > > This is the current behavior of accounting for memfds, and I suspect
> > > any kind of shared memory.
> > >
> > > If cgroup A creates a memfd, maps and faults in pages, shares the
> > > memfd with cgroup B and then A unmaps and closes the memfd, then
> > > cgroup A is still charged for the pages it faulted in.
> >
> > As we discussed, as long as the memory is swappable then eventually
> > memory pressure on cgroup A will evict the memfd pages and then cgroup
> > B will swap it in and be charged for it.
> 
> I am not familiar with memfd, but based on
> mem_cgroup_swapin_charge_folio() it seems like if cgroup B swapped in
> the pages they will remain charged to cgroup A, unless cgroup A is
> removed/offlined. Am I missing something?

Ah, I don't know, Tejun said:

"but it can converge when page usage transfers across cgroups
if needed."

Which I assumed was swap related but I don't know how convergence
works.

Jason