References: <Y/T+pw25oGmKqz1k@nvidia.com> <Y/T/bkcYc9Krw4rE@slm.duckdns.org>
 <Y/UEkNn0O65Pfi4e@nvidia.com> <Y/UIURDjR9pv+gzx@slm.duckdns.org>
 <Y/Ua6VcNe/DFh7X4@nvidia.com> <Y/UfS8TDIXhUlJ/I@slm.duckdns.org>
 <Y/UiQmuVwh2eqrfA@nvidia.com> <87o7pmnd0p.fsf@nvidia.com>
 <Y/YRJNwwvqp7nKKt@nvidia.com> <87k009nvnr.fsf@nvidia.com>
 <Y/at3iYz/xBSPPM+@infradead.org>
User-agent: mu4e 1.8.10; emacs 28.2
From:   Alistair Popple <apopple@nvidia.com>
To:     Christoph Hellwig <hch@infradead.org>
Cc:     Jason Gunthorpe <jgg@nvidia.com>, Tejun Heo <tj@kernel.org>,
        Michal Hocko <mhocko@suse.com>,
        Yosry Ahmed <yosryahmed@google.com>, linux-mm@kvack.org,
        cgroups@vger.kernel.org, linux-kernel@vger.kernel.org,
        jhubbard@nvidia.com, tjmercier@google.com, hannes@cmpxchg.org,
        surenb@google.com, mkoutny@suse.com, daniel@ffwll.ch,
        "Daniel P . Berrange" <berrange@redhat.com>,
        Alex Williamson <alex.williamson@redhat.com>,
        Zefan Li <lizefan.x@bytedance.com>,
        Andrew Morton <akpm@linux-foundation.org>
Subject: Re: [PATCH 14/19] mm: Introduce a cgroup for pinned memory
Date:   Thu, 23 Feb 2023 11:35:54 +1100
In-reply-to: <Y/at3iYz/xBSPPM+@infradead.org>
Message-ID: <87fsaxnrqa.fsf@nvidia.com>
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?yLgT7/XTqULLC/3pEsuI+WrzdM2G/7wITQ2gXngBf2UpYPdoArHSgW6xnv1U?=
 =?us-ascii?Q?Rvsf8WlZZLXcyIy/Ge/9QoV6sR3dNG0h0Fd3UGsOVeHcJ1btdEZ2WoH0/iK9?=
 =?us-ascii?Q?GrnSDs7F2TdOX1hrEvAOGbWbqpj80CAFw8jwlcvB83hKG/Zh8Q47S64ck3Pf?=
 =?us-ascii?Q?sohDYw+nQIaaxaHBHYK544GGZSbC7NsKM8gpR5cPWqRHQRP4vUS7yTZojBzy?=
 =?us-ascii?Q?XQqcgJTT3DfsHw99uyAM7+k9+hq4PISNluwPCub2/KAozcSNjWXAoDFViqd+?=
 =?us-ascii?Q?qTdZy1g6Wdzjn0dnJezw5mId/LvfTOUgw4YW3hll+1hpV9/ZOucIo/JsMgy5?=
 =?us-ascii?Q?0EDfE9HTmKxwOfc4LqyxMgsICkm4L519a8SwR+TXY2zvvkPXfeB8a3p0WwqY?=
 =?us-ascii?Q?BEJDxKW7PuyCk7yNJDaXl23G/ArtEJbl+YTfw7YTdJBxxJ7YKb+uZrg2aGFD?=
 =?us-ascii?Q?gOGvHq9phVDT1udmIz50gX7acJjGU8KE51S+ltvQGf9wy5kAFawneLUz0rYH?=
 =?us-ascii?Q?THFoeykOnOSUteWFuj+Y22QvUpG4Aqzju2zMhwnx635ITRF7c+e+m5D/QddN?=
 =?us-ascii?Q?jZNMNLwP+L2M30QbS+c204pmz66IC8Un8xB+qD3Z3RMjB/AoHbo/pVUMrS2x?=
 =?us-ascii?Q?/VnoiJYWHcDUbc86UMw1oTuev88SAeRuil1m56fda7VQfI2mFdu6t5o4B0nO?=
 =?us-ascii?Q?aQiSYTAQeKyN9sRqaU2DaPSbF2STNhzfrAtKvSTfVE7BrskkUHm/EcZlgqbV?=
 =?us-ascii?Q?uTFs7Mg8/mGro43wJJM/F6l3hnQz8iUwbBVplD3Wh8qhPxFlBKKSsE3Dmek0?=
 =?us-ascii?Q?8d+LTQ8SVF2Y4Dl9i5cb/+9+/FEnhaSZmB8Eqw37V7P50czCkG3aW6bdFvGj?=
 =?us-ascii?Q?3OHRpk/atXKATp3YvU6A7TiYVgFxEtw+qtYhxtqy9f//zXf1aUJTu/Crpy1g?=
 =?us-ascii?Q?arwNGLfBf+SALFWKAD83aYDG++lLHe8VxAg5gR+g468Nbn5Y64NO17E9wTPz?=
 =?us-ascii?Q?jXowd0f2LvSN7pIVBaDX4JWrmK/5AbDaeULHMktke1F5dBv9hpd9/xbDmXCU?=
 =?us-ascii?Q?R2LF+c6NzrR4/C6x7JGlXO/Gw9qdCfMAB7Kf9jPwf2y+rWVIhJPuThxeM4LQ?=
 =?us-ascii?Q?ulF3hOHVLWBps4bImf/Bu381EEKseZbrwjcqI1O0O/1yT/SW3xyW070SCJaY?=
 =?us-ascii?Q?JD3bKSHwZKzdt/s/ZUrZ40zcSUkCTg5o6yxrRzic3/QJpB6lRHyg2eGzqo4f?=
 =?us-ascii?Q?oFil+vh5kaIaUlgrrpWBr0744NP+JLo9jvMVe4FRcOB6m+AIeUtRQ9PrxtWk?=
 =?us-ascii?Q?ccyJXbrLMETNo32snaDL7WkW9qkN0PoXkXC0NNfQGFHwdBOgsVmPaAhaYye0?=
 =?us-ascii?Q?UP+I2kHK/3PPrCylJ6Nf9mtSOiByn+STxSG3IqAvcBNQegehM+D7nCGJSUbe?=
 =?us-ascii?Q?mJJYIfbGLbAASjFVXkcFgG26sNSkEfMYW7VJOwtat0mnS/P6BAKzpjqkvtj8?=
 =?us-ascii?Q?89VdbOUdeMRoCRC9qORae5QB9sBD2zV1KTDt4mEFyCAFsDI+JhmpOBcEMtXm?=
 =?us-ascii?Q?zWPZtYvhLK0CzlHx26/9TpnikPhaJWGOKcVU0c8I?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0c808221-5ac3-4f02-64d9-08db1535e902
X-MS-Exchange-CrossTenant-AuthSource: BYAPR12MB3176.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Feb 2023 00:35:49.2411
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 46JxylTl6Wkb4IWfOuMnCidEJgvbxd3muLPpwSGRcpkHzf8wKQLFKlH2F7aYqoWH+Kh52d8Slny4R4Ldefwb8g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB8354
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


Christoph Hellwig <hch@infradead.org> writes:

> On Thu, Feb 23, 2023 at 09:59:35AM +1100, Alistair Popple wrote:
>> The idea was every driver already needs to allocate a pages array to
>> pass to pin_user_pages(), and by necessity drivers have to keep a
>> reference to the contents of that in one form or another. So
>> conceptually the equivalent of:
>> 
>> struct vm_account {
>>        struct list_head possible_pinners;
>>        struct mem_cgroup *memcg;
>>        struct pages **pages;
>>        [...]
>> };
>> 
>> Unpinnig involves finding a new owner by traversing the list of
>> page->memcg_data->possible_pinners and iterating over *pages[] to figure
>> out if that vm_account actually has this page pinned or not and could
>> own it.
>> 
>> Agree this is costly though. And I don't think all drivers keep the
>> array around so "iterating over *pages[]" may need to be a callback.
>
> Is pinning in this context referring to FOLL_LONGTERM pins or any
> FOLL_PIN?  In the latter case block based direct I/O does not keep
> the pages array around, and also is absolutely not willing to pay
> for the overhead.

Good point. I was primarily targeting FOLL_LONGTERM users. I'm not too
familiar with block based direct I/O but from what I can tell it
currently doesn't respect any kind of RLIMIT anyway so I guess the
requirment to limit pinned pages there isn't so revelant.