Date:   Thu, 23 Feb 2023 13:31:44 -0400
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Daniel =?utf-8?B?UC4gQmVycmFuZ8Op?= <berrange@redhat.com>
Cc:     Alistair Popple <apopple@nvidia.com>, Tejun Heo <tj@kernel.org>,
        Michal Hocko <mhocko@suse.com>,
        Yosry Ahmed <yosryahmed@google.com>, linux-mm@kvack.org,
        cgroups@vger.kernel.org, linux-kernel@vger.kernel.org,
        jhubbard@nvidia.com, tjmercier@google.com, hannes@cmpxchg.org,
        surenb@google.com, mkoutny@suse.com, daniel@ffwll.ch,
        Alex Williamson <alex.williamson@redhat.com>,
        Zefan Li <lizefan.x@bytedance.com>,
        Andrew Morton <akpm@linux-foundation.org>
Subject: Re: [PATCH 14/19] mm: Introduce a cgroup for pinned memory
Message-ID: <Y/ejAKU7l8DFrkR8@nvidia.com>
References: <Y/UEkNn0O65Pfi4e@nvidia.com>
 <Y/UIURDjR9pv+gzx@slm.duckdns.org>
 <Y/Ua6VcNe/DFh7X4@nvidia.com>
 <Y/UfS8TDIXhUlJ/I@slm.duckdns.org>
 <Y/UiQmuVwh2eqrfA@nvidia.com>
 <87o7pmnd0p.fsf@nvidia.com>
 <Y/YRJNwwvqp7nKKt@nvidia.com>
 <87k009nvnr.fsf@nvidia.com>
 <Y/bHNO7A8T3QQ5T+@nvidia.com>
 <Y/ct88JBeQuSmCuj@redhat.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <Y/ct88JBeQuSmCuj@redhat.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ekVZdE5ySkcwN2VxQzhxVFJPYXlDemx4NS9TRVNxcFJMbEtEbDZmdUZqc1RC?=
 =?utf-8?B?eHIwVFJmU2xJbWtadzUyL3VreGZFcko4ZlN3dEZWaWtaVTF5SHVVV1RyeTNO?=
 =?utf-8?B?SFo2Z2prZkN1Q3FKQjhhUXFMUTNQVnZGTlVEYWtQWGlKUll0TkNnQnJMZEU2?=
 =?utf-8?B?RFNLNVhETVhoRnc1TjI5Ti9lMG5IdkhDQk5kYnFJMnZaYzIvN2RsOE5jT0xw?=
 =?utf-8?B?MFRMdVpQdTdFUkllMWxmZXJFK09hZmpRVHc3NmRCZzA2OGdQdUl2RTRlOG5O?=
 =?utf-8?B?ZXYzZlhiTUs5cjE2ZVlrYVk2WVprYzJkTGpiRTkzK2w1UW9jYzBBWWMzZjRP?=
 =?utf-8?B?SkdPYnpjL3VmQU5NSzdNeWthd2xJK1pvRk1pMk5SVFM5SHlmdTdWZWFiYXgx?=
 =?utf-8?B?Qys2ckxrUW4rdVNFK21rUHZlTE1mSWluekdPOVE4cWtXVlJzNnBGQXFJVDho?=
 =?utf-8?B?NnFBekdZVklkUXNSR2JPcGZ5OXYrR0dYTDZCWHhtLzRCRU5GTEhkeUI0Qlpx?=
 =?utf-8?B?YnVZZXczZDA1TEM1U1JsazUrQ1F0VHBadzFiQzJERHRsQlRSUDRmUVdlUHBK?=
 =?utf-8?B?c096Y2FRWGFGczNHWnVrc3QxaGxZWWJhTXp4Sy9RQ1lJZzVicUFLME9DbDFX?=
 =?utf-8?B?WFg2RHlLZ3VBeFFiWXNQUk9yRCtUNkhkZ2ZhTWFiM3owMXhiOXpvV1ptRU1G?=
 =?utf-8?B?bGhESGhvdlVEbkI0UVpHSVJ3UUxKMlN4bEZqRTNkaElWd1BHVHYyaExJb3pn?=
 =?utf-8?B?MHFVWlk5V25sVDh4d1Y3U0NOVmYxci9SazBqOWlpVS9aQk9jNGxRTlJwdDFz?=
 =?utf-8?B?NlU5WEVBRFprYmduZTFobEpScTdiYVJ2Q3MwMVVRbko0R29yR2tUZER2VU1M?=
 =?utf-8?B?UmIrYTd3VmVQM2JJNFFKU2hUY1Z5U01qMGUwZEdkY3FKb1RFNmc4QXY2ZWE3?=
 =?utf-8?B?SnhTRFR3N0R1TVBKLzZPQlF0dzQvZ09nUzZPMitDWlhyZU1ZOEl6YUZPeTlZ?=
 =?utf-8?B?MFhoNDJEdlluQk9wcFNQbnhSdk5ob2cxREc0ZnhNN0xuMlJQa3FYdTRqWmNI?=
 =?utf-8?B?TFFTMFpkMllBQWdNVFUxK0FJWjk1aW43RlFhUVp5djltdWx3cHdnelI3REZr?=
 =?utf-8?B?dGhZeXpQdFlLcTJHd3c5dnV1QW1kQ0czdW5aZ0g4R1dldkZoaVFwWi9QN2Mv?=
 =?utf-8?B?YzhteEFLRjIweDI0R1h6amdSTVJFMXdSdWJTMDhMcEowU0tocjI4aTdRNmNG?=
 =?utf-8?B?STJQL29jREdlYXhFOGFEUldGcjV4aVAvcTd5RDJoUWRZSkpyZHMyTkVSYUlC?=
 =?utf-8?B?TXF6V2NLK3poRUNBamUrRFVKM0J0ZGpOV1BLVW9iS1Y0dm5mdGVVLzlBeWRL?=
 =?utf-8?B?VDhVT0Y3eksvb0lmTWduUmI2NnNORHJtaTlzaHFmRDc2Tnhsa2FySi9UMTUr?=
 =?utf-8?B?blovTTVaMGhZTU5tNnNDckVRczNJTmhlTHIzU3Y2MVpIQ3BIazlTbHZLVzRY?=
 =?utf-8?B?Sm52dGRyNGhSUUQ2aEppK2NCenpwbkdYUVMxSWUzbUwvSkhzeUt4RzJscER1?=
 =?utf-8?B?c0g1RUhoWHFOZVJMQ3krd05XNjZvV0RqVk1UNmFxVHRoaGNLRityeitJajQz?=
 =?utf-8?B?WXpyb2dqT0JPK1l3SVZxcVJkcnNQb2YxTHo1a1pnNWxmZEZRZi9YYjlDdWxP?=
 =?utf-8?B?VGs5bnRsMFVhclJXWTNiUnJUODRMQlF6T2NzQXpCSXJPQ1JkMW4zNS9RYzBF?=
 =?utf-8?B?MEthakRIeXpsZVZ1UXVjcUkvZmcvd3B3ZmdzZDhSNlhlekxRK2ExMndTSmd2?=
 =?utf-8?B?dGZaVUxWV05OVkFUY3pUNzZHL0Q1SzRVLzNVMlBwYUNQOXlKcHg5WkxqTG5r?=
 =?utf-8?B?NlFhT0xzcy9lUFhyRGFyTVBLdEdPajV6dGNHSEpjM0hJMnpOaTE0NktjSWZk?=
 =?utf-8?B?cjhWVFhFZnkyY0RhTCtxN1pXNGQ2cExaZ1VDUUU4bTJrZ0hJakQxUFdNYlUr?=
 =?utf-8?B?Yjl6UGpoQkJtWTU2V2ZaZjkzenZKTDBEQ0VVRmovbTBSVzh5NW5yUlk5dVp6?=
 =?utf-8?B?RUhNTFdzbkNBZEJBdTNreHVnQ1RYYmtoak1EZDJvdURUUFh0NVJ5WDVCREtT?=
 =?utf-8?Q?vQUSCjdUZ8q4l2/WhgW/egqJf?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c78b3368-3cb6-4c23-3e45-08db15c3d67b
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Feb 2023 17:31:46.5670
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: suIID+pjHBI6M1n5cHQGCCiMPWYHG/rLN5n2ckEezgpmhggv7trQ7uWVVdfG/v4F
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7847
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thu, Feb 23, 2023 at 09:12:19AM +0000, Daniel P. Berrangé wrote:

> The memory limits we can set on VMs are somewhat limited. In general
> we prefer to avoid setting any hard per-VM memory cap by default.

So if you don't use hard limits on the memcg..

But to do this with a hard limit:
 
> We do still have to apply some tuning for VFIO, around what amount
> of memory it is allowed to lock, but that is not so bad as we just
> need to allow it to lock guest RAM which is known + an finite extra
> amount, so don't need to take account of all of QEMU's memory
> allocations in general.

Will need its own controller, right?

Jason