From:   Alistair Popple <apopple@nvidia.com>
To:     linux-mm@kvack.org, cgroups@vger.kernel.org
Cc:     linux-kernel@vger.kernel.org, jgg@nvidia.com, jhubbard@nvidia.com,
        tjmercier@google.com, hannes@cmpxchg.org, surenb@google.com,
        mkoutny@suse.com, daniel@ffwll.ch,
        "Daniel P . Berrange" <berrange@redhat.com>,
        Alex Williamson <alex.williamson@redhat.com>,
        Alistair Popple <apopple@nvidia.com>
Subject: [PATCH 18/19] mm/mmap: Charge locked memory to pins cgroup
Date:   Mon,  6 Feb 2023 18:47:55 +1100
Message-Id: <fa58b745a597a32b434b0d92d55cf0b97317cfec.1675669136.git-series.apopple@nvidia.com>
In-Reply-To: <cover.c238416f0e82377b449846dbb2459ae9d7030c8e.1675669136.git-series.apopple@nvidia.com>
References: <cover.c238416f0e82377b449846dbb2459ae9d7030c8e.1675669136.git-series.apopple@nvidia.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?8l7kC3OaK8BVTwVGt5goAI8wanQmAamsLKwN4NQpQIf4Omil0phmccbSqFeG?=
 =?us-ascii?Q?1Zj6ueoJx6xMDXquKDcQe5Er2cYvwVL4XX+2VHao5jZOwFkVnR28N1Sn4fn0?=
 =?us-ascii?Q?tSRX1+dzFIipP5IPwtwykErOR0e3SkceMaAVzAQtlErlGG3oyjrFLk8oqRyf?=
 =?us-ascii?Q?Sp6gJpZZw+8PQWV9V1LO6JuAIUrR/CTzXD79JZnPtEBmhASDkMQHjJoUQ5TO?=
 =?us-ascii?Q?xO5ONGXsnwI+n6lyNQm5W9G/fVdR3JBwGneg3qghzZf8j50sqW5ZOOiTAa4k?=
 =?us-ascii?Q?xg8orZXUEuTgdmYGeAgWNpCtE2HQSKk1vvxJEJYsHzROorPqUshOjTR6tdE+?=
 =?us-ascii?Q?QrGL5cuytzC+/2qr+X1Pit8TjfmyvELisUPWATl7QwbXwk4yOOBRCnCbW2FN?=
 =?us-ascii?Q?SIzcYCDmt39TTs4w6RPni5BgIjHahkFqf5q4NFxlmctuxuHZ5eFe4T11pSkB?=
 =?us-ascii?Q?RpqdK1cVGKu0J9ulk2cBYqJ9BNm8uW8eLEYWTXUG+C8fZ89hUQxsappHro1K?=
 =?us-ascii?Q?basRoaW5vum5BgCfWUBHNcU5aMcGJVQX8rrPQ5MwtgFllVkVtAj7RWLpfe0F?=
 =?us-ascii?Q?60pGmjJyN6g+97czYDhflfKCZW7gbDXBJP8l8MFpLc6fNTJNNtm6jhsK9ZYy?=
 =?us-ascii?Q?yS3KRgKwRF+gRJk2BBcjoi0Djtpx2dOJlX+F4Ioo8q4GvL0cKkLrXbH5g6Q5?=
 =?us-ascii?Q?KNBtmmNjf3YpDhxpekbGluthLLUc2v5CCpKr9/ArAtsUb089j0p6582e7zMi?=
 =?us-ascii?Q?kLUgOQhOJ79dy9LNIY3hqH1wKTO5ieCBb/VsE5LpJpP4+WmZ0bKchrCLR2lE?=
 =?us-ascii?Q?QShlUg4cSvKBX36Zo33V0/f1MtH/lr8xbj+BrwvlrpHbSHHelW3vZf9S7bVl?=
 =?us-ascii?Q?qXuCzjKH/ulZdQDcAw+MvUsTD4vddgzeEBIE5+BoB4NxoR+FJbzLNPBGF1xv?=
 =?us-ascii?Q?/OC2WXsNaOXRWGO24ASTDFKdpJBRWDSAs51qCJ2LqdhVJqwQZ6HOr5zz8VG3?=
 =?us-ascii?Q?PTKVvQUKbflh2hf0LzJ0UDeGJ/+5xt81MysKO6VIcEElzItaP+lrrwX9ooJ+?=
 =?us-ascii?Q?sTQWdI8Htq9Lxus9hnccpTFqmaVqQfxIhDIfSyqxv2qiaAUmWYENztI8W/pd?=
 =?us-ascii?Q?TYAvSX+6Dmj4U3QQguzMoHuY/A9xU0ZXsadHFEhGh0KF2W8A5LPt4WSB6Uol?=
 =?us-ascii?Q?J8T1uaNfmqZHKrMjZ9P004V0HSzhoV3uo6ZD0Pa8I73QgtzYTsKMjTacKcbX?=
 =?us-ascii?Q?ZgbPhGioZvLA4EZTdjWrS2lzZ5HJPAT/FO8Gm2C5UWy/2Y4ndsjX+Haq9k5c?=
 =?us-ascii?Q?07U8wL0Le846YqGhqna+JnRUPv+xoJNaKPqpiutkX9qoF7mRn31zGwTrz8YY?=
 =?us-ascii?Q?XtQL+by/WxJZOa6XPJSR9TAIG0g9xFEIVNuPlXyCkNJv2CptGfhmkYmsH+3G?=
 =?us-ascii?Q?r4cF1eOWtFDCPDAkkQPXJV+ubu4+1Q4mQMhH6v5DN3cF2YZ6EV5MphqpPQOl?=
 =?us-ascii?Q?r9u3GhFBBwt0x4RCVlUeFqdEoU88XsrsQxekmUkMCAP54hc8+2jswdSuEFxk?=
 =?us-ascii?Q?vPyie5g73YQgJQXm/7CKzyMT0q01G3Bv2NS8CUew?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 427e0896-7913-4612-52a1-08db0816c76f
X-MS-Exchange-CrossTenant-AuthSource: BYAPR12MB3176.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Feb 2023 07:50:13.2727
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: tD8S4PSSN2NXQP7cLnr9ez07RFo1dGSONbQ2/RD76uWwk7J/TTGeB0H+KUhyxz4kuaksMFWXf+J2eqKvEedMsA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8573
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

account_locked_vm() is used to account memory to mm->locked_vm. This
adds accounting to the pins cgorup as it behaves similarly and should
be accounted against the same global limit if set.

This means memory must now be unaccounted for correctly, as the cgroup
typically outlives both the mm and the task. It is assumed that
callers of account_locked_vm() only do accounting against the current
task. Callers that need to do accounting against remote tasks should
use account_pinned_vm() and associated struct vm_account to hold
references to the cgroup.

Signed-off-by: Alistair Popple <apopple@nvidia.com>
Cc: linux-mm@kvack.org
Cc: linux-kernel@vger.kernel.org
---
 mm/util.c | 24 +++++++++++++++++++++++-
 1 file changed, 23 insertions(+), 1 deletion(-)

diff --git a/mm/util.c b/mm/util.c
index 1ca0dfe..755bada 100644
--- a/mm/util.c
+++ b/mm/util.c
@@ -589,15 +589,21 @@ int __account_locked_vm(struct mm_struct *mm, unsigned long pages,
 			struct task_struct *task, bool bypass_rlim)
 {
 	unsigned long locked_vm, limit;
+	struct pins_cgroup *pins_cg = get_pins_cg(task);
 	int ret = 0;
 
 	mmap_assert_write_locked(mm);
 
+	if (pins_cg && !pins_try_charge(pins_cg, pages))
+		return -ENOMEM;
+
 	locked_vm = mm->locked_vm;
 	if (!bypass_rlim) {
 		limit = task_rlimit(task, RLIMIT_MEMLOCK) >> PAGE_SHIFT;
-		if (locked_vm + pages > limit)
+		if (locked_vm + pages > limit) {
+			pins_uncharge(pins_cg, pages);
 			ret = -ENOMEM;
+		}
 	}
 
 	if (!ret)
@@ -607,6 +613,12 @@ int __account_locked_vm(struct mm_struct *mm, unsigned long pages,
 		 (void *)_RET_IP_, pages << PAGE_SHIFT, locked_vm << PAGE_SHIFT,
 		task_rlimit(task, RLIMIT_MEMLOCK), ret ? " - exceeded" : "");
 
+	pr_debug("%s: [%d] caller %ps %lu %lu/%lu%s\n", __func__, task->pid,
+		 (void *)_RET_IP_, pages << PAGE_SHIFT, locked_vm << PAGE_SHIFT,
+		task_rlimit(task, RLIMIT_MEMLOCK), ret ? " - exceeded" : "");
+
+	if (pins_cg)
+		put_pins_cg(pins_cg);
 	return ret;
 }
 EXPORT_SYMBOL_GPL(__account_locked_vm);
@@ -622,8 +634,18 @@ void __unaccount_locked_vm(struct mm_struct *mm, unsigned long pages)
 {
 	unsigned long locked_vm = mm->locked_vm;
 
+	/*
+	 * TODO: Convert book3s vio to use pinned vm to ensure
+	 * unaccounting happens to the correct cgroup.
+	 */
+	struct pins_cgroup *pins_cg = get_pins_cg(current);
+
 	mmap_assert_write_locked(mm);
 	WARN_ON_ONCE(pages > locked_vm);
+	if (pins_cg) {
+		pins_uncharge(pins_cg, pages);
+		put_pins_cg(pins_cg);
+	}
 	mm->locked_vm = locked_vm - pages;
 }
 EXPORT_SYMBOL_GPL(__unaccount_locked_vm);
-- 
git-series 0.9.1