Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
From:   Imran Khan <imran.f.khan@oracle.com>
To:     tj@kernel.org, gregkh@linuxfoundation.org
Cc:     linux-kernel@vger.kernel.org
Subject: [PATCH v6 5/7] kernfs: Use a per-fs rwsem to protect per-fs list of kernfs_super_info.
Date:   Mon, 14 Feb 2022 23:03:20 +1100
Message-Id: <20220214120322.2402628-6-imran.f.khan@oracle.com>
In-Reply-To: <20220214120322.2402628-1-imran.f.khan@oracle.com>
References: <20220214120322.2402628-1-imran.f.khan@oracle.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Yoa5lTx/sfdKBWNYfhsOnl4G98AtMNkS/18GonjXU6wbL/lvszbvqpzdivzF?=
 =?us-ascii?Q?11FQS7LkI2I8h59eYV4Box6KQFJQ2LE1VMJmBiOxHvww+iJwV1ilqLGG1l/w?=
 =?us-ascii?Q?zpUxecuarviEmu3oUyLW4xPU4MNyP6KEkYyuZcQkRS5ch+JxrqMktZRB2GTn?=
 =?us-ascii?Q?gRbDCmthoMN6i0jBMPw74M5TygY0qZtd7dpO6rgfgWUmHylxNUWTniF42409?=
 =?us-ascii?Q?Ax1Xz8w0XZM0e08IxvS5CL3zpGDVhKrxOEd5pXmHq+vx70km+SEEOb0Hui54?=
 =?us-ascii?Q?ipGTS6QmunOsGjB/dLW+XD4kDJ6dB+x+0HcCLKuMsEroFFfMJ+1GGJpAWAYW?=
 =?us-ascii?Q?ag29xYYbLu0sR0HxkvPr+ms3l+6GfYuHlX7f3utjetqWaSUN8DbYHj4x1Vas?=
 =?us-ascii?Q?Nz+wyfvOqwfPrJNdx/vUzZU1+Q17ek42zeY5bOGnu+UIJd9JSBmE+Mv9fH0E?=
 =?us-ascii?Q?9S3dl/ChChS80X4J8dXsVQ7Ii0oprCXmcykx2UOT/NPGDAUSGdNoaQ79q3Qq?=
 =?us-ascii?Q?++W8TDIx2J9TeZKFvfAPc/BKwYq6jxm+QLp0HWG1uF0G83ZuucgxoLavenUj?=
 =?us-ascii?Q?3jJyMi6p0KDtebD5BoSQ+39hz0VdT4i82DNsXLxy43yuRSUlmsxccvSOlJaO?=
 =?us-ascii?Q?ed55z9OEbytowHzGCaxEigrMYOeYtEYPeSpnFxjQxMIeL2E2VqvsHp7X1r3j?=
 =?us-ascii?Q?wLwCjFnTWfTbw28ySaTftuE5+XfQkFruQKxXefYmlcfRDC1fxMxLEnEvwOWa?=
 =?us-ascii?Q?eU/S6p2hunczCJjWrib/oK4ZuuwY7kLQExwSFZdm7IV6dlwiR0p31N1H+qMn?=
 =?us-ascii?Q?N3iisZ7ZMMOVjsvd9VQRWZTPORzl5aNYI4XgRsKrQvwuOMOPAu7IYj7N78zi?=
 =?us-ascii?Q?nc23edagPnoYy49eKIas0Wb+b5/GxChobKAHCdqS98OMzZGwkaFVwwBSlQ1D?=
 =?us-ascii?Q?H/p636Dfp7i13ByIZUkezfeJIFOWWoShAQSFsP4119TD7hQ68d8gvo9juyjQ?=
 =?us-ascii?Q?CtyC/OhQdvB4oZddGFgsDE6bROnrYcj7IdC/1/4TJyr/i7PsMRJyrkAV7wRq?=
 =?us-ascii?Q?RrzaNr27IhOeWyOmlhu9IjRqy9bc8bRSzZKO+/4Gvl/g+gox0m7Lon7bqTK3?=
 =?us-ascii?Q?AcFV9CVw8C4AugOF4fx1aAm5On1sjK87AO0StiNqMfz6E4yHqluJ9cqjnl5F?=
 =?us-ascii?Q?CDbYCAjO1fGKqGtxyKkGrLwOfj/WsEve0Qw7lhqvrXzk+QWK+mv7QZc6wptf?=
 =?us-ascii?Q?+8YCd1ZJlQ1tUhDQEhDABqkZ+3w40n2xiFWR6LFC+MOBLUV1dVR/F+A2DzlF?=
 =?us-ascii?Q?4b2UCqCBpgE3ms3hzodkJYcD4SkTWFOhV/e9emJUb3y1EsfoePOixi18Ok/A?=
 =?us-ascii?Q?N1SQuBhGEXsH4wlikyR3q1VNbN2nbCsaTew3XFMCy1356I9pj+mKnXoO+K0/?=
 =?us-ascii?Q?BzkN450tuE41gIT+Ci5mALxnW2aQxyq5ZdEF3MR9M3Ns/kKxCzINS/JQsEDp?=
 =?us-ascii?Q?qNDu2KCO3evqBXPLfkIbQmInmianJx67qeKUzgoK2m4Zy0tnvfU/RwUVuL/v?=
 =?us-ascii?Q?xjSdEE3yl89B+Y60rXP8fwXwQE4dmcEoN/TLf9dfBB8WL0agMgTmddpHZi5u?=
 =?us-ascii?Q?WonjKfANmlJHH+pC6iKlOWM=3D?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 90fe73a0-bbb9-4833-75e6-08d9efb20bd4
X-MS-Exchange-CrossTenant-AuthSource: CO1PR10MB4468.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Feb 2022 12:03:43.2587
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: H/NC50f+nSY553S+aifsRb9NaZupoRJbUrdABQ5FNMY3jdrFW8QkoAcRFI36gwqCGp4habd9McJxYaZpnhjOpg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB4191
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10257 signatures=673431
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 bulkscore=0 suspectscore=0
 phishscore=0 malwarescore=0 mlxscore=0 mlxlogscore=999 adultscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2201110000
 definitions=main-2202140073
X-Proofpoint-ORIG-GUID: dOLXksAU45Ceh0F1ieJLGK5upFKiwjVd
X-Proofpoint-GUID: dOLXksAU45Ceh0F1ieJLGK5upFKiwjVd
X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,RDNS_NONE,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE
	autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
	lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Right now per-fs kernfs_rwsem protects list of kernfs_super_info instances
for a kernfs_root. Since kernfs_rwsem is used to synchronize several other
operations across kernfs and since most of these operations don't impact
kernfs_super_info, we can use a separate per-fs rwsem to synchronize access
to list of kernfs_super_info.
This helps in reducing contention around kernfs_rwsem and also allows
operations that change/access list of kernfs_super_info to proceed without
contending for kernfs_rwsem.

Signed-off-by: Imran Khan <imran.f.khan@oracle.com>
---
 fs/kernfs/dir.c        | 1 +
 fs/kernfs/file.c       | 2 ++
 fs/kernfs/mount.c      | 8 ++++----
 include/linux/kernfs.h | 1 +
 4 files changed, 8 insertions(+), 4 deletions(-)

diff --git a/fs/kernfs/dir.c b/fs/kernfs/dir.c
index e6d9772ddb4ca..dc769301ac96b 100644
--- a/fs/kernfs/dir.c
+++ b/fs/kernfs/dir.c
@@ -917,6 +917,7 @@ struct kernfs_root *kernfs_create_root(struct kernfs_syscall_ops *scops,
 	idr_init(&root->ino_idr);
 	init_rwsem(&root->kernfs_rwsem);
 	INIT_LIST_HEAD(&root->supers);
+	init_rwsem(&root->supers_rwsem);
 
 	/*
 	 * On 64bit ino setups, id is ino.  On 32bit, low 32bits are ino.
diff --git a/fs/kernfs/file.c b/fs/kernfs/file.c
index f3ecc6fe8aedc..af2046bc63aa1 100644
--- a/fs/kernfs/file.c
+++ b/fs/kernfs/file.c
@@ -859,6 +859,7 @@ static void kernfs_notify_workfn(struct work_struct *work)
 	/* kick fsnotify */
 	down_write(&root->kernfs_rwsem);
 
+	down_write(&root->supers_rwsem);
 	list_for_each_entry(info, &kernfs_root(kn)->supers, node) {
 		struct kernfs_node *parent;
 		struct inode *p_inode = NULL;
@@ -894,6 +895,7 @@ static void kernfs_notify_workfn(struct work_struct *work)
 
 		iput(inode);
 	}
+	up_write(&root->supers_rwsem);
 
 	up_write(&root->kernfs_rwsem);
 	kernfs_put(kn);
diff --git a/fs/kernfs/mount.c b/fs/kernfs/mount.c
index 809b738739b18..d35142226c340 100644
--- a/fs/kernfs/mount.c
+++ b/fs/kernfs/mount.c
@@ -347,9 +347,9 @@ int kernfs_get_tree(struct fs_context *fc)
 		}
 		sb->s_flags |= SB_ACTIVE;
 
-		down_write(&root->kernfs_rwsem);
+		down_write(&root->supers_rwsem);
 		list_add(&info->node, &info->root->supers);
-		up_write(&root->kernfs_rwsem);
+		up_write(&root->supers_rwsem);
 	}
 
 	fc->root = dget(sb->s_root);
@@ -376,9 +376,9 @@ void kernfs_kill_sb(struct super_block *sb)
 	struct kernfs_super_info *info = kernfs_info(sb);
 	struct kernfs_root *root = info->root;
 
-	down_write(&root->kernfs_rwsem);
+	down_write(&root->supers_rwsem);
 	list_del(&info->node);
-	up_write(&root->kernfs_rwsem);
+	up_write(&root->supers_rwsem);
 
 	/*
 	 * Remove the superblock from fs_supers/s_instances
diff --git a/include/linux/kernfs.h b/include/linux/kernfs.h
index 7ee0595b315a2..84653c609a5c0 100644
--- a/include/linux/kernfs.h
+++ b/include/linux/kernfs.h
@@ -265,6 +265,7 @@ struct kernfs_root {
 
 	wait_queue_head_t	deactivate_waitq;
 	struct rw_semaphore	kernfs_rwsem;
+	struct rw_semaphore	supers_rwsem;
 };
 
 struct kernfs_open_file {
-- 
2.30.2