Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <8b0a1619-1e39-fc3a-1226-f3b167e64646@oracle.com>
Date:   Thu, 27 Jul 2023 15:35:58 +1000
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0)
 Gecko/20100101 Thunderbird/102.13.0
Subject: Re: [PATCH 1/2] kernfs: dont take i_lock on inode attr read
Content-Language: en-US
From:   Imran Khan <imran.f.khan@oracle.com>
To:     Ian Kent <raven@themaw.net>,
        Anders Roxell <anders.roxell@linaro.org>
Cc:     Arnd Bergmann <arnd@arndb.de>, Tejun Heo <tj@kernel.org>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        Minchan Kim <minchan@kernel.org>,
        Eric Sandeen <sandeen@sandeen.net>,
        Alexander Viro <viro@zeniv.linux.org.uk>,
        Rick Lindsley <ricklind@linux.vnet.ibm.com>,
        David Howells <dhowells@redhat.com>,
        Miklos Szeredi <miklos@szeredi.hu>,
        Carlos Maiolino <cmaiolino@redhat.com>,
        linux-fsdevel <linux-fsdevel@vger.kernel.org>,
        Kernel Mailing List <linux-kernel@vger.kernel.org>,
        elver@google.com
References: <166606025456.13363.3829702374064563472.stgit@donald.themaw.net>
 <166606036215.13363.1288735296954908554.stgit@donald.themaw.net>
 <Y2BMonmS0SdOn5yh@slm.duckdns.org> <20221221133428.GE69385@mutt>
 <7815c8da-7d5f-c2c5-9dfd-7a77ac37c7f7@themaw.net>
 <e25ee08c-7692-4042-9961-a499600f0a49@app.fastmail.com>
 <9e35cf66-79ef-1f13-dc6b-b013c73a9fc6@themaw.net>
 <db933d76-1432-f671-8712-d94de35277d8@themaw.net> <20230718190009.GC411@mutt>
 <76fcd1fe-b5f5-dd6b-c74d-30c2300f3963@themaw.net>
 <ce407424e98bf5f2b186df5d28dd5749a6cbfa45.camel@themaw.net>
 <15eddad0-e73b-2686-b5ba-eaacc57b8947@themaw.net>
 <3505769d-9e7a-e76d-3aa7-286d689345b6@oracle.com>
In-Reply-To: <3505769d-9e7a-e76d-3aa7-286d689345b6@oracle.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?bXlxSkNRdUcxMzgvRGJrN1NIOXBaditmb3k0N3RDajlVN0lqRlQ0WFl4WHAz?=
 =?utf-8?B?ZW9XWXNxUk5xalFPenRsK0ovblAvTnZrREFwWFpiTlhNUzlCU01sVklpQVBI?=
 =?utf-8?B?eGtCUVFEdGV6T0xCZ25HSDd5YU1DQnVML3pLdzFkSUtYQ2tDaHEwOWx5YnE3?=
 =?utf-8?B?eWxlVlZnQ2hiajk2aU1YNk5lRlNON05kaW0zQmhpc1JGUVJkamFQUjZGaDlC?=
 =?utf-8?B?MzhlTFlRb1hhU0dWcUNab0JxdkJBMCtTNEFSVitheFl2NWk2UGR5MW1iZ2Iw?=
 =?utf-8?B?bjNzYko5eG9tdzNiZjdwOHgzdzY4VktUbDU0SnR1MDdiSmg3dGNKQ3VyTnJm?=
 =?utf-8?B?RXU4WCt4RnNVOWZtM0tuN2Q0REczQWtBL3JvdXN5czMwUUNTa3Z3RVhXSEd5?=
 =?utf-8?B?UnRQWXg3UmN4Y01HYzhzZ0o0MFpTS0ZFa3ZQWjV3Qm1BUE1OMUVmOG0rd1Nm?=
 =?utf-8?B?TVBKcURYamlHdTVFcVB4dktxZWIxOGxGZm1VZ2F1RlhKSmR5b1pLV0JUZ2lX?=
 =?utf-8?B?eGtBcjdnc2ticW5qMG5zYzJLTmRLaTRmRnhRM0RHUkN5aCs0NmZDVTBRV282?=
 =?utf-8?B?WFBMYy9ReDJPVnRMMGFlVjNoTHBvTExxYnQrcmR5b2JJUlYwaHZmTG9tWHY4?=
 =?utf-8?B?UGt5OExkOVFTNncyMUJxaGU3TXVCMVZJTkJRVm1FUlZ0aWt1eWtmdVUyVEVF?=
 =?utf-8?B?NkpMZU81dXMzWDBzSTlHbUR6azNocmg1a2pzZWprYmNuaWxiUzZ5aTNMci9H?=
 =?utf-8?B?WlpFeks3clpIVG5sVGZNNnQvRkRGZTdRNEhtR1FlZDhuYWIvOTJLYW1xR1lL?=
 =?utf-8?B?MnZEczJkU3o2bFUzUjRJdFJOQXdZSmN4RW9aWUZRdWZ5UXAxWk9vSUxyNUE2?=
 =?utf-8?B?VENQYjZ3SG5saFZCWnNWMEI1Rm5jcWZpREowcFg5YjdkVHpPeUg1ZGZJenNl?=
 =?utf-8?B?OFVZOEhzREpLTXZiT1Myc2tobXVMMmVwbDNUdnZabUczeEF2NzNOU3BsbGMy?=
 =?utf-8?B?bURBOW94aWJzeGUydytqNlRDSVVwWVhSdXBnMlVmZkkxSURyeUVlN1J2dDFG?=
 =?utf-8?B?aTBzNS9GbytORmtYTjFqbWJJV2dnODVONzFLQ1hWenB4cUZmS3Zpd29mZVVa?=
 =?utf-8?B?Z005SlFFb0Z6UlJKSTFzSmhCai81ektBc3ZmTm1CVUZDY2RiVFJSRXVFRlpx?=
 =?utf-8?B?Ly93YkFIM1MrRTdFSEFEeXhCa09TZzFNRVFsQVBvVmlKdkl1RzJFQmwxYml3?=
 =?utf-8?B?VEZuVW50eWdmQnFuTEZRZkUySGJSRnVpN1k5MStYSGpremtWbDZpa3pPS0Qv?=
 =?utf-8?B?N0trSjVYOVhnMDFRa2JtSU1YZ09Wa3VoWHpOS3hBVFNtendyWGhtZmFKYjli?=
 =?utf-8?B?L3J4dzFBQTRlaXJCWk1wYjZHS0JZb0RtR1BWVUIxOFJlUEkrQms1eFV6QWEy?=
 =?utf-8?B?bWM0VXdBRjhic1VWY05VSitWK0RQY25GdHJPcWJDTXpTZmhhenZkdnhjTG1G?=
 =?utf-8?B?VnF6WGdFa21PZHJDK2owRzhRMFA5Um1qdEx6QXVsR1M1dHpKQWdZWmlkQTcw?=
 =?utf-8?B?ZmtZUlJjRWovYUVrMkdKeFkxUkhsdVdBK2cxVTNJQStOQS9Ib3JXQUlvSXBN?=
 =?utf-8?B?NzczbXRVTnBETEhpOGo4aXd3T090Qnpkd3NQMUxaVFdySDlLaWZWRDF1ZW16?=
 =?utf-8?B?blVaY3F3K24rZ0hpa0FCQXVZaTVnRmplM1dkQThpd1Bxc0NTSno5Y2pBb0Uz?=
 =?utf-8?B?bGlDVmZnbEtYNHFoWlZCS244azMrdkZ2Q0pXQ3NORmRnSUx2aUFPS3JRWHJi?=
 =?utf-8?B?bTBlMkZxWm96R2phSFVKdDVGNEp1cXdLT21MT1NUZ0FldmZ3cE56NUNycWds?=
 =?utf-8?B?QUVUNEtHTzVUSTA3OVBpT05BQ1ptWlVuNHpYV2p2SlpxcnBqeGhxaGgwN3B3?=
 =?utf-8?B?WGI4ZTlnUFI3eDI3em1rWEIyaHNkUGllcjJIb3N6RStveU5mZnArY3BWUzdO?=
 =?utf-8?B?TFFjdjRSRHB6YWxNdnJXSDNIcTFlLzJNMU5jY3hWczRaMUFyOUo4Tm55QXpY?=
 =?utf-8?B?VDFGZlVkNjZTeVhNSXFYSWdNWWRJMElGZ1BORU0xR3NKclZhOThpSmxMRjR6?=
 =?utf-8?Q?A0wM7n+j7fiv/uMeaYjfHxHZk?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: =?utf-8?B?K1BvUWhURkVBK21BTFVWY3lvd003NGZ3aGs1Z1g0WFBBWXI1RUs5N2dRQW84?=
 =?utf-8?B?YWQxUmdJS0lxekZwd2dqMzB0cWtmeE4yVEo1TE92bUkxK0drcmowanZMQ1h5?=
 =?utf-8?B?a09ZUUNyUUgrSGEyRm0raFAyV0NHUElIWVdra3dzM1gxRC9MMzA5MzhDZFJ2?=
 =?utf-8?B?U3ZwNW14ZGdIcVdRMkkzZ2FqdUNuK21lTGV0Kys3aDNQeVBnUVluNnppd295?=
 =?utf-8?B?NVh3UVRHOVIxeDYweDZ2ZlBacDQ3d3NaNFBESFk0RGQ2NzR1aGlJWXpUUHZh?=
 =?utf-8?B?ekhjcGxqaXo5RjBya0IzbHZzT3RHL1NoOWlhemdGL29FZ1gwRm1adUg1d20x?=
 =?utf-8?B?NjlPZnlhNEpJQmJrcnF3dC9zb2RUMzVDOUVHN2RwdEVWdjE4Z3RleWEvcEI1?=
 =?utf-8?B?VVFCcFpYV2prTnF0VEM4YU5HcHNmSnExSzVNTFg2UUhFRVVsS3NwUkxuaWxS?=
 =?utf-8?B?NDhqRk5zdEJ0bVp4MEJBbU41bG1HcDhBQ0FzV2FGVlhmaG8zRThDZmorSlpx?=
 =?utf-8?B?UTlIQWZoRTVNRGNuUWFHZlIxdXNOdmhEbjQvcTdCRmVHQUhLRkdqWUM1cFhE?=
 =?utf-8?B?dmVGTHJsazJ3WGg2V01VR1h0WXA0cjIxcnJjZFBXYnFCVkxYcmdEaHhMeUdO?=
 =?utf-8?B?Ui9IT1JJVFJkYXdzUGVVb1hjY2JhRGhMZEt2YlBRNzhvWnU4Y1ZFTDF0U0F0?=
 =?utf-8?B?aWx5L2ZDUWJTd3RtK25oUVhITVhEVkNITkdZa09leWR1U0RPK3VOeWxENHRj?=
 =?utf-8?B?NW9hMGRZYytpSmlNbzJsR2g5NFhkWU55OG5SY1VPem1KSzlEbzYrVWtCdWNL?=
 =?utf-8?B?MVRqd3NQcmdFa0dQZXcrK2RiaHVjOVBjRmRnM29XMXFHT3hsYVFQRHA2QjNz?=
 =?utf-8?B?SnUwNlNXcVF1Rk1rdWRablJqM3l4ZE9EYnNkbk5BOEJhbEEyVWNrNElCTXM1?=
 =?utf-8?B?Q2ljcFlEUDJWT2hiYmtUc2U2RERMWm5iK05ibHMyakE4SmhROXdELzB1S2t1?=
 =?utf-8?B?V2hhSi8rSEtYdWU2MXY3T2dpdStKQUx6cndmYS9LamlIZk90elpEUTRIQUo2?=
 =?utf-8?B?eTJvTkNMUDVZWlg0dm1RRTUrNEtibUM2S1JFbEpHUkc4VllDOTExVEd3R2du?=
 =?utf-8?B?UlFvNmNXemFZcHp4YTVVTGlqcHhkcDFTbThxMnJRbURGaTI1RWhKWCtQSnc1?=
 =?utf-8?B?ZFkxUkc0R2xidU9WM0RsQkJOYzhld2ltYUNnSVg3VFFxLy81em9oWlBCanNp?=
 =?utf-8?B?Y29Za1VDMmRTdUdPZEkrcmViZWpLUWkxQmI2UG96Zkg3SnBxVXVZZFZnQXFK?=
 =?utf-8?B?cGZWRkhUZGtBQUNFWDRDVWpVOE1VdXpPWXdkaWNaSmRMa0VpREJEU3dEbTRV?=
 =?utf-8?B?N2JUVTNLVXZ2RUx2c2Y2L0dqcUlPVkI1cWdQaHhWK1BZdWRVd2ZPUFRFN1dk?=
 =?utf-8?B?R0txU2JjKzVkbzEzOWVacDdMWTU1RElyaldsR0tGUXcwYmRpbElPNEhmN2hS?=
 =?utf-8?B?YWdySnYxZWprbHFzZGJqS0hVR2hVK3IzalRhQm13REVGYWs3bThIZEJIem8y?=
 =?utf-8?B?ak0wdz09?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c42c771f-e69a-40f2-4b21-08db8e635f4a
X-MS-Exchange-CrossTenant-AuthSource: CO1PR10MB4468.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Jul 2023 05:36:05.5853
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: YsRFM/DvKG9gXOZ3DDfQwYBm8ZoZScUbMTXtcIQOT1WXNXVKpZV3kO9my3wIk14xRGnEPIScfPZnXdrm05KE0A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR10MB6994
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26
 definitions=2023-07-26_08,2023-07-26_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 spamscore=0 adultscore=0
 phishscore=0 bulkscore=0 malwarescore=0 mlxscore=0 mlxlogscore=999
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2306200000
 definitions=main-2307270049
X-Proofpoint-ORIG-GUID: kzn0WBUkNA25jBQtM6LLSo_wH9_OCFfs
X-Proofpoint-GUID: kzn0WBUkNA25jBQtM6LLSo_wH9_OCFfs
X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hello again Ian,
I take back my previous comment :).

On 27/7/2023 2:30 pm, Imran Khan wrote:
> Hello Ian,
> Sorry for late reply. I was about to reply this week.
> 
> On 27/7/2023 10:38 am, Ian Kent wrote:
>> On 20/7/23 10:03, Ian Kent wrote:
>>> On Wed, 2023-07-19 at 12:23 +0800, Ian Kent wrote:
> 
> [...]
>>> I do see a problem with recent changes.
>>>
>>> I'll send this off to Greg after I've done some testing (primarily just
>>> compile and function).
>>>
>>> Here's a patch which describes what I found.
>>>
>>> Comments are, of course, welcome, ;)
>>
>> Anders I was hoping you would check if/what lockdep trace
>>
>> you get with this patch.
>>
>>
>> Imran, I was hoping you would comment on my change as it
>>
>> relates to the kernfs_iattr_rwsem changes.
>>
>>
>> Ian
>>
>>>
>>> kernfs: fix missing kernfs_iattr_rwsem locking
>>>
>>> From: Ian Kent <raven@themaw.net>
>>>
>>> When the kernfs_iattr_rwsem was introduced a case was missed.
>>>
>>> The update of the kernfs directory node child count was also protected
>>> by the kernfs_rwsem and needs to be included in the change so that the
>>> child count (and so the inode n_link attribute) does not change while
>>> holding the rwsem for read.
>>>
> 
> kernfs direcytory node's child count changes in kernfs_(un)link_sibling and
> these are getting invoked while adding (kernfs_add_one),
> removing(__kernfs_remove) or moving (kernfs_rename_ns)a node. Each of these
> operations proceed under kernfs_rwsem and I see each invocation of
> kernfs_link/unlink_sibling during the above mentioned operations is happening
> under kernfs_rwsem.
> So the child count should still be protected by kernfs_rwsem and we should not
> need to acquire kernfs_iattr_rwsem in kernfs_link/unlink_sibling.
> 

kernfs_refresh_inode can still race against kernfs_link/unlink_siblings. So your
change looks good to me.
My tests are not showing any issues either. ( I tested on 4.14 and 5.4 kernels
as well).

Fee free to add my RB.

Reviewed-by: Imran Khan <imran.f.khan@oracle.com>

> Kindly let me know your thoughts. I would still like to see new lockdep traces
> with this change.
> 
> Thanks,
> Imran
> 
>>> Fixes: 9caf696142 (kernfs: Introduce separate rwsem to protect inode
>>> attributes)
>>>
>>> Signed-off-by: Ian Kent <raven@themaw.net>
>>> Cc: Anders Roxell <anders.roxell@linaro.org>
>>> Cc: Imran Khan <imran.f.khan@oracle.com>
>>> Cc: Arnd Bergmann <arnd@arndb.de>
>>> Cc: Minchan Kim <minchan@kernel.org>
>>> Cc: Eric Sandeen <sandeen@sandeen.net>
>>> ---
>>>   fs/kernfs/dir.c |    4 ++++
>>>   1 file changed, 4 insertions(+)
>>>
>>> diff --git a/fs/kernfs/dir.c b/fs/kernfs/dir.c
>>> index 45b6919903e6..6e84bb69602e 100644
>>> --- a/fs/kernfs/dir.c
>>> +++ b/fs/kernfs/dir.c
>>> @@ -383,9 +383,11 @@ static int kernfs_link_sibling(struct kernfs_node
>>> *kn)
>>>       rb_insert_color(&kn->rb, &kn->parent->dir.children);
>>>         /* successfully added, account subdir number */
>>> +    down_write(&kernfs_root(kn)->kernfs_iattr_rwsem);
>>>       if (kernfs_type(kn) == KERNFS_DIR)
>>>           kn->parent->dir.subdirs++;
>>>       kernfs_inc_rev(kn->parent);
>>> +    up_write(&kernfs_root(kn)->kernfs_iattr_rwsem);
>>>         return 0;
>>>   }
>>> @@ -408,9 +410,11 @@ static bool kernfs_unlink_sibling(struct
>>> kernfs_node *kn)
>>>       if (RB_EMPTY_NODE(&kn->rb))
>>>           return false;
>>>   +    down_write(&kernfs_root(kn)->kernfs_iattr_rwsem);
>>>       if (kernfs_type(kn) == KERNFS_DIR)
>>>           kn->parent->dir.subdirs--;
>>>       kernfs_inc_rev(kn->parent);
>>> +    up_write(&kernfs_root(kn)->kernfs_iattr_rwsem);
>>>         rb_erase(&kn->rb, &kn->parent->dir.children);
>>>       RB_CLEAR_NODE(&kn->rb);
>>>