Message-ID: <c90b250e-822f-9f3b-8327-a0296e38cd1b@ya.ru>
Date:   Thu, 9 Mar 2023 22:34:18 +0300
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.8.0
Subject: Re: [PATCH v4 7/8] mm: vmscan: remove shrinker_rwsem from
 synchronize_shrinkers()
Content-Language: en-US
To:     Qi Zheng <zhengqi.arch@bytedance.com>,
        =?UTF-8?Q?Christian_K=c3=b6nig?= <christian.koenig@amd.com>,
        akpm@linux-foundation.org, hannes@cmpxchg.org, shakeelb@google.com,
        mhocko@kernel.org, roman.gushchin@linux.dev, muchun.song@linux.dev,
        david@redhat.com, shy828301@gmail.com, rppt@kernel.org
Cc:     sultan@kerneltoast.com, dave@stgolabs.net,
        penguin-kernel@I-love.SAKURA.ne.jp, paulmck@kernel.org,
        linux-mm@kvack.org, linux-kernel@vger.kernel.org
References: <20230307065605.58209-1-zhengqi.arch@bytedance.com>
 <20230307065605.58209-8-zhengqi.arch@bytedance.com>
 <bc829246-4606-cc8d-4a4f-da285255f5a3@ya.ru>
 <07078623-b7ef-ceb4-eccc-8872a4067273@bytedance.com>
 <d527005e-5f55-a6f7-21ce-817076860e99@amd.com>
 <85f9e200-dabe-7340-b76d-6525988054aa@bytedance.com>
From:   Kirill Tkhai <tkhai@ya.ru>
In-Reply-To: <85f9e200-dabe-7340-b76d-6525988054aa@bytedance.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Precedence: bulk

On 09.03.2023 11:32, Qi Zheng wrote:
> Hi Christian,
> 
> On 2023/3/9 16:11, Christian König wrote:
>> Am 09.03.23 um 08:06 schrieb Qi Zheng:
>>> Hi Kirill,
>>>
>>> On 2023/3/9 06:39, Kirill Tkhai wrote:
>>>> On 07.03.2023 09:56, Qi Zheng wrote:
>>>>> Now there are no readers of shrinker_rwsem, so
>>>>> synchronize_shrinkers() does not need to hold the
>>>>> writer of shrinker_rwsem to wait for all running
>>>>> shinkers to complete, synchronize_srcu() is enough.
>>>>>
>>>>> Signed-off-by: Qi Zheng <zhengqi.arch@bytedance.com>
>>>>> ---
>>>>>   mm/vmscan.c | 8 ++------
>>>>>   1 file changed, 2 insertions(+), 6 deletions(-)
>>>>>
>>>>> diff --git a/mm/vmscan.c b/mm/vmscan.c
>>>>> index 7aaf6f94ac1b..ac7ab4aa344f 100644
>>>>> --- a/mm/vmscan.c
>>>>> +++ b/mm/vmscan.c
>>>>> @@ -796,15 +796,11 @@ EXPORT_SYMBOL(unregister_shrinker);
>>>>>   /**
>>>>>    * synchronize_shrinkers - Wait for all running shrinkers to complete.
>>>>>    *
>>>>> - * This is equivalent to calling unregister_shrink() and register_shrinker(),
>>>>> - * but atomically and with less overhead. This is useful to guarantee that all
>>>>> - * shrinker invocations have seen an update, before freeing memory, similar to
>>>>> - * rcu.
>>>>> + * This is useful to guarantee that all shrinker invocations have seen an
>>>>> + * update, before freeing memory.
>>>>>    */
>>>>>   void synchronize_shrinkers(void)
>>>>>   {
>>>>> -    down_write(&shrinker_rwsem);
>>>>> -    up_write(&shrinker_rwsem);
>>>>>       atomic_inc(&shrinker_srcu_generation);
>>>>>       synchronize_srcu(&shrinker_srcu);
>>>>>   }
>>>>
>>>> Just curious, callers of synchronize_shrinkers() don't want to have parallel register_shrinker() and unregister_shrink() are completed?
>>>> Here we only should wait for parallel shrink_slab(), correct?
>>>
>>> I think yes.
>>>
>>> The synchronize_shrinkers() is currently only used by TTM pool.
>>>
>>> In TTM pool, a shrinker named "drm-ttm_pool" is registered, and
>>> the scan_objects callback will pick an entry from its own shrinker_list:
>>>
>>> ttm_pool_shrink
>>> --> spin_lock(&shrinker_lock);
>>>     pt = list_first_entry(&shrinker_list, typeof(*pt), shrinker_list);
>>>     list_move_tail(&pt->shrinker_list, &shrinker_list);
>>>     spin_unlock(&shrinker_lock);
>>>
>>> These entries have been removed from shrinker_list before calling
>>> synchronize_shrinkers():
>>>
>>> ttm_pool_fini
>>> --> ttm_pool_type_fini
>>>     --> spin_lock(&shrinker_lock);
>>>     list_del(&pt->shrinker_list);
>>>     spin_unlock(&shrinker_lock);
>>>     synchronize_shrinkers
>>>
>>> So IIUC, we only need to wait for the parallel shrink_slab() here. Like
>>> its comment says:
>>>
>>> /* We removed the pool types from the LRU, but we need to also make sure
>>>  * that no shrinker is concurrently freeing pages from the pool.
>>>  */
>>
>> Yes your analyses is completely correct.
>>
>> I just didn't wanted to add another SRCU into the critical code paths of the TTM pool just for device hot plug when I have that functionality already.
>>
>> We just make sure that no shrinker is running in parallel with destruction of the pool. Registering and unregistering is harmless.
> 
> That's great, thanks for confirming.
> 
> Thanks,
> Qi

Christian and Qi, thanks for the explanation.

>>
>> Regards,
>> Christian.
>>
>>>
>>> + CC: Christian König :)
>>>
>>> Thanks,
>>> Qi
>>
>>