Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Valentin Schneider <vschneid@redhat.com>
To:     Lai Jiangshan <jiangshanlai@gmail.com>
Cc:     linux-kernel@vger.kernel.org, Tejun Heo <tj@kernel.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Frederic Weisbecker <frederic@kernel.org>,
        Juri Lelli <juri.lelli@redhat.com>,
        Phil Auld <pauld@redhat.com>,
        Marcelo Tosatti <mtosatti@redhat.com>
Subject: Re: [PATCH v6 4/4] workqueue: Unbind kworkers before sending them
 to exit()
In-Reply-To: <CAJhGHyB06u=mtbouGBw2CbLX9+gXPSPqHrHZHvorzF+8cPrG+Q@mail.gmail.com>
References: <20221128183109.446754-1-vschneid@redhat.com>
 <20221128183109.446754-5-vschneid@redhat.com>
 <CAJhGHyB06u=mtbouGBw2CbLX9+gXPSPqHrHZHvorzF+8cPrG+Q@mail.gmail.com>
Date:   Thu, 01 Dec 2022 10:37:30 +0000
Message-ID: <xhsmh8rjrl89h.mognet@vschneid.remote.csb>
MIME-Version: 1.0
Content-Type: text/plain
Precedence: bulk

On 01/12/22 11:01, Lai Jiangshan wrote:
> On Tue, Nov 29, 2022 at 2:31 AM Valentin Schneider <vschneid@redhat.com> wrote:
>
>> @@ -3627,8 +3668,11 @@ static bool wq_manager_inactive(struct worker_pool *pool)
>>  static void put_unbound_pool(struct worker_pool *pool)
>>  {
>>         DECLARE_COMPLETION_ONSTACK(detach_completion);
>> +       struct list_head cull_list;
>>         struct worker *worker;
>>
>> +       INIT_LIST_HEAD(&cull_list);
>> +
>>         lockdep_assert_held(&wq_pool_mutex);
>>
>>         if (--pool->refcnt)
>> @@ -3651,17 +3695,19 @@ static void put_unbound_pool(struct worker_pool *pool)
>>          * Because of how wq_manager_inactive() works, we will hold the
>>          * spinlock after a successful wait.
>>          */
>> +       mutex_lock(&wq_pool_attach_mutex);
>>         rcuwait_wait_event(&manager_wait, wq_manager_inactive(pool),
>>                            TASK_UNINTERRUPTIBLE);
>>         pool->flags |= POOL_MANAGER_ACTIVE;
>
> Hello, Valentin
>
> I'm afraid it might deadlock here.
>
> If put_unbound_pool() is called while manage_workers() is sleeping
> on allocating memory, put_unbound_pool() will get the wq_pool_attach_mutex
> earlier than the manager which prevents the manager from getting the
> lock to attach the newly created worker and deadlock.
>

Well spotted, I can see it now.

> I think mutex_lock(&wq_pool_attach_mutex) can be moved into
> wq_manager_inactive(), and handle it in the same way as pool->lock.
>

That looks sane enough, I'll try to tweak my tests to get the manager
involved to test this out. Thanks!