Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <a51a0a2e-d50d-c172-9403-034b13dff1a4@redhat.com>
Date:   Tue, 12 Jul 2022 23:02:48 -0400
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.10.0
Subject: Re: [PATCH] sched/schedutil: Fix deadlock between cpuset and cpu
 hotplug when using schedutil
Content-Language: en-US
To:     Steven Rostedt <rostedt@goodmis.org>, Tejun Heo <tj@kernel.org>
Cc:     Qais Yousef <qais.yousef@arm.com>,
        Xuewen Yan <xuewen.yan@unisoc.com>, rafael@kernel.org,
        viresh.kumar@linaro.org, mingo@redhat.com, peterz@infradead.org,
        juri.lelli@redhat.com, vincent.guittot@linaro.org,
        dietmar.eggemann@arm.com, bsegall@google.com, mgorman@suse.de,
        bristot@redhat.com, linux-kernel@vger.kernel.org,
        ke.wang@unisoc.com, xuewyan@foxmail.com, linux-pm@vger.kernel.org
References: <20220705123705.764-1-xuewen.yan@unisoc.com>
 <20220711174629.uehfmqegcwn2lqzu@wubuntu> <YsyO9GM9mCydaybo@slm.duckdns.org>
 <20220711171130.6390600b@gandalf.local.home>
From:   Waiman Long <longman@redhat.com>
In-Reply-To: <20220711171130.6390600b@gandalf.local.home>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 7/11/22 17:11, Steven Rostedt wrote:
> On Mon, 11 Jul 2022 10:58:28 -1000
> Tejun Heo <tj@kernel.org> wrote:
>
>> I don't think lockdep would be able to track CPU1 -> CPU2 dependency here
>> unfortunately.
>>
>>> AFAIU:
>>>
>>>
>>> CPU0                                     CPU1                                   CPU2
>>>
>>> // attach task to a different
>>> // cpuset cgroup via sysfs
>>> __acquire(cgroup_threadgroup_rwsem)
>>>
>>>                                           // pring up CPU2 online
>>>                                           __acquire(cpu_hotplug_lock)
>>>                                           // wait for CPU2 to come online
> Should there be some annotation here that tells lockdep that CPU1 is now
> blocked on CPU2?
>
> Then this case would be caught by lockdep.
>
> -- Steve
>
>
>>>                                                                                  // bringup cpu online
>>>                                                                                  // call cpufreq_online() which tries to create sugov kthread
>>> __acquire(cpu_hotplug_lock)                                                     copy_process()
>>>                                                                                     cgroup_can_fork()
>>>                                                                                        cgroup_css_set_fork()
>>>                                                                                        __acquire(cgroup_threadgroup_rwsem)
>>> // blocks forever                        // blocks forever                            // blocks forever

Actually, the dependency can probably be coded by calling 
lockdep_acquire_cpus_lock() in cpu2 (task 2) before acquiring 
cgroup_threadgroup_rwsem to indicate the dependency between CPU1 and 
CPU2 (task 1 and task 2). lockdep_release_cpus_lock() can then called 
after release the rwsem.

Cheers,
Longman