Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
From:   Joel Fernandes <joel@joelfernandes.org>
Mime-Version: 1.0 (1.0)
Subject: Re: [PATCH v2 RESEND] tick/nohz: Fix cpu_is_hotpluggable() by checking with nohz subsystem
Date:   Wed, 25 Jan 2023 01:57:49 -0500
Message-Id: <1F1F75F7-DAB2-4DCC-B366-E1AC45606816@joelfernandes.org>
References: <CAABZP2wa_ZTHUr9tH_6OSpr+TgNACo4kMu3eawsGV5qkCDoAKg@mail.gmail.com>
Cc:     paulmck@kernel.org, linux-kernel@vger.kernel.org,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        Frederic Weisbecker <frederic@kernel.org>,
        Will Deacon <will@kernel.org>, Marc Zyngier <maz@kernel.org>,
        rcu <rcu@vger.kernel.org>, stable@vger.kernel.org,
        Frederic Weisbecker <fweisbec@gmail.com>,
        Ingo Molnar <mingo@kernel.org>,
        "Rafael J. Wysocki" <rafael@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>
In-Reply-To: <CAABZP2wa_ZTHUr9tH_6OSpr+TgNACo4kMu3eawsGV5qkCDoAKg@mail.gmail.com>
To:     Zhouyi Zhou <zhouzhouyi@gmail.com>
Precedence: bulk


> On Jan 24, 2023, at 7:13 PM, Zhouyi Zhou <zhouzhouyi@gmail.com> wrote:
>=20
> =EF=BB=BFOn Wed, Jan 25, 2023 at 6:56 AM Paul E. McKenney <paulmck@kernel.=
org> wrote:
>>=20
>>> On Tue, Jan 24, 2023 at 05:31:26PM +0000, Joel Fernandes (Google) wrote:=

>>> For CONFIG_NO_HZ_FULL systems, the tick_do_timer_cpu cannot be offlined.=

>>> However, cpu_is_hotpluggable() still returns true for those CPUs. This c=
auses
>>> torture tests that do offlining to end up trying to offline this CPU cau=
sing
>>> test failures. Such failure happens on all architectures.
>>>=20
>>> Fix it by asking the opinion of the nohz subsystem on whether the CPU ca=
n
>>> be hotplugged.
>>>=20
>>> [ Apply Frederic Weisbecker feedback on refactoring tick_nohz_cpu_down()=
. ]
>>>=20
>>> For drivers/base/ portion:
>>> Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
>>>=20
>>> Cc: Frederic Weisbecker <frederic@kernel.org>
>>> Cc: "Paul E. McKenney" <paulmck@kernel.org>
>>> Cc: Zhouyi Zhou <zhouzhouyi@gmail.com>
>>> Cc: Will Deacon <will@kernel.org>
>>> Cc: Marc Zyngier <maz@kernel.org>
>>> Cc: rcu <rcu@vger.kernel.org>
>>> Cc: stable@vger.kernel.org
>>> Fixes: 2987557f52b9 ("driver-core/cpu: Expose hotpluggability to the res=
t of the kernel")
>>> Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
>>=20
>> Queued for further review and testing, thank you both!
>>=20
>> It might be a few hours until it becomes publicly visible, but it will
>> get there.
> A new round of rcutorture test on fixed linux-5.15.y[3] has been
> performed in the PPC VM of Open Source Lab of Oregon State University
> [1], which will last about 29 hours. The test result on original
> linux-5.15.y is at [2].
>=20
> [1] http://140.211.169.189/linux-stable-rc/tools/testing/selftests/rcutort=
ure/res/2023.01.25-00.04.30-torture/
> [2] http://140.211.169.189/linux-stable-rc/tools/testing/selftests/rcutort=
ure/res/2023.01.18-13.22.39-torture/
> [3] http://140.211.169.189/linux-stable-rc/
>=20
> Happy to benefit the community and this is a fruitful learning journey ;-)=


Thanks a lot Zhouyi! After testing something your reply can also optionally i=
nclude:

Tested-by: Zhouyi Zhou <zhouzhoui@gmail.com>

Thanks.

Have fun at the university. I miss my university days.

 - Joel


>=20
> Cheers
> Zhouyi
>>=20
>>                                                        Thanx, Paul
>>=20
>>> ---
>>> Sorry, resending with CC to stable.
>>>=20
>>> drivers/base/cpu.c       |  3 ++-
>>> include/linux/tick.h     |  2 ++
>>> kernel/time/tick-sched.c | 11 ++++++++---
>>> 3 files changed, 12 insertions(+), 4 deletions(-)
>>>=20
>>> diff --git a/drivers/base/cpu.c b/drivers/base/cpu.c
>>> index 55405ebf23ab..450dca235a2f 100644
>>> --- a/drivers/base/cpu.c
>>> +++ b/drivers/base/cpu.c
>>> @@ -487,7 +487,8 @@ static const struct attribute_group *cpu_root_attr_g=
roups[] =3D {
>>> bool cpu_is_hotpluggable(unsigned int cpu)
>>> {
>>>      struct device *dev =3D get_cpu_device(cpu);
>>> -     return dev && container_of(dev, struct cpu, dev)->hotpluggable;
>>> +     return dev && container_of(dev, struct cpu, dev)->hotpluggable
>>> +             && tick_nohz_cpu_hotpluggable(cpu);
>>> }
>>> EXPORT_SYMBOL_GPL(cpu_is_hotpluggable);
>>>=20
>>> diff --git a/include/linux/tick.h b/include/linux/tick.h
>>> index bfd571f18cfd..9459fef5b857 100644
>>> --- a/include/linux/tick.h
>>> +++ b/include/linux/tick.h
>>> @@ -216,6 +216,7 @@ extern void tick_nohz_dep_set_signal(struct task_str=
uct *tsk,
>>>                                   enum tick_dep_bits bit);
>>> extern void tick_nohz_dep_clear_signal(struct signal_struct *signal,
>>>                                     enum tick_dep_bits bit);
>>> +extern bool tick_nohz_cpu_hotpluggable(unsigned int cpu);
>>>=20
>>> /*
>>>  * The below are tick_nohz_[set,clear]_dep() wrappers that optimize off-=
cases
>>> @@ -280,6 +281,7 @@ static inline void tick_nohz_full_add_cpus_to(struct=
 cpumask *mask) { }
>>>=20
>>> static inline void tick_nohz_dep_set_cpu(int cpu, enum tick_dep_bits bit=
) { }
>>> static inline void tick_nohz_dep_clear_cpu(int cpu, enum tick_dep_bits b=
it) { }
>>> +static inline bool tick_nohz_cpu_hotpluggable(unsigned int cpu) { retur=
n true; }
>>>=20
>>> static inline void tick_dep_set(enum tick_dep_bits bit) { }
>>> static inline void tick_dep_clear(enum tick_dep_bits bit) { }
>>> diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
>>> index 9c6f661fb436..63e3e8ebcd64 100644
>>> --- a/kernel/time/tick-sched.c
>>> +++ b/kernel/time/tick-sched.c
>>> @@ -510,7 +510,7 @@ void __init tick_nohz_full_setup(cpumask_var_t cpuma=
sk)
>>>      tick_nohz_full_running =3D true;
>>> }
>>>=20
>>> -static int tick_nohz_cpu_down(unsigned int cpu)
>>> +bool tick_nohz_cpu_hotpluggable(unsigned int cpu)
>>> {
>>>      /*
>>>       * The tick_do_timer_cpu CPU handles housekeeping duty (unbound
>>> @@ -518,8 +518,13 @@ static int tick_nohz_cpu_down(unsigned int cpu)
>>>       * CPUs. It must remain online when nohz full is enabled.
>>>       */
>>>      if (tick_nohz_full_running && tick_do_timer_cpu =3D=3D cpu)
>>> -             return -EBUSY;
>>> -     return 0;
>>> +             return false;
>>> +     return true;
>>> +}
>>> +
>>> +static int tick_nohz_cpu_down(unsigned int cpu)
>>> +{
>>> +     return tick_nohz_cpu_hotpluggable(cpu) ? 0 : -EBUSY;
>>> }
>>>=20
>>> void __init tick_nohz_init(void)
>>> --
>>> 2.39.1.405.gd4c25cc71f-goog
>>>=20