Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Nathan Lynch <nathanl@linux.ibm.com>
To:     Valentin Schneider <valentin.schneider@arm.com>
Cc:     srikar@linux.vnet.ibm.com, peterz@infradead.org,
        linux-kernel@vger.kernel.org, clg@kaod.org, mingo@kernel.org,
        linuxppc-dev@lists.ozlabs.org,
        Michael Ellerman <mpe@ellerman.id.au>
Subject: Re: [PATCH] powerpc/smp: do not decrement idle task preempt count
 in CPU offline
In-Reply-To: <87h7dijo4m.mognet@arm.com>
References: <20211015145548.2269836-1-nathanl@linux.ibm.com>
 <87h7dijo4m.mognet@arm.com>
Date:   Fri, 15 Oct 2021 12:02:33 -0500
Message-ID: <875ytyz0eu.fsf@linux.ibm.com>
MIME-Version: 1.0
Content-Type: text/plain
Precedence: bulk

Valentin Schneider <valentin.schneider@arm.com> writes:

> On 15/10/21 09:55, Nathan Lynch wrote:
>> With PREEMPT_COUNT=y, when a CPU is offlined and then onlined again, we
>> get:
>>
>> BUG: scheduling while atomic: swapper/1/0/0x00000000
>> no locks held by swapper/1/0.
>> CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.15.0-rc2+ #100
>> Call Trace:
>>  dump_stack_lvl+0xac/0x108
>>  __schedule_bug+0xac/0xe0
>>  __schedule+0xcf8/0x10d0
>>  schedule_idle+0x3c/0x70
>>  do_idle+0x2d8/0x4a0
>>  cpu_startup_entry+0x38/0x40
>>  start_secondary+0x2ec/0x3a0
>>  start_secondary_prolog+0x10/0x14
>>
>> This is because powerpc's arch_cpu_idle_dead() decrements the idle task's
>> preempt count, for reasons explained in commit a7c2bb8279d2 ("powerpc:
>> Re-enable preemption before cpu_die()"), specifically "start_secondary()
>> expects a preempt_count() of 0."
>>
>> However, since commit 2c669ef6979c ("powerpc/preempt: Don't touch the idle
>> task's preempt_count during hotplug") and commit f1a0a376ca0c ("sched/core:
>> Initialize the idle task with preemption disabled"), that justification no
>> longer holds.
>>
>> The idle task isn't supposed to re-enable preemption, so remove the
>> vestigial preempt_enable() from the CPU offline path.
>>
>
> Humph, I got confused because 2c669ef6979c explicitly mentions hotplug,
> but that's the hotplug machinery which is already involved for bringing up
> the secondaries at boot time.
>
> IIUC your issue here is the preempt_count being messed up when
> hot-unplugging a CPU, which leads to fireworks during hotplug

That's right.

> (IOW I didn't
> test my last patch against hotplug - my bad!)
>
> Reviewed-by: Valentin Schneider <valentin.schneider@arm.com>

No worries and thank you for reviewing.


>> Tested with pseries and powernv in qemu, and pseries on PowerVM.
>>
>> Fixes: 2c669ef6979c ("powerpc/preempt: Don't touch the idle task's preempt_count during hotplug")
>> Fixes: f1a0a376ca0c ("sched/core: Initialize the idle task with preemption disabled")
>
> I think only the first Fixes: is needed.

OK, I'll re-send with that changed as well as your r-b. Thanks.