Received-SPF: pass (google.com: domain of linux-kernel+bounces-207495-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
From: Thomas Gleixner <tglx@linutronix.de>
To: Steven Rostedt <rostedt@goodmis.org>
Cc: Sam Sun <samsun1006219@gmail.com>, linux-kernel@vger.kernel.org,
 x86@kernel.org, syzkaller-bugs@googlegroups.com, peterz@infradead.org,
 jpoimboe@kernel.org, jbaron@akamai.com, ardb@kernel.org, mingo@redhat.com,
 Borislav Petkov <bp@alien8.de>, dave.hansen@linux.intel.com,
 hpa@zytor.com, xrivendell7@gmail.com, Greg Kroah-Hartman
 <gregkh@linuxfoundation.org>, Tejun Heo <tj@kernel.org>
Subject: Re: [Linux kernel bug] WARNING in static_key_slow_inc_cpuslocked
In-Reply-To: <87h6e2xdg1.ffs@tglx>
References: <20240609090431.3af238bc@gandalf.local.home>
 <87o78axlbm.ffs@tglx> <20240609102530.0a292b07@rorschach.local.home>
 <87le3exfx2.ffs@tglx> <87h6e2xdg1.ffs@tglx>
Date: Sun, 09 Jun 2024 21:39:36 +0200
Message-ID: <87ed95ykg7.ffs@tglx>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain

On Sun, Jun 09 2024 at 18:56, Thomas Gleixner wrote:

> On Sun, Jun 09 2024 at 18:02, Thomas Gleixner wrote:
>> On Sun, Jun 09 2024 at 10:25, Steven Rostedt wrote:
>> Well the bug is there to detect inconsistency and that clearly works :)
>>
>> But I clearly can't read, because the jump label operations are
>> serialized via jump_label_mutex. Hrm...
>
> Ok. Now I found if for real. It's in the jump label core:
>
> CPU0                            CPU1
>
> static_key_slow_dec()
>  static_key_slow_try_dec()
>
>    key->enabled == 1
>    val = atomic_fetch_add_unless(&key->enabled, -1, 1);
>    if (val == 1)
>    	return false;
>
>    jump_label_lock();
>    if (atomic_dec_and_test(&key->enabled)) {
>       --> key->enabled == 0
>       __jump_label_update()
>
>                                 static_key_slow_dec()
>                                  static_key_slow_try_dec()
>
>                                     key->enabled == 0
>                                     val = atomic_fetch_add_unless(&key->enabled, -1, 1);
>
>                                     --> key->enabled == -1 <- FAIL
>
> static_key_slow_try_dec() is buggy. It needs similar logic as
> static_key_slow_try_inc() to work correctly.
>
> It's not only the 0, key->enabled can be -1 when the other CPU is in the
> slow path of enabling it.
>
> I'll send a patch after testing it.

That's fixable, but it does not cure all of it.

set_attr_rdpmc() really needs serialization otherwise it can end up with
unbalanced operations.

CPU0                                    CPU1

x86_pmu.attr_rdpmc == 0

if (val != x86_pmu.attr_rdpmc) {
   if (val == 0)
   	...
   else if (x86_pmu.attr_rdpmc == 0)
     static_branch_dec(&rdpmc_never_available_key);

                                        if (val != x86_pmu.attr_rdpmc) {
                                           if (val == 0)
                                              	...
                                            else if (x86_pmu.attr_rdpmc == 0)
                               FAIL --->        static_branch_dec(&rdpmc_never_available_key);

No?