Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) client-ip=2620:137:e000::1:18;
Date:   Wed, 1 Jun 2022 11:49:37 -0400
From:   Phil Auld <pauld@redhat.com>
To:     Valentin Schneider <vschneid@redhat.com>
Cc:     linux-kernel@vger.kernel.org, Thomas Gleixner <tglx@linutronix.de>,
        Peter Zijlstra <peterz@infradead.org>
Subject: Re: [PATCH v2 1/2] cpuhp: make target_store() a nop when target ==
 state
Message-ID: <YpeKkVVp+/JWuxUi@lorien.usersys.redhat.com>
References: <20220526160615.7976-1-pauld@redhat.com>
 <20220526160615.7976-2-pauld@redhat.com>
 <xhsmhy1ynl3hr.mognet@vschneid.remote.csb>
 <20220527132156.GB26124@pauld.bos.csb>
 <xhsmho7zf8auj.mognet@vschneid.remote.csb>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <xhsmho7zf8auj.mognet@vschneid.remote.csb>
Precedence: bulk

On Mon, May 30, 2022 at 01:27:00PM +0100 Valentin Schneider wrote:
> On 27/05/22 09:22, Phil Auld wrote:
> > On Fri, May 27, 2022 at 10:38:24AM +0100 Valentin Schneider wrote:
> >> On 26/05/22 12:06, Phil Auld wrote:
> >> > writing the current state back in hotplug/target calls cpu_down()
> >> > which will set cpu dying even when it isn't and then nothing will
> >> > ever clear it. A stress test that reads values and writes them back
> >> > for all cpu device files in sysfs will trigger the BUG() in
> >> > select_fallback_rq once all cpus are marked as dying.
> >> >
> >> > kernel/cpu.c::target_store()
> >> >    ...
> >> >         if (st->state < target)
> >> >                 ret = cpu_up(dev->id, target);
> >> >         else
> >> >                 ret = cpu_down(dev->id, target);
> >> >
> >> > cpu_down() -> cpu_set_state()
> >> >     bool bringup = st->state < target;
> >> >     ...
> >> >     if (cpu_dying(cpu) != !bringup)
> >> >            set_cpu_dying(cpu, !bringup);
> >> >
> >> > Fix this by letting state==target fall through in the target_store()
> >> > conditional.
> >> >
> >>
> >> To go back on my data race paranoia: writes to both cpu$x/online and
> >> cpu$x/hotplug/target are serialized by device_hotplug_lock, and so are the
> >> exported kernel hotplug functions ({add, remove}_cpu()).
> >>
> >> That's not cpu_add_remove_lock as I was looking for, but that's still all
> >> under one lock, so I think we're good. Sorry for that!
> >>
> >
> > Right. This catches it up higher so that we don't get into the code that
> > starts actually changing things.  I wonder now in the state == target case
> > if we should make sure st->target == target.  With the second patch it's
> > less likely to be needed. Thoughts?
> >
> 
> Yeah, you could append a simple:
> 
>         else
>                 WARN_ON(st->state != target);

I was thinking more like:

      	 else 
                   if (st->target != target) st->target = target;

Since this is a write to the target field and we are not
doing one of the operations that will set target because 
state == target we should make sure target == target. Although
that could have its own issues, I suppose. But as I said
fixing the boot cpu should make it much less likely that
st->target != st->state once we have the hotplug lock. 
 
I don't see how that WARN would ever fire. We're under the lock
and nothing is re-reading the value of st->state anyway. Looks more
like a compiler sanity check :)


Cheers,
Phil


> > Maybe I'll include that if/when I have code to keep cpux/online in sync
> > with st->state and cpu_online_mask.
> 

--