Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 23 Nov 2022 09:03:37 -0800
From:   Josh Poimboeuf <jpoimboe@kernel.org>
To:     Andrew Cooper <Andrew.Cooper3@citrix.com>
Cc:     Peter Zijlstra <peterz@infradead.org>,
        "Paul E. McKenney" <paulmck@kernel.org>,
        "sfr@canb.auug.org.au" <sfr@canb.auug.org.au>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "jgross@suse.com" <jgross@suse.com>,
        "sstabellini@kernel.org" <sstabellini@kernel.org>,
        "boris.ostrovsky@oracle.com" <boris.ostrovsky@oracle.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Thomas Gleixner <tglx@linutronix.de>
Subject: Re: objtool warning for next-20221118
Message-ID: <20221123170337.keacggyvn4ykbtsw@treble>
References: <20221121040736.GA3738360@paulmck-ThinkPad-P17-Gen-1>
 <Y3thrQxfcvZSXFlr@hirez.programming.kicks-ass.net>
 <20221121145215.GF4001@paulmck-ThinkPad-P17-Gen-1>
 <20221122051605.4hcbslwxez2trdvt@treble>
 <Y3yJxQJ9DwxTx7km@hirez.programming.kicks-ass.net>
 <20221123012350.kelmmadh65lyswqz@treble>
 <Y33gAiFP2IpVdxJu@hirez.programming.kicks-ass.net>
 <2255c45f-d8ba-3f32-9d99-737ad3040dd7@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <2255c45f-d8ba-3f32-9d99-737ad3040dd7@citrix.com>
Precedence: bulk

On Wed, Nov 23, 2022 at 10:52:09AM +0000, Andrew Cooper wrote:
> > Well, if you return from arch_cpu_idle_dead() you're back in the idle
> > loop -- exactly where you would be if you were to bootstrap the whole
> > CPU -- provided you have it remember the whole state (easier with a
> > vCPU).

play_dead() really needs sane semantics.  Not only does it introduce a
surprise to the offlining code in do_idle(), it also skips the entire
hotplug state machine.  Not sure if that introduces any bugs, but at the
very least it's subtle and surprising.

> > But maybe I'm missing something, lets add Xen folks on.
> 
> Calling VCPUOP_down on oneself always succeeds, but all it does is
> deschedule the vCPU.
> 
> It can be undone at a later point by a different vcpu issuing VCPUOP_up
> against the previously-downed CPU, at which point the vCPU gets rescheduled.
> 
> This is why the VCPUOP_down hypercall returns normally.  All state
> really is intact.
> 
> As for what Linux does, this is how xen_pv_cpu_up() currently behaves. 
> If you want to make Xen behave more everything else, then bug a BUG()
> after VCPUOP_down, and adjust xen_pv_cpu_up() to skip its initialised
> check and always use VCPUOP_initialise to bring the vCPU back online.

Or we could do what sev_es_play_dead() does and just call start_cpu0()
after the hypercall returns?

Though, start_cpu0() seems very suspect, it just uses the stack of the
idle task of whatever CPU happened to be last brought up via cpu_up().
Which means we now have two idle tasks sharing the same stack?  How is
start_cpu0() not broken???

-- 
Josh