Date:   Wed, 1 Feb 2023 08:57:27 -0800
From:   Josh Poimboeuf <jpoimboe@kernel.org>
To:     Mark Rutland <mark.rutland@arm.com>
Cc:     Petr Mladek <pmladek@suse.com>,
        Joe Lawrence <joe.lawrence@redhat.com>, kvm@vger.kernel.org,
        "Michael S. Tsirkin" <mst@redhat.com>,
        Peter Zijlstra <peterz@infradead.org>, netdev@vger.kernel.org,
        Jiri Kosina <jikos@kernel.org>, linux-kernel@vger.kernel.org,
        virtualization@lists.linux-foundation.org,
        "Seth Forshee (DigitalOcean)" <sforshee@digitalocean.com>,
        live-patching@vger.kernel.org, Miroslav Benes <mbenes@suse.cz>
Subject: Re: [PATCH 0/2] vhost: improve livepatch switching for heavily
 loaded vhost worker kthreads
Message-ID: <20230201165727.lnywx6zyefbqbrke@treble>
References: <Y9OpTtqWjAkC2pal@hirez.programming.kicks-ass.net>
 <20230127165236.rjcp6jm6csdta6z3@treble>
 <20230127170946.zey6xbr4sm4kvh3x@treble>
 <20230127221131.sdneyrlxxhc4h3fa@treble>
 <Y9e6ssSHUt+MUvum@hirez.programming.kicks-ass.net>
 <Y9gOMCWGmoc5GQMj@FVFF77S0Q05N>
 <20230130194823.6y3rc227bvsgele4@treble>
 <Y9jr0fP7DtA9Of1L@FVFF77S0Q05N>
 <20230131163832.z46ihurbmjcwuvck@treble>
 <Y9pInB8KvcyhAwDa@FVFF77S0Q05N>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <Y9pInB8KvcyhAwDa@FVFF77S0Q05N>
Precedence: bulk

On Wed, Feb 01, 2023 at 11:10:20AM +0000, Mark Rutland wrote:
> On Tue, Jan 31, 2023 at 08:38:32AM -0800, Josh Poimboeuf wrote:
> > On Tue, Jan 31, 2023 at 10:22:09AM +0000, Mark Rutland wrote:
> > > > Hm, it might be nice if our out-of-line static call implementation would
> > > > automatically do a static key check as part of static_call_cond() for
> > > > NULL-type static calls.
> > > > 
> > > > But the best answer is probably to just add inline static calls to
> > > > arm64.  Is the lack of objtool the only thing blocking that?
> > > 
> > > The major issues were branch range limitations (and needing the linker to add
> > > PLTs),
> > 
> > Does the compiler do the right thing (e.g., force PLT) if the branch
> > target is outside the translation unit?  I'm wondering if we could for
> > example use objtool to help enforce such rules at the call site.
> 
> It's the linker (rather than the compiler) that'll generate the PLT if the
> caller and callee are out of range at link time. There are a few other issues
> too (e.g. no guarnatee that the PLT isn't used by multiple distinct callers,
> CMODX patching requirements), so we'd have to generate a pseudo-PLT ourselves
> at build time with a patching-friendly code sequence. Ard had a prototype for
> that:
> 
>   https://lore.kernel.org/linux-arm-kernel/20211105145917.2828911-1-ardb@kernel.org/
> 
> ... but that was sufficiently painful that we went with the current static key
> approach:
> 
>   https://lore.kernel.org/all/20211109172408.49641-1-mark.rutland@arm.com/

Thanks for the background.

Was there a reason for putting it out-of-line rather than directly in
_cond_resched()?

If it were inline then it wouldn't be that much different from the
static called version and I wonder if we could simplify by just using
the static key for all PREEMPT_DYNAMIC configs.

> > > If we knew each call-site would only call a particular function or skip the
> > > call, then we could do better (and would probably need something like objtool
> > > to NOP that out at compile time), but since we don't know the callee at build
> > > time we can't ensure we have a PLT in range when necessary.
> > 
> > Unfortunately most static calls have multiple destinations.
> 
> Sure, but here we're just enabling/disabling a call, which we could treat
> differently, or wrap at a different level within the scheduler code. I'm happy
> to take a look at that.

I can try to emulate what you did for PREEMPT_DYNAMIC.  I'll Cc you on
my actual patch to come soon-ish.

> > And most don't have the option of being NULL.
> 
> Oh, I was under the impression that all could be disabled/skipped, which is
> what a NULL target implied.

I guess what I was trying to say is that if the target can be NULL, the
call site has to use static_call_cond() to not break the
!HAVE_STATIC_CALL case.  But most call sites use static_call().

-- 
Josh