Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Fri, 9 Nov 2018 09:10:28 -0600
From:   Josh Poimboeuf <jpoimboe@redhat.com>
To:     Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc:     Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        the arch/x86 maintainers <x86@kernel.org>,
        Andy Lutomirski <luto@kernel.org>,
        Steven Rostedt <rostedt@goodmis.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Ingo Molnar <mingo@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        Masami Hiramatsu <mhiramat@kernel.org>,
        Jason Baron <jbaron@akamai.com>, Jiri Kosina <jkosina@suse.cz>,
        David Laight <David.Laight@aculab.com>,
        Borislav Petkov <bp@alien8.de>
Subject: Re: [RFC PATCH 1/3] static_call: Add static call infrastructure
Message-ID: <20181109151028.faifw66enzye32gg@treble>
References: <cover.1541711457.git.jpoimboe@redhat.com>
 <3cf04e113d71c9f8e4be95fb84a510f085aa4afa.1541711457.git.jpoimboe@redhat.com>
 <CAKv+Gu9bMkFEx_GxH3uLm0gveQurp7ELDi6C5NvpugufPFMbRA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <CAKv+Gu9bMkFEx_GxH3uLm0gveQurp7ELDi6C5NvpugufPFMbRA@mail.gmail.com>
User-Agent: NeoMutt/20180716
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Fri, Nov 09, 2018 at 02:39:17PM +0100, Ard Biesheuvel wrote:
> > +       for (site = start; site < stop; site++) {
> > +               struct static_call_key *key = static_call_key(site);
> > +               unsigned long addr = static_call_addr(site);
> > +
> > +               if (list_empty(&key->site_mods)) {
> > +                       struct static_call_mod *mod;
> > +
> > +                       mod = kzalloc(sizeof(*mod), GFP_KERNEL);
> > +                       if (!mod) {
> > +                               WARN(1, "Failed to allocate memory for static calls");
> > +                               return;
> > +                       }
> > +
> > +                       mod->sites = site;
> > +                       list_add_tail(&mod->list, &key->site_mods);
> > +
> > +                       /*
> > +                        * The trampoline should no longer be used.  Poison it
> > +                        * it with a BUG() to catch any stray callers.
> > +                        */
> > +                       arch_static_call_poison_tramp(addr);
> 
> This patches the wrong thing: the trampoline is at key->func not addr.

If you look at the x86 implementation, it actually does poison the
trampoline.

The address of the trampoline isn't actually known here.  key->func
isn't the trampoline address; it's the destination func address.

So instead I passed the address of the call instruction.  The arch code
then reads the instruction to find the callee (the trampoline).

The code is a bit confusing.  To make it more obvious, maybe we should
add another arch function to read the call destination.  Then this code
can pass that into arch_static_call_poison_tramp().

> However, patching it here means we poison it before all users are
> patched. I added this on top
> 
> diff --git a/kernel/static_call.c b/kernel/static_call.c
> index 599ebc6fc4f1..d9562329bec6 100644
> --- a/kernel/static_call.c
> +++ b/kernel/static_call.c
> @@ -248,6 +248,7 @@ static void __init static_call_init(void)
>         struct static_call_site *start = __start_static_call_sites;
>         struct static_call_site *stop  = __stop_static_call_sites;
>         struct static_call_site *site;
> +       struct static_call_key *prev_key = NULL;
> 
>         if (start == stop) {
>                 pr_warn("WARNING: empty static call table\n");
> @@ -279,7 +280,9 @@ static void __init static_call_init(void)
>                          * The trampoline should no longer be used.  Poison it
>                          * it with a BUG() to catch any stray callers.
>                          */
> -                       arch_static_call_poison_tramp(addr);
> +                       if (prev_key)
> +
> arch_static_call_poison_tramp((unsigned long)prev_key->func);
> +                       prev_key = key;
>                 }
> 
>                 arch_static_call_transform(addr, key->func);

While it does indeed poison the trampoline before all users are patched,
I had been thinking that it didn't really matter because this is before
the other CPUs have been booted.

But I believe interrupts are enabled at this point during the boot, so
it would indeed be wise to poison it afterwards, in case an irq handler
makes a static call.

-- 
Josh