Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
MIME-Version: 1.0
References: <cover.1648049113.git.andreyknvl@google.com> <0bb72ea8fa88ef9ae3508c23d993952a0ae6f0f9.1648049113.git.andreyknvl@google.com>
 <YkV1ORaR97g45Fag@FVFF77S0Q05N>
In-Reply-To: <YkV1ORaR97g45Fag@FVFF77S0Q05N>
From:   Andrey Konovalov <andreyknvl@gmail.com>
Date:   Tue, 5 Apr 2022 17:38:07 +0200
Message-ID: <CA+fCnZc0--X_bQDEr+3kgimFL3zGm-kBL-5Tx6KLYybUd3zEzA@mail.gmail.com>
Subject: Re: [PATCH v2 3/4] arm64: implement stack_trace_save_shadow
To:     Mark Rutland <mark.rutland@arm.com>
Cc:     andrey.konovalov@linux.dev, Marco Elver <elver@google.com>,
        Alexander Potapenko <glider@google.com>,
        Catalin Marinas <catalin.marinas@arm.com>,
        Will Deacon <will@kernel.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        Dmitry Vyukov <dvyukov@google.com>,
        Andrey Ryabinin <ryabinin.a.a@gmail.com>,
        kasan-dev <kasan-dev@googlegroups.com>,
        Vincenzo Frascino <vincenzo.frascino@arm.com>,
        Sami Tolvanen <samitolvanen@google.com>,
        Peter Collingbourne <pcc@google.com>,
        Evgenii Stepanov <eugenis@google.com>,
        Florian Mayer <fmayer@google.com>,
        Linux Memory Management List <linux-mm@kvack.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Andrey Konovalov <andreyknvl@google.com>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Thu, Mar 31, 2022 at 11:32 AM Mark Rutland <mark.rutland@arm.com> wrote:
>
> This doesn't do any of the trampoline repatinting (e.g. for kretprobes or
> ftrace graph caller) that the regular unwinder does, so if either of those are
> in use this is going to produce bogus results.

Responded on the cover letter wrt this.

> > +noinline notrace int arch_stack_walk_shadow(unsigned long *store,
> > +                                         unsigned int size,
> > +                                         unsigned int skipnr)
> > +{
> > +     unsigned long *scs_top, *scs_base, *scs_next;
> > +     unsigned int len = 0, part;
> > +
> > +     preempt_disable();
>
> This doesn't look necessary; it's certinaly not needed for the regular unwinder.
>
> Critically, in the common case of unwinding just the task stack, we don't need
> to look at any of the per-cpu stacks, and so there's no need to disable
> preemption. See the stack nesting logic in the regular unwinder.

The common unwinder doesn't access per-cpu variables, so
preempt_disable() is not required.

Although, in this case, the per-cpu variable is read-only, so
preempt_disable() is probably also not required. Unless LOCKDEP or
some other tools complain about this.

> If we *do* need to unwind per-cpu stacks, we figure that out and verify our
> countext *at* the transition point.

I'm not sure I understand this statement. You mean we need to keep the
currently relevant SCS stack base and update it in interrupt handlers?
This will require modifying the entry code.

> > +
> > +     /* Get the SCS pointer. */
> > +     asm volatile("mov %0, x18" : "=&r" (scs_top));
>
> Does the compiler guarantee where this happens relative to any prologue
> manipulation of x18?
>
> This seems like something we should be using a compilar intrinsic for, or have
> a wrapper that passes this in if necessary.

This is a good point, I'll investigate this.

> > +
> > +     /* The top SCS slot is empty. */
> > +     scs_top -= 1;
> > +
> > +     /* Handle SDEI and hardirq frames. */
> > +     for (part = 0; part < ARRAY_SIZE(scs_parts); part++) {
> > +             scs_next = *this_cpu_ptr(scs_parts[part].saved);
> > +             if (scs_next) {
> > +                     scs_base = *this_cpu_ptr(scs_parts[part].base);
> > +                     if (walk_shadow_stack_part(scs_top, scs_base, store,
> > +                                                size, &skipnr, &len))
> > +                             goto out;
> > +                     scs_top = scs_next;
> > +             }
> > +     }
>
> We have a number of portential stack nesting orders (and may need to introduce
> more stacks in future), so I think we need to be more careful with this. The
> regular unwinder handles that dynamically.

I'll rewrite this part based on the other comments, so let's discuss it then.

Thanks!