Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <c2bfe1aa390777b10d810d2b76a35b97fbd32a1c.camel@intel.com>
 <CALCETrUBDUKTcmmYD7BpZkL3869ELvha1PqOcScw4M-B_DQdiA@mail.gmail.com>
 <7653c6c74a4eee18b8bdc8262e0c0b5b95f9d771.camel@intel.com> <d4b9d6a1-d413-b044-1d4a-512bbbf06d78@intel.com>
In-Reply-To: <d4b9d6a1-d413-b044-1d4a-512bbbf06d78@intel.com>
From:   "H.J. Lu" <hjl.tools@gmail.com>
Date:   Sat, 18 Jul 2020 16:04:46 -0700
Message-ID: <CAMe9rOo9ivMP4b4qYGWvtEzKT-h68iPecjqp3wZzQCpVqE9FsA@mail.gmail.com>
Subject: Re: Random shadow stack pointer corruption
To:     Dave Hansen <dave.hansen@intel.com>
Cc:     Yu-cheng Yu <yu-cheng.yu@intel.com>,
        Andy Lutomirski <luto@kernel.org>,
        LKML <linux-kernel@vger.kernel.org>, X86 ML <x86@kernel.org>,
        Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>,
        Ingo Molnar <mingo@redhat.com>,
        "Ravi V. Shankar" <ravi.v.shankar@intel.com>,
        Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
        Tony Luck <tony.luck@intel.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Peter Zijlstra <peterz@infradead.org>,
        Weijiang Yang <weijiang.yang@intel.com>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Sat, Jul 18, 2020 at 3:41 PM Dave Hansen <dave.hansen@intel.com> wrote:
>
> On 7/18/20 11:24 AM, Yu-cheng Yu wrote:
> > On Sat, 2020-07-18 at 11:00 -0700, Andy Lutomirski wrote:
> >> On Sat, Jul 18, 2020 at 10:58 AM Yu-cheng Yu <yu-cheng.yu@intel.com> wrote:
> >>> Hi,
> >>>
> >>> My shadow stack tests start to have random shadow stack pointer corruption after
> >>> v5.7 (excluding).  The symptom looks like some locking issue or the kernel is
> >>> confused about which CPU a task is on.  In later tip/master, this can be
> >>> triggered by creating two tasks and each does continuous
> >>> pthread_create()/pthread_join().  If the kernel has max_cpus=1, the issue goes
> >>> away.  I also checked XSAVES/XRSTORS, but this does not seem to be an issue
> >>> coming from there.
> >>
> >> What do you mean "shadow stack pointer corruption"?  Is SSP itself
> >> corrupt while running in the kernel?  Is one of the MSRs getting
> >> corrupted?  Is the memory to which the shadow stack points getting
> >> corrupted? Is the CPU rejecting an attempt to change SSP?
> >
> > What I see is, a new thread after ret_from_fork() and iret back to ring-3,
> > its shadow stack pointer (MSR_IA32_PL3_SSP) is corrupted.
>
> Does corrupt mean random?  Or is it a valid stack address, just not for
> _this_ thread?  Or NULL?  Or is it a kernel address?  Have you tried
> tracing *ALL* the WRMSR's and XRSTOR's that write to the MSR?

Another data point.  When memory corruption happened, there was no
core dump at all.  We verified that core dump was enabled and we did
get core dump for other programs.


-- 
H.J.