Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Thu, 21 Mar 2019 16:35:50 -0400
From:   Steven Rostedt <rostedt@goodmis.org>
To:     Andrew Cooper <andrew.cooper3@citrix.com>
Cc:     Andy Lutomirski <luto@kernel.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        Jan Beulich <jbeulich@suse.com>,
        Boris Ostrovsky <boris.ostrovsky@oracle.com>,
        Juergen Gross <jgross@suse.com>,
        Stefano Stabellini <sstabellini@kernel.org>,
        LKML <linux-kernel@vger.kernel.org>,
        "H. Peter Anvin" <hpa@zytor.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@kernel.org>, Borislav Petkov <bp@alien8.de>,
        Joel Fernandes <joel@joelfernandes.org>,
        He Zhe <zhe.he@windriver.com>
Subject: Re: [RFC][PATCH] tracing/x86: Save CR2 before tracing irqsoff on
 error_entry
Message-ID: <20190321163550.21b29194@gandalf.local.home>
In-Reply-To: <b6d80345-4153-13d4-a63d-4760764d7c98@citrix.com>
References: <20190320221534.165ab87b@oasis.local.home>
        <20190321083317.GL6058@hirez.programming.kicks-ass.net>
        <20190321090241.GL6521@hirez.programming.kicks-ass.net>
        <20190321104517.GM6521@hirez.programming.kicks-ass.net>
        <20190321093242.4a948198@gandalf.local.home>
        <20190321172203.GS5996@hirez.programming.kicks-ass.net>
        <CALCETrXqmjJt7azNoEtxs2eBkDd-vjUAYvrjNh_RD30DAvksBw@mail.gmail.com>
        <CAHk-=whSeiDXVvhie_bBPqDw99JNJMdTuX8ef82BhNanyo7zRw@mail.gmail.com>
        <CALCETrVaCZKjXj9ZqtvQgLk2DqXwNCh7b5DHvSnNAORsqJXCcQ@mail.gmail.com>
        <CAHk-=wjEHqY1+DxJ-_oNWTuTNQwMPtJKJh_HE=e1F60V+TwUfQ@mail.gmail.com>
        <20190321183713.GX5996@hirez.programming.kicks-ass.net>
        <CALCETrW-Jn=nWOgYUdMhoS_Z6vYNmfYzt6P54FBOaWzp4kPvQQ@mail.gmail.com>
        <b6d80345-4153-13d4-a63d-4760764d7c98@citrix.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Thu, 21 Mar 2019 20:00:19 +0000
Andrew Cooper <andrew.cooper3@citrix.com> wrote:

> I'm afraid that I've not got the original context of this thread, but
> I'm going to guess that something is resulting in a #PF before %cr2
> before it gets saved on the #PF path, and using a PVOP causes things to
> explode?

Actually it had nothing to do with PVOPs. It was tracing that caused
it. We added tracepoints to irqs off and on (with a config enabled),
and there's an option to allow taking user stack traces at any
tracepoint. The tracing of irqs off happened at the start of the page
fault entry, and by tracing the userspace stack (which can also fault)
we corrupted the cr2 for the original page fault.

The work now is to store that cr2 early on, but that requires doing it
in assembly, and that's where PVOPs makes the solution hard.

-- Steve


> 
> If it helps in the short term, Xen will trap and emulate a mov from cr2
> instruction correctly.  Performance will surely suck, but it might be an
> option if we need to rethink how this information is made available to
> guests.
> 
> Thanks,
> 
> ~Andrew