Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <20200407011259.GA72735@juliacomputing.com> <2A931F48-D28F-46F3-827F-FF7F4D5D3E66@amacapital.net>
 <CABV8kRyi-5wyiCV3HsPfFx6x1_icV72BSy+5eK8UC3UCexTSCA@mail.gmail.com>
 <CAP045AohDoxZ+tvm+hBWWXgXz6N-z0+UWj--_o1AiVRubWYm6A@mail.gmail.com> <20200407123348.GV20730@hirez.programming.kicks-ass.net>
In-Reply-To: <20200407123348.GV20730@hirez.programming.kicks-ass.net>
From:   Keno Fischer <keno@juliacomputing.com>
Date:   Tue, 7 Apr 2020 09:52:42 -0400
Message-ID: <CABV8kRx=7fFvHqQ9AX-1P9EBhXg5sH4xjoPKz6qkiQK4tHFUbA@mail.gmail.com>
Subject: Re: [RFC PATCH v2] x86/arch_prctl: Add ARCH_SET_XCR0 to set XCR0 per-thread
To:     Peter Zijlstra <peterz@infradead.org>
Cc:     Kyle Huey <me@kylehuey.com>, Andy Lutomirski <luto@amacapital.net>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>,
        "maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT)" <x86@kernel.org>,
        "H. Peter Anvin" <hpa@zytor.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>,
        Andi Kleen <andi@firstfloor.org>,
        Kyle Huey <khuey@kylehuey.com>,
        "Robert O'Callahan" <robert@ocallahan.org>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

> > It's mentioned elsewhere, but I want to emphasize that the return
> > value of xgetbv is the big one because the dynamic linker uses this.
> > rr trace portability is essentially limited to machines with identical
> > xcr0 values because of it.
>
> I'm thinking just exposing that value is doable in a much less
> objectionable fashion, no?

Hi Peter,

I'm not sure I understand what you're asking,
but let me attempt to provide an answer anyway.
If I'm off the mark in what you would like to know,
please let me know and I'll try my best to get back
to you.

rr's operating principle relies upon every instruction
having deterministic and reproducible behavior,
every time they're executed and across machines.
That means literally bitwise identical updates to the
x86 register state. Most instructions do that given
identical register state - of course some don't by
design like rdtsc. Those instructions get trapped
and emulated (we're very lucky that doing so is
possible for all such instructions of practical
interest on Intel hardware). xcr0 puts us in a bit of
a bind here, because it modifies the user-visble
behavior of instructions (in the three ways I mentioned).
The xgetbv behavior is indeed the most problematic.
If there was a way to selectively trap
xgetbv/xsave/xrestor and emulate it, that would likely
prove sufficient (even just xgetbv may be sufficient,
but I'd have to do further work to validate that).
However, I don't think it's possible to trap these
instructions without also disabling the corresponding
xstate components, which we do not want, since
those instructions do actually need to get executed.

Thanks,
Keno