Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Content-Type: text/plain;
        charset=utf-8
Mime-Version: 1.0 (1.0)
Subject: Re: [RFC] x86: Speculative execution warnings
From:   Andy Lutomirski <luto@amacapital.net>
In-Reply-To: <EB9EEC92-A513-44B4-9377-56691916BF5D@vmware.com>
Date:   Tue, 14 May 2019 10:15:21 -0700
Cc:     Paul Turner <pjt@google.com>,
        the arch/x86 maintainers <x86@kernel.org>,
        Borislav Petkov <bp@alien8.de>,
        LKML <linux-kernel@vger.kernel.org>,
        Andy Lutomirsky <luto@kernel.org>,
        Ingo Molnar <mingo@kernel.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Jann Horn <jannh@google.com>
Content-Transfer-Encoding: quoted-printable
Message-Id: <3944C0B1-D0C4-4D2F-B055-69313CFD73F2@amacapital.net>
References: <20190510192514.19301-1-namit@vmware.com> <CAPM31RJ_vQsLp3nK5nhq0U8J+x_9w=aV+TtPGj7vdtiOKPpw8g@mail.gmail.com> <EB9EEC92-A513-44B4-9377-56691916BF5D@vmware.com>
To:     Nadav Amit <namit@vmware.com>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk


On May 14, 2019, at 10:00 AM, Nadav Amit <namit@vmware.com> wrote:

>> On May 14, 2019, at 1:00 AM, Paul Turner <pjt@google.com> wrote:
>>=20
>> From: Nadav Amit <namit@vmware.com>
>> Date: Fri, May 10, 2019 at 7:45 PM
>> To: <x86@kernel.org>
>> Cc: Borislav Petkov, <linux-kernel@vger.kernel.org>, Nadav Amit, Andy
>> Lutomirsky, Ingo Molnar, Peter Zijlstra, Thomas Gleixner, Jann Horn
>>=20
>>> It may be useful to check in runtime whether certain assertions are
>>> violated even during speculative execution. This can allow to avoid
>>> adding unnecessary memory fences and at the same time check that no data=

>>> leak channels exist.
>>>=20
>>> For example, adding such checks can show that allocating zeroed pages
>>> can return speculatively non-zeroed pages (the first qword is not
>>> zero).  [This might be a problem when the page-fault handler performs
>>> software page-walk, for example.]
>>>=20
>>> Introduce SPEC_WARN_ON(), which checks in runtime whether a certain
>>> condition is violated during speculative execution. The condition should=

>>> be computed without branches, e.g., using bitwise operators. The check
>>> will wait for the condition to be realized (i.e., not speculated), and
>>> if the assertion is violated, a warning will be thrown.
>>>=20
>>> Warnings can be provided in one of two modes: precise and imprecise.
>>> Both mode are not perfect. The precise mode does not always make it easy=

>>> to understand which assertion was broken, but instead points to a point
>>> in the execution somewhere around the point in which the assertion was
>>> violated.  In addition, it prints a warning for each violation (unlike
>>> WARN_ONCE() like behavior).
>>>=20
>>> The imprecise mode, on the other hand, can sometimes throw the wrong
>>> indication, specifically if the control flow has changed between the
>>> speculative execution and the actual one. Note that it is not a
>>> false-positive, it just means that the output would mislead the user to
>>> think the wrong assertion was broken.
>>>=20
>>> There are some more limitations. Since the mechanism requires an
>>> indirect branch, it should not be used in production systems that are
>>> susceptible for Spectre v2. The mechanism requires TSX and performance
>>> counters that are only available in skylake+. There is a hidden
>>> assumption that TSX is not used in the kernel for anything else, other
>>> than this mechanism.
>>=20
>> Nice trick!
>=20
> =E2=80=9CIllusion." [ ignore if you don=E2=80=99t know the reference ]
>=20
>>=20
>> Can you eliminate the indirect call by forcing an access fault to
>> abort the transaction instead, e.g. "cmove 0, $1=E2=80=9D?
>>=20
>> (If this works, it may also allow support on older architectures as
>> the RTM_RETIRED.ABORT* events go back further I believe?)
>=20
> I don=E2=80=99t think it would work. The whole problem is that we need a c=
ounter
> that is updated during execution and not retirement. I tried several
> counters and could not find other appropriate ones.
>=20
> The idea behind the implementation is to affect the control flow through
> data dependency. I may be able to do something similar without an indirect=

> branch. I=E2=80=99ll take a page, put the XABORT on the page and make the p=
age NX.
> Then, a direct jump would go to this page. The conditional-mov would chang=
e
> the PTE to X if the assertion is violated. There should be a page-walk eve=
n
> if the CPU finds the entry in the TLB, since this entry is NX.
>=20

I think you only get a page walk if the TLB entry is not-present.  I=E2=80=99=
d be a bit surprised if the CPU is willing to execute, even speculatively, f=
rom speculatively written data. Good luck!=