Received-SPF: pass (google.com: domain of linux-kernel+bounces-59824-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Date: Fri, 9 Feb 2024 10:43:38 -0800
In-Reply-To: <CAHk-=wi3p5C1n03UYoQhgVDJbh_0ogCpwbgVGnOdGn6RJ6hnKA@mail.gmail.com>
Precedence: bulk
Mime-Version: 1.0
References: <20240208220604.140859-1-seanjc@google.com> <CAKwvOdk_obRUkD6WQHhS9uoFVe3HrgqH5h+FpqsNNgmj4cmvCQ@mail.gmail.com>
 <DM6PR02MB40587AD6ABBF1814E9CCFA7CB84B2@DM6PR02MB4058.namprd02.prod.outlook.com>
 <CAHk-=wi3p5C1n03UYoQhgVDJbh_0ogCpwbgVGnOdGn6RJ6hnKA@mail.gmail.com>
Message-ID: <ZcZyWrawr1NUCiQZ@google.com>
Subject: Re: [PATCH] Kconfig: Explicitly disable asm goto w/ outputs on gcc-11
 (and earlier)
From: Sean Christopherson <seanjc@google.com>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: "Andrew Pinski (QUIC)" <quic_apinski@quicinc.com>, Nick Desaulniers <ndesaulniers@google.com>, 
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, Masahiro Yamada <masahiroy@kernel.org>, 
	Peter Zijlstra <peterz@infradead.org>, "kvm@vger.kernel.org" <kvm@vger.kernel.org>
Content-Type: text/plain; charset="us-ascii"

On Fri, Feb 09, 2024, Linus Torvalds wrote:
> On Fri, 9 Feb 2024 at 09:14, Andrew Pinski (QUIC)
> <quic_apinski@quicinc.com> wrote:
> >
> > So the exact versions of GCC where this is/was fixed are:
> > 12.4.0 (not released yet)
> > 13.2.0
> > 14.1.0 (not released yet)
> 
> Looking at the patch that the bugzilla says is the fix, it *looks*
> like it's just the "mark volatile" that is missing.
> 
> But Sean says that  even if we mark "asm goto" as volatile manually,
> it still fails.
> 
> So there seems to be something else going on in addition to just the volatile.

Aha!  Yeah, there's a second bug that set things up so that the "not implicitly
volatile" bug could rear its head.  (And now I feel less bad for not suspecting
the compiler sooner, because it didn't occur to me that gcc could possibly think
the asm blob had no used outputs).

With "volatile" forced, gcc generates code for the asm blob, but doesn't actually
consume the output of the VMREAD.  As a result, the optimization pass sees the
unused output and throws it away because the blob isn't treated as volatile.

   vmread %rax,%rax       <= output register is unused
   jbe    0xffffffff8109994a <sync_vmcs02_to_vmcs12+1898>
   xor    %r12d,%r12d     <= one of the "return 0" statements
   mov    %r12,0xf0(%rbx) <= store the output

> Side note: the reason we have that "asm_volatile_goto()" define in the
> kernel is that we *used* to have a _different_ workaround for a gcc
> bug in this area:
> 
>  /*
>   * GCC 'asm goto' miscompiles certain code sequences:
>   *
>   *   http://gcc.gnu.org/bugzilla/show_bug.cgi?id=58670
>   *
>   * Work it around via a compiler barrier quirk suggested by Jakub Jelinek.
>   *
>   * (asm goto is automatically volatile - the naming reflects this.)
>   */
>  #define asm_volatile_goto(x...) do { asm goto(x); asm (""); } while (0)
> 
> and looking at that (old) bugzilla there seems to be a lot of "seems
> to be fixed", but it's not entirely clear.
> 
> We've removed that workaround in commit 43c249ea0b1e ("compiler-gcc.h:
> remove ancient workaround for gcc PR 58670"), I'm wondering if maybe
> that removal was a bit optimistic.

FWIW, reverting that does restore correct behavior on gcc-11.

Note, this is 100% reproducible across multiple systems, though AFAICT it's
somewhat dependent on the .config.  Holler if anyone wants the .config.