Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 11 Aug 2023 19:59:57 -0700
In-Reply-To: <20230811180528.GJZNZ4aIHCn3zMaida@fat_crate.local>
Mime-Version: 1.0
References: <20230811155255.250835-1-seanjc@google.com> <20230811180528.GJZNZ4aIHCn3zMaida@fat_crate.local>
Message-ID: <ZNb1rXGWfyM0XdjB@google.com>
Subject: Re: [PATCH] x86/retpoline: Don't clobber RFLAGS during srso_safe_ret()
From:   Sean Christopherson <seanjc@google.com>
To:     Borislav Petkov <bp@alien8.de>
Cc:     Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>,
        Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
        linux-kernel@vger.kernel.org, Srikanth Aithal <sraithal@amd.com>,
        kvm@vger.kernel.org, Paolo Bonzini <pbonzini@redhat.com>
Content-Type: text/plain; charset="us-ascii"
Precedence: bulk

On Fri, Aug 11, 2023, Borislav Petkov wrote:
> On Fri, Aug 11, 2023 at 08:52:55AM -0700, Sean Christopherson wrote:
> > A major motivation for doing fast emulation is to leverage the CPU to
> > handle consumption and manipulation of arithmetic flags, i.e. RFLAGS is
> > both an input and output to the target of the call.  fastop() collects
> > the RFLAGS result by pushing RFLAGS onto the stack and popping them back
> > into a variable (held in RDI in this case)
> > 
> >   asm("push %[flags]; popf; " CALL_NOSPEC " ; pushf; pop %[flags]\n"
> 
> Right, and I've tested this countless times with gcc-built host and
> guest.
> 
> But Nathan's case where the host is built with gcc but the guest with
> clang, would trigger this. And as he confirms, that fixes it so I wonder
> what is the difference in code generation to make this rFLAGS corruption
> noticeable in that particular configuration.

Might be I/O APIC accesses?  Unless things have changed, the I/O APIC code uses
a struct overlay to access the I/O APIC, i.e. when doing emulated MMIO accesses.
If clang generates an ADD or whatever and consumes flags, e.g. instead of a
straight MOV, that would explain the problems.