Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Thomas Gleixner <tglx@linutronix.de>
To:     Alexander Graf <graf@amazon.com>, X86 ML <x86@kernel.org>
Cc:     Andy Lutomirski <luto@kernel.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Andrew Cooper <andrew.cooper3@citrix.com>,
        "Paul E. McKenney" <paulmck@kernel.org>,
        Alexandre Chartre <alexandre.chartre@oracle.com>,
        Frederic Weisbecker <frederic@kernel.org>,
        Paolo Bonzini <pbonzini@redhat.com>,
        Sean Christopherson <sean.j.christopherson@intel.com>,
        Masami Hiramatsu <mhiramat@kernel.org>,
        Petr Mladek <pmladek@suse.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Joel Fernandes <joel@joelfernandes.org>,
        Boris Ostrovsky <boris.ostrovsky@oracle.com>,
        Juergen Gross <jgross@suse.com>,
        Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
        Josh Poimboeuf <jpoimboe@redhat.com>,
        Will Deacon <will@kernel.org>,
        Tom Lendacky <thomas.lendacky@amd.com>,
        Wei Liu <wei.liu@kernel.org>,
        Michael Kelley <mikelley@microsoft.com>,
        Jason Chen CJ <jason.cj.chen@intel.com>,
        Zhao Yakui <yakui.zhao@intel.com>,
        "Peter Zijlstra \(Intel\)" <peterz@infradead.org>,
        Avi Kivity <avi@scylladb.com>,
        "Herrenschmidt\, Benjamin" <benh@amazon.com>, robketr@amazon.de,
        amos@scylladb.com, Brian Gerst <brgerst@gmail.com>,
        stable@vger.kernel.org
Subject: Re: [PATCH] x86/irq: Preserve vector in orig_ax for APIC code
In-Reply-To: <20200826115357.3049-1-graf@amazon.com>
References: <20200826115357.3049-1-graf@amazon.com>
Date:   Wed, 26 Aug 2020 16:27:54 +0200
Message-ID: <87k0xlv5w5.fsf@nanos.tec.linutronix.de>
MIME-Version: 1.0
Content-Type: text/plain
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Wed, Aug 26 2020 at 13:53, Alexander Graf wrote:
> Commit 633260fa143 ("x86/irq: Convey vector as argument and not in ptregs")
> changed the handover logic of the vector identifier from ~vector in orig_ax
> to purely register based. Unfortunately, this field has another consumer
> in the APIC code which the commit did not touch. The net result was that
> IRQ balancing did not work and instead resulted in interrupt storms, slowing
> down the system.

The net result is an observationof the symptom but that does not explain
what the underlying technical issue is.

> This patch restores the original semantics that orig_ax contains the vector.
> When we receive an interrupt now, the actual vector number stays stored in
> the orig_ax field which then gets consumed by the APIC code.
>
> To ensure that nobody else trips over this in the future, the patch also adds
> comments at strategic places to warn anyone who would refactor the code that
> there is another consumer of the field.
>
> With this patch in place, IRQ balancing works as expected and performance
> levels are restored to previous levels.

There's a lot of 'This patch and we' in that changelog. Care to grep
for 'This patch' in Documentation/process/ ?

> diff --git a/arch/x86/entry/entry_32.S b/arch/x86/entry/entry_32.S
> index df8c017..22e829c 100644
> --- a/arch/x86/entry/entry_32.S
> +++ b/arch/x86/entry/entry_32.S
> @@ -727,7 +727,7 @@ SYM_CODE_START_LOCAL(asm_\cfunc)
>  	ENCODE_FRAME_POINTER
>  	movl	%esp, %eax
>  	movl	PT_ORIG_EAX(%esp), %edx		/* get the vector from stack */
> -	movl	$-1, PT_ORIG_EAX(%esp)		/* no syscall to restart */
> +	/* keep vector on stack for APIC's irq_complete_move() */

Yes that's fixing your observed wreckage, but it introduces a worse one.

user space
  -> interrupt
       push vector into orig_ax (values are in the ranges of 0-127 and -128 - 255
                                 except for the system vectors which do
                                 not go through this code)
      handle()
      ...
      exit_to_user_mode_loop()
         arch_do_signal()
            /* Did we come from a system call? */
	    if (syscall_get_nr(current, regs) >= 0) {

               ---> BOOM for any vector 0-127 because syscall_get_nr()
                         resolves to regs->orig_ax

Going to be fun to debug.

The below nasty hack cures it, but I hate it with a passion. I'll look
deeper for a sane variant.

Thanks,

        tglx
---
--- a/arch/x86/kernel/irq.c
+++ b/arch/x86/kernel/irq.c
@@ -246,7 +246,9 @@ DEFINE_IDTENTRY_IRQ(common_interrupt)
 
 	desc = __this_cpu_read(vector_irq[vector]);
 	if (likely(!IS_ERR_OR_NULL(desc))) {
+		regs->orig_ax = (unsigned long)vector;
 		handle_irq(desc, regs);
+		regs->orig_ax = -1;
 	} else {
 		ack_APIC_irq();