Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [PATCH] x86/irq: Preserve vector in orig_ax for APIC code
To:     Thomas Gleixner <tglx@linutronix.de>, X86 ML <x86@kernel.org>
CC:     Andy Lutomirski <luto@kernel.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Andrew Cooper <andrew.cooper3@citrix.com>,
        "Paul E. McKenney" <paulmck@kernel.org>,
        Alexandre Chartre <alexandre.chartre@oracle.com>,
        Frederic Weisbecker <frederic@kernel.org>,
        Paolo Bonzini <pbonzini@redhat.com>,
        Sean Christopherson <sean.j.christopherson@intel.com>,
        Masami Hiramatsu <mhiramat@kernel.org>,
        Petr Mladek <pmladek@suse.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Joel Fernandes <joel@joelfernandes.org>,
        Boris Ostrovsky <boris.ostrovsky@oracle.com>,
        Juergen Gross <jgross@suse.com>,
        Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
        Josh Poimboeuf <jpoimboe@redhat.com>,
        "Will Deacon" <will@kernel.org>,
        Tom Lendacky <thomas.lendacky@amd.com>,
        Wei Liu <wei.liu@kernel.org>,
        Michael Kelley <mikelley@microsoft.com>,
        Jason Chen CJ <jason.cj.chen@intel.com>,
        Zhao Yakui <yakui.zhao@intel.com>,
        "Peter Zijlstra (Intel)" <peterz@infradead.org>,
        Avi Kivity <avi@scylladb.com>,
        "Herrenschmidt, Benjamin" <benh@amazon.com>, <robketr@amazon.de>,
        <amos@scylladb.com>, Brian Gerst <brgerst@gmail.com>,
        <stable@vger.kernel.org>
References: <20200826115357.3049-1-graf@amazon.com>
 <87k0xlv5w5.fsf@nanos.tec.linutronix.de>
From:   Alexander Graf <graf@amazon.com>
Message-ID: <fd87a87d-7d8a-9959-6c81-f49003a43c21@amazon.com>
Date:   Wed, 26 Aug 2020 18:33:46 +0200
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:78.0)
 Gecko/20100101 Thunderbird/78.1.1
MIME-Version: 1.0
In-Reply-To: <87k0xlv5w5.fsf@nanos.tec.linutronix.de>
Content-Language: en-US
Content-Type: text/plain; charset="windows-1252"; format="flowed"
Content-Transfer-Encoding: quoted-printable
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk


On 26.08.20 16:27, Thomas Gleixner wrote:
> =

> On Wed, Aug 26 2020 at 13:53, Alexander Graf wrote:
>> Commit 633260fa143 ("x86/irq: Convey vector as argument and not in ptreg=
s")
>> changed the handover logic of the vector identifier from ~vector in orig=
_ax
>> to purely register based. Unfortunately, this field has another consumer
>> in the APIC code which the commit did not touch. The net result was that
>> IRQ balancing did not work and instead resulted in interrupt storms, slo=
wing
>> down the system.
> =

> The net result is an observationof the symptom but that does not explain
> what the underlying technical issue is.
> =

>> This patch restores the original semantics that orig_ax contains the vec=
tor.
>> When we receive an interrupt now, the actual vector number stays stored =
in
>> the orig_ax field which then gets consumed by the APIC code.
>>
>> To ensure that nobody else trips over this in the future, the patch also=
 adds
>> comments at strategic places to warn anyone who would refactor the code =
that
>> there is another consumer of the field.
>>
>> With this patch in place, IRQ balancing works as expected and performance
>> levels are restored to previous levels.
> =

> There's a lot of 'This patch and we' in that changelog. Care to grep
> for 'This patch' in Documentation/process/ ?
> =

>> diff --git a/arch/x86/entry/entry_32.S b/arch/x86/entry/entry_32.S
>> index df8c017..22e829c 100644
>> --- a/arch/x86/entry/entry_32.S
>> +++ b/arch/x86/entry/entry_32.S
>> @@ -727,7 +727,7 @@ SYM_CODE_START_LOCAL(asm_\cfunc)
>>        ENCODE_FRAME_POINTER
>>        movl    %esp, %eax
>>        movl    PT_ORIG_EAX(%esp), %edx         /* get the vector from st=
ack */
>> -     movl    $-1, PT_ORIG_EAX(%esp)          /* no syscall to restart */
>> +     /* keep vector on stack for APIC's irq_complete_move() */
> =

> Yes that's fixing your observed wreckage, but it introduces a worse one.
> =

> user space
>    -> interrupt
>         push vector into orig_ax (values are in the ranges of 0-127 and -=
128 - 255
>                                   except for the system vectors which do
>                                   not go through this code)
>        handle()
>        ...
>        exit_to_user_mode_loop()
>           arch_do_signal()
>              /* Did we come from a system call? */
>              if (syscall_get_nr(current, regs) >=3D 0) {
> =

>                 ---> BOOM for any vector 0-127 because syscall_get_nr()
>                           resolves to regs->orig_ax
> =

> Going to be fun to debug.

Hah, that's the code flow I was looking for to understand why the value =

was negative in the first place. Thanks a lot for pointing it out!

> =

> The below nasty hack cures it, but I hate it with a passion. I'll look
> deeper for a sane variant.

An alternative (that doesn't make the code easier to read, but would fix =

the issue at hand) would be touse a pushq imm16 with vector | 0x8000 =

instead to always make the value negative, no?


Alex


Amazon Development Center Germany GmbH
Krausenstr. 38
10117 Berlin
Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss
Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B
Sitz: Berlin
Ust-ID: DE 289 237 879