Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   David Laight <David.Laight@ACULAB.COM>
To:     'Thomas Gleixner' <tglx@linutronix.de>,
        Alexander Graf <graf@amazon.com>, X86 ML <x86@kernel.org>
CC:     Andy Lutomirski <luto@kernel.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Andrew Cooper <andrew.cooper3@citrix.com>,
        "Paul E. McKenney" <paulmck@kernel.org>,
        Alexandre Chartre <alexandre.chartre@oracle.com>,
        Frederic Weisbecker <frederic@kernel.org>,
        Paolo Bonzini <pbonzini@redhat.com>,
        Sean Christopherson <sean.j.christopherson@intel.com>,
        Masami Hiramatsu <mhiramat@kernel.org>,
        Petr Mladek <pmladek@suse.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Joel Fernandes <joel@joelfernandes.org>,
        Boris Ostrovsky <boris.ostrovsky@oracle.com>,
        Juergen Gross <jgross@suse.com>,
        Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
        Josh Poimboeuf <jpoimboe@redhat.com>,
        "Will Deacon" <will@kernel.org>,
        Tom Lendacky <thomas.lendacky@amd.com>,
        Wei Liu <wei.liu@kernel.org>,
        Michael Kelley <mikelley@microsoft.com>,
        Jason Chen CJ <jason.cj.chen@intel.com>,
        Zhao Yakui <yakui.zhao@intel.com>,
        "Peter Zijlstra (Intel)" <peterz@infradead.org>,
        Avi Kivity <avi@scylladb.com>,
        "Herrenschmidt, Benjamin" <benh@amazon.com>,
        "robketr@amazon.de" <robketr@amazon.de>,
        "amos@scylladb.com" <amos@scylladb.com>,
        Brian Gerst <brgerst@gmail.com>,
        "stable@vger.kernel.org" <stable@vger.kernel.org>,
        "Alex bykov" <alex.bykov@scylladb.com>
Subject: RE: x86/irq: Unbreak interrupt affinity setting
Thread-Topic: x86/irq: Unbreak interrupt affinity setting
Thread-Index: AQHWe+aKb+AhwM2rPkq6/MK3Hcp5nKlK5iEA
Date:   Wed, 26 Aug 2020 21:37:00 +0000
Message-ID: <db3e28b59d404f55aff83120c077d6f6@AcuMS.aculab.com>
References: <20200826115357.3049-1-graf@amazon.com>
 <87k0xlv5w5.fsf@nanos.tec.linutronix.de>
 <fd87a87d-7d8a-9959-6c81-f49003a43c21@amazon.com>
 <87blixuuny.fsf@nanos.tec.linutronix.de>
 <873649utm4.fsf@nanos.tec.linutronix.de>
 <87wo1ltaxz.fsf@nanos.tec.linutronix.de>
In-Reply-To: <87wo1ltaxz.fsf@nanos.tec.linutronix.de>
Accept-Language: en-GB, en-US
Content-Language: en-US
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

From: Thomas Gleixner
> Sent: 26 August 2020 21:22
...
> Moving interrupts on x86 happens in several steps. A new vector on a
> different CPU is allocated and the relevant interrupt source is
> reprogrammed to that. But that's racy and there might be an interrupt
> already in flight to the old vector. So the old vector is preserved until
> the first interrupt arrives on the new vector and the new target CPU. Once
> that happens the old vector is cleaned up, but this cleanup still depends
> on the vector number being stored in pt_regs::orig_ax, which is now -1.

I suspect that it is much more 'racy' than that for PCI-X interrupts.
On the hardware side there is an interrupt disable bit, and address
and a value.
To raise an interrupt the hardware must write the value to the address.

If the cpu needs to move an interrupt both the address and value
need changing, but the cpu wont write the address and value using
the same TLP, so the hardware could potentially write a value to
the wrong address.
Worse than that, the hardware could easily only look at the address
and value in the clocks after checking the interrupt is enabled.
So masking the interrupt immediately prior to changing the vector
info may not be enough.

It is likely that a read-back of the mask before updating the vector
is enough.

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)