Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   David Laight <David.Laight@ACULAB.COM>
To:     'Thomas Gleixner' <tglx@linutronix.de>,
        'Alexander Graf' <graf@amazon.com>, 'X86 ML' <x86@kernel.org>
CC:     'Andy Lutomirski' <luto@kernel.org>,
        'LKML' <linux-kernel@vger.kernel.org>,
        'Andrew Cooper' <andrew.cooper3@citrix.com>,
        "'Paul E. McKenney'" <paulmck@kernel.org>,
        'Alexandre Chartre' <alexandre.chartre@oracle.com>,
        'Frederic Weisbecker' <frederic@kernel.org>,
        'Paolo Bonzini' <pbonzini@redhat.com>,
        'Sean Christopherson' <sean.j.christopherson@intel.com>,
        'Masami Hiramatsu' <mhiramat@kernel.org>,
        'Petr Mladek' <pmladek@suse.com>,
        'Steven Rostedt' <rostedt@goodmis.org>,
        'Joel Fernandes' <joel@joelfernandes.org>,
        'Boris Ostrovsky' <boris.ostrovsky@oracle.com>,
        'Juergen Gross' <jgross@suse.com>,
        "'Mathieu Desnoyers'" <mathieu.desnoyers@efficios.com>,
        'Josh Poimboeuf' <jpoimboe@redhat.com>,
        'Will Deacon' <will@kernel.org>,
        'Tom Lendacky' <thomas.lendacky@amd.com>,
        'Wei Liu' <wei.liu@kernel.org>,
        'Michael Kelley' <mikelley@microsoft.com>,
        'Jason Chen CJ' <jason.cj.chen@intel.com>,
        "'Zhao Yakui'" <yakui.zhao@intel.com>,
        "'Peter Zijlstra (Intel)'" <peterz@infradead.org>,
        'Avi Kivity' <avi@scylladb.com>,
        "'Herrenschmidt, Benjamin'" <benh@amazon.com>,
        "'robketr@amazon.de'" <robketr@amazon.de>,
        "'amos@scylladb.com'" <amos@scylladb.com>,
        'Brian Gerst' <brgerst@gmail.com>,
        "'stable@vger.kernel.org'" <stable@vger.kernel.org>,
        'Alex bykov' <alex.bykov@scylladb.com>
Subject: RE: x86/irq: Unbreak interrupt affinity setting
Thread-Topic: x86/irq: Unbreak interrupt affinity setting
Thread-Index: AQHWe+aKb+AhwM2rPkq6/MK3Hcp5nKlK5iEAgAAGXqA=
Date:   Wed, 26 Aug 2020 21:47:23 +0000
Message-ID: <42ae8716e425495c964ae7372bd7ff52@AcuMS.aculab.com>
References: <20200826115357.3049-1-graf@amazon.com>
 <87k0xlv5w5.fsf@nanos.tec.linutronix.de>
 <fd87a87d-7d8a-9959-6c81-f49003a43c21@amazon.com>
 <87blixuuny.fsf@nanos.tec.linutronix.de>
 <873649utm4.fsf@nanos.tec.linutronix.de>
 <87wo1ltaxz.fsf@nanos.tec.linutronix.de>
 <db3e28b59d404f55aff83120c077d6f6@AcuMS.aculab.com>
In-Reply-To: <db3e28b59d404f55aff83120c077d6f6@AcuMS.aculab.com>
Accept-Language: en-GB, en-US
Content-Language: en-US
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

From: David Laight
> Sent: 26 August 2020 22:37
> 
> From: Thomas Gleixner
> > Sent: 26 August 2020 21:22
> ...
> > Moving interrupts on x86 happens in several steps. A new vector on a
> > different CPU is allocated and the relevant interrupt source is
> > reprogrammed to that. But that's racy and there might be an interrupt
> > already in flight to the old vector. So the old vector is preserved until
> > the first interrupt arrives on the new vector and the new target CPU. Once
> > that happens the old vector is cleaned up, but this cleanup still depends
> > on the vector number being stored in pt_regs::orig_ax, which is now -1.
> 
> I suspect that it is much more 'racy' than that for PCI-X interrupts.
> On the hardware side there is an interrupt disable bit, and address
> and a value.
> To raise an interrupt the hardware must write the value to the address.
> 
> If the cpu needs to move an interrupt both the address and value
> need changing, but the cpu wont write the address and value using
> the same TLP, so the hardware could potentially write a value to
> the wrong address.
> Worse than that, the hardware could easily only look at the address
> and value in the clocks after checking the interrupt is enabled.
> So masking the interrupt immediately prior to changing the vector
> info may not be enough.
> 
> It is likely that a read-back of the mask before updating the vector
> is enough.

But not enough to assume you won't receive an interrupt after reading
back that interrupts are masked.

(I've implemented the hardware side for an fpga ...)

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)