Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Sat, 25 Jul 2020 13:08:31 +0100
Message-ID: <875zabyeyo.wl-maz@kernel.org>
From:   Marc Zyngier <maz@kernel.org>
To:     Thomas Gleixner <tglx@linutronix.de>
Cc:     John Keeping <john@metanate.com>,
        LKML <linux-kernel@vger.kernel.org>, x86@kernel.org,
        Ben Herrenschmidt <benh@amazon.com>,
        Ali Saidi <alisaidi@amazon.com>,
        linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH V2] genirq/affinity: Handle affinity setting on inactive interrupts correctly
In-Reply-To: <87h7twu1cp.fsf@nanos.tec.linutronix.de>
References: <87k0z2s2q3.fsf@nanos.tec.linutronix.de>
        <877dv2rv25.fsf@nanos.tec.linutronix.de>
        <20200724182422.27ddced6.john@metanate.com>
        <87h7twu1cp.fsf@nanos.tec.linutronix.de>
User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue)
 FLIM/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL/10.8 EasyPG/1.0.0 Emacs/26.3
 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO)
MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue")
Content-Type: text/plain; charset=US-ASCII
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Hi both,

On Fri, 24 Jul 2020 21:03:50 +0100,
Thomas Gleixner <tglx@linutronix.de> wrote:
> 
> John,
> 
> John Keeping <john@metanate.com> writes:
> > On Fri, 17 Jul 2020 18:00:02 +0200
> > Thomas Gleixner <tglx@linutronix.de> wrote:
> > It seems that this patch breaks perf events on RK3288 because the PMU
> > interrupts that should be per-cpu are now all on CPU0 so no events are
> > collected from CPUs 1-3 and those interrupts are killed as spurious
> > after a few seconds.

SPI-backed PMUs. Urgh...

> >
> > I'm seeing this on 4.19.134 and 5.4.53 but as far as I can tell the
> > relevant code hasn't changed through to next-20200723.  Reverting the
> > backport of this change fixes the problem.
> 
> Bah.
> 
> > It looks like what happens is that because the interrupts are not
> > per-CPU in the hardware, armpmu_request_irq() calls irq_force_affinity()
> > while the interrupt is deactivated and then request_irq() with
> > IRQF_PERCPU | IRQF_NOBALANCING.
> >
> > Now when irq_startup() runs with IRQ_STARTUP_NORMAL, it calls
> > irq_setup_affinity() which returns early because IRQF_PERCPU and
> > IRQF_NOBALANCING are set, leaving the interrupt on its original CPU.
> 
> Right. My brain tricked me to believe that we made activation mandatory,
> but that's not.
> 
> I have some ideas for a trivial generic way to solve this without
> undoing the commit in question and without going through all the irq
> chip drivers. So far everything I came up with is butt ugly. Maybe Marc
> has some brilliant idea.

Not really. We have contradicting behaviours here, where some
interrupts want to see the set_affinity early (the above case), and
some cannot handle that (x86 vectors and the GICv3 ITS). We could key
it on the presence of an activate callback, but it feels fragile.

I'll follow up on your patch in the next email, which seems like a
sensible approach.

	M.

-- 
Without deviation from the norm, progress is not possible.