Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Sat, 10 Oct 2020 11:13:07 -0400
From:   Masayoshi Mizuma <msys.mizuma@gmail.com>
To:     Marc Zyngier <maz@kernel.org>
Cc:     Sumit Garg <sumit.garg@linaro.org>, catalin.marinas@arm.com,
        will@kernel.org, mark.rutland@arm.com, daniel.thompson@linaro.org,
        jason@lakedaemon.net, kgdb-bugreport@lists.sourceforge.net,
        dianders@chromium.org, linux-kernel@vger.kernel.org,
        linux-arm-kernel@lists.infradead.org, jason.wessel@windriver.com,
        tglx@linutronix.de, julien.thierry.kdev@gmail.com
Subject: Re: [PATCH v4 1/5] arm64: Add framework to turn IPI as NMI
Message-ID: <20201010151307.vq74if4mndjn4nhm@gabell>
References: <1599830924-13990-1-git-send-email-sumit.garg@linaro.org>
 <1599830924-13990-2-git-send-email-sumit.garg@linaro.org>
 <20201010015855.vksetnj4luft5enc@gabell>
 <877drypgqb.wl-maz@kernel.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <877drypgqb.wl-maz@kernel.org>
Precedence: bulk

On Sat, Oct 10, 2020 at 10:34:04AM +0100, Marc Zyngier wrote:
> On Sat, 10 Oct 2020 02:58:55 +0100,
> Masayoshi Mizuma <msys.mizuma@gmail.com> wrote:
> 
> [...]
> 
> > > +void ipi_nmi_setup(int cpu)
> > > +{
> > > +	if (!ipi_desc)
> > > +		return;
> > 
> > ipi_nmi_setup() may be called twice for CPU0:
> > 
> >   set_smp_ipi_range => set_smp_ipi_nmi => ipi_nmi_setup
> >                     => ipi_setup => ipi_nmi_setup
> > 
> > Actually, I got the following error message via the second ipi_nmi_setup():
> > 
> >   GICv3: Pseudo-NMIs enabled using relaxed ICC_PMR_EL1 synchronisation
> >   GICv3: Cannot set NMI property of enabled IRQ 8
> >   genirq: Failed to setup NMI delivery: irq 8
> > 
> > Why don't we have a check to prevent that? Like as:
> > 
> >        if (cpumask_test_cpu(cpu, ipi_desc->percpu_enabled))
> >                return;
> 
> That's definitely the wrong thing to do. prepare_nmi_setup() shouldn't
> be called twice, and papering over it isn't acceptable.

Got it. How about moving ipi_nmi_setup() from ipi_setup() to
secondary_start_kernel() ? so that CPU0 can call ipi_nmi_setup() only
from set_smp_ipi_nmi().

--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -245,6 +245,7 @@ asmlinkage notrace void secondary_start_kernel(void)
        notify_cpu_starting(cpu);
 
        ipi_setup(cpu);
+       ipi_nmi_setup(cpu);
 
        store_cpu_topology(cpu);
        numa_add_cpu(cpu);
@@ -966,8 +967,6 @@ static void ipi_setup(int cpu)
 
        for (i = 0; i < nr_ipi; i++)
                enable_percpu_irq(ipi_irq_base + i, 0);
-
-       ipi_nmi_setup(cpu);
 }
 
 #ifdef CONFIG_HOTPLUG_CPU

Thanks,
Masa