Received-SPF: pass (google.com: domain of linux-kernel+bounces-169734-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Date: Mon, 6 May 2024 14:27:50 +0300
From: Serge Semin <fancer.lancer@gmail.com>
To: Borislav Petkov <bp@alien8.de>
Cc: Michal Simek <michal.simek@amd.com>, 
	Alexander Stein <alexander.stein@ew.tq-group.com>, Tony Luck <tony.luck@intel.com>, 
	James Morse <james.morse@arm.com>, Mauro Carvalho Chehab <mchehab@kernel.org>, 
	Robert Richter <rric@kernel.org>, Dinh Nguyen <dinguyen@kernel.org>, 
	Punnaiah Choudary Kalluri <punnaiah.choudary.kalluri@xilinx.com>, Arnd Bergmann <arnd@arndb.de>, 
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>, linux-arm-kernel@lists.infradead.org, linux-edac@vger.kernel.org, 
	linux-kernel@vger.kernel.org, Sherry Sun <sherry.sun@nxp.com>, Borislav Petkov <bp@suse.de>
Subject: Re: [PATCH v5 01/20] EDAC/synopsys: Fix ECC status data and IRQ
 disable race condition
Message-ID: <vugkhnu5c7so7dk3z2cuhlbu66gv6skvicuseblrmkzyttnnlr@lqzqvysk6wbl>
References: <20240222181324.28242-1-fancer.lancer@gmail.com>
 <20240222181324.28242-2-fancer.lancer@gmail.com>
 <20240415183616.GDZh1zoFsBzvAEduRo@fat_crate.local>
 <szcie4giwjykne4su6uu5wsmtsl3e3jd53rjfiwir6hm3ju7as@6eqh2xmj35ie>
 <20240421100712.GAZiTlUOm1hrLQvaMi@fat_crate.local>
 <whgp2xx4dv3szezz3bvmgutgazz6kvie3q7rgpr35zqzuzsygk@wppqzusteru4>
 <20240506102029.GGZjiu7TKP9FVp-2Sb@fat_crate.local>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240506102029.GGZjiu7TKP9FVp-2Sb@fat_crate.local>

On Mon, May 06, 2024 at 12:20:29PM +0200, Borislav Petkov wrote:
> On Thu, Apr 25, 2024 at 03:52:38PM +0300, Serge Semin wrote:
> > Even if we get to add the spin-lock serializing the ECCCLR writes it
> > won't solve the problem since the IRQ-disabler critical section could
> > be executed a bit before the IRQ-handler critical section so the later
> > one will just re-enable the IRQs disabled by the former one.
> > 
> > Here is what is suggested in my patch to fix the problem:
> > 
> >      IRQ-handler                        |    IRQ-disabler
> >                                         |
> > zynqmp_get_error_info:                  |
> >                                         | lock_irqsave
> >                                         | ECCCLR = 0; // disable IRQs
> >                                         | unlock_irqrestore
> >  lock_irqsave;                          |
> >  tmp = ECCCLR | clear_sts_bits;         |
> >  ECCCLR = tmp;                          |
> >  unlock_irqrestore;                     |
> 

> <--- I'm presuming here the IRQ-disabler will reenable interrupts at
> some point?
> 
> Otherwise we have the same problem as before when interrupts remain off
> after the IRQ handler has run.

In the sketch above the IRQ-disabler is the method which disables the
IRQ in the concurrent manner. After my patch is applied the
IRQ-handler will no longer touch the IRQ enable/disable bits, but will
preserve them as is:
-       clearval = ECC_CTRL_CLR_CE_ERR | ECC_CTRL_CLR_CE_ERRCNT;
-       clearval |= ECC_CTRL_CLR_UE_ERR | ECC_CTRL_CLR_UE_ERRCNT;
+       spin_lock_irqsave(&priv->reglock, flags);
+
+       clearval = readl(base + ECC_CLR_OFST) |
+                  ECC_CTRL_CLR_CE_ERR | ECC_CTRL_CLR_CE_ERRCNT |
+                  ECC_CTRL_CLR_UE_ERR | ECC_CTRL_CLR_UE_ERRCNT;
        writel(clearval, base + ECC_CLR_OFST);
-       writel(0x0, base + ECC_CLR_OFST);
+
+       spin_unlock_irqrestore(&priv->reglock, flags);

Thus there won't be need in the IRQs re-enabling later in the handler:

@@ -576,8 +601,6 @@ static irqreturn_t intr_handler(int irq, void *dev_id)
        /* v3.0 of the controller does not have this register */
        if (!(priv->p_data->quirks & DDR_ECC_INTR_SELF_CLEAR))
                writel(regval, priv->baseaddr + DDR_QOS_IRQ_STAT_OFST);
-       else
-               enable_intr(priv);

So the only IRQ-disabler left in the driver - disable_intr() - will be
called from the device/driver remove() function. The ECCCLR CSR access
will be guarded with the spin-lock in the IRQ-disabler and in the
IRQ-handler. So it will be safe to have them executed concurrently.

> 
> Other than that, yes, I see it, we will need the locking.
> 
> Thanks for elaborating.

Always welcome. Glad we've settled this.)

-Serge(y)

> 
> -- 
> Regards/Gruss,
>     Boris.
> 
> https://people.kernel.org/tglx/notes-about-netiquette