Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 21 Nov 2022 13:50:51 -0500
From:   "Gabriel L. Somlo" <gsomlo@gmail.com>
To:     Jiri Slaby <jirislaby@kernel.org>
Cc:     linux-kernel@vger.kernel.org, linux-serial@vger.kernel.org,
        gregkh@linuxfoundation.org, kgugala@antmicro.com,
        mholenko@antmicro.com, joel@jms.id.au,
        david.abdurachmanov@gmail.com, florent@enjoy-digital.fr,
        geert@linux-m68k.org, ilpo.jarvinen@linux.intel.com
Subject: Re: [PATCH v5 12/14] serial: liteuart: add IRQ support for the RX
 path
Message-ID: <Y3vIi2OF5t4IrCS1@errol.ini.cmu.edu>
References: <20221118145512.509950-1-gsomlo@gmail.com>
 <20221118145512.509950-13-gsomlo@gmail.com>
 <abb7d87a-46c7-fe7f-f943-4287d15a537f@kernel.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <abb7d87a-46c7-fe7f-f943-4287d15a537f@kernel.org>
Precedence: bulk

Hi Jiri,

On Mon, Nov 21, 2022 at 09:54:34AM +0100, Jiri Slaby wrote:
> On 18. 11. 22, 15:55, Gabriel Somlo wrote:
> > Add support for IRQ-driven RX. Support for the TX path will be added
> > in a separate commit.
> > 
> > Signed-off-by: Gabriel Somlo <gsomlo@gmail.com>
> > Reviewed-by: Ilpo J?rvinen <ilpo.jarvinen@linux.intel.com>
> > ---
> > 
> > Changes from v4:
> >    - using dev_err() instead of a combo of pr_err() and pr_fmt()
> >    - dropped "get irq" comment in probe()
> > 
> > > Changes from v3:
> > >    - add shadow irq register to support polling mode and avoid reading
> > >      hardware mmio irq register to learn which irq flags are enabled
> > >      - this also simplifies both liteuart_interrupt() and liteuart_startup()
> > 
> >   drivers/tty/serial/liteuart.c | 76 +++++++++++++++++++++++++++++++----
> >   1 file changed, 69 insertions(+), 7 deletions(-)
> > 
> > diff --git a/drivers/tty/serial/liteuart.c b/drivers/tty/serial/liteuart.c
> > index 8a6e176be08e..678c37c952cf 100644
> > --- a/drivers/tty/serial/liteuart.c
> > +++ b/drivers/tty/serial/liteuart.c
> > @@ -7,6 +7,7 @@
> >   #include <linux/bits.h>
> >   #include <linux/console.h>
> > +#include <linux/interrupt.h>
> >   #include <linux/litex.h>
> >   #include <linux/module.h>
> >   #include <linux/of.h>
> > @@ -46,6 +47,7 @@ struct liteuart_port {
> >   	struct uart_port port;
> >   	struct timer_list timer;
> >   	u32 id;
> > +	u8 irq_reg;
> >   };
> >   #define to_liteuart_port(port)	container_of(port, struct liteuart_port, port)
> > @@ -76,6 +78,19 @@ static void liteuart_putchar(struct uart_port *port, unsigned char ch)
> >   	litex_write8(port->membase + OFF_RXTX, ch);
> >   }
> > +static void liteuart_update_irq_reg(struct uart_port *port, bool set, u8 mask)
> > +{
> > +	struct liteuart_port *uart = to_liteuart_port(port);
> > +
> > +	if (set)
> > +		uart->irq_reg |= mask;
> > +	else
> > +		uart->irq_reg &= ~mask;
> > +
> > +	if (port->irq)
> > +		litex_write8(port->membase + OFF_EV_ENABLE, uart->irq_reg);
> > +}
> > +
> >   static void liteuart_stop_tx(struct uart_port *port)
> >   {
> >   }
> > @@ -129,13 +144,27 @@ static void liteuart_rx_chars(struct uart_port *port)
> >   	tty_flip_buffer_push(&port->state->port);
> >   }
> > +static irqreturn_t liteuart_interrupt(int irq, void *data)
> > +{
> > +	struct liteuart_port *uart = data;
> > +	struct uart_port *port = &uart->port;
> > +	u8 isr;
> > +
> > +	spin_lock(&port->lock);
> > +	isr = litex_read8(port->membase + OFF_EV_PENDING) & uart->irq_reg;
> > +	if (isr & EV_RX)
> > +		liteuart_rx_chars(port);
> > +	spin_unlock(&port->lock);
> > +
> > +	return IRQ_RETVAL(isr);
> > +}
> > +
> >   static void liteuart_timer(struct timer_list *t)
> >   {
> >   	struct liteuart_port *uart = from_timer(uart, t, timer);
> >   	struct uart_port *port = &uart->port;
> > -	liteuart_rx_chars(port);
> > -
> > +	liteuart_interrupt(0, port);
> 
> Are you sure spin_lock() is safe from this path? I assume so, but have you
> thought about it?

I checked and at that point `in_serving_softirq()` is true.

*However*, after studying spin_lock() and friends for a while, I'm
not quite clear on whether that unequivocally translates
to "yes, we're safe" :)

As such, I'm inclined to switch to `spin_lock_irqsave()` and
`spin_unlock_irqrestore()` even in the interrupt handler, which is
explicitly stated to be "safe from any context":
https://www.kernel.org/doc/html/v4.15/kernel-hacking/locking.html#cheat-sheet-for-locking

The alternative could be to set `TIMER_IRQSAFE` in `timer_setup()`,
but no other tty driver seems to be doing that, so I'd be a bit off
the beaten path there... :)

Please do let me know what you think about this, particularly if you
consider going the spin_lock_irqsave-everywhere-just-to-be-safe route 
overkill... :)
 
> >   	mod_timer(&uart->timer, jiffies + uart_poll_timeout(port));
> >   }
> > @@ -161,19 +190,46 @@ static unsigned int liteuart_get_mctrl(struct uart_port *port)
> >   static int liteuart_startup(struct uart_port *port)
> >   {
> >   	struct liteuart_port *uart = to_liteuart_port(port);
> > +	unsigned long flags;
> > +	int ret;
> > -	/* disable events */
> > -	litex_write8(port->membase + OFF_EV_ENABLE, 0);
> > +	if (port->irq) {
> > +		ret = request_irq(port->irq, liteuart_interrupt, 0,
> > +				  KBUILD_MODNAME, uart);
> 
> Just asking: cannot the irq be shared?

Given the way LiteX gateware is currently generated, each
irq-triggering device is given its own separate line. I don't think
setting the IRQF_SHARED flag actually *hurts* anything (no difference
in behavior while testing), but I don't think it's needed ATM.

> > +		if (ret) {
> > +			dev_err(port->dev,
> > +				"line %d irq %d failed: switch to polling\n",
> > +				port->line, port->irq);
> 
> That is, it should be only dev_warn(), or?

Makes sense, will use dev_warn() in v6.

Please LMK what you think about spin_lock[_irqsave] (and IRQF_SHARED),
and I'll send out v6 with all the necessary chances right after that.

Thanks much,
--Gabriel

> > +			port->irq = 0;
> > +		}
> > +	}
> 
> thanks,
> -- 
> js
> suse labs
>