2020-10-13 11:19:37

by Tian Tao

[permalink] [raw]
Subject: [PATCH] vfio/platform: Replace spin_lock_irqsave by spin_lock in hard IRQ

It is redundant to do irqsave and irqrestore in hardIRQ context.

Signed-off-by: Tian Tao <[email protected]>
---
drivers/vfio/platform/vfio_platform_irq.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/drivers/vfio/platform/vfio_platform_irq.c b/drivers/vfio/platform/vfio_platform_irq.c
index c5b09ec..24fd6c5 100644
--- a/drivers/vfio/platform/vfio_platform_irq.c
+++ b/drivers/vfio/platform/vfio_platform_irq.c
@@ -139,10 +139,9 @@ static int vfio_platform_set_irq_unmask(struct vfio_platform_device *vdev,
static irqreturn_t vfio_automasked_irq_handler(int irq, void *dev_id)
{
struct vfio_platform_irq *irq_ctx = dev_id;
- unsigned long flags;
int ret = IRQ_NONE;

- spin_lock_irqsave(&irq_ctx->lock, flags);
+ spin_lock(&irq_ctx->lock);

if (!irq_ctx->masked) {
ret = IRQ_HANDLED;
@@ -152,7 +151,7 @@ static irqreturn_t vfio_automasked_irq_handler(int irq, void *dev_id)
irq_ctx->masked = true;
}

- spin_unlock_irqrestore(&irq_ctx->lock, flags);
+ spin_unlock(&irq_ctx->lock);

if (ret == IRQ_HANDLED)
eventfd_signal(irq_ctx->trigger, 1);
--
2.7.4


2020-10-14 06:56:09

by Alex Williamson

[permalink] [raw]
Subject: Re: [PATCH] vfio/platform: Replace spin_lock_irqsave by spin_lock in hard IRQ

On Tue, 13 Oct 2020 10:00:58 +0800
Tian Tao <[email protected]> wrote:

> It is redundant to do irqsave and irqrestore in hardIRQ context.

But this function is also called from non-IRQ context. Thanks,

Alex

> Signed-off-by: Tian Tao <[email protected]>
> ---
> drivers/vfio/platform/vfio_platform_irq.c | 5 ++---
> 1 file changed, 2 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/vfio/platform/vfio_platform_irq.c b/drivers/vfio/platform/vfio_platform_irq.c
> index c5b09ec..24fd6c5 100644
> --- a/drivers/vfio/platform/vfio_platform_irq.c
> +++ b/drivers/vfio/platform/vfio_platform_irq.c
> @@ -139,10 +139,9 @@ static int vfio_platform_set_irq_unmask(struct vfio_platform_device *vdev,
> static irqreturn_t vfio_automasked_irq_handler(int irq, void *dev_id)
> {
> struct vfio_platform_irq *irq_ctx = dev_id;
> - unsigned long flags;
> int ret = IRQ_NONE;
>
> - spin_lock_irqsave(&irq_ctx->lock, flags);
> + spin_lock(&irq_ctx->lock);
>
> if (!irq_ctx->masked) {
> ret = IRQ_HANDLED;
> @@ -152,7 +151,7 @@ static irqreturn_t vfio_automasked_irq_handler(int irq, void *dev_id)
> irq_ctx->masked = true;
> }
>
> - spin_unlock_irqrestore(&irq_ctx->lock, flags);
> + spin_unlock(&irq_ctx->lock);
>
> if (ret == IRQ_HANDLED)
> eventfd_signal(irq_ctx->trigger, 1);

Subject: RE: [PATCH] vfio/platform: Replace spin_lock_irqsave by spin_lock in hard IRQ



> -----Original Message-----
> From: Alex Williamson [mailto:[email protected]]
> Sent: Wednesday, October 14, 2020 10:32 AM
> To: tiantao (H) <[email protected]>
> Cc: [email protected]; [email protected]; [email protected];
> [email protected]; Song Bao Hua (Barry Song)
> <[email protected]>; Linuxarm <[email protected]>
> Subject: Re: [PATCH] vfio/platform: Replace spin_lock_irqsave by spin_lock in
> hard IRQ
>
> On Tue, 13 Oct 2020 10:00:58 +0800
> Tian Tao <[email protected]> wrote:
>
> > It is redundant to do irqsave and irqrestore in hardIRQ context.
>
> But this function is also called from non-IRQ context. Thanks,

It seems you mean
vfio_platform_set_irqs_ioctl() ->
vfio_platform_set_irq_trigger ->
handler() ?

so, will it be better to move the irqsave out of the vfio_automasked_irq_handler()
and put it to where the function is called in non-IRQ context?

I mean:

irqhandler()
{
spin_lock() //without irqsave
spin_unlock()
}

Non-irq context which is calling this handler:
irqsave();
irqhandler();
irqrestore();

Anyway, if it is called in IRQ context, it is redundant to do irqsave.

>
> Alex
>
> > Signed-off-by: Tian Tao <[email protected]>
> > ---
> > drivers/vfio/platform/vfio_platform_irq.c | 5 ++---
> > 1 file changed, 2 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/vfio/platform/vfio_platform_irq.c
> b/drivers/vfio/platform/vfio_platform_irq.c
> > index c5b09ec..24fd6c5 100644
> > --- a/drivers/vfio/platform/vfio_platform_irq.c
> > +++ b/drivers/vfio/platform/vfio_platform_irq.c
> > @@ -139,10 +139,9 @@ static int vfio_platform_set_irq_unmask(struct
> vfio_platform_device *vdev,
> > static irqreturn_t vfio_automasked_irq_handler(int irq, void *dev_id)
> > {
> > struct vfio_platform_irq *irq_ctx = dev_id;
> > - unsigned long flags;
> > int ret = IRQ_NONE;
> >
> > - spin_lock_irqsave(&irq_ctx->lock, flags);
> > + spin_lock(&irq_ctx->lock);
> >
> > if (!irq_ctx->masked) {
> > ret = IRQ_HANDLED;
> > @@ -152,7 +151,7 @@ static irqreturn_t vfio_automasked_irq_handler(int
> irq, void *dev_id)
> > irq_ctx->masked = true;
> > }
> >
> > - spin_unlock_irqrestore(&irq_ctx->lock, flags);
> > + spin_unlock(&irq_ctx->lock);
> >
> > if (ret == IRQ_HANDLED)
> > eventfd_signal(irq_ctx->trigger, 1);

Thanks
Barry

2020-10-14 08:16:04

by Alex Williamson

[permalink] [raw]
Subject: Re: [PATCH] vfio/platform: Replace spin_lock_irqsave by spin_lock in hard IRQ

On Wed, 14 Oct 2020 00:15:13 +0000
"Song Bao Hua (Barry Song)" <[email protected]> wrote:

> > -----Original Message-----
> > From: Alex Williamson [mailto:[email protected]]
> > Sent: Wednesday, October 14, 2020 10:32 AM
> > To: tiantao (H) <[email protected]>
> > Cc: [email protected]; [email protected]; [email protected];
> > [email protected]; Song Bao Hua (Barry Song)
> > <[email protected]>; Linuxarm <[email protected]>
> > Subject: Re: [PATCH] vfio/platform: Replace spin_lock_irqsave by spin_lock in
> > hard IRQ
> >
> > On Tue, 13 Oct 2020 10:00:58 +0800
> > Tian Tao <[email protected]> wrote:
> >
> > > It is redundant to do irqsave and irqrestore in hardIRQ context.
> >
> > But this function is also called from non-IRQ context. Thanks,
>
> It seems you mean
> vfio_platform_set_irqs_ioctl() ->
> vfio_platform_set_irq_trigger ->
> handler() ?

Yes.

> so, will it be better to move the irqsave out of the vfio_automasked_irq_handler()
> and put it to where the function is called in non-IRQ context?
>
> I mean:
>
> irqhandler()
> {
> spin_lock() //without irqsave
> spin_unlock()
> }
>
> Non-irq context which is calling this handler:
> irqsave();
> irqhandler();
> irqrestore();
>
> Anyway, if it is called in IRQ context, it is redundant to do irqsave.

What's the advantage? You're saying it's redundant, is it also wrong?
If it's not wrong and only redundant, what's the tangible latency
difference in maintaining a separate IRQ context handler without the
irqsave/restore? Thanks,

Alex

> > > Signed-off-by: Tian Tao <[email protected]>
> > > ---
> > > drivers/vfio/platform/vfio_platform_irq.c | 5 ++---
> > > 1 file changed, 2 insertions(+), 3 deletions(-)
> > >
> > > diff --git a/drivers/vfio/platform/vfio_platform_irq.c
> > b/drivers/vfio/platform/vfio_platform_irq.c
> > > index c5b09ec..24fd6c5 100644
> > > --- a/drivers/vfio/platform/vfio_platform_irq.c
> > > +++ b/drivers/vfio/platform/vfio_platform_irq.c
> > > @@ -139,10 +139,9 @@ static int vfio_platform_set_irq_unmask(struct
> > vfio_platform_device *vdev,
> > > static irqreturn_t vfio_automasked_irq_handler(int irq, void *dev_id)
> > > {
> > > struct vfio_platform_irq *irq_ctx = dev_id;
> > > - unsigned long flags;
> > > int ret = IRQ_NONE;
> > >
> > > - spin_lock_irqsave(&irq_ctx->lock, flags);
> > > + spin_lock(&irq_ctx->lock);
> > >
> > > if (!irq_ctx->masked) {
> > > ret = IRQ_HANDLED;
> > > @@ -152,7 +151,7 @@ static irqreturn_t vfio_automasked_irq_handler(int
> > irq, void *dev_id)
> > > irq_ctx->masked = true;
> > > }
> > >
> > > - spin_unlock_irqrestore(&irq_ctx->lock, flags);
> > > + spin_unlock(&irq_ctx->lock);
> > >
> > > if (ret == IRQ_HANDLED)
> > > eventfd_signal(irq_ctx->trigger, 1);
>
> Thanks
> Barry
>

Subject: RE: [PATCH] vfio/platform: Replace spin_lock_irqsave by spin_lock in hard IRQ



> -----Original Message-----
> From: Alex Williamson [mailto:[email protected]]
> Sent: Wednesday, October 14, 2020 1:50 PM
> To: Song Bao Hua (Barry Song) <[email protected]>
> Cc: tiantao (H) <[email protected]>; [email protected];
> [email protected]; [email protected]; [email protected];
> Linuxarm <[email protected]>
> Subject: Re: [PATCH] vfio/platform: Replace spin_lock_irqsave by spin_lock in
> hard IRQ
>
> On Wed, 14 Oct 2020 00:15:13 +0000
> "Song Bao Hua (Barry Song)" <[email protected]> wrote:
>
> > > -----Original Message-----
> > > From: Alex Williamson [mailto:[email protected]]
> > > Sent: Wednesday, October 14, 2020 10:32 AM
> > > To: tiantao (H) <[email protected]>
> > > Cc: [email protected]; [email protected]; [email protected];
> > > [email protected]; Song Bao Hua (Barry Song)
> > > <[email protected]>; Linuxarm <[email protected]>
> > > Subject: Re: [PATCH] vfio/platform: Replace spin_lock_irqsave by spin_lock
> in
> > > hard IRQ
> > >
> > > On Tue, 13 Oct 2020 10:00:58 +0800
> > > Tian Tao <[email protected]> wrote:
> > >
> > > > It is redundant to do irqsave and irqrestore in hardIRQ context.
> > >
> > > But this function is also called from non-IRQ context. Thanks,
> >
> > It seems you mean
> > vfio_platform_set_irqs_ioctl() ->
> > vfio_platform_set_irq_trigger ->
> > handler() ?
>
> Yes.
>
> > so, will it be better to move the irqsave out of the
> vfio_automasked_irq_handler()
> > and put it to where the function is called in non-IRQ context?
> >
> > I mean:
> >
> > irqhandler()
> > {
> > spin_lock() //without irqsave
> > spin_unlock()
> > }
> >
> > Non-irq context which is calling this handler:
> > irqsave();
> > irqhandler();
> > irqrestore();
> >
> > Anyway, if it is called in IRQ context, it is redundant to do irqsave.
>
> What's the advantage? You're saying it's redundant, is it also wrong?

It is not wrong and it doesn't make any malfunction. It just takes a couple of
instruction cycles to do save/restore and irq-disable/enable of cpu, which
is useless in irq context.

So the advantage is that we are going to remove some redundant instruction
cycles. And if the irq handler is called very often, we speed up the system.

> If it's not wrong and only redundant, what's the tangible latency
> difference in maintaining a separate IRQ context handler without the
> irqsave/restore? Thanks,

For this question, maybe need some benchmark to get answer. If the irqhandler
is not called that often, I agree it might be not worth to maintain two pieces of
code.

>
> Alex
>
> > > > Signed-off-by: Tian Tao <[email protected]>
> > > > ---
> > > > drivers/vfio/platform/vfio_platform_irq.c | 5 ++---
> > > > 1 file changed, 2 insertions(+), 3 deletions(-)
> > > >
> > > > diff --git a/drivers/vfio/platform/vfio_platform_irq.c
> > > b/drivers/vfio/platform/vfio_platform_irq.c
> > > > index c5b09ec..24fd6c5 100644
> > > > --- a/drivers/vfio/platform/vfio_platform_irq.c
> > > > +++ b/drivers/vfio/platform/vfio_platform_irq.c
> > > > @@ -139,10 +139,9 @@ static int vfio_platform_set_irq_unmask(struct
> > > vfio_platform_device *vdev,
> > > > static irqreturn_t vfio_automasked_irq_handler(int irq, void *dev_id)
> > > > {
> > > > struct vfio_platform_irq *irq_ctx = dev_id;
> > > > - unsigned long flags;
> > > > int ret = IRQ_NONE;
> > > >
> > > > - spin_lock_irqsave(&irq_ctx->lock, flags);
> > > > + spin_lock(&irq_ctx->lock);
> > > >
> > > > if (!irq_ctx->masked) {
> > > > ret = IRQ_HANDLED;
> > > > @@ -152,7 +151,7 @@ static irqreturn_t
> vfio_automasked_irq_handler(int
> > > irq, void *dev_id)
> > > > irq_ctx->masked = true;
> > > > }
> > > >
> > > > - spin_unlock_irqrestore(&irq_ctx->lock, flags);
> > > > + spin_unlock(&irq_ctx->lock);
> > > >
> > > > if (ret == IRQ_HANDLED)
> > > > eventfd_signal(irq_ctx->trigger, 1);
> >

Thanks
Barry