Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20220903161309.32848-4-apatel@ventanamicro.com> <20220904080601.2405-1-hdanton@sina.com>
In-Reply-To: <20220904080601.2405-1-hdanton@sina.com>
From:   Anup Patel <apatel@ventanamicro.com>
Date:   Mon, 14 Nov 2022 14:26:44 +0530
Message-ID: <CAK9=C2XJdEP=h4dQFKOSBcZMMH-OZRGyWy_B333UYbFMDGMoGQ@mail.gmail.com>
Subject: Re: [PATCH v9 3/7] genirq: Add mechanism to multiplex a single HW IPI
To:     Hillf Danton <hdanton@sina.com>
Cc:     linux-kernel@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Sun, Sep 4, 2022 at 1:36 PM Hillf Danton <hdanton@sina.com> wrote:
>
> On 3 Sep 2022 21:43:05 +0530 Anup Patel <apatel@ventanamicro.com> wrote:
> >
> > +static void ipi_mux_send_mask(struct irq_data *d, const struct cpumask *mask)
> > +{
> > +     u32 ipi_bit = BIT(irqd_to_hwirq(d));
> > +     struct cpumask pmask = { 0 };
> > +     unsigned long pending;
> > +     int cpu;
> > +
> > +     for_each_cpu(cpu, mask) {
> > +             pending = atomic_fetch_or_release(ipi_bit,
> > +                                     per_cpu_ptr(&ipi_mux_bits, cpu));
> > +
> > +             /*
> > +              * The atomic_fetch_or_release() above must complete before
> > +              * the atomic_read() below to avoid racing ipi_mux_unmask().
> > +              */
> > +             smp_mb__after_atomic();
> > +
> > +             if (!(pending & ipi_bit) &&
> > +                 (atomic_read(per_cpu_ptr(&ipi_mux_enable, cpu)) & ipi_bit))
> > +                     cpumask_set_cpu(cpu, &pmask);
> > +     }
> > +
> > +     /* Trigger the parent IPI */
> > +     ipi_mux_ops->ipi_mux_send(ipi_mux_parent_virq, ipi_mux_data, &pmask);
> > +}
>
> Can you shed some light on the reasons why a pending TLB ipi for flushing
> page3 on CPU2 prevents TLB ipi from being sent to CPU2 for flushing page9.

The "!(pending & ipi_bit)" check in the above "if ()" statement seems to be
causing this. I am seeing CPU soft-lockup due to the  "!(pending & ipi_bit)"
check on a QEMU RISC-V virt machine with large CPUs.

I will quickly send v11 with the check removed.

Thanks,
Anup