Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <20200915115609.85106-1-qianjun.kernel@gmail.com>
 <20200915115609.85106-5-qianjun.kernel@gmail.com> <878scz89tl.fsf@nanos.tec.linutronix.de>
 <20200925004207.GE19346@lenoir>
In-Reply-To: <20200925004207.GE19346@lenoir>
From:   jun qian <qianjun.kernel@gmail.com>
Date:   Mon, 28 Sep 2020 18:51:48 +0800
Message-ID: <CAKc596Km6kjQcp2MJmH9BZLY_7i7yFmHDmRnaJGsm4WzUNjwaA@mail.gmail.com>
Subject: Re: [PATCH V7 4/4] softirq: Allow early break the softirq processing loop
To:     Frederic Weisbecker <frederic@kernel.org>
Cc:     Thomas Gleixner <tglx@linutronix.de>, peterz@infradead.org,
        will@kernel.org, luto@kernel.org, linux-kernel@vger.kernel.org,
        Yafang Shao <laoar.shao@gmail.com>, qais.yousef@arm.com,
        Uladzislau Rezki <urezki@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

Frederic Weisbecker <frederic@kernel.org> =E4=BA=8E2020=E5=B9=B49=E6=9C=882=
5=E6=97=A5=E5=91=A8=E4=BA=94 =E4=B8=8A=E5=8D=888:42=E5=86=99=E9=81=93=EF=BC=
=9A
>
> On Thu, Sep 24, 2020 at 05:37:42PM +0200, Thomas Gleixner wrote:
> > Subject: softirq; Prevent starvation of higher softirq vectors
> [...]
> > +     /*
> > +      * Word swap pending to move the not yet handled bits of the prev=
ious
> > +      * run first and then clear the duplicates in the newly raised on=
es.
> > +      */
> > +     swahw32s(&cur_pending);
> > +     pending =3D cur_pending & ~(cur_pending << SIRQ_PREV_SHIFT);
> > +
> >       for_each_set_bit(vec_nr, &pending, NR_SOFTIRQS) {
> >               int prev_count;
> >
> > +             vec_nr &=3D SIRQ_VECTOR_MASK;
>
> Shouldn't NR_SOFTIRQS above protect from that?
>
> >               __clear_bit(vec_nr, &pending);
> >               kstat_incr_softirqs_this_cpu(vec_nr);
> >
> [...]
> > +     } else {
> > +             /*
> > +              * Retain the unprocessed bits and swap @cur_pending back
> > +              * into normal ordering
> > +              */
> > +             cur_pending =3D (u32)pending;
> > +             swahw32s(&cur_pending);
> > +             /*
> > +              * If the previous bits are done move the low word of
> > +              * @pending into the high word so it's processed first.
> > +              */
> > +             if (!(cur_pending & SIRQ_PREV_MASK))
> > +                     cur_pending <<=3D SIRQ_PREV_SHIFT;
>
> If the previous bits are done and there is no timeout, should
> we consider to restart a loop?
>
> A common case would be to enter do_softirq() with RCU_SOFTIRQ set
> in the SIRQ_PREV_MASK and NET_RX_SOFTIRQ set in the normal mask.
>
> You would always end up processing the RCU_SOFTIRQ here and trigger
> ksoftirqd for the NET_RX_SOFTIRQ.

yes, I found that this problem also exists in our project. The RCU
softirq may cost
9ms, that will delay the net_rx/tx softirq to process, Peter's branch
maybe can slove
the problem
git://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git core/softirq

>
> Although that's probably no big deal as we should be already in ksoftirqd
> if we processed prev bits. We are just going to iterate the kthread loop
> instead of the do_softirq loop. Probably no real issue then...
>
>
> >
> > +             /* Merge the newly pending ones into the low word */
> > +             cur_pending |=3D new_pending;
> > +     }
> > +     set_softirq_pending(cur_pending);
> >       wakeup_softirqd();
> >  out:
> >       lockdep_softirq_end(in_hardirq);
>