Message-ID: <1515782670.7007.3.camel@gmx.de>
Subject: Re: [RFC 1/2] softirq: Defer net rx/tx processing to ksoftirqd
 context
From: Mike Galbraith <efault@gmx.de>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>,
        Eric Dumazet <edumazet@google.com>,
        Dmitry Safonov <dima@arista.com>,
        Frederic Weisbecker <frederic@kernel.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Dmitry Safonov <0x7f454c46@gmail.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        David Miller <davem@davemloft.net>,
        Frederic Weisbecker <fweisbec@gmail.com>,
        Hannes Frederic Sowa <hannes@stressinduktion.org>,
        Ingo Molnar <mingo@kernel.org>,
        "Levin, Alexander (Sasha Levin)" <alexander.levin@verizon.com>,
        Paolo Abeni <pabeni@redhat.com>,
        "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
        Radu Rendec <rrendec@arista.com>,
        Rik van Riel <riel@redhat.com>,
        Stanislaw Gruszka <sgruszka@redhat.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Wanpeng Li <wanpeng.li@hotmail.com>
Date: Fri, 12 Jan 2018 19:44:30 +0100
In-Reply-To: <1515780912.21898.45.camel@gmx.de>
References: <CANn89i+mVmzrZ14Kttt=J0wsDOMHhm8CHiMRLQwEZXMxiVpftg@mail.gmail.com>
         <20180111163204.GE6176@hirez.programming.kicks-ass.net>
         <CA+55aFwc3CP-sKOyVvaLab3azmr3LnPfADnGJXDcxYz9dT75=A@mail.gmail.com>
         <CANn89i+ZTLtA5ZLRAbCgM_Cx-2xiwRbDXM4x=-QiM78r5ptcqg@mail.gmail.com>
         <CA+55aFyZPzkjwkLXWWXp3KUfLD7MUtGxSu1Q6vc0O5i9Ea6ZKw@mail.gmail.com>
         <CANn89iJzekwx_Hs0t0O==+gwAfqMyVHBg=gemayZZJXb4bYJdQ@mail.gmail.com>
         <CA+55aFx+1tFpnLBXjZKoYsMMVPakeP8nycyfMpF7agUXz_kGkQ@mail.gmail.com>
         <CANn89i+ehJg_7YfOCicgv_EuQWR6Xn7GHi+g5=atigeXDeNMHw@mail.gmail.com>
         <CA+55aFwA1skftujPWmuQJq_s-EG=PP+mFiuUiZNBar=deYNu3Q@mail.gmail.com>
         <1515734588.10188.36.camel@gmx.de>
         <20180112101336.GT3040@hirez.programming.kicks-ass.net>
         <CA+55aFygPtKM0fOGu1HNdZ9TCgk5hJ3r78fX03KYUT=0npPiEw@mail.gmail.com>
         <1515779049.21898.23.camel@gmx.de>
         <CA+55aFxRF=9xav_XEMizV2vqiX6S4SigBxFTrXwWTRrTKG1Yvw@mail.gmail.com>
         <1515780912.21898.45.camel@gmx.de>
Content-Type: text/plain; charset="ISO-8859-15"
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org

On Fri, 2018-01-12 at 19:15 +0100, Mike Galbraith wrote:
> On Fri, 2018-01-12 at 09:51 -0800, Linus Torvalds wrote:
> > On Fri, Jan 12, 2018 at 9:44 AM, Mike Galbraith <efault@gmx.de> wrote:
> > >
> > > Nah, a misunderstanding happened.  RT that still offers full threading
> > > creates per-softirq threads per cpu.  The regular trees split ksoftirqd
> > > into only two threads per cpu, one processes timer/hrtimer softriqs,
> > > the other processes the rest.
> > 
> > Ok, that sounds like it should work, but it also sounds like it's very
> > specific to RT itself.
> > 
> > For example, the dvb issue was not about the timer softirqs, but about
> > the tasklet ones.
> > 
> > So maybe we wouldn't need to split it for _every_ softirq, but we'd
> > need to split it more than just along the timer case.
> > 
> > And it does sound a bit excessive to have ten fixed threads for every
> > CPU. The days when tens of CPU's meant "huge system" are gone. These
> > days it can be a phone.
> 
> Yeah, it is excessive more often than not.  You get to prioritize, and
> segregate, which is nice, but you pay for it.

BTW, much of the softirq load in RT is processed by the raising task.

      tbench_srv-6985  [000] d...112   293.902511: softirq_raise: vec=3 [action=NET_RX]
      tbench_srv-6985  [000] .....13   293.902511: softirq_entry: vec=3 [action=NET_RX]
      tbench_srv-6985  [000] .....13   293.902515: softirq_exit: vec=3 [action=NET_RX]
          tbench-6984  [003] d...112   293.902520: softirq_raise: vec=3 [action=NET_RX]
          tbench-6984  [003] .....13   293.902520: softirq_entry: vec=3 [action=NET_RX]
          tbench-6984  [003] .....13   293.902523: softirq_exit: vec=3 [action=NET_RX]