Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Wed, 17 Jan 2018 15:30:49 -0500 (EST)
Message-Id: <20180117.153049.1803664333084879932.davem@davemloft.net>
To:     efault@gmx.de
Cc:     torvalds@linux-foundation.org, peterz@infradead.org,
        edumazet@google.com, dima@arista.com, frederic@kernel.org,
        linux-kernel@vger.kernel.org, 0x7f454c46@gmail.com,
        akpm@linux-foundation.org, fweisbec@gmail.com,
        hannes@stressinduktion.org, mingo@kernel.org,
        alexander.levin@verizon.com, pabeni@redhat.com,
        paulmck@linux.vnet.ibm.com, rrendec@arista.com, riel@redhat.com,
        sgruszka@redhat.com, tglx@linutronix.de, wanpeng.li@hotmail.com
Subject: Re: [RFC 1/2] softirq: Defer net rx/tx processing to ksoftirqd
 context
From:   David Miller <davem@davemloft.net>
In-Reply-To: <1515782670.7007.3.camel@gmx.de>
References: <CA+55aFxRF=9xav_XEMizV2vqiX6S4SigBxFTrXwWTRrTKG1Yvw@mail.gmail.com>
        <1515780912.21898.45.camel@gmx.de>
        <1515782670.7007.3.camel@gmx.de>
Mime-Version: 1.0
Content-Type: Text/Plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

From: Mike Galbraith <efault@gmx.de>
Date: Fri, 12 Jan 2018 19:44:30 +0100

> On Fri, 2018-01-12 at 19:15 +0100, Mike Galbraith wrote:
>> On Fri, 2018-01-12 at 09:51 -0800, Linus Torvalds wrote:
>> > On Fri, Jan 12, 2018 at 9:44 AM, Mike Galbraith <efault@gmx.de> wrote:
>> > >
>> > > Nah, a misunderstanding happened.  RT that still offers full threading
>> > > creates per-softirq threads per cpu.  The regular trees split ksoftirqd
>> > > into only two threads per cpu, one processes timer/hrtimer softriqs,
>> > > the other processes the rest.
>> > 
>> > Ok, that sounds like it should work, but it also sounds like it's very
>> > specific to RT itself.
>> > 
>> > For example, the dvb issue was not about the timer softirqs, but about
>> > the tasklet ones.
>> > 
>> > So maybe we wouldn't need to split it for _every_ softirq, but we'd
>> > need to split it more than just along the timer case.
>> > 
>> > And it does sound a bit excessive to have ten fixed threads for every
>> > CPU. The days when tens of CPU's meant "huge system" are gone. These
>> > days it can be a phone.
>> 
>> Yeah, it is excessive more often than not.  You get to prioritize, and
>> segregate, which is nice, but you pay for it.
> 
> BTW, much of the softirq load in RT is processed by the raising task.
> 
>       tbench_srv-6985  [000] d...112   293.902511: softirq_raise: vec=3 [action=NET_RX]
>       tbench_srv-6985  [000] .....13   293.902511: softirq_entry: vec=3 [action=NET_RX]
>       tbench_srv-6985  [000] .....13   293.902515: softirq_exit: vec=3 [action=NET_RX]
>           tbench-6984  [003] d...112   293.902520: softirq_raise: vec=3 [action=NET_RX]
>           tbench-6984  [003] .....13   293.902520: softirq_entry: vec=3 [action=NET_RX]
>           tbench-6984  [003] .....13   293.902523: softirq_exit: vec=3 [action=NET_RX]

And this is because tbench runs over loopback, which triggers softirqs in
the context of whatever generates the loopback packet transmit.  Which in
this case is the tbench process calling sendmsg().

I wanted to chime in about this earlier, and make it clear that it isn't
just IRQs that can trigger softirqs.  User context actions in the kernel
can trigger softirqs too.