MIME-Version: 1.0
In-Reply-To: <1515734588.10188.36.camel@gmx.de>
References: <20180109133623.10711-1-dima@arista.com> <20180109133623.10711-2-dima@arista.com>
 <CANn89iK3M97MN0Pf3nXb+UAqqhUWOdSthHRBTYCwP75Ax_hO8Q@mail.gmail.com>
 <1515620880.3350.44.camel@arista.com> <CA+55aFyKKt4_5RT9RT8ZH-W26hC8=AvRYf8YxBm98dGSWwFs8g@mail.gmail.com>
 <20180111032232.GA11633@lerouge> <CA+55aFx_3zwQJ0YbDCL4YxpWEWhcEZfJnn42LzWBWDi3h1VdGA@mail.gmail.com>
 <20180111044456.GC11633@lerouge> <1515681091.3039.21.camel@arista.com>
 <CANn89i+mVmzrZ14Kttt=J0wsDOMHhm8CHiMRLQwEZXMxiVpftg@mail.gmail.com>
 <20180111163204.GE6176@hirez.programming.kicks-ass.net> <CA+55aFwc3CP-sKOyVvaLab3azmr3LnPfADnGJXDcxYz9dT75=A@mail.gmail.com>
 <CANn89i+ZTLtA5ZLRAbCgM_Cx-2xiwRbDXM4x=-QiM78r5ptcqg@mail.gmail.com>
 <CA+55aFyZPzkjwkLXWWXp3KUfLD7MUtGxSu1Q6vc0O5i9Ea6ZKw@mail.gmail.com>
 <CANn89iJzekwx_Hs0t0O==+gwAfqMyVHBg=gemayZZJXb4bYJdQ@mail.gmail.com>
 <CA+55aFx+1tFpnLBXjZKoYsMMVPakeP8nycyfMpF7agUXz_kGkQ@mail.gmail.com>
 <CANn89i+ehJg_7YfOCicgv_EuQWR6Xn7GHi+g5=atigeXDeNMHw@mail.gmail.com>
 <CA+55aFwA1skftujPWmuQJq_s-EG=PP+mFiuUiZNBar=deYNu3Q@mail.gmail.com> <1515734588.10188.36.camel@gmx.de>
From: Linus Torvalds <torvalds@linux-foundation.org>
Date: Fri, 12 Jan 2018 09:24:14 -0800
Message-ID: <CA+55aFxO4dBQzZJZ+wm5CPD_CE=1HpB1x0uz6kCs6vdu8d6b6Q@mail.gmail.com>
Subject: Re: [RFC 1/2] softirq: Defer net rx/tx processing to ksoftirqd context
To: Mike Galbraith <efault@gmx.de>
Cc: Eric Dumazet <edumazet@google.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Dmitry Safonov <dima@arista.com>,
        Frederic Weisbecker <frederic@kernel.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Dmitry Safonov <0x7f454c46@gmail.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        David Miller <davem@davemloft.net>,
        Frederic Weisbecker <fweisbec@gmail.com>,
        Hannes Frederic Sowa <hannes@stressinduktion.org>,
        Ingo Molnar <mingo@kernel.org>,
        "Levin, Alexander (Sasha Levin)" <alexander.levin@verizon.com>,
        Paolo Abeni <pabeni@redhat.com>,
        "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
        Radu Rendec <rrendec@arista.com>,
        Rik van Riel <riel@redhat.com>,
        Stanislaw Gruszka <sgruszka@redhat.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Wanpeng Li <wanpeng.li@hotmail.com>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org

On Thu, Jan 11, 2018 at 9:23 PM, Mike Galbraith <efault@gmx.de> wrote:
> On Thu, 2018-01-11 at 12:22 -0800, Linus Torvalds wrote:
>>
>> So maybe we could get rid of the per-cpu ksoftirqd entirely, and
>> replace it with with per-cpu and per-softirq workqueues?
>
> How would that be better than what RT used to do, and I still do for my
> RT kernels via boot option, namely split ksoftirqd into per-softirq
> threads.

So I think testing and playing around with things is probably the way
to go, but the  reason I would suggest at least looking at workqueues
is that I feel that we already have a *ton* of percpu threads, and
multiplying the ksoftirqd threads by a factor of ten (or however many
softirqs we have) sounds insane.

And yes, they do need to be percpu. Networking wants to do all that
packet handling concurrently across CPUs, so we can't just replace the
per-cpu ksoftirqd threads with per-softirq threads: we really do want
both per-cpu _and_ per-softirq.

And honestly, that sounds insane to do with dedicated threads, most of
which do basically nothing at all.. And it's what the workqueue
threads are designed for - expanding as needed, and not tying up a
dedicated thread for every little work.

But code and testing talks,

            Linus