Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
In-Reply-To: <20180117180713.GA17735@lerouge>
References: <1516077640-19718-1-git-send-email-frederic@kernel.org>
 <20180117145620.213cb5ad@vento.lan> <20180117180713.GA17735@lerouge>
From:   Linus Torvalds <torvalds@linux-foundation.org>
Date:   Wed, 17 Jan 2018 15:56:43 -0800
Message-ID: <CA+55aFzojENpf+-fEhkNL_RaQbtC0FoB6x7DDXqZCScCOAUmQg@mail.gmail.com>
Subject: Re: [RFC PATCH 0/5] softirq: Per vector threading v2
To:     Frederic Weisbecker <frederic@kernel.org>
Cc:     Mauro Carvalho Chehab <mchehab@s-opensource.com>,
        LKML <linux-kernel@vger.kernel.org>,
        Levin Alexander <alexander.levin@verizon.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Hannes Frederic Sowa <hannes@stressinduktion.org>,
        "Paul E . McKenney" <paulmck@linux.vnet.ibm.com>,
        Wanpeng Li <wanpeng.li@hotmail.com>,
        Dmitry Safonov <dima@arista.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Eric Dumazet <edumazet@google.com>,
        Radu Rendec <rrendec@arista.com>,
        Ingo Molnar <mingo@kernel.org>,
        Stanislaw Gruszka <sgruszka@redhat.com>,
        Paolo Abeni <pabeni@redhat.com>,
        Rik van Riel <riel@redhat.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        David Miller <davem@davemloft.net>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Wed, Jan 17, 2018 at 10:07 AM, Frederic Weisbecker
<frederic@kernel.org> wrote:
>
> I see, so you may want to test (possibly much) higher values of MAX_SOFTIRQ_RESTART,
> such as 50 or 100.

I suspect the "number of softiqs per jiffy" is hardly interesting at all.

We used to allow up to 2mS or ten iterations per _invocation_, never
mind per timer tick.

I thought you were going to actally account for time, but I don't
think you ended up doing that.

Maybe time isn't necessarily the thing to do, but just pure "count per
jiffy" seems very bad.

What I might suggest using instead:

 - do it by time. This may be too expensive, though. Keeping track of
ns-level timing per invocation can be nasty.

 - do it by "we got a new softirq event while handling another softirq
event". That was our old count per invocation, except you could do it
per softirq, and just allow *one* (ie keep a bitmask of "I've already
handled this softirq", and if the restart results in it being
triggered *again* you say "ok, I'll just move this to a workqueue"

 - .. something else?

I'd suggest trying the "if we get a new softirq event that we've
already seen while we were already handling softirq events" thing.
That should really take care of the networking case of "90% time spend
in softirq handling during packet storms" thing. If we spend that much
time on softirqs, we *will* get a new softirq while handling an old
one occasionally.

            Linus