Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F1280217AB
Date:   Wed, 24 Jan 2018 03:12:01 +0100
From:   Frederic Weisbecker <frederic@kernel.org>
To:     Dmitry Safonov <dima@arista.com>
Cc:     Paolo Abeni <pabeni@redhat.com>,
        LKML <linux-kernel@vger.kernel.org>,
        Levin Alexander <alexander.levin@verizon.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Mauro Carvalho Chehab <mchehab@s-opensource.com>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        Hannes Frederic Sowa <hannes@stressinduktion.org>,
        "Paul E . McKenney" <paulmck@linux.vnet.ibm.com>,
        Wanpeng Li <wanpeng.li@hotmail.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Andrew Morton <akpm@linux-foundation.org>,
        Radu Rendec <rrendec@arista.com>,
        Ingo Molnar <mingo@kernel.org>,
        Stanislaw Gruszka <sgruszka@redhat.com>,
        Rik van Riel <riel@redhat.com>,
        Eric Dumazet <edumazet@google.com>,
        David Miller <davem@davemloft.net>
Subject: Re: [RFC PATCH 0/4] softirq: Per vector threading v3
Message-ID: <20180124021159.GB11302@lerouge>
References: <1516376774-24076-1-git-send-email-frederic@kernel.org>
 <1516702432.2554.37.camel@redhat.com>
 <1516710733.2762.19.camel@arista.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <1516710733.2762.19.camel@arista.com>
User-Agent: Mutt/1.5.24 (2015-08-30)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Tue, Jan 23, 2018 at 12:32:13PM +0000, Dmitry Safonov wrote:
> On Tue, 2018-01-23 at 11:13 +0100, Paolo Abeni wrote:
> > Hi,
> > 
> > On Fri, 2018-01-19 at 16:46 +0100, Frederic Weisbecker wrote:
> > > As per Linus suggestion, this take doesn't limit the number of
> > > occurences
> > > per jiffy anymore but instead defers a vector to workqueues as soon
> > > as
> > > it gets re-enqueued on IRQ tail.
> > > 
> > > No tunable here, so testing should be easier.
> > > 
> > > git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-
> > > dynticks.git
> > > 	softirq/thread-v3
> > > 
> > > HEAD: 6835e92cbd70ef4a056987d2e1ed383b294429d4
> > 
> > I tested this series in the UDP flood scenario, binding the user
> > space
> > process receiving the packets on the same CPU processing the related
> > IRQ, and the tput sinks nearly to 0, like before Eric's patch.
> > 
> > The perf tool says that almost all the softirq processing is done
> > inside the workqueue, but the user space process is scheduled very
> > rarely, while before this series, in this scenario, ksoftirqd and the
> > user space process got a fair share of the CPU time.
> 
> It get a fair share of the CPU time only on some lucky platforms (maybe
> on the most). On other not-so-lucky platforms it was also unfair - as
> I've previously described by the reason of slow raising softirqs :-/

Indeed your case looks pretty special, as IRQs don't get the opportunity to
interrupt softirqs but they fire right after.

I'm wondering if you shouldn't try threaded IRQs. Having softirqs always
running in a thread instead of hardirq tail might help (through threadirqs=
kernel param).