Received-SPF: pass (google.com: domain of linux-kernel+bounces-214566-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Precedence: bulk
MIME-Version: 1.0
References: <20240612170303.3896084-1-bigeasy@linutronix.de>
 <20240612170303.3896084-9-bigeasy@linutronix.de> <20240612131829.2e33ca71@rorschach.local.home>
 <20240614082758.6pSMV3aq@linutronix.de>
In-Reply-To: <20240614082758.6pSMV3aq@linutronix.de>
From: Eric Dumazet <edumazet@google.com>
Date: Fri, 14 Jun 2024 10:38:15 +0200
Message-ID: <CANn89i+YfdmKSMgHni4ogMDq0BpFQtjubA0RxXcfZ8fpgV5_fw@mail.gmail.com>
Subject: Re: [PATCH v6 net-next 08/15] net: softnet_data: Make xmit.recursion
 per task.
To: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Steven Rostedt <rostedt@goodmis.org>, linux-kernel@vger.kernel.org, 
	netdev@vger.kernel.org, "David S. Miller" <davem@davemloft.net>, 
	Daniel Bristot de Oliveira <bristot@kernel.org>, Boqun Feng <boqun.feng@gmail.com>, 
	Daniel Borkmann <daniel@iogearbox.net>, Frederic Weisbecker <frederic@kernel.org>, 
	Ingo Molnar <mingo@redhat.com>, Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>, 
	Peter Zijlstra <peterz@infradead.org>, Thomas Gleixner <tglx@linutronix.de>, 
	Waiman Long <longman@redhat.com>, Will Deacon <will@kernel.org>, Ben Segall <bsegall@google.com>, 
	Daniel Bristot de Oliveira <bristot@redhat.com>, Dietmar Eggemann <dietmar.eggemann@arm.com>, 
	Juri Lelli <juri.lelli@redhat.com>, Mel Gorman <mgorman@suse.de>, 
	Valentin Schneider <vschneid@redhat.com>, Vincent Guittot <vincent.guittot@linaro.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Fri, Jun 14, 2024 at 10:28=E2=80=AFAM Sebastian Andrzej Siewior
<bigeasy@linutronix.de> wrote:
>
> On 2024-06-12 13:18:29 [-0400], Steven Rostedt wrote:
> > On Wed, 12 Jun 2024 18:44:34 +0200
> > Sebastian Andrzej Siewior <bigeasy@linutronix.de> wrote:
> >
> > > Softirq is preemptible on PREEMPT_RT. Without a per-CPU lock in
> > > local_bh_disable() there is no guarantee that only one device is
> > > transmitting at a time.
> > > With preemption and multiple senders it is possible that the per-CPU
> > > recursion counter gets incremented by different threads and exceeds
> > > XMIT_RECURSION_LIMIT leading to a false positive recursion alert.
> > >
> > > Instead of adding a lock to protect the per-CPU variable it is simple=
r
> > > to make the counter per-task. Sending and receiving skbs happens alwa=
ys
> > > in thread context anyway.
> > >
> > > Having a lock to protected the per-CPU counter would block/ serialize=
 two
> > > sending threads needlessly. It would also require a recursive lock to
> > > ensure that the owner can increment the counter further.
> > >
> > > Make the recursion counter a task_struct member on PREEMPT_RT.
> >
> > I'm curious to what would be the harm to using a per_task counter
> > instead of per_cpu outside of PREEMPT_RT. That way, we wouldn't have to
> > have the #ifdef.
>
> There should be a hole on !RT, too so we shouldn't gain weight. The
> limit is set to 8 so an u8 would be enough. The counter is only accessed
> with BH-disabled so it will be used only in one context since it can't
> schedule().
>
> I think it should work fine. netdev folks, you want me to remove that
> ifdef and use a per-Task counter unconditionally?

It depends if this adds another cache line miss/dirtying or not.

What about other fields from softnet_data.xmit ?