Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 18 Jul 2022 23:24:57 +0200
From:   Peter Zijlstra <peterz@infradead.org>
To:     Tariq Toukan <ttoukan.linux@gmail.com>
Cc:     Tariq Toukan <tariqt@nvidia.com>,
        "David S. Miller" <davem@davemloft.net>,
        Saeed Mahameed <saeedm@nvidia.com>,
        Jakub Kicinski <kuba@kernel.org>,
        Ingo Molnar <mingo@redhat.com>,
        Juri Lelli <juri.lelli@redhat.com>,
        Eric Dumazet <edumazet@google.com>,
        Paolo Abeni <pabeni@redhat.com>, netdev@vger.kernel.org,
        Gal Pressman <gal@nvidia.com>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        linux-kernel@vger.kernel.org
Subject: Re: [PATCH net-next V2 2/2] net/mlx5e: Improve remote NUMA
 preferences used for the IRQ affinity hints
Message-ID: <YtXPqTM2fH+MUKH7@worktop.programming.kicks-ass.net>
References: <20220718124315.16648-1-tariqt@nvidia.com>
 <20220718124315.16648-3-tariqt@nvidia.com>
 <YtVlDiLTPxm312u+@worktop.programming.kicks-ass.net>
 <2fc99d26-f804-ad34-1fd7-90cfb123b426@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <2fc99d26-f804-ad34-1fd7-90cfb123b426@gmail.com>
Precedence: bulk

On Mon, Jul 18, 2022 at 10:49:21PM +0300, Tariq Toukan wrote:

> > > +	first = cpumask_local_spread(0, dev->priv.numa_node);
> > 
> > Arguably you want something like:
> > 
> > 	first = cpumask_any(cpumask_of_node(dev->priv.numa_node));
> 
> Any doesn't sound like what I'm looking for, I'm looking for first.
> I do care about the order within the node, so it's more like
> cpumask_first(cpumask_of_node(dev->priv.numa_node));
> 
> Do you think this has any advantage over cpumask_local_spread, if used only
> during the setup phase of the driver?

Only for the poor sod trying to read this code ;-) That is, I had no
idea what cpumask_local_spread() does, while cpumask_first() is fairly
obvious.

> > > @@ -830,8 +887,7 @@ static int comp_irqs_request(struct mlx5_core_dev *dev)
> > >   		ret = -ENOMEM;
> > >   		goto free_irqs;
> > >   	}
> > > -	for (i = 0; i < ncomp_eqs; i++)
> > > -		cpus[i] = cpumask_local_spread(i, dev->priv.numa_node);
> > > +	mlx5_set_eqs_cpus(dev, cpus, ncomp_eqs);
> > 
> > So you change this for mlx5, what about the other users of
> > cpumask_local_spread() ?
> 
> I took a look at the different netdev users.
> While some users have similar use case to ours (affinity hints), many others
> use cpumask_local_spread in other flows (XPS setting, ring allocations,
> etc..).
> 
> Moving them to use the newly exposed API needs some deeper dive into their
> code, especially due to the possible undesired side-effects.
> 
> I prefer not to include these changes in my series for now, but probably
> contribute it in a followup work.

Fair enough.