Received-SPF: pass (google.com: domain of linux-kernel+bounces-58632-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Date: Thu, 8 Feb 2024 11:41:26 -0800
From: Joe Damato <jdamato@fastly.com>
To: Tariq Toukan <ttoukan.linux@gmail.com>
Cc: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, tariqt@nvidia.com,
	rrameshbabu@nvidia.com, Saeed Mahameed <saeedm@nvidia.com>,
	Leon Romanovsky <leon@kernel.org>,
	"David S. Miller" <davem@davemloft.net>,
	Eric Dumazet <edumazet@google.com>,
	Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>,
	Richard Cochran <richardcochran@gmail.com>,
	Gal Pressman <gal@nvidia.com>,
	Vadim Fedorenko <vadim.fedorenko@linux.dev>,
	"open list:MELLANOX MLX5 core VPI driver" <linux-rdma@vger.kernel.org>
Subject: Re: [PATCH net-next v3] net/mlx5e: link NAPI instances to queues and
 IRQs
Message-ID: <20240208194125.GA3166@fastly.com>
References: <20240208030702.27296-1-jdamato@fastly.com>
 <8c083e6d-5fcd-4557-88dd-0f95acdbc747@gmail.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <8c083e6d-5fcd-4557-88dd-0f95acdbc747@gmail.com>
User-Agent: Mutt/1.5.24 (2015-08-30)

On Thu, Feb 08, 2024 at 09:13:25PM +0200, Tariq Toukan wrote:
> 
> 
> On 08/02/2024 5:07, Joe Damato wrote:
> >Make mlx5 compatible with the newly added netlink queue GET APIs.
> >
> >Signed-off-by: Joe Damato <jdamato@fastly.com>
> >---
> >v2 -> v3:
> >   - Fix commit message subject
> >   - call netif_queue_set_napi in mlx5e_ptp_activate_channel and
> >     mlx5e_ptp_deactivate_channel to enable/disable NETDEV_QUEUE_TYPE_RX for
> >     the PTP channel.
> >   - Modify mlx5e_activate_txqsq and mlx5e_deactivate_txqsq to set
> >     NETDEV_QUEUE_TYPE_TX which should take care of all TX queues including
> >     QoS/HTB and PTP.
> >   - Rearrange mlx5e_activate_channel and mlx5e_deactivate_channel for
> >     better ordering when setting and unsetting NETDEV_QUEUE_TYPE_RX NAPI
> >     structs
> >
> >v1 -> v2:
> >   - Move netlink NULL code to mlx5e_deactivate_channel
> >   - Move netif_napi_set_irq to mlx5e_open_channel and avoid storing the
> >     irq, after netif_napi_add which itself sets the IRQ to -1
> >
> >  drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c  | 3 +++
> >  drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 7 +++++++
> >  2 files changed, 10 insertions(+)
> >
> >diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c b/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c
> >index 078f56a3cbb2..fbbc287d924d 100644
> >--- a/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c
> >+++ b/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c
> >@@ -927,6 +927,8 @@ void mlx5e_ptp_activate_channel(struct mlx5e_ptp *c)
> >  	int tc;
> >  	napi_enable(&c->napi);
> >+	netif_queue_set_napi(c->netdev, c->rq.ix, NETDEV_QUEUE_TYPE_RX,
> >+			     &c->napi);
> >  	if (test_bit(MLX5E_PTP_STATE_TX, c->state)) {
> >  		for (tc = 0; tc < c->num_tc; tc++)
> >@@ -951,6 +953,7 @@ void mlx5e_ptp_deactivate_channel(struct mlx5e_ptp *c)
> >  			mlx5e_deactivate_txqsq(&c->ptpsq[tc].txqsq);
> >  	}
> >+	netif_queue_set_napi(c->netdev, c->rq.ix, NETDEV_QUEUE_TYPE_RX, NULL);
> >  	napi_disable(&c->napi);
> >  }
> >diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
> >index c8e8f512803e..2f1792854dd5 100644
> >--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
> >+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
> >@@ -1806,6 +1806,7 @@ void mlx5e_activate_txqsq(struct mlx5e_txqsq *sq)
> >  	set_bit(MLX5E_SQ_STATE_ENABLED, &sq->state);
> >  	netdev_tx_reset_queue(sq->txq);
> >  	netif_tx_start_queue(sq->txq);
> >+	netif_queue_set_napi(sq->channel->netdev, sq->txq_ix, NETDEV_QUEUE_TYPE_TX, &sq->channel->napi);
> 
> Might be called with channel==NULL.
> For example for PTP.
> 
> Prefer sq->netdev and sq->cq.napi.

OK, I'll make that change for the v4 which I'll send sometime tomorrow.

Thanks.

> >  }
> >  void mlx5e_tx_disable_queue(struct netdev_queue *txq)
> >@@ -1819,6 +1820,7 @@ void mlx5e_deactivate_txqsq(struct mlx5e_txqsq *sq)
> >  {
> >  	struct mlx5_wq_cyc *wq = &sq->wq;
> >+	netif_queue_set_napi(sq->channel->netdev, sq->txq_ix, NETDEV_QUEUE_TYPE_TX, NULL);
> 
> Same here.
> 
> >  	clear_bit(MLX5E_SQ_STATE_ENABLED, &sq->state);
> >  	synchronize_net(); /* Sync with NAPI to prevent netif_tx_wake_queue. */
> >@@ -2560,6 +2562,7 @@ static int mlx5e_open_channel(struct mlx5e_priv *priv, int ix,
> >  	c->lag_port = mlx5e_enumerate_lag_port(priv->mdev, ix);
> >  	netif_napi_add(netdev, &c->napi, mlx5e_napi_poll);
> >+	netif_napi_set_irq(&c->napi, irq);
> >  	err = mlx5e_open_queues(c, params, cparam);
> >  	if (unlikely(err))
> >@@ -2602,12 +2605,16 @@ static void mlx5e_activate_channel(struct mlx5e_channel *c)
> >  		mlx5e_activate_xsk(c);
> >  	else
> >  		mlx5e_activate_rq(&c->rq);
> >+
> >+	netif_queue_set_napi(c->netdev, c->ix, NETDEV_QUEUE_TYPE_RX, &c->napi);
> >  }
> >  static void mlx5e_deactivate_channel(struct mlx5e_channel *c)
> >  {
> >  	int tc;
> >+	netif_queue_set_napi(c->netdev, c->ix, NETDEV_QUEUE_TYPE_RX, NULL);
> >+
> >  	if (test_bit(MLX5E_CHANNEL_STATE_XSK, c->state))
> >  		mlx5e_deactivate_xsk(c);
> >  	else