Received-SPF: pass (google.com: domain of linux-kernel+bounces-67485-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Date: Thu, 15 Feb 2024 19:18:50 +0100
From: Dragos Tatulea <dtatulea@nvidia.com>
To: Jonah Palmer <jonah.palmer@oracle.com>
Cc: mst@redhat.com, jasowang@redhat.com, xuanzhuo@linux.alibaba.com, 
	eperezma@redhat.com, si-wei.liu@oracle.com, virtualization@lists.linux-foundation.org, 
	boris.ostrovsky@oracle.com, leiyang@redhat.com, linux-kernel@vger.kernel.org
Subject: Re: [PATCH RESEND] vdpa/mlx5: Allow CVQ size changes
Message-ID: <psyjxz3jk6qqj3xiqlrl6dkmmrhuxu6te2pveko72s7zkdn46v@w6uthanhjhnr>
References: <20240215174647.3885093-1-jonah.palmer@oracle.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240215174647.3885093-1-jonah.palmer@oracle.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?lLAcdbuLPzmHy/HSGbKQP1FnVftiVSwKBTji1DiLcpSz0pVft1yqs6UgwsgJ?=
 =?us-ascii?Q?hcV/dmA+szgW3PcUh9loWzHhIrJk/GZfhPNpfnBmUwyBaKeFW3kwvbofojFA?=
 =?us-ascii?Q?Y2xqWPDvl/Um4c0UHCVHMsMtuz7jDLjKkPVFd+h+LiNW8Hk902FkPlBdz2Dm?=
 =?us-ascii?Q?s5sr4NPep4bEpKA+lTZ2Mi/Nl9BSoGn6gFM6RzsJs7rDt7pVGSYgdBLXXndW?=
 =?us-ascii?Q?SvRyBM9ofBiPjXvCYaCIRYaRzmOLm3dJ+pDAYCegXm6qJpRjbWs3aWz+bpcK?=
 =?us-ascii?Q?xU3qKCSOfCHSPUj0Rd04UsMfTV0+zF8Gf8t0cr6F+cJTswT09lrayEcUe1dR?=
 =?us-ascii?Q?2vzBN3qjR0/eNBP2o8qy2MOAkwaVIXbHb0H1riQ82yKNiv/1tG5grVZCffJV?=
 =?us-ascii?Q?bSgg7noVMHYdTzDBhPccMc3yMogri4tAbzXQenFfsHUYFqp990tlru+vX10W?=
 =?us-ascii?Q?6apDzbEEuAGmNrQRRnowJjJWyLsNlm3TD6t9ecd4AwsWKnKjo2VaT3vn19fk?=
 =?us-ascii?Q?SczHlRLPQb6mFwOwAtaIe6/FWnPB5QBi36rQhqr00lnRLVVQ4Oj6EV+COCJn?=
 =?us-ascii?Q?W4yi2S5eMpBGa+qzv+jccL1ZmFjRZnN7DBxKKjZ8Bc10GZW28VmUPl/+3hRC?=
 =?us-ascii?Q?Fp0ebI3TdQiag3pwmJUiI8zFU2+isoGVmCkS9tB2UhpBjr5XfU8AflaKDRVv?=
 =?us-ascii?Q?iDO8oQhRljbkv93qvGOheLQ2F6yqdNbrP0Fl6xK1EfEIFRCxypaA91jL+znq?=
 =?us-ascii?Q?nnayY2laMYxbg5gEOwA4U9oHGslo1xzsDoW1SYU36CUsnHVAjdGY4fva/+zA?=
 =?us-ascii?Q?Pbwz2nmWQzL+xEESZq77Yw5aPVOUIXogcmv8RyNLuYWuW8UBLXHa5SsUNy/Q?=
 =?us-ascii?Q?LYRVLG8/Ymc0O6dy9bvzIIjUnyMU4kPEi9yJALKIY2/PmMTJijox8JiGC6nH?=
 =?us-ascii?Q?PjAGqG2eP6co0E0ElJuuhmZkSKtbUP4SeAYxvU0cHZtwmdyD4UqkYsAdTYaA?=
 =?us-ascii?Q?lMTuKAi6s65zQeO3Xr3/q7NtNInzjuDNCpyk0RpCy5J1/mLrHwkj2ZeqfiKH?=
 =?us-ascii?Q?XeP71GDnahK/1Pyu3ke39GWG81hfu2cLYVbOw9O41fL75wmqdk6zBNxgCYtt?=
 =?us-ascii?Q?2/4i1TudL+5KPQxTVFMr42U7JYY4GznqBczZSb3kDy8de1RvwAyh5dRhjhuV?=
 =?us-ascii?Q?0gZZYcmIshjazL5UOhaoB8ZdOz3O7IdSrBRDd5r9T8jAzeavqOtSaFdn28Kr?=
 =?us-ascii?Q?BbhIt9wo66vb3Jjt9FMAcLUpyurDZKMSD47FzqCFXMOh2E5K0QEt+I3e69ts?=
 =?us-ascii?Q?A3wmQKpG0FRl0X+Kvpxc6oYnhyqgq9QoCTuVqlMV6iRqvptv7KbsM5OtiC1C?=
 =?us-ascii?Q?jLOnpK08SVj4UBMBTiRDKYBInvOAx3LX6U1KvcpHOyGBelZ9QcVZkp3ZHouk?=
 =?us-ascii?Q?Sh47v9bwlNDZOyvUQsMclHMvLY2OR4Q2/EUnVZFcn1I8tysBvxSsUADx+BlD?=
 =?us-ascii?Q?/AQzR8/iH6UsQaFNkjh+y2gXH/qFUqLgPBgggcy/2A+XapHw80ki23ZcWJZ1?=
 =?us-ascii?Q?YbfX1ho8lCeZc1ErpsDeRtslDnAlWo/T4JJ/PFh6+cTRc2MNpJX/6audwTHg?=
 =?us-ascii?Q?2lBmCgwvTpC2CtSdbQHxPt8mjQ/l1Qpbbd+0puV7x0/j?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0bc26c83-a4c6-49f2-600c-08dc2e529148
X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB5565.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Feb 2024 18:18:54.0789
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: eG0BAQ5cteA0bwP1Y/rUGgqesra9c8gK9EH98T01nYZ8kdAIbrxW3Ue4AFq82LPumOmGhR32fZKGp1NGLn3t2g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB7703

On 02/15, Jonah Palmer wrote:
> The MLX driver was not updating its control virtqueue size at set_vq_num
> and instead always initialized to MLX5_CVQ_MAX_ENT (16) at
> setup_cvq_vring.
> 
> Qemu would try to set the size to 64 by default, however, because the
> CVQ size always was initialized to 16, an error would be thrown when
> sending >16 control messages (as used-ring entry 17 is initialized to 0).
> For example, starting a guest with x-svq=on and then executing the
> following command would produce the error below:
> 
>  # for i in {1..20}; do ifconfig eth0 hw ether XX:xx:XX:xx:XX:XX; done
> 
>  qemu-system-x86_64: Insufficient written data (0)
>  [  435.331223] virtio_net virtio0: Failed to set mac address by vq command.
>  SIOCSIFHWADDR: Invalid argument
> 
> Signed-off-by: Jonah Palmer <jonah.palmer@oracle.com>

Thanks for the patch Jonah!
> ---
>  drivers/vdpa/mlx5/net/mlx5_vnet.c | 10 ++++++++--
>  1 file changed, 8 insertions(+), 2 deletions(-)
> 
> diff --git a/drivers/vdpa/mlx5/net/mlx5_vnet.c b/drivers/vdpa/mlx5/net/mlx5_vnet.c
> index 778821bab7d9..c74de1fe6a94 100644
> --- a/drivers/vdpa/mlx5/net/mlx5_vnet.c
> +++ b/drivers/vdpa/mlx5/net/mlx5_vnet.c
> @@ -2276,9 +2276,15 @@ static void mlx5_vdpa_set_vq_num(struct vdpa_device *vdev, u16 idx, u32 num)
>  	struct mlx5_vdpa_net *ndev = to_mlx5_vdpa_ndev(mvdev);
>  	struct mlx5_vdpa_virtqueue *mvq;
>  
> -	if (!is_index_valid(mvdev, idx) || is_ctrl_vq_idx(mvdev, idx))
> +	if (!is_index_valid(mvdev, idx))
>  		return;
>  
> +        if (is_ctrl_vq_idx(mvdev, idx)) {
> +                struct mlx5_control_vq *cvq = &mvdev->cvq;
Nit: add a space between variable declarations and usage.

> +                cvq->vring.vring.num = num;
> +                return;
> +        }
> +
>  	mvq = &ndev->vqs[idx];
>  	mvq->num_ent = num;
>  }
> @@ -2963,7 +2969,7 @@ static int setup_cvq_vring(struct mlx5_vdpa_dev *mvdev)
>  		u16 idx = cvq->vring.last_avail_idx;
>  
>  		err = vringh_init_iotlb(&cvq->vring, mvdev->actual_features,
> -					MLX5_CVQ_MAX_ENT, false,
> +					cvq->vring.vring.num, false,
>  					(struct vring_desc *)(uintptr_t)cvq->desc_addr,
>  					(struct vring_avail *)(uintptr_t)cvq->driver_addr,
>  					(struct vring_used *)(uintptr_t)cvq->device_addr);

Besides the nit:

Acked-by: Dragos Tatulea <dtatulea@nvidia.com>

> -- 
> 2.39.3
>