Received-SPF: pass (google.com: domain of linux-kernel+bounces-99603-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Message-ID: <7d7f80fe-ceea-4bff-a01a-f1b12f31fcf2@cornelisnetworks.com>
Date: Mon, 11 Mar 2024 18:22:10 -0400
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH net-next] IB/hfi1: allocate dummy net_device dynamically
Content-Language: en-US
To: Breno Leitao <leitao@debian.org>
Cc: Jason Gunthorpe <jgg@ziepe.ca>, Leon Romanovsky <leon@kernel.org>,
 kuba@kernel.org, keescook@chromium.org,
 "open list:HFI1 DRIVER" <linux-rdma@vger.kernel.org>,
 open list <linux-kernel@vger.kernel.org>
References: <20240308182951.2137779-1-leitao@debian.org>
 <6460dd4b-9b65-49df-beaf-05412e42f706@cornelisnetworks.com>
 <Ze8j8yWyXCQtwcOJ@gmail.com>
From: Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com>
In-Reply-To: <Ze8j8yWyXCQtwcOJ@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?SDcxWlhxK2l2VktHelFKczdIbUd2dHU3WlZGYW5ET1pkdERYVk1MQXlUKzJD?=
 =?utf-8?B?dldnMk5QalNDMDhxQ3FUUWQ5S3ordVBTZ2Qxd3NKZ2ViZDNWQ2wxeWNISnZP?=
 =?utf-8?B?TXUySmNTMmFvOVhhVnBFTXNueit4MHo5c3RJdnhIS1I4L3RSZEorL2IrdlU4?=
 =?utf-8?B?QllnVUVnSFZ2c3UweGtwUFZ5ZnpNNTM0aTN2d3RodnFaVXBrbzMraGZLRDVX?=
 =?utf-8?B?S3RnTGF2OHgzakhVaXovb0kxKzY5WEVxclhwWXhoUVFzSTZLTVpVNDZnR1cr?=
 =?utf-8?B?K3FCTElJa0VuSzRuTGUvMU1jK2EzaHFicXhVeDZEYUZDZytRSzNWZ2t1YUgw?=
 =?utf-8?B?Q3piSWZpUzNxdTl5cW5kM3BnbjBNaWpJdFp5aml2ZlR3SHVqb0U3cG1iMng4?=
 =?utf-8?B?SStoZVVmaFhuTHB6eHk3OFU2T3VNOWMxb3ZoM0xRVVlhOFJ3clFwdkVCMTIx?=
 =?utf-8?B?bFhCckFoMFhGbDNodEdQak1BTG9Wd09CWjBxRFc4TUZhdkxFeHo1WGZ3Rndw?=
 =?utf-8?B?cVdyc1dvYTVNWHplSU1zaitnazA3NFBEU1ZmOExrQkpwRjJUMjlCRDFta3JD?=
 =?utf-8?B?SlZkRlRSWGVWVkFRWkpWVi9MSmprRjZLdC9VM25COU5mbmNIWWxhQnFyL2x2?=
 =?utf-8?B?ekhuNzFYd1ZoMm5CTTEzMlBxK1JpekxnRGlLL2hQS2VVcGZWT3BNWVFySjM2?=
 =?utf-8?B?c2h1YTdqcDZuN3ZrYUw4WFd5SWp1ZS9mdTNBaWk0S3NRbW5lSkEvNnNUWDQv?=
 =?utf-8?B?VmU2S3NxazJ3RTN2WFd6SmVEQmRESFgyS29VU3dpeXMrUFI0V1pFM1Qrenha?=
 =?utf-8?B?TGF6bnVpQm5WTTNIVzUvMWxTM1R3MEdMcTEwOGhhVVJqWlZScHp2ZHBhbllx?=
 =?utf-8?B?cjBZVkQrQkpQOWpMZFE2YThaZmVYOTVVMVorc3RPU2xYTm5xeDdlWlp5bi9w?=
 =?utf-8?B?VWFNYk9sQUlKWTFLS09OYTdCcVRMYnhNS3VCT0M5QUFlVVRRQ2dPdkg5N09a?=
 =?utf-8?B?QVYrYkoyRjBnNWRVbEhPdlNzd1BrMmxWcjI1R3ZVR0VlcTQvdEI1QUIrK29p?=
 =?utf-8?B?SUI3Z0RIbWNzM2NYclZpUldoTjJacytUTEdVYVBWOWc3Tjl0ekhwa3RlQW4v?=
 =?utf-8?B?c293MHAycHdLMnErcHpvSkFYaE9yYnlFZk5lL3liWjJJaGU5ZGpMcmRxTEdR?=
 =?utf-8?B?R0JVajk5ZEw1bjFocnRnZWVJaVZzcDZsY3ZxL2lubUdoOElhU1d3RE80ZG0w?=
 =?utf-8?B?TnhnTmFzcFM3QXVYYzdjVDBOQitnYWFwdkdycXBKWTRzSmZoYVN4Wi9vWUpt?=
 =?utf-8?B?VVAyNkwwbE5RMndkdlc1NTJPWFp1ZU1IdlprT3d1SGFvL2IyVHpKeGo3SVlE?=
 =?utf-8?B?WTZwUGNPZjU1Zzc4QTEvbkpFK2dBS09xakl2dVJXN0RjWk1HRnV3dTljelBr?=
 =?utf-8?B?c1V5bG1FUkpVRkFWUW02YVFhUkZxcHY3c0NYYXYyTXhQcHFxcXZnVG41eXhB?=
 =?utf-8?B?dVl4TGIvb2NadE5aaXhPSHAwQUJua29oVmtPb1VIdDg5OTNZd0hLZ0lUNTdn?=
 =?utf-8?B?R1FVbmh5aC9WSU5WbzE3TlJYbUQyV1prQzByZ0dUQ0Jjam9wcDlZa1FESDgy?=
 =?utf-8?B?cFNLNjlNNlgrdVRLSlJHU3BSaVA2MzZGK0cyTnpnK2xLUjNYdmdoOTV6Nnp5?=
 =?utf-8?B?dHFKd0JLQW52TWFpL1RKRnRuek04ZjdTbXR0NEt4S0RqVFQxaDJpT3FlTGxK?=
 =?utf-8?B?YWlhVVM1ekx1bHoxYUpVMmtINmJRU0xhT0pJaTdLYXhRMElBWVVSdnZIaHVE?=
 =?utf-8?B?aVF0dkFnZEhRMk1KdFFDL1NBR3c3bWNZMTlRZEVtVnlTemRuOWppUDJwbDlJ?=
 =?utf-8?B?OS92R2FMRGYwOHljYXZFdDRReThJUGNid2dKNENiSVJxYWc3eDBZcE1WMy9L?=
 =?utf-8?B?ckFNTVBMR2xkeTFESXlVb291U2tlMSt4R3JSSjRQdWJFWEFwdFVTNkUrcWxT?=
 =?utf-8?B?OFJQYXdkVXJjbFloY0twcy85Ukh0bzhMdjJKZVd5eWMvdGY2a2xGeG5WdlJk?=
 =?utf-8?B?cTBhYlp3ZFhreVFYbDBOZ0t0NDZJb3Z3VEdMTWovTGd1bWw5bzA2UkJpZmpU?=
 =?utf-8?B?aFlyRHc4eElZTGVRQXZtbmNBcVZYaURaWlFGQXBmd0VZcXY1U1lsYzlGQW8w?=
 =?utf-8?Q?sc0tf5BCuU6UwOmwHSTyJ4Q=3D?=
X-OriginatorOrg: cornelisnetworks.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1fe8881f-60d4-4cac-3998-08dc4219b275
X-MS-Exchange-CrossTenant-AuthSource: BL0PR01MB4817.prod.exchangelabs.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Mar 2024 22:22:11.6485
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4dbdb7da-74ee-4b45-8747-ef5ce5ebe68a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: jeg6REARDPnJ9TNOJMk64NZa3JHNsRBwR6qI3H6OkPk+eX6BWxT1fkPKfV1ARbl5VAtQ1/5V6fK1RUX0Hcb4zN0hE8gfLQW/Zo2nm7lFoS2UE779eRGPZ76ToU9/LC6t
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR01MB7716

On 3/11/24 11:32 AM, Breno Leitao wrote:
> Hello Dennis,
> 
> On Mon, Mar 11, 2024 at 08:05:45AM -0400, Dennis Dalessandro wrote:
>> On 3/8/24 1:29 PM, Breno Leitao wrote:
>>> struct net_device shouldn't be embedded into any structure, instead,
>>> the owner should use the priv space to embed their state into net_device.
>>>
>>> Embedding net_device into structures prohibits the usage of flexible
>>> arrays in the net_device structure. For more details, see the discussion
>>> at [1].
>>>
>>> Un-embed the net_device from struct iwl_trans_pcie by converting it
>>> into a pointer. Then use the leverage alloc_netdev() to allocate the
>>> net_device object at iwl_trans_pcie_alloc.
>>
>> What does an Omni-Path Architecture driver from Cornelis Networks have to do
>> with an Intel wireless driver?
> 
> That is an oversight. I will fix it in v2. Sorry about it.
> 
>>> The private data of net_device becomes a pointer for the struct
>>> iwl_trans_pcie, so, it is easy to get back to the iwl_trans_pcie parent
>>> given the net_device object.
>>>
>>> [1] https://lore.kernel.org/all/20240229225910.79e224cf@kernel.org/
>>>
>>> Signed-off-by: Breno Leitao <leitao@debian.org>
>>> ---
>>>  drivers/infiniband/hw/hfi1/netdev.h    | 2 +-
>>>  drivers/infiniband/hw/hfi1/netdev_rx.c | 9 +++++++--
>>>  2 files changed, 8 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/drivers/infiniband/hw/hfi1/netdev.h b/drivers/infiniband/hw/hfi1/netdev.h
>>> index 8aa074670a9c..07c8f77c9181 100644
>>> --- a/drivers/infiniband/hw/hfi1/netdev.h
>>> +++ b/drivers/infiniband/hw/hfi1/netdev.h
>>> @@ -49,7 +49,7 @@ struct hfi1_netdev_rxq {
>>>   *		When 0 receive queues will be freed.
>>>   */
>>>  struct hfi1_netdev_rx {
>>> -	struct net_device rx_napi;
>>> +	struct net_device *rx_napi;
>>>  	struct hfi1_devdata *dd;
>>>  	struct hfi1_netdev_rxq *rxq;
>>>  	int num_rx_q;
>>> diff --git a/drivers/infiniband/hw/hfi1/netdev_rx.c b/drivers/infiniband/hw/hfi1/netdev_rx.c
>>> index 720d4c85c9c9..5c26a69fa2bb 100644
>>> --- a/drivers/infiniband/hw/hfi1/netdev_rx.c
>>> +++ b/drivers/infiniband/hw/hfi1/netdev_rx.c
>>> @@ -188,7 +188,7 @@ static int hfi1_netdev_rxq_init(struct hfi1_netdev_rx *rx)
>>>  	int i;
>>>  	int rc;
>>>  	struct hfi1_devdata *dd = rx->dd;
>>> -	struct net_device *dev = &rx->rx_napi;
>>> +	struct net_device *dev = rx->rx_napi;
>>>  
>>>  	rx->num_rx_q = dd->num_netdev_contexts;
>>>  	rx->rxq = kcalloc_node(rx->num_rx_q, sizeof(*rx->rxq),
>>> @@ -360,7 +360,11 @@ int hfi1_alloc_rx(struct hfi1_devdata *dd)
>>>  	if (!rx)
>>>  		return -ENOMEM;
>>>  	rx->dd = dd;
>>> -	init_dummy_netdev(&rx->rx_napi);
>>> +	rx->rx_napi = alloc_netdev(sizeof(struct iwl_trans_pcie *),
>>> +				   "dummy", NET_NAME_UNKNOWN,
>>> +				   init_dummy_netdev);
>>
>> Again with the iwl stuff? Please do not stuff to the mailing list that doesn't
>> even compile....
>>
>>  CC [M]  drivers/infiniband/hw/hfi1/verbs.o
>>   CC [M]  drivers/infiniband/hw/hfi1/verbs_txreq.o
>>   CC [M]  drivers/infiniband/hw/hfi1/vnic_main.o
>> In file included from ./include/net/sock.h:46,
>>                  from ./include/linux/tcp.h:19,
>>                  from ./include/linux/ipv6.h:95,
>>                  from ./include/net/ipv6.h:12,
>>                  from ./include/rdma/ib_verbs.h:25,
>>                  from ./include/rdma/ib_hdrs.h:11,
>>                  from drivers/infiniband/hw/hfi1/hfi.h:29,
>>                  from drivers/infiniband/hw/hfi1/sdma.h:15,
>>                  from drivers/infiniband/hw/hfi1/netdev_rx.c:11:
>> drivers/infiniband/hw/hfi1/netdev_rx.c: In function ‘hfi1_alloc_rx’:
>> drivers/infiniband/hw/hfi1/netdev_rx.c:365:36: error: passing argument 4 of
>> ‘alloc_netdev_mqs’ from incompatible pointer type
>> [-Werror=incompatible-pointer-types]
>>   365 |                                    init_dummy_netdev);
>>       |                                    ^~~~~~~~~~~~~~~~~
>>       |                                    |
>>       |                                    int (*)(struct net_device *)
>> ./include/linux/netdevice.h:4632:63: note: in definition of macro ‘alloc_netdev’
>>  4632 |         alloc_netdev_mqs(sizeof_priv, name, name_assign_type, setup, 1, 1)
>>       |                                                               ^~~~~
>> ./include/linux/netdevice.h:4629:44: note: expected ‘void (*)(struct net_device
>> *)’ but argument is of type ‘int (*)(struct net_device *)’
>>  4629 |                                     void (*setup)(struct net_device *),
>>       |                                     ~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~
>>   CC [M]  drivers/infiniband/hw/hfi1/vnic_sdma.o
> 
> Sorry, this patch is against net-next and you probably tested in Linus'
> upstream.
> 
> You need to have d160c66cda0ac8614 ("net: Do not return value from
> init_dummy_netdev()"), which is in net-next, and has this important
> change that is necessary for this patch:
> 
>     -int init_dummy_netdev(struct net_device *dev);
>     +void init_dummy_netdev(struct net_device *dev);
> 
> If you are OK with a v2, I will fix the topics reported in this thread.

You also use struct iwl_trans_pcie in hfi1 code. Fix that up, and make sure the
patch builds and I'll give it a test.

-Denny

> Thank you
> Breno