Received-SPF: pass (google.com: domain of linux-kernel+bounces-196052-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Message-ID: <cfb8307f-39cc-4aaa-a394-295234dbbe95@amperemail.onmicrosoft.com>
Date: Thu, 30 May 2024 19:51:32 -0400
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 3/3] mctp pcc: Implement MCTP over PCC Transport
To: Jakub Kicinski <kuba@kernel.org>, admiyo@os.amperecomputing.com
Cc: Jeremy Kerr <jk@codeconstruct.com.au>,
 Matt Johnston <matt@codeconstruct.com.au>,
 "David S. Miller" <davem@davemloft.net>, Eric Dumazet <edumazet@google.com>,
 Paolo Abeni <pabeni@redhat.com>, netdev@vger.kernel.org,
 linux-kernel@vger.kernel.org
References: <20240513173546.679061-1-admiyo@os.amperecomputing.com>
 <20240528191823.17775-1-admiyo@os.amperecomputing.com>
 <20240528191823.17775-4-admiyo@os.amperecomputing.com>
 <20240528194557.7a8f522d@kernel.org>
Content-Language: en-US
From: Adam Young <admiyo@amperemail.onmicrosoft.com>
In-Reply-To: <20240528194557.7a8f522d@kernel.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?YW5zV2JFL3VKdHlnd2FneVcwZzJDTlRhc01qbUZhbThiOExVOURxalNQblJj?=
 =?utf-8?B?cEFNTzVuQ2VTekJ6d2pHcWE3WnBGY3djR3RrUUh4TE9vT0VoalVIaGRXSkFt?=
 =?utf-8?B?cXhOK2hpakp3eVFBdTVJRjE3bWNtcGFaQkhYUlNkRWpySTRTQkJPK2NnNlY4?=
 =?utf-8?B?OE5FU1AxRDRCamdaZ3QxL255NFQ1SWVMcXFhSjJHOVBuMVprcUZsempuOTJr?=
 =?utf-8?B?eE5aUDM2ZGdhdlltR2xqb0NlTWZ5azN2ZDdxSWFnRGRJVW9ZdGg1WVllRVhX?=
 =?utf-8?B?Um9TZS94d0ZEVGFZTThFWndGb2hYdmZLbVRTSFZXMFhOUnY0VmtNZjZ4WEgv?=
 =?utf-8?B?UXFkMmFGdGZwNERXV0tFZUhoMFhnVkl3M1lIK041Vmw4Y0ZwK1kvUFlWSFpY?=
 =?utf-8?B?Z0E3TDJ4cXFLcWJ3eThEZElKSDBmNERuMmM1RElsbGJPd1lqNk9Xd0hkdll3?=
 =?utf-8?B?Z1N1eWJQRjFkRWNtamlnV1dtY21GRklqekU2U05odVkySWdhUTBDdUZvbDdt?=
 =?utf-8?B?NGRMRGxpT3ZqUGg3dmJrRVhnMDUwRG9ONDNDVjNNcWtjeFV2M2dpdVhTRnA2?=
 =?utf-8?B?OUtYL0FubTNPMDRDZUZuVURwRERXZE13TExzR0dJUW9iZHR6MC80aFhBbS9Y?=
 =?utf-8?B?dEpmbEp4aG4rSHY2cFZCalZGUzRqTnBZY1dYTGVKNnFaVjZyOER1emZTVFlv?=
 =?utf-8?B?MXhHYk9DVDVwazdob01QVnBWeEx4bjFpMG4rQjU2b2RkamJmVHNFaENQbmNM?=
 =?utf-8?B?MnNLeGdPZ3AxRW5sUWg0UUdKb0JubTA1d0FBYU9NNDFpYk41U3hEWG5yQS9M?=
 =?utf-8?B?NXN3eHFtcGNWVmV3bllwZjFTWmNETmRPRzlsalRjNkdpZ205SUxLSndKQ1Mz?=
 =?utf-8?B?RURMQXdvYmdpRDIzTHZXeGlocWpRSkF0OGF3aVpOSENkVUFILytZMFpEcm9N?=
 =?utf-8?B?a3VHclZNRlJuUXRQTXBPQjhNL0lWUDZNTFEzdHlJVHR6UU5nSjBGUUhEaFA4?=
 =?utf-8?B?alR0NWRBRm9JbEpVZkpKa2tUSU1jTHFNVDN6YjhCTUxLOThTR3hVdVdXQ1JL?=
 =?utf-8?B?UGVZampmd1BzZ2VVRDhidGlQNlpHQ28yRmtPNllPVm9JRlR1ZGxka1BFZ3J5?=
 =?utf-8?B?ODdHYng3QzlDVkR3N243cGZGU0d3elVUbURhbXRnSjJzT2N4ZnNjZURGVXVU?=
 =?utf-8?B?aU9QZFZiekRDVTh6NWZ5Nm5tdkRhM2gxdW12MlR2cmhiaEk0R05Ddm5KSG4v?=
 =?utf-8?B?WTZNaGRqWGFEWFI2U25sUDRYNkttZWl3QmZ6OURhYUZhU2hhcnl3dGlRYk5L?=
 =?utf-8?B?Uk8yblZDWXdqZm1GSzV1NVhqamZEZlhTOXZzRkp1d3RFUkQwdUFIMUJBREtn?=
 =?utf-8?B?ZlJET1hsMllsd3huemh4dk1BSTZGVnMyemwzWThqaFl1NVNrZE1WZFNJNmhN?=
 =?utf-8?B?Z3hjb25yQUdKQTcxbHdhYkJCZzM2Q3VjakRqdnk5NTRuV1hRZlBmLzBiMDhs?=
 =?utf-8?B?eU9Rcnd2UE5pZ1V6Uk12bzFRZnNkOVpFYVpxRGdVMlV4MUxPbDgyUVRwNFY4?=
 =?utf-8?B?WDU1SHRkV1pNY21qSVRNZHNjbUtzc2srN0dXdFVYSFB1SDViZ2NxRmYxcUlC?=
 =?utf-8?B?V21JRFhFVHFST1R5NFpCZFVqcjl5TzBEVFVHa2FHay9pOEVaektqUGhiS1hr?=
 =?utf-8?B?eGgzTTlQNFdwZ3JBb0FldmZGMVhxTWs5RVQrZWRNTTFaQmVYaHQyam5LVWho?=
 =?utf-8?B?STVJOU96MVdlUlY2R2lUK29Jd2V6dlVWMmFqZ0Y2aWFuSWZ0V3RLQnY1blV3?=
 =?utf-8?B?RHFRbGV3NVMwY0pJR2FsVzJ6MzJSY3E2NjZ3YVV1RUpYWFlMUXlXRGRnc2wv?=
 =?utf-8?B?ZEQzTkhSV0lWUjVmVW5uZ1NqdE9UdWx5bUV3WVNuajkzU2dlbUhoRGxIODk4?=
 =?utf-8?B?QWhYdkJVSHVYUStxalYzd3BkVmNZWnB4MzNscFlnNGFqK1N3cVcxOXVlc2p3?=
 =?utf-8?B?Vmh3a3J0WUh6Q3RTbUUxMldyQlZLTXpNbkhvamErZjkzTFVpeDFQeUN4Y014?=
 =?utf-8?B?b1BrK2FJL2ErOFJTbVZjc1J0dHpDQWRMZmwvdnQ5MGZZc0lXQTI3NjFoR3dH?=
 =?utf-8?B?Z1lhM05mVytEbFpLcU1ud09pcE83VDJFeWd1RUVWRW84UXBDWUpYRmhtYmpS?=
 =?utf-8?B?L21YOWFuaTRTZDNJT08xQ01hSlE4NU8zWmd0THlPSkVGSXFaN2MwSGJrYnNQ?=
 =?utf-8?Q?GbZUnmvTCDMETCtaqanA8davI7dUClaAHzDul5RQzM=3D?=
X-OriginatorOrg: amperemail.onmicrosoft.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 387d03c0-3349-4b1b-3d73-08dc8103700a
X-MS-Exchange-CrossTenant-AuthSource: SA0PR01MB6171.prod.exchangelabs.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 May 2024 23:51:34.4993
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: /FEfoY8zQ9je6bdgKxEYheRnZPiZq0UdsZHWrqaOus8uVJGeee2UtB+mXmnE9WO5h+OYoq4RjlrMTs0e5AkP6URDmK5oxF84rbfJMiag1kT1cRSCqgqHF0jDj82ENM/C
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR01MB8630


On 5/28/24 22:45, Jakub Kicinski wrote:
> On Tue, 28 May 2024 15:18:23 -0400 admiyo@os.amperecomputing.com wrote:
>> From: Adam Young <admiyo@amperecomputing.com>
>>
>> Implementation of DMTF DSP:0292
>> Management Control Transport Protocol(MCTP)  over
>> Platform Communication Channel(PCC)
>>
>> MCTP devices are specified by entries in DSDT/SDST and
>> reference channels specified in the PCCT.
>>
>> Communication with other devices use the PCC based
>> doorbell mechanism.
> Missing your SoB, but please wait for more feedback before reposting.
Sorry, must have dropped it when redoing the last patch,  It was on the 
original and will be on the next version.
>
>> +#include <net/pkt_sched.h>
> Hm, what do you need this include for?
Gets the symbolic constant
>
>> +#define SPDM_VERSION_OFFSET 1
>> +#define SPDM_REQ_RESP_OFFSET 2
>> +#define MCTP_PAYLOAD_LENGTH 256
>> +#define MCTP_CMD_LENGTH 4
>> +#define MCTP_PCC_VERSION     0x1 /* DSP0253 defines a single version: 1 */
>> +#define MCTP_SIGNATURE "MCTP"
>> +#define SIGNATURE_LENGTH 4
>> +#define MCTP_HEADER_LENGTH 12
>> +#define MCTP_MIN_MTU 68
>> +#define PCC_MAGIC 0x50434300
>> +#define PCC_DWORD_TYPE 0x0c
> Could you align the values using tabs?
Will do
>
>> +static void mctp_pcc_client_rx_callback(struct mbox_client *c, void *buffer)
>> +{
>> +	struct mctp_pcc_ndev *mctp_pcc_dev;
>> +	struct mctp_skb_cb *cb;
>> +	struct sk_buff *skb;
>> +	u32 length_offset;
>> +	u32 flags_offset;
>> +	void *skb_buf;
>> +	u32 data_len;
>> +	u32 flags;
>> +
>> +	mctp_pcc_dev = container_of(c, struct mctp_pcc_ndev, inbox_client);
>> +	length_offset = offsetof(struct mctp_pcc_hdr, length);
>> +	data_len = readl(mctp_pcc_dev->pcc_comm_inbox_addr + length_offset) +
>> +		   MCTP_HEADER_LENGTH;
>> +
>> +	skb = netdev_alloc_skb(mctp_pcc_dev->mdev.dev, data_len);
>> +	if (!skb) {
>> +		mctp_pcc_dev->mdev.dev->stats.rx_dropped++;
>> +		return;
>> +	}
>> +	mctp_pcc_dev->mdev.dev->stats.rx_packets++;
>> +	mctp_pcc_dev->mdev.dev->stats.rx_bytes += data_len;
> Please implement ndo_get_stats64, use of the core dev stats in drivers
> is deprecated:
>
>   *	@stats:		Statistics struct, which was left as a legacy, use
>   *			rtnl_link_stats64 instead

Thanks.  Was not aware.  The other MCTP drivers need this as well.


>
>> +	skb->protocol = htons(ETH_P_MCTP);
>> +	skb_buf = skb_put(skb, data_len);
>> +	memcpy_fromio(skb_buf, mctp_pcc_dev->pcc_comm_inbox_addr, data_len);
>> +	skb_reset_mac_header(skb);
>> +	skb_pull(skb, sizeof(struct mctp_pcc_hdr));
>> +	skb_reset_network_header(skb);
>> +	cb = __mctp_cb(skb);
>> +	cb->halen = 0;
>> +	skb->dev =  mctp_pcc_dev->mdev.dev;
> netdev_alloc_skb() already sets dev
>
>> +	netif_rx(skb);
>> +
>> +	flags_offset = offsetof(struct mctp_pcc_hdr, flags);
>> +	flags = readl(mctp_pcc_dev->pcc_comm_inbox_addr + flags_offset);
>> +	mctp_pcc_dev->in_chan->ack_rx = (flags & 1) > 0;
>> +}
>> +
>> +static netdev_tx_t mctp_pcc_tx(struct sk_buff *skb, struct net_device *ndev)
>> +{
>> +	struct mctp_pcc_hdr pcc_header;
>> +	struct mctp_pcc_ndev *mpnd;
>> +	void __iomem *buffer;
>> +	unsigned long flags;
>> +	int rc;
>> +
>> +	netif_stop_queue(ndev);
> Why?

I saw this in other network drivers, both Ethernet and MCTP.  As I 
understand it, without stopping the queue, we could take another packet 
before this one is sent and we only have one buffer;  that said, it 
should be protected by the spin lock.

I am tempted to leave this in here, but move all of the statistics into 
the spin lock.  Is there a reason to not stop the queue?

>
>> +	ndev->stats.tx_bytes += skb->len;
>> +	ndev->stats.tx_packets++;
>> +	mpnd = (struct mctp_pcc_ndev *)netdev_priv(ndev);
>> +
>> +	spin_lock_irqsave(&mpnd->lock, flags);
>> +	buffer = mpnd->pcc_comm_outbox_addr;
>> +	pcc_header.signature = PCC_MAGIC;
>> +	pcc_header.flags = 0x1;
>> +	memcpy(pcc_header.mctp_signature, MCTP_SIGNATURE, SIGNATURE_LENGTH);
>> +	pcc_header.length = skb->len + SIGNATURE_LENGTH;
>> +	memcpy_toio(buffer, &pcc_header, sizeof(struct mctp_pcc_hdr));
>> +	memcpy_toio(buffer + sizeof(struct mctp_pcc_hdr), skb->data, skb->len);
>> +	rc = mpnd->out_chan->mchan->mbox->ops->send_data(mpnd->out_chan->mchan,
>> +							 NULL);
>> +	spin_unlock_irqrestore(&mpnd->lock, flags);
>> +
>> +	dev_consume_skb_any(skb);
>> +	netif_start_queue(ndev);
>> +	if (!rc)
>> +		return NETDEV_TX_OK;
>> +	return NETDEV_TX_BUSY;
>> +}
>> +
>> +static const struct net_device_ops mctp_pcc_netdev_ops = {
>> +	.ndo_start_xmit = mctp_pcc_tx,
>> +	.ndo_uninit = NULL
> No need to init things to NULL
>
>> +static void mctp_pcc_driver_remove(struct acpi_device *adev)
>> +{
>> +	struct mctp_pcc_ndev *mctp_pcc_dev = NULL;
>> +	struct list_head *ptr;
>> +	struct list_head *tmp;
>> +
>> +	list_for_each_safe(ptr, tmp, &mctp_pcc_ndevs) {
>> +		struct net_device *ndev;
>> +
>> +		mctp_pcc_dev = list_entry(ptr, struct mctp_pcc_ndev, next);
>> +		if (adev && mctp_pcc_dev->acpi_device == adev)
>> +			continue;
>> +
>> +		mctp_pcc_dev->cleanup_channel(mctp_pcc_dev->out_chan);
>> +		mctp_pcc_dev->cleanup_channel(mctp_pcc_dev->in_chan);
>> +		ndev = mctp_pcc_dev->mdev.dev;
>> +		if (ndev)
>> +			mctp_unregister_netdev(ndev);
>> +		list_del(ptr);
>> +		if (adev)
>> +			break;
>> +	}
>> +};
> spurious ;
>
>
>> +	.owner = THIS_MODULE,
>> +
> suprious new line
>
>> +};
>> +