Received-SPF: pass (google.com: domain of linux-kernel+bounces-213073-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Message-ID: <2c22ba85-2338-4f16-b3c2-70c4270cd96b@intel.com>
Date: Thu, 13 Jun 2024 13:05:58 +0200
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH iwl-next 12/12] idpf: use libeth Rx buffer management for
 payload buffer
To: Simon Horman <horms@kernel.org>
CC: <intel-wired-lan@lists.osuosl.org>, Tony Nguyen
	<anthony.l.nguyen@intel.com>, "David S. Miller" <davem@davemloft.net>, "Eric
 Dumazet" <edumazet@google.com>, Jakub Kicinski <kuba@kernel.org>, Paolo Abeni
	<pabeni@redhat.com>, Mina Almasry <almasrymina@google.com>,
	<nex.sw.ncis.osdt.itp.upstreaming@intel.com>, <netdev@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>, Dan Carpenter <dan.carpenter@linaro.org>
References: <20240528134846.148890-1-aleksander.lobakin@intel.com>
 <20240528134846.148890-13-aleksander.lobakin@intel.com>
 <20240601090842.GZ491852@kernel.org>
From: Alexander Lobakin <aleksander.lobakin@intel.com>
Content-Language: en-US
In-Reply-To: <20240601090842.GZ491852@kernel.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Yjl6WWRqdnVMSWxuMUlLS3RMNXNtUHdCM2ZZUjNzTHROb3c2cDBIenpCVHlF?=
 =?utf-8?B?ZE9VNzhCUVBoOEtlWnUxaGhobW1UQjREUktNWU8rN3l4L0hEcGJseGQzVXhW?=
 =?utf-8?B?eGhSMnVYNXRHZmhLbElsd21zdjM5MEZIVjgzem1NbXI3cDBPSGluaTQrR25z?=
 =?utf-8?B?eUo3Wkp6dVY3dFNOMHl0SW1KVGVjWlB0VURVZWlOQUhZTUtXRDZQSFpKZHgr?=
 =?utf-8?B?UVJtT0ZIcUlqRmZMMXRNOXREakozaVlubnFzZXpOM2lENUt3eWdNUEhIR2hB?=
 =?utf-8?B?N1M5VFBEcUtHVGN4dExhN1JKNTJmMFZRNHZTbWVBMXVid2NpRFFqc0sydklX?=
 =?utf-8?B?ZFkyb2dJU0psTUs5ZFMxd01SM2UxOElWY2IvZ3pvWU54bk52dnhKcGxyYmQ5?=
 =?utf-8?B?Z25GQ0lLaTJjVWRkdnkyR3RGblVhZzlVUi9Db2RXeWV0Z2VUb0Fja29kU09s?=
 =?utf-8?B?UTJoM1d4WGlJcHNHMUlTWnFYL1dlbHdsU3MvZGw2elcrVERyRWRrUFo5Y0Nn?=
 =?utf-8?B?T3dXUjk0RlkvT2xWUGpxTEkwOTg4c0xmNERXdmppaWMvSGlaY1J1bDJ6OWRB?=
 =?utf-8?B?akxTL3Jma0VZZEhQalNDQ2Y5SHZNRTBUTGVHSzBBN203V0RzalJneUdOdW5J?=
 =?utf-8?B?NmtGemEzbU5BRmlVTnduY0srUUluZlNVbnFyN3p4aGNzVnk3K2xaQ0YvQXNi?=
 =?utf-8?B?cmF5a2hHaHY2QXk4WWVmVC9kcVJmQU1pelB1Y04vNG1tNnI2L2JwU0ROeFNE?=
 =?utf-8?B?YUtibFJCdlRSdURkazRQSnp1SzM1WHkrMUJ5Wll4WWd0dGxjd01FdURyMTdU?=
 =?utf-8?B?aDBZYkh6QjlQYVU0YkgwOXNBaXhCRDJhYURsTXNRWUxMQ1k5UHFIMHdybGJp?=
 =?utf-8?B?Q1NzRHFjcXI2NzZSQ29xMk8vWjFmUkx5Q0FXRm1aNXJWUytXc2kxeG5DK1Zs?=
 =?utf-8?B?OHE1UFIvdXJzcXlJdlhSUjF5L2xOd0h1WTlDelBUdU9oSEE0MzFEanZSMFp5?=
 =?utf-8?B?OXhvWnRHd1hKYURCRTFib2hGYkU5UE1VTm9lRjc3RXQrK3BpVm81dHgvVjBV?=
 =?utf-8?B?THVEZHZ4SGEzWEZBeHd1ZlF5QzZJbzFnWS9yM2xmNm11cmJYRnFjOVlJQ2p1?=
 =?utf-8?B?NWtJOThuUVZlV3VKaU50K21rUDJ2S2wwK040ZWNtVWI3SDRnc05SMHM4QTlh?=
 =?utf-8?B?UTlDUGRkdm56a0lBcGkvSGtWOFJmY3hiMkI4aTVMWlhWc2VOc0hxS3pGN0gz?=
 =?utf-8?B?aDZsWEtzNUJBMTdZTGVHb0diVXlOc0hGaTdMUGhNa1poUGhkY1VzSEN1b2tl?=
 =?utf-8?B?a3NOMnd2MDlnUFJUVlpBY2RUa0dFY0hTOGJ4bDZXa0diSjdSbWNJbUd0T3Y4?=
 =?utf-8?B?Q25PRm8zTjdBa2FVcWJaV1AxQU1Zckg4VlgwSm92Qm9obGU5T2l1alJLeDFI?=
 =?utf-8?B?SW5ZZmFvY2FZcWtqUXZQRU1kYXVsTWlXK0xuNVNETW1QSmd4RjRKWEQwZ2h5?=
 =?utf-8?B?Q2s2ZnpETnhyUEd5bXNjendjeGRGTDQ2eGdzOTJQSEl3NzhyejFkZzJSODJD?=
 =?utf-8?B?NGtGanNRTldLK3B4T09MVkJKdWN4ckcwU1BObll0Tkc0T3V3empUd3pUcjRS?=
 =?utf-8?B?Mm00OWVsb1lyNDdEYjNXRk84YWtzZjR0RWJPeDFybk9tak00ajNwTnI0Zkpx?=
 =?utf-8?B?ZWExZFRBd2RHS2xYRXVjanA0NkgzS1VXQitWd2FrajBzRkFoS2Evdk1rTnB4?=
 =?utf-8?B?dUhCQWpqY25PbnhGOGtLdXR2UndMdXpuZStDM05Ya2hFWWNFb3FTRTY0T2hF?=
 =?utf-8?B?alV2MElKM094OENLTllNdkJYb3Vvc01SSkpaZlhzZjhETndmV3oyMkFIM2Ra?=
 =?utf-8?B?U3QzZ1NFN3Rtb1VnRVNnUk4zMzRyM0dFQ3Y1OUFRZDc5SDVhZGlabjQ3RUk5?=
 =?utf-8?B?UHBXOGdiSXBlMHN5bnNrNFhWeDhManIzQ2dIRDBCN3dvT3d3SnBMelNtbVpP?=
 =?utf-8?B?TG9Mc3ZUbUV3MkF0Ym9Ed3BiWGQrQnVuL1Z1M25YbmNUSFRLdHpCNGpodnF0?=
 =?utf-8?B?aFZJMmJXU1lsYmMxdjJuUDBQN05jN2Y4VWpDUG1QY0poQUV2S2FmbkVTWVZD?=
 =?utf-8?B?SlVFM2ZuUXJlTnJGcmZyYmVIZktkek85dmoxNXNJZ2R4Z0MzcjFLdFFQZ2Z4?=
 =?utf-8?Q?T3CjP0shGQCPIuNoFgR1SEA=3D?=
X-MS-Exchange-CrossTenant-Network-Message-Id: bd2deb8b-d288-46e6-1b25-08dc8b98de16
X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB8718.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jun 2024 11:06:25.7886
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 9fHf7B4XplKKrp8CurovZ7HNN4w/3QJAT66EMvri4aYfMOsX26rGXeTV6Aq9k3gc2nlsMQYu4IK3oE4J3XHp23tO+Z15oYdXk/YRPGyVSOI=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR11MB6997
X-OriginatorOrg: intel.com

From: Simon Horman <horms@kernel.org>
Date: Sat, 1 Jun 2024 10:08:42 +0100

> + Dan Carpenter
> 
> On Tue, May 28, 2024 at 03:48:46PM +0200, Alexander Lobakin wrote:
>> idpf uses Page Pool for data buffers with hardcoded buffer lengths of
>> 4k for "classic" buffers and 2k for "short" ones. This is not flexible
>> and does not ensure optimal memory usage. Why would you need 4k buffers
>> when the MTU is 1500?
>> Use libeth for the data buffers and don't hardcode any buffer sizes. Let
>> them be calculated from the MTU for "classics" and then divide the
>> truesize by 2 for "short" ones. The memory usage is now greatly reduced
>> and 2 buffer queues starts make sense: on frames <= 1024, you'll recycle
>> (and resync) a page only after 4 HW writes rather than two.
>>
>> Signed-off-by: Alexander Lobakin <aleksander.lobakin@intel.com>
> 
> ...
> 
>> diff --git a/drivers/net/ethernet/intel/idpf/idpf_txrx.c b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
> 
> ...
> 
> Hi Alexander,
> 
> The code above the hunk below, starting at line 3321, is:
> 
> 		if (unlikely(!hdr_len && !skb)) {
> 			hdr_len = idpf_rx_hsplit_wa(hdr, rx_buf, pkt_len);
> 			pkt_len -= hdr_len;
> 			u64_stats_update_begin(&rxq->stats_sync);
> 			u64_stats_inc(&rxq->q_stats.hsplit_buf_ovf);
> 			u64_stats_update_end(&rxq->stats_sync);
> 		}
> 		if (libeth_rx_sync_for_cpu(hdr, hdr_len)) {
> 			skb = idpf_rx_build_skb(hdr, hdr_len);
> 			if (!skb)
> 				break;
> 			u64_stats_update_begin(&rxq->stats_sync);
> 			u64_stats_inc(&rxq->q_stats.hsplit_pkts);
> 			u64_stats_update_end(&rxq->stats_sync);
> 		}
> 
>> @@ -3413,24 +3340,24 @@ static int idpf_rx_splitq_clean(struct idpf_rx_queue *rxq, int budget)
>>  		hdr->page = NULL;
>>  
>>  payload:
>> -		if (pkt_len) {
>> -			idpf_rx_sync_for_cpu(rx_buf, pkt_len);
>> -			if (skb)
>> -				idpf_rx_add_frag(rx_buf, skb, pkt_len);
>> -			else
>> -				skb = idpf_rx_construct_skb(rxq, rx_buf,
>> -							    pkt_len);
>> -		} else {
>> -			idpf_rx_put_page(rx_buf);
>> -		}
>> +		if (!libeth_rx_sync_for_cpu(rx_buf, pkt_len))
>> +			goto skip_data;
>> +
>> +		if (skb)
>> +			idpf_rx_add_frag(rx_buf, skb, pkt_len);
>> +		else
>> +			skb = idpf_rx_build_skb(rx_buf, pkt_len);
>>  
>>  		/* exit if we failed to retrieve a buffer */
>>  		if (!skb)
>>  			break;
>>  
>> -		idpf_rx_post_buf_refill(refillq, buf_id);
>> +skip_data:
>> +		rx_buf->page = NULL;
>>  
>> +		idpf_rx_post_buf_refill(refillq, buf_id);
>>  		IDPF_RX_BUMP_NTC(rxq, ntc);
>> +
>>  		/* skip if it is non EOP desc */
>>  		if (!idpf_rx_splitq_is_eop(rx_desc))
>>  			continue;
> 
> The code following this hunk, ending at line 3372, looks like this:
> 
> 		/* pad skb if needed (to make valid ethernet frame) */
> 		if (eth_skb_pad(skb)) {
> 			skb = NULL;
> 			continue;
> 		}
> 		/* probably a little skewed due to removing CRC */
> 		total_rx_bytes += skb->len;
> 
> Smatch warns that:
> .../idpf_txrx.c:3372 idpf_rx_splitq_clean() error: we previously assumed 'skb' could be null (see line 3321)
> 
> I think, but am not sure, this is because it thinks skb might
> be NULL at the point where "goto skip_data;" is now called above.
> 
> Could you look into this?

This is actually a good catch. skb indeed could be NULL and we needed to
check that in the same condition where !eop is checked.
Fixed already in my tree, so it will be fixed in v2. Thanks for catching!

(BTW I fixed that in iavf when submitting the libeth series, but forgot
 to fix that here lol >_<)
(Also, it was implicitly fixed in the later commits where I convert skb
 to xdp_buff here, so I didn't catch this one)

Thanks,
Olek