Received-SPF: pass (google.com: domain of linux-kernel+bounces-213070-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Message-ID: <b110726e-d496-4975-8089-57a4931da47d@intel.com>
Date: Thu, 13 Jun 2024 13:03:00 +0200
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH iwl-next 03/12] idpf: split &idpf_queue into 4
 strictly-typed queue structures
To: Simon Horman <horms@kernel.org>
CC: <intel-wired-lan@lists.osuosl.org>, Tony Nguyen
	<anthony.l.nguyen@intel.com>, "David S. Miller" <davem@davemloft.net>, "Eric
 Dumazet" <edumazet@google.com>, Jakub Kicinski <kuba@kernel.org>, Paolo Abeni
	<pabeni@redhat.com>, Mina Almasry <almasrymina@google.com>,
	<nex.sw.ncis.osdt.itp.upstreaming@intel.com>, <netdev@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>
References: <20240528134846.148890-1-aleksander.lobakin@intel.com>
 <20240528134846.148890-4-aleksander.lobakin@intel.com>
 <20240601085308.GY491852@kernel.org>
From: Alexander Lobakin <aleksander.lobakin@intel.com>
Content-Language: en-US
In-Reply-To: <20240601085308.GY491852@kernel.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RWd1RUUwQ3I0RS9VYXN2ZmNlMDU2QmhxMFdNenpFWnpUUWhBc29MRk92Umh4?=
 =?utf-8?B?MlliRWI4elErT0tlMUxXV0h1YldKTVV4SXhvU1BDcW5xOE12dTEvWFY5dzZk?=
 =?utf-8?B?Y01YRmthTTVuRWJWbmdYWVQ0ZTAvUFpkL3AvTFhiL2h1S1BmRTdoVlA4d2wx?=
 =?utf-8?B?VDliR1FoZFlkbURwamdIZFd0Mmh5QjU3eGFDd0s2elM3MlIzRlE3NHB6K2RS?=
 =?utf-8?B?bHFGdzBWd3VFTjNDekNVUGJ2b202TnBGaWVvOUN6aEwxbmR4bmt0QXBlUitI?=
 =?utf-8?B?RUgyZVBFY0dwb0lIbXNVeDZ1ck1Cc1U2aG1HNXZVV3VBZkY0SWRXd0wycmhn?=
 =?utf-8?B?RGFZMnpVT1FXKzRuMHAvSGJNR3lzOURkUG9aWHRCRWYxK2xud0dCaFRzWFcv?=
 =?utf-8?B?VDAyTS9hZ25vVDZTb1M1bVhITG9jYk5aM3JFcGJycU5LVVFHc3dLdTJ3bWZ6?=
 =?utf-8?B?QXViaXgwWHRLK01IbG54UGZTQzh0bmtuTWdXa0U1OFVUcjNRclpsMTFYb0Ez?=
 =?utf-8?B?ZzU2NkVRamgxWjMxbmVEZ2dNK2w1WGxsVUFXdnRhWjF5VDdyN096cGYzb01E?=
 =?utf-8?B?UlFHQkNzd3EyMHBZb1l6d1J1RklDRVJ6c2V3SWNhdyt4K2R5Y3N4ZE5sSU0r?=
 =?utf-8?B?T0RQeVBqbUM2UFZaUEN4OHJ6cTJmUDQzUlhFNHBCenJHRmFIdThOdGdZdVAy?=
 =?utf-8?B?QVo5SXBGT2NZOVoxa09jYXFydGVnZ1BObXdqTXB2ZDlSbEtXNEhRdW5WUUFM?=
 =?utf-8?B?eUF6T2lFU0lxQzNOMXJiZ010RlhRWnBBeHBKb2cxMCtaTmo0S3JwN2xINDBR?=
 =?utf-8?B?U05NMWIwS1ptS3AyY1VaQkZpRFBIU3NTSUt1RmxrWUx6SU53K3M5aEM1dE1B?=
 =?utf-8?B?cmc3U3FNN213UUZGZXhKY1J2cHQ3SWY2V3JyQmlIYm5rS05OZ0V0UHdTY2xz?=
 =?utf-8?B?dkNSLzE4bDZyRXI1WHhhOHVPM2Q3T3J2NTVCZGNuQkVvTnZsdm9hVWtCYVhX?=
 =?utf-8?B?SWY5ZUE2Y3NOb1RqRXdkdi93azV5aVRpT0VDQ3V5MG1rTHZBWW53TmRlb0M3?=
 =?utf-8?B?YTkyTitPVDV4ZGFFZnVBREFPd1JiUnZIQ1NUSk5Dc3FKaERsb1JEY1IxRHg2?=
 =?utf-8?B?cUFBN0lYZlFwSTE2bnlsR3phZzdiSkZWcmorQk1NVkFJclFMZ0ltOUlWRm5t?=
 =?utf-8?B?NFFjOHZ6TXpTVFNLMXN0YnZTQjNsODI0bmFRS25EMEdKWnFmeDZWOTI4OTE5?=
 =?utf-8?B?cGxjTkxsZTlneUk1UkZlNVJ4bUZ0L0FjT3FXT1dzZXVyelBzYWs2Uk5YZ0th?=
 =?utf-8?B?dWF6aDV5V1JpU3h3WmVjNEwvUTFsWklDbVlLem1tTnJRWDNFd2o2dnNaYXAr?=
 =?utf-8?B?NkloZmpNQjkydS9oN3RPSHc1TDE5RndIdmFRV2NJdDV5T1MxMDBQK3pMTmpC?=
 =?utf-8?B?T2szUUVZVU0xSWorZnlESGhBbjZaalEzRmNxbDVpbFZnVTZ6ZmlLQnRWV1Zz?=
 =?utf-8?B?UjdoMWJzZHViUzFVV2NVUkp2clVabTd5VEk1OVJ0NDU1ajIwcXNRaDFESUZv?=
 =?utf-8?B?YXVCOWJTaXc3aDFmcG5pRFpkYS8yS1pSTDQ0VndHdVdXQWNUVThkZnUvQTFJ?=
 =?utf-8?B?RzJvWUxsRkNjUzQ0a3ExN0FTR2pmbThaOFB3b3lBSCtZUUpGS3ZobTJ1Y2VT?=
 =?utf-8?B?ZVVNRjAyVkNqSXU5WU1Kb0VoV1A2QnlPY1BiWTNWZ3VUQ2Nwd0ZmV1grZzdt?=
 =?utf-8?B?MWdnTGNuS2k1SUdVRFV4UnN4UHAxdlpETEhVK2lrSmdkN1ExRXNaeHl0UFEx?=
 =?utf-8?B?OS81L0RrZTdiTU5qN08reStLNEpZdHpLc29ubldSQWpFS2lDY25tVFBwdDVT?=
 =?utf-8?B?TldWVndaMWJoMGcyOERlVUVrUW83UUwvYjhSS1NmUldVSEJIdTJVZTNvbHNB?=
 =?utf-8?B?cnJjQlBTN3R2QlE5VUZ6c2JVK0pMMDZ4T2JtRUIzMElNRHdpNlNtRitsZ3BL?=
 =?utf-8?B?Rmg0bHdpNGI1NEh6MS9vSXpTU1ZkMXo1TnJubkNFeGc4R2hDRHkrY0xlZmRW?=
 =?utf-8?B?NEE1WjFOOUJ3K0dTTjVQc01sMDZzTzlvSFA0djlQZFh0TjFFc3dJaVN1T0c0?=
 =?utf-8?B?UzIyTWVBTWlZOXBmN1dwNkZXQ2N5SmNVYmUwSXBIWUN0THVjSkhZUVdIVXRi?=
 =?utf-8?B?Smc9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: f693441c-7c86-41d0-617b-08dc8b9873dc
X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB8718.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Jun 2024 11:03:27.7031
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: NhJfmFhb4KJBk8hremop55ygGUdiFJiq7ovdmMZf2ckBHiswa19XP2UVtK9e0x+Z8dGybjlKnOr4kcb+7DgpROkEEvQsYeHAPWDq5f1I4BM=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB7279
X-OriginatorOrg: intel.com

From: Simon Horman <horms@kernel.org>
Date: Sat, 1 Jun 2024 09:53:08 +0100

> On Tue, May 28, 2024 at 03:48:37PM +0200, Alexander Lobakin wrote:
>> Currently, sizeof(struct idpf_queue) is 32 Kb.
>> This is due to the 12-bit hashtable declaration at the end of the queue.
>> This HT is needed only for Tx queues when the flow scheduling mode is
>> enabled. But &idpf_queue is unified for all of the queue types,
>> provoking excessive memory usage.
>> The unified structure in general makes the code less effective via
>> suboptimal fields placement. You can't avoid that unless you make unions
>> each 2 fields. Even then, different field alignment etc., doesn't allow
>> you to optimize things to the limit.
>> Split &idpf_queue into 4 structures corresponding to the queue types:
>> RQ (Rx queue), SQ (Tx queue), FQ (buffer queue), and CQ (completion
>> queue). Place only needed fields there and shortcuts handy for hotpath.
>> Allocate the abovementioned hashtable dynamically and only when needed,
>> keeping &idpf_tx_queue relatively short (192 bytes, same as Rx). This HT
>> is used only for OOO completions, which aren't really hotpath anyway.
>> Note that this change must be done atomically, otherwise it's really
>> easy to get lost and miss something.
>>
>> Signed-off-by: Alexander Lobakin <aleksander.lobakin@intel.com>
> 
> ...
> 
>> diff --git a/drivers/net/ethernet/intel/idpf/idpf_txrx.c b/drivers/net/ethernet/intel/idpf/idpf_txrx.c
> 
> ...
> 
>> @@ -1158,20 +1325,22 @@ static void idpf_rxq_set_descids(struct idpf_vport *vport, struct idpf_queue *q)
>>   */
>>  static int idpf_txq_group_alloc(struct idpf_vport *vport, u16 num_txq)
>>  {
>> -	bool flow_sch_en;
>> -	int err, i;
>> +	bool split, flow_sch_en;
>> +	int i;
>>  
>>  	vport->txq_grps = kcalloc(vport->num_txq_grp,
>>  				  sizeof(*vport->txq_grps), GFP_KERNEL);
>>  	if (!vport->txq_grps)
>>  		return -ENOMEM;
>>  
>> +	split = idpf_is_queue_model_split(vport->txq_model);
>>  	flow_sch_en = !idpf_is_cap_ena(vport->adapter, IDPF_OTHER_CAPS,
>>  				       VIRTCHNL2_CAP_SPLITQ_QSCHED);
>>  
>>  	for (i = 0; i < vport->num_txq_grp; i++) {
>>  		struct idpf_txq_group *tx_qgrp = &vport->txq_grps[i];
>>  		struct idpf_adapter *adapter = vport->adapter;
>> +		struct idpf_txq_stash *stashes;
>>  		int j;
>>  
>>  		tx_qgrp->vport = vport;
>> @@ -1180,45 +1349,62 @@ static int idpf_txq_group_alloc(struct idpf_vport *vport, u16 num_txq)
>>  		for (j = 0; j < tx_qgrp->num_txq; j++) {
>>  			tx_qgrp->txqs[j] = kzalloc(sizeof(*tx_qgrp->txqs[j]),
>>  						   GFP_KERNEL);
>> -			if (!tx_qgrp->txqs[j]) {
>> -				err = -ENOMEM;
>> +			if (!tx_qgrp->txqs[j])
>>  				goto err_alloc;
>> -			}
>> +		}
>> +
>> +		if (split && flow_sch_en) {
>> +			stashes = kcalloc(num_txq, sizeof(*stashes),
>> +					  GFP_KERNEL);
> 
> Hi Alexander,
> 
> Here stashes is assigned a memory allocation and
> then then assigned to tx_qgrp->stashes a few lines below...
> 
>> +			if (!stashes)
>> +				goto err_alloc;
>> +
>> +			tx_qgrp->stashes = stashes;
>>  		}
>>  
>>  		for (j = 0; j < tx_qgrp->num_txq; j++) {
>> -			struct idpf_queue *q = tx_qgrp->txqs[j];
>> +			struct idpf_tx_queue *q = tx_qgrp->txqs[j];
>>  
>>  			q->dev = &adapter->pdev->dev;
>>  			q->desc_count = vport->txq_desc_count;
>>  			q->tx_max_bufs = idpf_get_max_tx_bufs(adapter);
>>  			q->tx_min_pkt_len = idpf_get_min_tx_pkt_len(adapter);
>> -			q->vport = vport;
>> +			q->netdev = vport->netdev;
>>  			q->txq_grp = tx_qgrp;
>> -			hash_init(q->sched_buf_hash);
>>  
>> -			if (flow_sch_en)
>> -				set_bit(__IDPF_Q_FLOW_SCH_EN, q->flags);
>> +			if (!split) {
>> +				q->clean_budget = vport->compln_clean_budget;
>> +				idpf_queue_assign(CRC_EN, q,
>> +						  vport->crc_enable);
>> +			}
>> +
>> +			if (!flow_sch_en)
>> +				continue;
>> +
>> +			if (split) {
> 
> ... but here elements of stashes seem to be assigned to q->stash
> without stashes having being initialised.
> 
> Flagged by Smatch

Hi! Yes, I saw the report, but isn't it a false positive?

Allocation happens when `split && flow_sch_en`, and here we have

			if (!flow_sch_en)
				continue;

			if (split)
				// assign

IOW the assignment can't happen without the allocation?

> 
>> +				q->stash = &stashes[j];
>> +				hash_init(q->stash->sched_buf_hash);
>> +			}
>> +
>> +			idpf_queue_set(FLOW_SCH_EN, q);

Thanks,
Olek