Received-SPF: pass (google.com: domain of linux-kernel+bounces-138451-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Message-ID: <36a067fa-75c0-4e7d-b704-229deea2d440@intel.com>
Date: Wed, 10 Apr 2024 13:49:59 +0200
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH net-next v9 7/9] libeth: add Rx buffer management
To: Przemek Kitszel <przemyslaw.kitszel@intel.com>
CC: "David S. Miller" <davem@davemloft.net>, Eric Dumazet
	<edumazet@google.com>, Jakub Kicinski <kuba@kernel.org>, Paolo Abeni
	<pabeni@redhat.com>, Alexander Duyck <alexanderduyck@fb.com>, Yunsheng Lin
	<linyunsheng@huawei.com>, Jesper Dangaard Brouer <hawk@kernel.org>, "Ilias
 Apalodimas" <ilias.apalodimas@linaro.org>, Christoph Lameter <cl@linux.com>,
	Vlastimil Babka <vbabka@suse.cz>, Andrew Morton <akpm@linux-foundation.org>,
	<nex.sw.ncis.osdt.itp.upstreaming@intel.com>, <netdev@vger.kernel.org>,
	<intel-wired-lan@lists.osuosl.org>, <linux-mm@kvack.org>,
	<linux-kernel@vger.kernel.org>
References: <20240404154402.3581254-1-aleksander.lobakin@intel.com>
 <20240404154402.3581254-8-aleksander.lobakin@intel.com>
 <45eb2bf1-e7b0-4045-82b3-93b9f81b7988@intel.com>
 <bebf1a1a-e4ec-4ec0-9d01-57a51bcf14ca@intel.com>
 <9d389192-63c4-44e8-b863-6323b45aec0f@intel.com>
From: Alexander Lobakin <aleksander.lobakin@intel.com>
Content-Language: en-US
In-Reply-To: <9d389192-63c4-44e8-b863-6323b45aec0f@intel.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Nm9KTkNCNlduZ1F0MUJUb01LK2tjcTNPMDdUTGxzUW81Umd2bUF4Zy8zaW5v?=
 =?utf-8?B?UWFLaUc0QkNjQlBoZldwUGlPL2k3RThhZ01YZGNGaWpxZDFOUW13K0JTYW1C?=
 =?utf-8?B?UmUvT1JVWElEblpybmp2bldYR2ZjWktGQ29XQjRKUnBTT21RNzh4NjBEZllv?=
 =?utf-8?B?STliZmRYZ2xmQXMvU3I2eXNQU0ZQVEQ2S2VHelQ0YUU4U0dYSmN1SVk2T3RD?=
 =?utf-8?B?ZjFWUVVCNUVONTFUbDNqNFdTNEs1RDh4V1RGKzVHQ2tSYkRpaWhhaFZPMGlZ?=
 =?utf-8?B?RW1VWW5yRTN6VHRNelVDUHp6M0swbEVJMHJENFl5V2lGdXd2T1VVbHlzaGQr?=
 =?utf-8?B?eXFaM0Urd21rOXR4ZGcrNnJGMlNZNTFzQ1A1ZE1IYTRhL1VFM0dHZVdoVmx3?=
 =?utf-8?B?YlFSNkxkck5jMEd3dk1jMHpraVR2Q1VQZmxWT1NBL2czb1lyV0hscXJnam03?=
 =?utf-8?B?dWNtclp4d1YwM1FiaU9aZk5oMVY4ajV1NlVZRUIwWEo0aUcvaUVDTXlFQTJN?=
 =?utf-8?B?WUpxVEt4QXhwL1BQZ25STFU4Qzc5RVNCWWhQNUlaNG5qU2pnQzdPajZIcWVG?=
 =?utf-8?B?MWxyUDVSNWt5MFg2VXg3MFFKZUdrMWpsc1BBZUlieTBVQ0ZwZXJoK1NRd3ZI?=
 =?utf-8?B?Rm1wcHVVNTIrK0hEWURWRjZoeE52TldJOWdtWGVLcUhPKzV2ZWxRTGFiRDBv?=
 =?utf-8?B?SUk5cjRoQ0ZmOGlRZUM0Mmo3Sm1VdHhOcy94OUlTRlp6VTM2UERuOE8zZTRT?=
 =?utf-8?B?T2pHWS9STTFXRGVZVnppZ241K2FwZWtIU0RKRlQwZUdCMVkwZzFMNG1kVThi?=
 =?utf-8?B?Ky91eEd5eEZETlBWU0Vzd1RvNkh6US9yTytkMWluYTQzc3JCUUt6Y3FOSzAz?=
 =?utf-8?B?TFFBNCtEeUVFU1JNcXVnTHpBWWJva2FaRHZDbkxWSnFTK01ySEJEVkhZYTFQ?=
 =?utf-8?B?UzlLWWxTN3AwWERlYW9LZStMTzU3NDBEZWtWOVZiNGE4VTlVeThtQWVIeXR0?=
 =?utf-8?B?Mng0L1o4UGl0SEdMMVRNbEtOR3BTUDJmK29DN1dSeVBEak9oRkpvTVp5eCtV?=
 =?utf-8?B?S2pXdUdQVTIrbTRpYkNMc3FlK3VFVGdQczRJTkgvSW1CQlo1NlNxcEpTUVEr?=
 =?utf-8?B?Wjl4WWliZGJlNld4UmR6MzJQZitqWE1hZW9kMll6cHFyYTIxRWMwWGJTS2Z3?=
 =?utf-8?B?SE82Q3NaSHlIaE43bThjR1NBT1ZFd3I1S1gvK2FXK0dSaHdhdzFmZTRkdThu?=
 =?utf-8?B?d2RXM3FHbG9hQnA5a2QxUDBENUpIaENKdWsweE4xb0RJTDVDLzJkbkVJOGRh?=
 =?utf-8?B?MUwzODlXcmtZaWJTY1ovUFVJYUdhRG5QTFMyTXhSTFFzdWEycVlRaXdBRFVo?=
 =?utf-8?B?VDN1MGhiK211V01QV28xUzZwd05tcE95YTdwZit6YmFObnJJM0FYZmZ1Vzlj?=
 =?utf-8?B?Vmh5WUxsMmhRMkszazdLV3h1aEM1NERVNWhyMzEwelZUVldVRld3Z1UzT0VO?=
 =?utf-8?B?K0xwSHduNmkwWmUvb1d4cTRUcnRqazBKRDE4T0JHTzhQMlo4V1RLaTA4cEd4?=
 =?utf-8?B?WlhmVFlCS3F3MEtKNlJUYXM4RzVTaVBpbzg4TFByTlBWd0MxWmgva2RuU3dX?=
 =?utf-8?B?YzdSVXVaa3Jwa0hSNXlJZUZMcE9HN24zQXlZU1c5RDNSR3JiMExYTkh1TDJi?=
 =?utf-8?B?RFBoeVZ1VFVldnNLTGdhanJqd2thM1Mrd0E5bzE3TXdFZ0JZSU8reXdSc2Yz?=
 =?utf-8?B?Mld0UHovZk10cnh3YjRDVFdKSmV5K0V5dzI1U1dlWnVrVnM0YWpjelY3eE9C?=
 =?utf-8?B?dkRacFZoQXJoRkt1VjU3U1dkc1Q0UFZRd1pHN1RXRWZwditETXZkRVhkTzFY?=
 =?utf-8?B?eGY3bkxNSFcvZWIwS1hZRy90OW03MEpEYlVWbStNeGM3cmZoSU5na280YjBM?=
 =?utf-8?B?TitXTXh0Qi9TcS9paUpnV0ZmbVY3SDFJWmIyMnMzQjlIT2JpMU40bjVKUTlJ?=
 =?utf-8?B?MWlnWTUyalJQKzNCT0ZQUVVOT3oxZXZseGJZVnJqdVVVQmdzaFZOWHI3V0xK?=
 =?utf-8?B?K2ZFZ0J6NVd0c2ZJaWwwaDNrTGwwL3lidXROVFFualB6bHhERWFYQjBSOVZt?=
 =?utf-8?B?UkdIZnRNSUxUTUVkWHZPeWZIYk1YV2V6SUN3VDcvMXBZRWlOY3lPcEVtOHBo?=
 =?utf-8?B?TlE9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: a57efcf5-8fd8-4a58-fff3-08dc5954a69e
X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB8718.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Apr 2024 11:52:08.9741
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: XsfnjREgLxg1/7mkF+xBS6WgKDUR3OynZleY7Anwfh3WWpSwIQ0tScaTj+dVcxLkCoJhVjvCu6JcLSbYm5x3odS9vD4oVaTgJTJ60kGyUbU=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB6053
X-OriginatorOrg: intel.com

From: Przemek Kitszel <przemyslaw.kitszel@intel.com>
Date: Tue, 9 Apr 2024 12:58:33 +0200

> On 4/8/24 11:09, Alexander Lobakin wrote:
>> From: Przemek Kitszel <przemyslaw.kitszel@intel.com>
>> Date: Fri, 5 Apr 2024 12:32:55 +0200
>>
>>> On 4/4/24 17:44, Alexander Lobakin wrote:
>>>> Add a couple intuitive helpers to hide Rx buffer implementation details
>>
>> [...]
>>
>>>> +struct libeth_fqe {
>>>> +    struct page        *page;
>>>> +    u32            offset;
>>>> +    u32            truesize;
>>>> +} __aligned_largest;
>>>> +
>>>> +/**
>>>> + * struct libeth_fq - structure representing a buffer queue
>>>> + * @fp: hotpath part of the structure
>>>> + * @pp: &page_pool for buffer management
>>>> + * @fqes: array of Rx buffers
>>>> + * @truesize: size to allocate per buffer, w/overhead
>>>> + * @count: number of descriptors/buffers the queue has
>>>> + * @buf_len: HW-writeable length per each buffer
>>>> + * @nid: ID of the closest NUMA node with memory
>>>> + */
>>>> +struct libeth_fq {
>>>> +    struct_group_tagged(libeth_fq_fp, fp,
>>>> +        struct page_pool    *pp;
>>>> +        struct libeth_fqe    *fqes;
>>>> +
>>>> +        u32            truesize;
>>>> +        u32            count;
>>>> +    );
>>>> +
>>>> +    /* Cold fields */
>>>> +    u32            buf_len;
>>>> +    int            nid;
>>>> +};
>>>
>>> [...]
>>>
>>> Could you please unpack the meaning of `fq` and `fqe` acronyms here?
>>
>> Rx:
>>
>> RQ -- receive queue, on which you get Rx DMA complete descriptors
>> FQ -- fill queue, the one you fill with free buffers
>>    FQE -- fill queue element, i.e. smth like "iavf_rx_buffer" or whatever
>>
>> Tx:
>>
>> SQ -- send queue, the one you fill with buffers to transmit
>>    SQE -- send queue element, i.e. "iavf_tx_buffer"
>> CQ -- completion queue, on which you get Tx DMA complete descriptors
>>
>> XDPSQ, XSkRQ etc. -- same as above, but for XDP / XSk
>>
>> I know that rxq, txq, bufq, complq is more common since it's been used
>> for years, but I like these "new" ones more :>
>>
> 
> Thank you, that sounds right. If you happen to sent v10, a bit of code
> comment with this info would be useful ;)

The current kdoc in front of each struct and function declaration is not
enough? :D

Thanks,
Olek