Received-SPF: pass (google.com: domain of linux-kernel+bounces-138558-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Message-ID: <b1438638-a6f9-4a88-b8fe-7bec28ba124f@intel.com>
Date: Wed, 10 Apr 2024 15:01:26 +0200
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH net-next v9 7/9] libeth: add Rx buffer management
To: Przemek Kitszel <przemyslaw.kitszel@intel.com>
CC: "David S. Miller" <davem@davemloft.net>, Eric Dumazet
	<edumazet@google.com>, Jakub Kicinski <kuba@kernel.org>, Paolo Abeni
	<pabeni@redhat.com>, Alexander Duyck <alexanderduyck@fb.com>, Yunsheng Lin
	<linyunsheng@huawei.com>, Jesper Dangaard Brouer <hawk@kernel.org>, "Ilias
 Apalodimas" <ilias.apalodimas@linaro.org>, Christoph Lameter <cl@linux.com>,
	Vlastimil Babka <vbabka@suse.cz>, Andrew Morton <akpm@linux-foundation.org>,
	<nex.sw.ncis.osdt.itp.upstreaming@intel.com>, <netdev@vger.kernel.org>,
	<intel-wired-lan@lists.osuosl.org>, <linux-mm@kvack.org>,
	<linux-kernel@vger.kernel.org>
References: <20240404154402.3581254-1-aleksander.lobakin@intel.com>
 <20240404154402.3581254-8-aleksander.lobakin@intel.com>
 <45eb2bf1-e7b0-4045-82b3-93b9f81b7988@intel.com>
 <bebf1a1a-e4ec-4ec0-9d01-57a51bcf14ca@intel.com>
 <9d389192-63c4-44e8-b863-6323b45aec0f@intel.com>
 <36a067fa-75c0-4e7d-b704-229deea2d440@intel.com>
 <61a952aa-1b3e-43c7-9f35-ab53e492305a@intel.com>
From: Alexander Lobakin <aleksander.lobakin@intel.com>
Content-Language: en-US
In-Reply-To: <61a952aa-1b3e-43c7-9f35-ab53e492305a@intel.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?M1hKTHJsbVRMVVNPbHUvVmovTW4rN2E3WWRPdUhMT3Rud1BieUluczVhSS93?=
 =?utf-8?B?VDZtMGxmTStFYkF3bE0yMWIxMXE2RlVKcmFrck8yMStkRGQvdm1BeDd0cEtE?=
 =?utf-8?B?bHVQNjFiaFFLOXNNU1dTOHl4ZytucjlkMU5QaFdrQ2lHZVdLMzRWbTRkU0Ez?=
 =?utf-8?B?VDRjakFvS2FFNy8yR0tLZjRybkp1RUlWZWFpNGs2R3VJZklPU1AvK1I2NjdZ?=
 =?utf-8?B?ZGlVczZaUmE2U2w2Zmxkc2p2bW4wYVBoN1dIU3RSMUFyZFBubFAxZkhHL3Fs?=
 =?utf-8?B?NHRGRmNmQll4c0YrcUNnVVVHTm9RVkxrR01XM2NyNStXQ1BmSHg0bXIvL1ZO?=
 =?utf-8?B?eWo2S29tMTVMeWhBZU81ZGU3bVR6R3Jld2FXTXhHVG56UGVpcjUweTNQaTk5?=
 =?utf-8?B?Yzh1cWgxa2pPQWNtclVLT1pnWXh3YmVnS0Y1SnBMTXdra213aWRSU0ZBcTNW?=
 =?utf-8?B?YzdXM2dsL3hvMnhWLzZ4d203VVJmaXBaT21UNVUrdXFpTWFVVFE0K2VMbWtm?=
 =?utf-8?B?ek40ZjNwaE1RNHVDTCsvTkF2bTI1TGRDam5CUXFsbGpIejN4WTg0RVVTUGd3?=
 =?utf-8?B?MTgzRmk2VVJXU2s1ZVN4UkIrYWlYc0ppdXJRR0dRV3RNTWlzUGFsNmlNejJH?=
 =?utf-8?B?VWxKZGhIOXllaXVJYXE2enNIYVRka2Q5OHdmaXVIWllHay84VVpuMUFJdWd3?=
 =?utf-8?B?REQ5OHI2TVF2WUlQMkYzOFJwd1gyajdvdFNNZk04OEhNUzMzTW1LVDBGcE1I?=
 =?utf-8?B?Z0s3M083cXcwby9jYVVtOENoOFl5RndjSFJVdHhoczRDNWJKamR2RTNJemhO?=
 =?utf-8?B?SmdFbnZPazZnMEhpYm0rQUVpNVpMVzhzWGdnQ1lkRks1biszbXZZejQ4SmNl?=
 =?utf-8?B?NXIzYzkvUFBGSzJXMnZaZmp6VUxxNFEzQ0tVZnBEdjFOazR1bmhiME9HT1Jq?=
 =?utf-8?B?RDZ4bldaTzg0UnlqaS9yOUdqWS9VMm9Kb1U0UjVhWGZlOUREaXl0blFObmtr?=
 =?utf-8?B?NjBvL1RYRnhhWlR5N2xucnBsQllqZ0NhNFkraFVlOGx1ZjdIb2VGVmJHbDI5?=
 =?utf-8?B?UGZpNDc2SW9vR0hyZUwyb3NwOVJUTWVTbFlLakhCT29FUGduR3d5UW53TGND?=
 =?utf-8?B?OC9VWCtxVkRMTlB2cjQ1djJoR2pJMFlMWmVJb1ZqbVhJOHFCR3dJU3JhWGFX?=
 =?utf-8?B?T2tISnZpS0d4bzUxNmNkRWV5aVh0M0ZRSlNHenhOcWlpTS8rbTJzS3h4U2ZS?=
 =?utf-8?B?azFjdWlUOEpaUjc3S09Pdm9CNjc1d0lycGVpazBId3ovTjh5eVFOdVdkdjVS?=
 =?utf-8?B?TE4zM09tUXFaYXF6UWEvVGdTeHhCRVVXWmdUL1F5UHNudE1leVIwUE9XWkNl?=
 =?utf-8?B?N2FWYnMxQXE3bUpWN1IzbXJvcEZidFg4OFlsTkFCSUtkN00xU0d6aEREQlZo?=
 =?utf-8?B?MGZ2QkR6NGhiTjQrdXdOSStwWEJRRDlSc1Y1WnR4Mnp3VXRpYTMxdmp1c2dB?=
 =?utf-8?B?MlZYYVJveVNkeHRGMk1NOVFBTEFTN1NjeHpIdGxjNFYxZEhlbTRhTzBSdU80?=
 =?utf-8?B?MjRXelJIR1FHeWhSUDlraFB0RFNhSTRXZlduTGNaVmlIZTlPcmlrd0dLdWFI?=
 =?utf-8?B?V2s3UUQxWWlBdko0SzdndGdQNmhTSTFYV0MyZzV0NG5tL1FtQU1KWkt4OXU0?=
 =?utf-8?B?SWhvT1BTaFh6MWJPY1F6T1ZVUk1zUUQ5dUJ0OTFFWm5iTmJuT2E5SGpieUl6?=
 =?utf-8?B?UCtHb2tZOExOQ1hRZTRLak9CWTEreHdFb1N2T0ZDWmpJNWkrZXJlZjFVOEJE?=
 =?utf-8?B?SVJkOHhjTjUwYmc5dkVNMk1uWlVQSGVETStZTFd6cGRpNEtWcHRQZ2FxY01Z?=
 =?utf-8?B?NVJTUElXRnJaN1U2d2ZqMy9UV0h0WDZ2c2U0Uml0WVdVNTBTaS9Ga00xMFUx?=
 =?utf-8?B?R0FsTjl4R2xjV2tHQjFnYWNnZ1NyTW9Fb1p6NStRWWFPbCtFRUNkTkx2Ym1T?=
 =?utf-8?B?WmFTQmJjQjk5ei9kTWZZUms5NXR2YzhvaXEzTUZJdlZrZ3NKUFFWWjJwaWlz?=
 =?utf-8?B?cXB3dXNQWStTNm1wWGUyNEU2MWxMWjdwUHh0TDkyUVZxWEd4U0dPUjJUZG5N?=
 =?utf-8?B?K3BNejRRZE5UR2pvTlNFNzZ5aHNxcDFvTGIrWE5qdUFDUkV3OEx0RjNqZ3dq?=
 =?utf-8?B?SkE9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: e1af9638-c95b-4f81-5a50-08dc595ea1f6
X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB8718.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Apr 2024 13:03:35.9970
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 1BdVvr2fxK+z+qD0cWVxvr+WFxrTiWwipDm75fF3XatCuE9hUn+MBh2khIJJtVuQJDGtsW0murJa8qGXtR6jK1vklVXzozVNTamnsxHTh54=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV3PR11MB8696
X-OriginatorOrg: intel.com

From: Przemek Kitszel <przemyslaw.kitszel@intel.com>
Date: Wed, 10 Apr 2024 15:01:32 +0200

> On 4/10/24 13:49, Alexander Lobakin wrote:
>> From: Przemek Kitszel <przemyslaw.kitszel@intel.com>
>> Date: Tue, 9 Apr 2024 12:58:33 +0200
>>
>>> On 4/8/24 11:09, Alexander Lobakin wrote:
>>>> From: Przemek Kitszel <przemyslaw.kitszel@intel.com>
>>>> Date: Fri, 5 Apr 2024 12:32:55 +0200
>>>>
>>>>> On 4/4/24 17:44, Alexander Lobakin wrote:
>>>>>> Add a couple intuitive helpers to hide Rx buffer implementation
>>>>>> details
>>>>
>>>> [...]
>>>>
>>>>>> +struct libeth_fqe {
>>>>>> +    struct page        *page;
>>>>>> +    u32            offset;
>>>>>> +    u32            truesize;
>>>>>> +} __aligned_largest;
>>>>>> +
>>>>>> +/**
>>>>>> + * struct libeth_fq - structure representing a buffer queue
>>>>>> + * @fp: hotpath part of the structure
>>>>>> + * @pp: &page_pool for buffer management
>>>>>> + * @fqes: array of Rx buffers
>>>>>> + * @truesize: size to allocate per buffer, w/overhead
>>>>>> + * @count: number of descriptors/buffers the queue has
>>>>>> + * @buf_len: HW-writeable length per each buffer
>>>>>> + * @nid: ID of the closest NUMA node with memory
>>>>>> + */
>>>>>> +struct libeth_fq {
>>>>>> +    struct_group_tagged(libeth_fq_fp, fp,
>>>>>> +        struct page_pool    *pp;
>>>>>> +        struct libeth_fqe    *fqes;
>>>>>> +
>>>>>> +        u32            truesize;
>>>>>> +        u32            count;
>>>>>> +    );
>>>>>> +
>>>>>> +    /* Cold fields */
>>>>>> +    u32            buf_len;
>>>>>> +    int            nid;
>>>>>> +};
>>>>>
>>>>> [...]
>>>>>
>>>>> Could you please unpack the meaning of `fq` and `fqe` acronyms here?
>>>>
>>>> Rx:
>>>>
>>>> RQ -- receive queue, on which you get Rx DMA complete descriptors
>>>> FQ -- fill queue, the one you fill with free buffers
>>>>     FQE -- fill queue element, i.e. smth like "iavf_rx_buffer" or
>>>> whatever
>>>>
>>>> Tx:
>>>>
>>>> SQ -- send queue, the one you fill with buffers to transmit
>>>>     SQE -- send queue element, i.e. "iavf_tx_buffer"
>>>> CQ -- completion queue, on which you get Tx DMA complete descriptors
>>>>
>>>> XDPSQ, XSkRQ etc. -- same as above, but for XDP / XSk
>>>>
>>>> I know that rxq, txq, bufq, complq is more common since it's been used
>>>> for years, but I like these "new" ones more :>
>>>>
>>>
>>> Thank you, that sounds right. If you happen to sent v10, a bit of code
>>> comment with this info would be useful ;)
>>
>> The current kdoc in front of each struct and function declaration is not
>> enough? :D
>>
>> Thanks,
>> Olek
> 
> I've asked my initial question just after reading it thrice, without
> your reply `FQE` was as meaningful as `ABC`

In the commit:

+ * struct libeth_fqe - structure representing an Rx buffer

:z

or you want me to expand FQ, FQE etc. abbrevs in the kdoc?

Thanks,
Olek