Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <95c5ba92-bccd-6a9a-5373-606a482e36a3@intel.com>
Date:   Mon, 10 Jul 2023 15:34:28 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.12.0
Subject: Re: [PATCH RFC net-next v4 6/9] iavf: switch to Page Pool
Content-Language: en-US
To:     Yunsheng Lin <yunshenglin0825@gmail.com>
CC:     Yunsheng Lin <linyunsheng@huawei.com>,
        "David S. Miller" <davem@davemloft.net>,
        Eric Dumazet <edumazet@google.com>,
        Jakub Kicinski <kuba@kernel.org>,
        Paolo Abeni <pabeni@redhat.com>,
        Maciej Fijalkowski <maciej.fijalkowski@intel.com>,
        Michal Kubiak <michal.kubiak@intel.com>,
        Larysa Zaremba <larysa.zaremba@intel.com>,
        Alexander Duyck <alexanderduyck@fb.com>,
        David Christensen <drc@linux.vnet.ibm.com>,
        "Jesper Dangaard Brouer" <hawk@kernel.org>,
        Ilias Apalodimas <ilias.apalodimas@linaro.org>,
        Paul Menzel <pmenzel@molgen.mpg.de>, <netdev@vger.kernel.org>,
        <intel-wired-lan@lists.osuosl.org>, <linux-kernel@vger.kernel.org>
References: <20230705155551.1317583-1-aleksander.lobakin@intel.com>
 <20230705155551.1317583-7-aleksander.lobakin@intel.com>
 <6b8bc66f-8a02-b6b4-92cc-f8aaf067abd8@huawei.com>
 <bc495d87-3968-495f-c672-bf1bab38524a@intel.com>
 <4946b9df-66ea-d184-b97c-0ba687e41df8@gmail.com>
From:   Alexander Lobakin <aleksander.lobakin@intel.com>
In-Reply-To: <4946b9df-66ea-d184-b97c-0ba687e41df8@gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?R1hRUFpYelhzR0xvK0lnSVMxRndxL3ozRUdLMXpGK05mRUpPZmRNQzlJSTI2?=
 =?utf-8?B?MDQwMVFERkhyL2Z3Z1FYbHBvT2hyMEIvUUpHall6bnFjSHNxbkp6N05ETGpG?=
 =?utf-8?B?V1hOWTJkS0FOQStFVTZ1bit5a2Z0R2dNd1NPY2xvQUlreGQ0UDB4ZG1EbmtW?=
 =?utf-8?B?L3hhS0hmTTAxVGY0dGx1Nk5XNk1nYWE4MS9kUzQzUHpCUEtRYXdzY3BYTGhU?=
 =?utf-8?B?NlU1aXkvUlJ2YW5aaEM4ZDlNbmxiMGx4TFVHdDlvQlNHazJiRWlPZ0M2dENo?=
 =?utf-8?B?dWRwRWdyK0VCczA5bG9hNU8yWTdGcGNLREFBNnlCenVDVS9oVW55dGRZT3RF?=
 =?utf-8?B?b1EzUCt0TW1IRHQrZG9ZdTlMOU45SjE5Z0dMQnJBbTR4YXVtMTBRaXViVkRQ?=
 =?utf-8?B?S2dvMVdYcVJlTFhYQXdCUm5uT3hBMktzOGorM0h5WWlsNTZSNitoTno1NXNh?=
 =?utf-8?B?b25LWG84MEJ5dDVZa3ZXQThKeFFHaHJ4ZkF0U01hamRUNUpKc1JNWEVkSTVw?=
 =?utf-8?B?SDJ5UTBIa3ZYYlJkV2s2VDk0eVd5WTZhVXpaQWc1eUo2VEZFRlExZFVmVUN6?=
 =?utf-8?B?a3EvNWh0dnR2Y1RBdENiZjloSy9ReDd4aVZFb0M4VWFaUURMOENtWXNlMXhC?=
 =?utf-8?B?OStpMXNOMTJySnRNRHlDLzg1KzlJYWptVUQ5eGd5NGhETjVFZy84alRlUE53?=
 =?utf-8?B?ZVR6b25QcWZWZGhmQ0J4d3RLZnFmWTlUQ25CaEdiTUVrSmxwV3lTOVFJUDRv?=
 =?utf-8?B?aFEyeUJGcnRTaFlBKzRORjZwWEtPNG9EenVPYTFnaGhsN1JJUWVlZTI4dEwz?=
 =?utf-8?B?dEsrcE5uOWI4aVhjcmkxb1NFeHNZVlRQdElvOENBU2x3Q2xDVm9BczNTbHh2?=
 =?utf-8?B?ZnFpR2hOME1aM3A0TEtiMzh1dFlkTDRMbjNCTkFiYXdPdFNGdHYzVG9nKy8r?=
 =?utf-8?B?Y3NBa1REd0E1WUZxQXMvdng0QnNUTXhXTFA0amg5Y2pYOEVmaXd1dHBnMFU5?=
 =?utf-8?B?eW9QSjVMSjgrVDJTa2xNM3JFSXhqNzdRZG13d1E3TER4ZHZJMDVHWjVNWmxw?=
 =?utf-8?B?VmxUUUlTUFNIVFQ5QTBHYkJQY3I1cDJxanlFZHBmdEl3ajJYSEhPRVpKaGxQ?=
 =?utf-8?B?OHcwVWRpSDZDVitPSlFvWmRYRzFNTmZVTEJCSVpkN3lnNk1takhSM3JtYVgw?=
 =?utf-8?B?eTROajNrV1ZKTHBwWUl6WjIrbFkyb1FEWGtDZmlFendYVFd4YUhBZTZLYTBL?=
 =?utf-8?B?ZUNESGlUbmJzM2h1OWVvY2tpeTRSQkkrMHlpU1JLVkdCWkF1VUwvQ1A3ZEs4?=
 =?utf-8?B?aEd6LzExanNuay9sVzNuVGYxVE9kcmoyM3NVbDYyMDQwNk0vWGZLOUZJMmxE?=
 =?utf-8?B?QkNXdFpxSmNra0lnRDAyNWtaL290bjF2SmhTZWlvZ09iajVjRi8yT3Bkb3o4?=
 =?utf-8?B?bkhvQzAxdWFhZHZiZUd2RzlDbmpqb2o0cFk5enl2cXh5c3ZOZEkxL05Fa0N1?=
 =?utf-8?B?K2dKQjVJWTdsOFBhUzJLd21UbmhCaFBvUFlKcVRDckZLSzNMa3ZEUTVJWXZP?=
 =?utf-8?B?aWQyRmEyaVZrWUwzOEl2UVk4WkxCRGNGd2NHVE5BM0xWNmYvT0JBUTZNdXRp?=
 =?utf-8?B?WlJ4ZDREMHF5NU43bkcrTFpza3RZd1JnU3p1OUhRcXhKeHFJV0dMSkd1dDBk?=
 =?utf-8?B?QjZCeVJjeWFSTndKMi80QXNLSnVUbjR3NFV3dnZWbjFEMG5rTlh6VUxxU0FI?=
 =?utf-8?B?SFJMWjY4SllUMkNJazV1OFppT29wZmp3TndJanJFYTh4QW5jZHlLcUJJSlcy?=
 =?utf-8?B?UHkzYnYwSEVseEJMRFY4a2Q5U2d5TXZPMDdyZWVBM0VEUTRRMzA5eDQ2NFBw?=
 =?utf-8?B?VEl4QUFSNit5TTN4Q1I3cUVYb2czWVVQaG9zV2c5NENCN2NvcXgxR2Z4S1Ju?=
 =?utf-8?B?alVsQ3VXQ3gyeGZOZlh3WGVOb2owYW04d3YrUlFyWHhZR1lITVgrTGFyQUlz?=
 =?utf-8?B?YWRmUEFtK0xZRUJCVmZhU29NUGhST2twd3MvUkNiL0lnR3VKL3ZmVmdQZmpn?=
 =?utf-8?B?OHlKOU1iUmdYMmhHOG5DQVp5ZWc4YU9ha0k4ZnU3eEpTOEhDcXBpNXJqdk5Y?=
 =?utf-8?B?VXY3b2hGZkRUS0hMcnExRnpBbGJWNTBMcFZ4Tkc4YTNFZUNRT3k1UVBZVits?=
 =?utf-8?B?T2c9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: b523e307-0173-4889-4747-08db814aab93
X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB3625.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Jul 2023 13:36:31.1641
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: B9Y09MGQ+RpOuOqnso1CcVC6BSgslzH2xpXQ2qhioUZcFTlYJUMZsZJi5kcJ4I0UTeu4iL8ys7prZLgmdi5miEqpdghVSyinrSFg0i3Bt5o=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR11MB7804
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-2.2 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,
        RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

From: Yunsheng Lin <yunshenglin0825@gmail.com>
Date: Sun, 9 Jul 2023 13:16:39 +0800

> On 2023/7/7 0:38, Alexander Lobakin wrote:
> 
> ...
>  
>>>
>>>>  /**
>>>> @@ -766,13 +742,19 @@ void iavf_free_rx_resources(struct iavf_ring *rx_ring)
>>>>   **/
>>>>  int iavf_setup_rx_descriptors(struct iavf_ring *rx_ring)
>>>>  {
>>>> -	struct device *dev = rx_ring->dev;
>>>> -	int bi_size;
>>>> +	struct page_pool *pool;
>>>> +
>>>> +	pool = libie_rx_page_pool_create(&rx_ring->q_vector->napi,
>>>> +					 rx_ring->count);
>>>
>>> If a page is able to be spilt between more than one desc, perhaps the
>>> prt_ring size does not need to be as big as rx_ring->count.
>>
>> But we doesn't know in advance, right? Esp. given that it's hidden in
>> the lib. But anyway, you can only assume that in regular cases if you
>> always allocate frags of the same size, PP will split pages when 2+
>> frags can fit there or return the whole page otherwise, but who knows
>> what might happen.
> 
> It seems intel driver is able to know the size of memory it needs when
> creating the ring/queue/napi/pp, maybe the driver only tell the libie
> how many descs does it use for queue, and libie can adjust it accordingly?

But libie can't say for sure how PP will split pages for it, right?

> 
>> BTW, with recent recycling optimization, most of recycling is done
>> directly through cache, not ptr_ring. So I'd even say it's safe to start
>> creating smaller ptr_rings in the drivers.
> 
> The problem is that we may use more memory than before for certain case
> if we don't limit the size of ptr_ring, unless we can ensure all of
> recycling is done directly through cache, not ptr_ring.
Also not sure I'm following =\

[...]

Thanks,
Olek