Message-ID: <85d803a2-a315-809a-5eff-13892aff5401@intel.com>
Date:   Wed, 15 Mar 2023 15:58:51 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.8.0
Subject: Re: [PATCH bpf-next v3 3/4] xdp: recycle Page Pool backed skbs built
 from XDP frames
Content-Language: en-US
To:     Jesper Dangaard Brouer <jbrouer@redhat.com>
CC:     Alexei Starovoitov <ast@kernel.org>,
        Daniel Borkmann <daniel@iogearbox.net>,
        Andrii Nakryiko <andrii@kernel.org>,
        Martin KaFai Lau <martin.lau@linux.dev>, <brouer@redhat.com>,
        Maciej Fijalkowski <maciej.fijalkowski@intel.com>,
        Larysa Zaremba <larysa.zaremba@intel.com>,
        =?UTF-8?Q?Toke_H=c3=b8iland-J=c3=b8rgensen?= <toke@redhat.com>,
        Song Liu <song@kernel.org>,
        Jesper Dangaard Brouer <hawk@kernel.org>,
        John Fastabend <john.fastabend@gmail.com>,
        Menglong Dong <imagedong@tencent.com>,
        "Mykola Lysenko" <mykolal@fb.com>,
        "David S. Miller" <davem@davemloft.net>,
        "Jakub Kicinski" <kuba@kernel.org>,
        Eric Dumazet <edumazet@google.com>,
        Paolo Abeni <pabeni@redhat.com>, <bpf@vger.kernel.org>,
        <netdev@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
        Freysteinn Alfredsson <Freysteinn.Alfredsson@kau.se>
References: <20230313215553.1045175-1-aleksander.lobakin@intel.com>
 <20230313215553.1045175-4-aleksander.lobakin@intel.com>
 <11e480dd-7969-7b58-440e-3207b98d0ac5@redhat.com>
From:   Alexander Lobakin <aleksander.lobakin@intel.com>
In-Reply-To: <11e480dd-7969-7b58-440e-3207b98d0ac5@redhat.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZkFkZGhGSHJ0Um5UOEdXWTVaektqYjg4WDFzakhoUGRNS2pYTCtESkhkRXRC?=
 =?utf-8?B?Ymsydmo1QXg3d0RRVzRzNVlGcXBTVHpvZm9hSDI2UnBuQW9tZ0t3VUsxUTly?=
 =?utf-8?B?SXA3cHhBLzl3bUN3ZnYzZ0ZBTVIrK3hhVUMvdzNVMDhFWFh4THVQUFlMQ2Qw?=
 =?utf-8?B?NjVyL2h3MEVIM0orOWxFRHRGNzNwQ3MxT3o5emgrUWpaVlk5Y0pQcUpQR293?=
 =?utf-8?B?Z01uQlFaa0xubXFROHExZW1lQm1NUGdiZnhIUmRsdmt3N3FpakhlMm5uRTVi?=
 =?utf-8?B?Y09vY3ZLZkdJOHA4c3ZjdG0raDFlM1d2dDZYaVhQOWphUHJqUkxSMTd2dHd1?=
 =?utf-8?B?RmgwUzlnTFBuczVIVWNGUEJOV2lNLzd6YzhuWDJuVjFJY0p5b01VUVR0SFBN?=
 =?utf-8?B?aHZVTFFoK3JMTXBvbUxSbUxWby9HaGNUOWtNK1FyNXpFRWluUkdqK2VYUHNz?=
 =?utf-8?B?R2RSSkdwSkJRa3RWOVZaTS8rS0phY0tHc3pqeFFBbFRuNDhDMnBaWFhmMHg2?=
 =?utf-8?B?dkw5aDU1dXNLWnhaaitGWTcweE05cXRTQ012SGRZOERGN3lZalZhZnB6Uith?=
 =?utf-8?B?VGZtejNDZ0tXZnJoUmlJajdWVTMrTHIxSSttVzF6VDVGTVJFMy9Oa05aZmYr?=
 =?utf-8?B?YXN3T09PaW1tZ0tlTjlGelphRlZLVnBrUkE3KzdvUG56VEJrTnlxdHg3UUkv?=
 =?utf-8?B?WWJGckVHb2xzTElHVzYvdTdrOXdKT0tpcUpkcXpROWRjbGo4VDhsd1pQRStK?=
 =?utf-8?B?TGxZZDg0TGlidExUMFZIMkFmcHczeHpBOUZWV3lseG1qOVZFb1FrTG9Ic2lq?=
 =?utf-8?B?R1lIRjVJYjVFNUUyREZpb1BtRUluRjRCY3FpbWpQdi9vMmozNVEwb20zZ2x3?=
 =?utf-8?B?NzA4VHB5VWJsSDdvOEVOdnh3ZS9YMEl5OGxqYU5xVzIvYi8vTjRnM0FDT2VF?=
 =?utf-8?B?YW8zdkV3b1lOK2l3M2MxaDlodkVaUm54T0ZYWlBKbzZPb0trdUlMd1FoUmFY?=
 =?utf-8?B?Vkp4K1dyamF4SUw4T250eEc5MHJrOGhjc1VxT2YzMGxMS0tnM3JBdWJlZFdN?=
 =?utf-8?B?Qks1Yk5WSVZXeWRVdC9JVlVzMjZPdDYwanl6L2hEU2RmU0gvdVNpdU53ZlRD?=
 =?utf-8?B?aE5rOXp4ZklVRFhUWHF1aEtqSFhRWUt1Sms3UzlOVTd0Zmx6QUJDQmgxa2l2?=
 =?utf-8?B?Nm44dy9DTTUzMzVlZEIxU2Roa0RPZTVDam9RaFBreTVwL1o2Y04zaEs2YmF1?=
 =?utf-8?B?bHRlZ1QwNDZOTnVmRXkrV2ZOWnlIb25GYzRFeFRkTzJjRWszRjgrNDNEcTVP?=
 =?utf-8?B?dzF0L0p2ZnFwbjQ2cHRNaVdwa0Y3R2Z5TWcxL2dJOExtK1J3WHlJTFlqL0k1?=
 =?utf-8?B?ZFpoM0NPY3lQUDJTVENZZzRDZUhlMzJmT0xySkM3SXplWTZFU2h0MXFxQTRq?=
 =?utf-8?B?ajk3cGI3UVdCWlN6ZUxJQnd2SlRDSURDVDBkZHMxMTZQY0dPdFBxemRHU212?=
 =?utf-8?B?czgwc25VdTQyQmRONEtncVpMSlJzRTFIbW10cGViZW5DVEJweXpzb3N4SnVO?=
 =?utf-8?B?Umt4a0xJTnRqdzVmSUZQb24xY0pBK2hhUjVDWElOM1FXbnRrOW9SMVlBUHVp?=
 =?utf-8?B?anpmVXhFbzN6Y24xTjlUUG50NXg5YTI4MnFWRmc1Q3pnVGsyWFRuOUtsSVh2?=
 =?utf-8?B?WDB5Y2psWm1VdUJTLy9SaElHbEloLzJCNWF5MUJYQW1XbXA0YjdYbDlNemJQ?=
 =?utf-8?B?SEtXeU1vSXZrczdkQVNjRE9WYnJ2dTA1TzdIMWg0Viswb3EyWGg4akh3RTJQ?=
 =?utf-8?B?V0hSK243UVZWNHVUaWhYT0xMOFlNVms4RzNqZjZUZld5aWRvVC8xMnBMVCtC?=
 =?utf-8?B?UEppbkpVcHhUa3F0NW56MStobFIxSlJWRmFMdWJFc1JjWmorcVdNQzA5TGNl?=
 =?utf-8?B?WlhYMVh2YXA3V0VkZmNJQTVzdXNIa0tUQ1AzSzFiTFR4MXRpME0xeDZpc29n?=
 =?utf-8?B?L3M5ZnovYkZHbExZWmlkbkplK3U3Z0dycy9qSXpDendCb25HTGJXLzhsekRo?=
 =?utf-8?B?ZFF5Rk9RVGdubWhTNmNBdUtyanBWbXY5YWhmWm9CbXJBUVZkOW1JaGd0M0lT?=
 =?utf-8?B?cjhYSnoyNzlVSnF1VnQ4ektvYnRnbThkWDIxMnc2dVBzT0Q0SnY2d3M1enNI?=
 =?utf-8?Q?/ZcWd1hvj+Is7oOAsgGQlUY=3D?=
X-MS-Exchange-CrossTenant-Network-Message-Id: fec07c2b-1cd3-47e8-4f4f-08db2565f84d
X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB3625.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Mar 2023 15:00:09.4391
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: dnlnhr1YDpWJDNMhsIxks5AKHwn+mjOHpGiWVDbyjIz6ZLfBdWRNBPuBEozrRC1KJBq1894v8BCNFuaFgbzai6NwuP+5VB09pqlHBByStDQ=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR11MB8370
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

From: Jesper Dangaard Brouer <jbrouer@redhat.com>
Date: Wed, 15 Mar 2023 15:55:44 +0100

> 
> On 13/03/2023 22.55, Alexander Lobakin wrote:
>> __xdp_build_skb_from_frame() state(d):
>>
>> /* Until page_pool get SKB return path, release DMA here */
>>
>> Page Pool got skb pages recycling in April 2021, but missed this
>> function.
>>
>> xdp_release_frame() is relevant only for Page Pool backed frames and it
>> detaches the page from the corresponding page_pool in order to make it
>> freeable via page_frag_free(). It can instead just mark the output skb
>> as eligible for recycling if the frame is backed by a pp. No change for
>> other memory model types (the same condition check as before).
>> cpumap redirect and veth on Page Pool drivers now become zero-alloc (or
>> almost).
>>
>> Signed-off-by: Alexander Lobakin <aleksander.lobakin@intel.com>
>> ---
>>   net/core/xdp.c | 4 ++--
>>   1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/net/core/xdp.c b/net/core/xdp.c
>> index 8c92fc553317..a2237cfca8e9 100644
>> --- a/net/core/xdp.c
>> +++ b/net/core/xdp.c
>> @@ -658,8 +658,8 @@ struct sk_buff *__xdp_build_skb_from_frame(struct
>> xdp_frame *xdpf,
>>        * - RX ring dev queue index    (skb_record_rx_queue)
>>        */
>>   -    /* Until page_pool get SKB return path, release DMA here */
>> -    xdp_release_frame(xdpf);
>> +    if (xdpf->mem.type == MEM_TYPE_PAGE_POOL)
>> +        skb_mark_for_recycle(skb);
> 
> I hope this is safe ;-) ... Meaning hopefully drivers does the correct
> thing when XDP_REDIRECT'ing page_pool pages.

Safe when it's done by the schoolbook. For now I'm observing only one
syzbot issue with test_run due to that it assumes yet another bunch
o'things I wouldn't rely on :D (separate subthread)

> 
> Looking for drivers doing weird refcnt tricks and XDP_REDIRECT'ing, I
> noticed the driver aquantia/atlantic (in aq_get_rxpages_xdp), but I now
> see this is not using page_pool, so it should be affected by this (but I
> worry if atlantic driver have a potential race condition for its refcnt
> scheme).

If we encounter some driver using Page Pool, but mangling refcounts on
redirect, we'll fix it ;)

> 
>>         /* Allow SKB to reuse area used by xdp_frame */
>>       xdp_scrub_frame(xdpf);
> 

Thanks,
Olek