Message-ID: <8d3a9feb-9ee5-4a49-330a-9a475e459228@intel.com>
Date:   Fri, 10 Feb 2023 13:29:21 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.7.1
Subject: Re: [PATCH bpf] bpf, test_run: fix &xdp_frame misplacement for
 LIVE_FRAMES
Content-Language: en-US
To:     =?UTF-8?Q?Toke_H=c3=b8iland-J=c3=b8rgensen?= <toke@redhat.com>
CC:     Alexei Starovoitov <ast@kernel.org>,
        Daniel Borkmann <daniel@iogearbox.net>,
        Andrii Nakryiko <andrii@kernel.org>,
        Martin KaFai Lau <martin.lau@linux.dev>,
        Song Liu <song@kernel.org>,
        Jesper Dangaard Brouer <hawk@kernel.org>,
        Jakub Kicinski <kuba@kernel.org>, <bpf@vger.kernel.org>,
        <netdev@vger.kernel.org>, <linux-kernel@vger.kernel.org>
References: <20230209172827.874728-1-alexandr.lobakin@intel.com>
 <87v8ka7gh5.fsf@toke.dk>
From:   Alexander Lobakin <alexandr.lobakin@intel.com>
In-Reply-To: <87v8ka7gh5.fsf@toke.dk>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TlZUaktZM3NEQS9PdkNuUDNYY2NvSSszdjZod21tdHhPdmhiSFphQWdZc1J4?=
 =?utf-8?B?OTY0QjM1cWZHeWtUYmhKbDdNVU9hK20zQk5WbGhlT0I3TkJ0REF0RUtSSDR2?=
 =?utf-8?B?ZkkxTnNTQWRoZS9tRjdVNVJ0VXgxdzkzSk5IbkRSeS9mQTRrR3k3Vms2ZkZ6?=
 =?utf-8?B?TzZQWVQwVEtPYmowTlBqOEhxcVYrU1pQMGczSXdhbEJmNFJkMW1ZaC8yNlh6?=
 =?utf-8?B?ZFJlQkp1TWlxUEZYbjhmWnEwVXZJWnFBeDZaWWh5UDJSRGFrU2srMy9vT3JJ?=
 =?utf-8?B?TDdDNG5TdEVBVXVFTThwVGkzYUpmYzVRckI1VHl4N1NaNEx3N3ZjSGdINkhr?=
 =?utf-8?B?TVhucmE3VTR3ZzJ2V1M0SHNQVHQ1VkFEaU9LcTRnaHBUcjdoYU1uUC9qblFX?=
 =?utf-8?B?TDdJY28zWUZ5L01lM1cvSHVkNlB0TnhuK0pTa3h1ellUM3JZNHY0aG1TY2xJ?=
 =?utf-8?B?a3hScWZRWDZ0VHVXRTIySUpGdW9wd21wM1VDZlNucWxnU0htck1RQld0UUl4?=
 =?utf-8?B?LzV4eldDelVjS0FocmNWeWZMREFvd0Y1TUt1VFpoVGdIRTFnYkVsSkowWmw2?=
 =?utf-8?B?MUNPTW9aTGRVcDRlZnAyVkdyNnBaQkNOQTRaR0Z4SUs5UmxsZjdOV2dBOGF1?=
 =?utf-8?B?OG5tVHAxc0lmOERqQS9vbHhjdXk1NG5KUVhrZlFvWEVVZDk2WDZJMVRZY3E2?=
 =?utf-8?B?elNJZkMwQUZZb1d4MGJrQ0hVdS9xQmFxZ2RWQ0xOd2NUbk9STytCNGxtZEZB?=
 =?utf-8?B?SytDcWtIT3ZOT3ZCRlFOdlg4UHJrRnRFSGNMbGpCY3NZSlpIK1VGRllVa1c1?=
 =?utf-8?B?S2VjSkpDd3BHNzJSamhGOWJEeVZKVDM1M2RnV250endnNGFqbFpSU3RNam02?=
 =?utf-8?B?M1pkWURSclZDd25LRy9FNFlJcEJZcTF1WDkwNDFSbisvRUdHRWlDOWEySkFR?=
 =?utf-8?B?cC9VL1hHby9IZW55clpmV2pWbDUzZW96UndwaGRNUTR3V2FxY3lGbWtydUdX?=
 =?utf-8?B?RmpxaHFUdzhEa0hnVHdsYjNpM3R4d3gvNmh0ZTljaEI2MkcyRExaVjVENEhi?=
 =?utf-8?B?Nnpha3ZUZi9NRGgzemtmMVJFSUtDd2RnOWY4QXRRY0hDME9qb3RGTjYrU0s4?=
 =?utf-8?B?bEpHSHY0Q1ZzeEZSQmpHaHRSSENzcEhWYTNFSW1vNzFBbEpsYzV6WWNBOVM4?=
 =?utf-8?B?UFFiUlZBdFE2RTkrdkdIM1p6amg1NHFmVUNQTUNXdDhYeHFYNE5FUFA2OW5o?=
 =?utf-8?B?Y0pjVDFSdk16cTZFTlhtMHZoWTRPNWU1aC9QbTdhbkhBbnNmenhHSEM4TFNh?=
 =?utf-8?B?aTdkL3pFc0V6eVJTZGtkOTdlczVOQWR2UnZQN21JeXpQYlZJYzBiRXRaYlJy?=
 =?utf-8?B?bDVWRkxEd1lna1BrMDB5NU44US9GRXdNMWtLdWMzUXNkZUNUNDR2WnJSWDdi?=
 =?utf-8?B?cTRPdVN1MzJYUHdkQmlUY1ZaWTkyQTdJSlRpR2thSmRqbnJyTTd0dElnSi9D?=
 =?utf-8?B?OGt0TDlYVmFqOG9lSGlJUFJuRkhwOFh2VUtIZXNYS0ZJTUUxT2w1YnFDUHpG?=
 =?utf-8?B?YUtieHFyZ1hqLzVHODF3MG1XNFp3WW9ac1VCeG9BcWRyT0tpS0dQMno5Uk1p?=
 =?utf-8?B?WXA2VUFJZ29tV29KSU55TnFOcmx0SFR1Qk42eGRtbkI4dk5RbUp4N2FTNVpi?=
 =?utf-8?B?cExCdFJ3RGYrc0tkakZ6MnhJa25Cb2xjSW1VMDZKbmFqK1I3RnRhQXFrVlAv?=
 =?utf-8?B?VUltcFZyblF6RU1CQnJ1V0owVGIvQjR6UGpqZDhnanBoN1RYc0ZvV1Nxcmlz?=
 =?utf-8?B?dEx0dnR5MVl5Y0IyRjVnbmI2QVE3R1VqOXdJeTF1K2ExR3dZV3Y5cGFIVU44?=
 =?utf-8?B?dHVYM09Bei9PRlk0QzFhZ0ppZW5jZWg0RDhJaDBkODZEdTJOeGNNbzJhMmhj?=
 =?utf-8?B?MGxCdGg0MnlqVHFXQkxxakxGZmN5MWZQYnBrbnhKZEkxNjF6MGFMMUVqeTZX?=
 =?utf-8?B?WXRYV21rU0ErS3J0aFphNVZBMGozcno3T29zdzZ3SVRXb3dYYXU5eXRxVEhG?=
 =?utf-8?B?SXlGcDBTVXFOSm1SR2NoR01RRngzSHZpY3QvbTZTSzRrc3hIcnA3Qm5nM0hn?=
 =?utf-8?B?M3VrU1l6MFArMkxKSVFveUR5NG5JRHVKZVVXLzM5WHB0UTRZU0tNNkxvUXpp?=
 =?utf-8?B?Nmc9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 3c80ba93-d604-44c8-b08e-08db0b628682
X-MS-Exchange-CrossTenant-AuthSource: BYAPR11MB3624.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Feb 2023 12:30:00.0406
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ljf1335uN0nXKbuVsRl/oDCKaY9dEhxWqbRqpReOsJ5V/k+rxDWauQ/qVlozagbDiZhlOaBP+CxQ7Oue5sYaC52SfgDS5Zci4jIasKVA1eg=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR11MB7920
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

From: Toke Høiland-Jørgensen <toke@redhat.com>
Date: Thu, 09 Feb 2023 21:04:38 +0100

> Alexander Lobakin <alexandr.lobakin@intel.com> writes:
> 
>> &xdp_buff and &xdp_frame are bound in a way that
>>
>> xdp_buff->data_hard_start == xdp_frame
>>
>> It's always the case and e.g. xdp_convert_buff_to_frame() relies on
>> this.
>> IOW, the following:
>>
>> 	for (u32 i = 0; i < 0xdead; i++) {
>> 		xdpf = xdp_convert_buff_to_frame(&xdp);
>> 		xdp_convert_frame_to_buff(xdpf, &xdp);
>> 	}
>>
>> shouldn't ever modify @xdpf's contents or the pointer itself.
>> However, "live packet" code wrongly treats &xdp_frame as part of its
>> context placed *before* the data_hard_start. With such flow,
>> data_hard_start is sizeof(*xdpf) off to the right and no longer points
>> to the XDP frame.
> 
> Oh, nice find!
> 
>> Instead of replacing `sizeof(ctx)` with `offsetof(ctx, xdpf)` in several
>> places and praying that there are no more miscalcs left somewhere in the
>> code, unionize ::frm with ::data in a flex array, so that both starts
>> pointing to the actual data_hard_start and the XDP frame actually starts
>> being a part of it, i.e. a part of the headroom, not the context.
>> A nice side effect is that the maximum frame size for this mode gets
>> increased by 40 bytes, as xdp_buff::frame_sz includes everything from
>> data_hard_start (-> includes xdpf already) to the end of XDP/skb shared
>> info.
> 
> I like the union approach, however...
> 
>> (was found while testing XDP traffic generator on ice, which calls
>>  xdp_convert_frame_to_buff() for each XDP frame)
>>
>> Fixes: b530e9e1063e ("bpf: Add "live packet" mode for XDP in BPF_PROG_RUN")
>> Signed-off-by: Alexander Lobakin <alexandr.lobakin@intel.com>
>> ---
>>  net/bpf/test_run.c | 13 ++++++++-----
>>  1 file changed, 8 insertions(+), 5 deletions(-)
>>
>> diff --git a/net/bpf/test_run.c b/net/bpf/test_run.c
>> index 2723623429ac..c3cce7a8d47d 100644
>> --- a/net/bpf/test_run.c
>> +++ b/net/bpf/test_run.c
>> @@ -97,8 +97,11 @@ static bool bpf_test_timer_continue(struct bpf_test_timer *t, int iterations,
>>  struct xdp_page_head {
>>  	struct xdp_buff orig_ctx;
>>  	struct xdp_buff ctx;
>> -	struct xdp_frame frm;
>> -	u8 data[];
>> +	union {
>> +		/* ::data_hard_start starts here */
>> +		DECLARE_FLEX_ARRAY(struct xdp_frame, frm);
>> +		DECLARE_FLEX_ARRAY(u8, data);
>> +	};
> 
> ...why does the xdp_frame need to be a flex array? Shouldn't this just be:
> 
>  +	union {
>  +		/* ::data_hard_start starts here */
>  +		struct xdp_frame frm;
>  +		DECLARE_FLEX_ARRAY(u8, data);
>  +	};
> 
> which would also get rid of the other three hunks of the patch?

That was my first thought. However, as I mentioned in between the lines
in the commitmsg, this doesn't decrease the sizeof(ctx), so we'd have to
replace those sizeofs with offsetof() in a couple places (-> the patch
length would be the same). So I went this way to declare that frm
doesn't belong to ctx but to the headroom.
I'm fine either way tho, so up to you guys.

> 
> -Toke
> 
Thanks,
Olek