Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.31 as permitted sender) client-ip=23.128.96.31;
Date:   Tue, 10 Oct 2023 18:35:31 +0200
From:   Larysa Zaremba <larysa.zaremba@intel.com>
To:     Stanislav Fomichev <sdf@google.com>
CC:     <bpf@vger.kernel.org>, Martin KaFai Lau <martin.lau@linux.dev>,
        "Alexei Starovoitov" <ast@kernel.org>,
        Daniel Borkmann <daniel@iogearbox.net>,
        Andrii Nakryiko <andrii@kernel.org>,
        Song Liu <song@kernel.org>,
        "Yonghong Song" <yonghong.song@linux.dev>,
        John Fastabend <john.fastabend@gmail.com>,
        KP Singh <kpsingh@kernel.org>, Hao Luo <haoluo@google.com>,
        Jiri Olsa <jolsa@kernel.org>, <linux-kernel@vger.kernel.org>,
        Maciej Fijalkowski <maciej.fijalkowski@intel.com>
Subject: Re: [PATCH bpf-next] selftests/bpf: add options and frags to
 xdp_hw_metadata
Message-ID: <ZSV9U0ti8otbZKen@lzaremba-mobl.ger.corp.intel.com>
References: <20231009160520.20831-1-larysa.zaremba@intel.com>
 <ZSQvMr3-lY9uTzn_@google.com>
 <ZST8OTwh+6y1S170@lzaremba-mobl.ger.corp.intel.com>
 <ZSV2HwOhuNr3XLbv@google.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <ZSV2HwOhuNr3XLbv@google.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?kygrb6Q94iQdQ0TedGcohm6r1zrH28ZpA5IrNLoD2Sg7crcaTcMOj9VnNxUU?=
 =?us-ascii?Q?2GVkIy324vW/OnSM5sjKni1vGbZgXFtTV1pWB1gXVd6yior0UGDdJv3NvkUY?=
 =?us-ascii?Q?tb4PI30TEJILBx3sQuqEX1TSNtuCNQEZPhScqgpczo0ZDbZy7Bf8GObSaKJO?=
 =?us-ascii?Q?ZJ0Ik8T3B3SEdJiNTzBGm84TFfxWU7fI4DG7LG+vdUscWqKpJ1Z03QNYqsQM?=
 =?us-ascii?Q?B3YakhJKk8wrnhL2QKpBWJEXSzvxQLfB1wqf+3b+GeX9izAQmWrGE7wJLk+a?=
 =?us-ascii?Q?0+9m+lb1tCDLaahkHa6lLQ0ROCp240rRwCZNielysoxl5/t0n65wRxYwMk7q?=
 =?us-ascii?Q?CAp4Bidpd3Ntk3OKUfGfi2VBga8cTDLqu7nFIe2ilo0gwnu45hulWGTUKzPg?=
 =?us-ascii?Q?4vuxkY6z8S+3XbF2qFHPgDBgQ9xhbLz4UHaAgJpDqMNKpa2SWnvlQS6YTTB2?=
 =?us-ascii?Q?pMEzcBoQ/xCvwnNivYfsFDCBLkFGag399IJR1cEVCl1IFKJciV7lBjXo8X2Z?=
 =?us-ascii?Q?TK+p6aK0huqc1axmXPbMx0Y2ho7RAZ5OK67fyA2vgZdKama7B9EJn/ccMYwe?=
 =?us-ascii?Q?6kB3YD55vJLA+7HKoc35mjGCNQWitJ2+xLwx6xqYmyhXRw5wNXLOUK9RZk1l?=
 =?us-ascii?Q?e7oUT0qNgrhSFyNhZHRjOQtoMhqeOhPCamfrBeMB54CBUSbDEUSsGvql5+Dq?=
 =?us-ascii?Q?Ixu1SLOEf8xKQmQ5HOtqoWZhxyE+1Dl+ogpcD+gRwH17mqnEH99sSb+V0Xp/?=
 =?us-ascii?Q?3ZhTKk8RcgIW94iXauRz6Zz6vXVKTTWYfuD/eURp1GFfVb1VmfxdZbQ9VRXY?=
 =?us-ascii?Q?nS7kCWQzTwP4j02e+p6zXzDqg2Xk3v0ed2giEic4KF4g82kI6vRBWOJhlhms?=
 =?us-ascii?Q?GVWa66Mv4Q28TARxuVLub6qZOGBGTkwO0j9+zX/xjs+RAkoZUZFVpQ/Jga7c?=
 =?us-ascii?Q?GNkp0NOZYDljF60xUgwBzENgxMbNMAQ66DFnkP5neTptbEh+E0voywo+7VA5?=
 =?us-ascii?Q?pNO+XIFsh4AJxKsSkeoHCu8jRLhZ8tXik8lEqBIpUmwfrOAbsulj3olxnQT7?=
 =?us-ascii?Q?aGYyD+m2ZGjQS34LmyHB3/A/eGyzcRZgMHIWHSpXOuOWGw36i4Co0D3E1B/N?=
 =?us-ascii?Q?bsqoQADcAyPvIxBpfTNrX+mg/OcsF8Iv129JXIfUSAV+wIy3tH69Z82tAsU8?=
 =?us-ascii?Q?+A9kJ+mw3AH2IWpJT2PZnY0+3aS09iO0RmDx8K8ZCTU+T1JGUiiKkQiFCx3p?=
 =?us-ascii?Q?dxiWg9sE+Y6nrAtnPnSjDnlcvSA31bLkWVxcCKLHHpo7I/1XKms5190AMK6P?=
 =?us-ascii?Q?C9/fx32VAPXcjTIJtwUDVycfHWdezjdgeBZOveZN6iXIAkCd6gnwSMhRtHRX?=
 =?us-ascii?Q?QZuKxCBFL09MQVT1vCQ/XTNNQRuqYIyZBhQ0TLAsJ9H3r07qHxVPZv4FLmTN?=
 =?us-ascii?Q?M4e8suw/9GktvM9M2b1U6yJIFbccX80lhBG5dMFREzJFRPNuWLPNH2sWlkei?=
 =?us-ascii?Q?wdDpjaa5mUm6RcOCkqOzGXqw7JWSngxCpzQXSsAEeUVXe9F4+y7/KlBJedf+?=
 =?us-ascii?Q?myt3zd6M0RKjolexqWIwbQDtZzReNck5q0SfR25y6zh7QwE6ZJ+I4LQdL9YI?=
 =?us-ascii?Q?gw=3D=3D?=
X-MS-Exchange-CrossTenant-Network-Message-Id: f6e4b677-7859-4837-2244-08dbc9aef06f
X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7529.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Oct 2023 16:35:39.9472
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Kl1dz4dPZHp9TFIJ43aAWq8oXTSyWvTJ6m6m8Lc/7MhEC2/wx3qUYn2bd9e8k2pnL4C1PisF5lN69NPlk+8i1kQSojjyEgDG5YHSEa+AJ7Q=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM3PR11MB8733
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=2.7 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Tue, 10 Oct 2023 09:36:01 -0700 (PDT)
X-Spam-Level: **

On Tue, Oct 10, 2023 at 09:04:47AM -0700, Stanislav Fomichev wrote:
> On 10/10, Larysa Zaremba wrote:
> > On Mon, Oct 09, 2023 at 09:49:54AM -0700, Stanislav Fomichev wrote:
> > > On 10/09, Larysa Zaremba wrote:
> > > > This is a follow-up to the commit 9b2b86332a9b ("bpf: Allow to use kfunc
> > > > XDP hints and frags together").
> > > > 
> > > > The are some possible implementations problems that may arise when
> > > > providing metadata specifically for multi-buffer packets, therefore there
> > > > must be a possibility to test such option separately.
> > > > 
> > > > Add an option to use multi-buffer AF_XDP xdp_hw_metadata and mark used XDP
> > > > program as capable to use frags.
> > > > 
> > > > As for now, xdp_hw_metadata accepts no options, so add simple option
> > > > parsing logic and a help message.
> > > > 
> > > > For quick reference, also add an ingress packet generation command to the
> > > > help message. The command comes from [0].
> > > > 
> > > > Example of output for multi-buffer packet:
> > > > 
> > > > xsk_ring_cons__peek: 1
> > > > 0xead018: rx_desc[15]->addr=10000000000f000 addr=f100 comp_addr=f000
> > > > rx_hash: 0x5789FCBB with RSS type:0x29
> > > > rx_timestamp:  1696856851535324697 (sec:1696856851.5353)
> > > > XDP RX-time:   1696856843158256391 (sec:1696856843.1583)
> > > > 	delta sec:-8.3771 (-8377068.306 usec)
> > > > AF_XDP time:   1696856843158413078 (sec:1696856843.1584)
> > > > 	delta sec:0.0002 (156.687 usec)
> > > > 0xead018: complete idx=23 addr=f000
> > > > xsk_ring_cons__peek: 1
> > > > 0xead018: rx_desc[16]->addr=100000000008000 addr=8100 comp_addr=8000
> > > > 0xead018: complete idx=24 addr=8000
> > > > xsk_ring_cons__peek: 1
> > > > 0xead018: rx_desc[17]->addr=100000000009000 addr=9100 comp_addr=9000 EoP
> > > > 0xead018: complete idx=25 addr=9000
> > > > 
> > > > Metadata is printed for the first packet only.
> > > > 
> > > > [0] https://lore.kernel.org/all/20230119221536.3349901-18-sdf@google.com/
> > > > 
> > > > Signed-off-by: Larysa Zaremba <larysa.zaremba@intel.com>
> > > > ---
> > > >  .../selftests/bpf/progs/xdp_hw_metadata.c     |  2 +-
> > > >  tools/testing/selftests/bpf/xdp_hw_metadata.c | 92 ++++++++++++++++---
> > > >  2 files changed, 79 insertions(+), 15 deletions(-)
> > > > 
> > > > diff --git a/tools/testing/selftests/bpf/progs/xdp_hw_metadata.c b/tools/testing/selftests/bpf/progs/xdp_hw_metadata.c
> > > > index 63d7de6c6bbb..8767d919c881 100644
> > > > --- a/tools/testing/selftests/bpf/progs/xdp_hw_metadata.c
> > > > +++ b/tools/testing/selftests/bpf/progs/xdp_hw_metadata.c
> > > > @@ -21,7 +21,7 @@ extern int bpf_xdp_metadata_rx_timestamp(const struct xdp_md *ctx,
> > > >  extern int bpf_xdp_metadata_rx_hash(const struct xdp_md *ctx, __u32 *hash,
> > > >  				    enum xdp_rss_hash_type *rss_type) __ksym;
> > > >  
> > > > -SEC("xdp")
> > > > +SEC("xdp.frags")
> > > >  int rx(struct xdp_md *ctx)
> > > >  {
> > > >  	void *data, *data_meta, *data_end;
> > > > diff --git a/tools/testing/selftests/bpf/xdp_hw_metadata.c b/tools/testing/selftests/bpf/xdp_hw_metadata.c
> > > > index 17c980138796..25225720346b 100644
> > > > --- a/tools/testing/selftests/bpf/xdp_hw_metadata.c
> > > > +++ b/tools/testing/selftests/bpf/xdp_hw_metadata.c
> > > > @@ -26,6 +26,7 @@
> > > >  #include <linux/sockios.h>
> > > >  #include <sys/mman.h>
> > > >  #include <net/if.h>
> > > > +#include <ctype.h>
> > > >  #include <poll.h>
> > > >  #include <time.h>
> > > >  
> > > > @@ -49,19 +50,29 @@ struct xsk {
> > > >  struct xdp_hw_metadata *bpf_obj;
> > > >  struct xsk *rx_xsk;
> > > >  const char *ifname;
> > > > +bool use_frags;
> > > >  int ifindex;
> > > >  int rxq;
> > > >  
> > > >  void test__fail(void) { /* for network_helpers.c */ }
> > > >  
> > > > -static int open_xsk(int ifindex, struct xsk *xsk, __u32 queue_id)
> > > > +static struct xsk_socket_config gen_socket_config(void)
> > > >  {
> > > > -	int mmap_flags = MAP_PRIVATE | MAP_ANONYMOUS | MAP_NORESERVE;
> > > > -	const struct xsk_socket_config socket_config = {
> > > > +	struct xsk_socket_config socket_config = {
> > > >  		.rx_size = XSK_RING_PROD__DEFAULT_NUM_DESCS,
> > > >  		.tx_size = XSK_RING_PROD__DEFAULT_NUM_DESCS,
> > > >  		.bind_flags = XDP_COPY,
> > > >  	};
> > > > +
> > > > +	if (use_frags)
> > > > +		socket_config.bind_flags |= XDP_USE_SG;
> > > > +	return socket_config;
> > > > +}
> > > 
> > > nit: why not drop const from socket_config and add this 'if (use_frags)'
> > > directly to open_xsk? Not sure separate function really buys us anything?
> > >
> > 
> > Considering there will also be ZC/copy option, I thought it would be good to 
> > separate socket config creation. After giving this a sencond thought though, 
> > for now options would control bind_flags only. What do you this about removing 
> > gen_socket_config(), but introducing get_bind_flags()?
> 
> In my pending series [0] I ended up adding bind_flags argument
> to open_xsk. Maybe do the same here? This also lets you drop
> global use_frags (if you move option parsing directly into main).
> 
> Or maybe add global bind_flags if you want to keep separate parsing
> routine (read_args)? Doesn't seem like we get anything by storing
> separate use_flags/use_copy and then construct bind_flags via extra
> get_bind_flags()?
>

I like the option with global bind_flags.
  
> 0: https://lore.kernel.org/bpf/20231003200522.1914523-10-sdf@google.com/