Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Shenwei Wang <shenwei.wang@nxp.com>
To:     Andrew Lunn <andrew@lunn.ch>
CC:     "David S. Miller" <davem@davemloft.net>,
        Eric Dumazet <edumazet@google.com>,
        Jakub Kicinski <kuba@kernel.org>,
        Paolo Abeni <pabeni@redhat.com>,
        Alexei Starovoitov <ast@kernel.org>,
        Daniel Borkmann <daniel@iogearbox.net>,
        Jesper Dangaard Brouer <hawk@kernel.org>,
        John Fastabend <john.fastabend@gmail.com>,
        "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "imx@lists.linux.dev" <imx@lists.linux.dev>
Subject: RE: [EXT] Re: [PATCH 1/1] net: fec: add initial XDP support
Thread-Topic: [EXT] Re: [PATCH 1/1] net: fec: add initial XDP support
Thread-Index: AQHY6K4PPMwFL4lhXUy4OBl+zI+fJK4fq0QAgAHOQoA=
Date:   Thu, 27 Oct 2022 01:50:54 +0000
Message-ID: <PAXPR04MB91856F5DC9B273037BBCB28789339@PAXPR04MB9185.eurprd04.prod.outlook.com>
References: <20221025201156.776576-1-shenwei.wang@nxp.com>
 <Y1heXnxgA6e5T8sr@lunn.ch>
In-Reply-To: <Y1heXnxgA6e5T8sr@lunn.ch>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?MYV6JaRMHrf5nSRM1ZdO4aZeiq1ikCV/sRO+M9nD0BvxQZPhSeMSI5xaaWI0?=
 =?us-ascii?Q?zA6vBQqONw6Cxs/FAhJX8QAZWIvD07j/wBYtUOE7APL4S8M9J06P+mW6dEaq?=
 =?us-ascii?Q?S9czz0stoKSxZDpf4zBmEK6tm1c3R0VzmbQL/D8uiZRLWMFv+J4V6q6LuRi6?=
 =?us-ascii?Q?NprDETV5ZNX1SyPIym2s2EkCeFq/MnHtl/+50bfk4tLpD+LfDO3P1mlhx2A4?=
 =?us-ascii?Q?nlfXpk2dAMsb1iVavK3S8x/XWMpZA2uIe4bT77+ykixi6Ri8MSuZ3sWacIMh?=
 =?us-ascii?Q?9tf8BTWDxhnA9MirvVWDfcHys+eb4ZhizHQE34ApgMv24w6a5IGCaRELTHpA?=
 =?us-ascii?Q?sdA8NT0F+jnPa5m/sv2JO+ulTelq0i5B7uj6T5eBWcRnt/CNa+KsiqBdn4Nu?=
 =?us-ascii?Q?RlRbhesyqs22iuc/SlbMGdwHvXEwc6Ls1YXvymtlOeVNcyxT96Et0/dNXAFK?=
 =?us-ascii?Q?byKfoBdOKf4uA+yVuY2YVp6ceDuG2T839+RKtUMmEjL3qAd8/W8IDUP098w1?=
 =?us-ascii?Q?QjW/g+4YmWUuxioIGzPzG2VvtBemAisPbzh1Xiv+R13w2VWdJZsjcXCTEzrt?=
 =?us-ascii?Q?jwg+2Zz1oGJZXy8J2XQ5F5PFEUeY6Rd9fZj4uHGRDWE2yxJruyEXhMskZAk4?=
 =?us-ascii?Q?bhRO3jDYN6XrELWHa0DE93weRnsoHfbUs92S9LTWlF5gb91rMdc6LXWNedMs?=
 =?us-ascii?Q?9qFmAiJ6mXj0y4mucdBZl3/OLa7AsCZDG1rgQdHmTraKTjyoYRcb4FLM4O8Z?=
 =?us-ascii?Q?+XkBrUvvWlvIyRhBxQzyipCc8fD2ttjFq0JWiixaQCovTJbuhH6jM/RijxD9?=
 =?us-ascii?Q?1OWBkscPaE9oo2oHAcakZcajQ0InfluJzmvBP4Pdr7vrmsQboQNbgYX2ab9j?=
 =?us-ascii?Q?WWD7EJR7oQXm52oBS/r2zhJh+GCDU/IePqK2I/aTeA1H/n6nqNpd/AkviYHf?=
 =?us-ascii?Q?ajcVs/4KgDeqyq62HqgQWVdPR9Q56zenx6YWUDQ9Rk6+sLOBQy9XKo8XQCaS?=
 =?us-ascii?Q?tZM6eYz3zoWne06M61TAf889zHvkk/FqrNGQceopxyrj2suU+CqHnnkYVM2D?=
 =?us-ascii?Q?VIgR6BlcT/Fvn9OJL0jJ1hPy3iDylIe8BqENnA5l6iXTSky8cboX7rBDXDk+?=
 =?us-ascii?Q?5TqP/dsVx3rt7EphiVZB+es7Yi0iR9pwHiT1hoe7DPj2tHYY7LXwbc8iKI4r?=
 =?us-ascii?Q?+wX7Cf8tE93f9MHF63syajdiQN0qY9b5+sPrDw1SNHO7GRGWAWzOtuvyXOpw?=
 =?us-ascii?Q?EHr2qXf78jvkhPyE2H76rxxbtTD2nJIVREctRYIaZe/4yaEyB805M28YAYyB?=
 =?us-ascii?Q?7igwym5/EkB/2dUMgFaff4wlYm33pxQhK2AFE/mQOUjCB8Yajx7zxfWP/XtL?=
 =?us-ascii?Q?QhEhgLsFr0LBsZnm0UHu0x+NL+dE4XYUuKbEj8GkpN32OvL83J8vchb3+zKq?=
 =?us-ascii?Q?C93nuljM8+wMfmDuiAMYFnWKe3L0Fk3dlpvRlQ3nDjlao+7YmkmiK9seiv4l?=
 =?us-ascii?Q?KoPZ5W7A6FFFFggcHvQx955BUDY9VM+AKFmCM3/94D9NzoZDHJS0y3RFna0r?=
 =?us-ascii?Q?OFXoZwCLJXVJ07HUNpzMrxhcXzOL3c6PumrSU9IQ?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: nxp.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: PAXPR04MB9185.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 24b2c399-2fcd-46c8-f49d-08dab7bdaf5e
X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Oct 2022 01:50:54.5128
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: fvYcOONOVVPxODA2Ivg09AbUQUhezTunNHblAjoJ6eoWbIUMiUjn9ilCekJihjUna5qTSXupPy+fCpAT3cg+Ng==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR04MB8335
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_PASS autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


> -----Original Message-----
> From: Andrew Lunn <andrew@lunn.ch>
> Sent: Tuesday, October 25, 2022 5:09 PM
> To: Shenwei Wang <shenwei.wang@nxp.com>
> Cc: David S. Miller <davem@davemloft.net>; Eric Dumazet
> <edumazet@google.com>; Jakub Kicinski <kuba@kernel.org>; Paolo Abeni
> <pabeni@redhat.com>; Alexei Starovoitov <ast@kernel.org>; Daniel Borkmann
> <daniel@iogearbox.net>; Jesper Dangaard Brouer <hawk@kernel.org>; John
> Fastabend <john.fastabend@gmail.com>; netdev@vger.kernel.org; linux-
> kernel@vger.kernel.org; imx@lists.linux.dev
> Subject: [EXT] Re: [PATCH 1/1] net: fec: add initial XDP support
>=20
> Caution: EXT Email
>=20
> > +#define FEC_ENET_XDP_PASS          0
> > +#define FEC_ENET_XDP_CONSUMED      BIT(0)
> > +#define FEC_ENET_XDP_TX            BIT(1)
> > +#define FEC_ENET_XDP_REDIR         BIT(2)
>=20
> I don't know XDP, so maybe a silly question. Are these action mutually ex=
clusive?
> Are these really bits, or should it be an enum?
> fec_enet_run_xdp() does not combine them as bits.
>=20
The bit here is to record the states that may required after completing the=
 XDP processing.
As the current implementation for XDP is not full, the other bit like FEC_E=
NET_XDP_TX is not=20
used for now. Generally it will require an extra action if a FEC_ENET_XDP_T=
X is returned.=20
Because we are processing a batch of packets together, those bits may get c=
ombined. It will=20
then responds to each bit accordingly.

> > +static int fec_enet_bpf(struct net_device *dev, struct netdev_bpf
> > +*bpf) {
> > +     struct fec_enet_private *fep =3D netdev_priv(dev);
> > +     bool is_run =3D netif_running(dev);
>=20
> You have the space, so maybe call it is_running.
>=20
> > +     struct bpf_prog *old_prog;
> > +     unsigned int dsize;
> > +     int i;
> > +
> > +     switch (bpf->command) {
> > +     case XDP_SETUP_PROG:
> > +             if (is_run)
> > +                     fec_enet_close(dev);
>=20
> fec_net_close() followed by fec_enet_open() is pretty expensive.  The PHY=
 is
> stopped and disconnected, and then connected and started. That will proba=
bly
> trigger an auto-neg, which takes around 1.5 seconds before the link is up=
 again.
>=20
> Maybe you should optimise this. I guess the real issue here is you need t=
o resize
> the RX ring. You need to be careful with that anyway. If the machine is u=
nder
> memory pressure, you might not be able to allocate the ring, resulting in=
 a
> broken interface. What is recommended for ethtool --set-ring is that you =
first
> allocate the new ring, and if that is successful, free the old ring. If t=
he allocation
> fails, you still have the old ring, and you can safely return -ENOMEM and=
 still
> have a working interface.
>=20
> So i think you can split this patch up into a few parts:
>=20
> XDP using the default ring size. Your benchmarks show it works, its just =
not
> optimal. But the resulting smaller patch will be easier to review.
>=20
> Add support for ethtool set-ring, which will allow you to pick apart the =
bits of
> fec_net_close() and fec_enet_open() which are needed for changing the rin=
gs.
> This might actually need a refactoring patch?
>=20

That sounds good. Let me think about it.

Thanks,
Shenwei

> And then add support for optimal ring size for XDP.
>=20
>     Andrew