Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Shenwei Wang <shenwei.wang@nxp.com>
To:     Alexander Lobakin <alexandr.lobakin@intel.com>
CC:     Andrew Lunn <andrew@lunn.ch>,
        "David S. Miller" <davem@davemloft.net>,
        Eric Dumazet <edumazet@google.com>,
        Jakub Kicinski <kuba@kernel.org>,
        Paolo Abeni <pabeni@redhat.com>,
        Alexei Starovoitov <ast@kernel.org>,
        Daniel Borkmann <daniel@iogearbox.net>,
        Jesper Dangaard Brouer <hawk@kernel.org>,
        John Fastabend <john.fastabend@gmail.com>,
        Wei Fang <wei.fang@nxp.com>,
        "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "imx@lists.linux.dev" <imx@lists.linux.dev>,
        kernel test robot <lkp@intel.com>
Subject: RE: [EXT] Re: [PATCH v3 1/1] net: fec: add xdp and page pool
 statistics
Thread-Topic: [EXT] Re: [PATCH v3 1/1] net: fec: add xdp and page pool
 statistics
Thread-Index: AQHY9eM2DcCGX+tMv0uN92sufUZdVq4+cwAAgAAGTYCAAA76UIAABgiAgABeOBCAAriEAIAAFxUQ
Date:   Wed, 16 Nov 2022 16:00:08 +0000
Message-ID: <PAXPR04MB9185FA86AEAF112D06132CC689079@PAXPR04MB9185.eurprd04.prod.outlook.com>
References: <20221111153505.434398-1-shenwei.wang@nxp.com>
 <20221114134542.697174-1-alexandr.lobakin@intel.com>
 <Y3JLz1niXbdVbRH9@lunn.ch>
 <PAXPR04MB91853D935E363E8A7E3ED7BF89059@PAXPR04MB9185.eurprd04.prod.outlook.com>
 <20221114152327.702592-1-alexandr.lobakin@intel.com>
 <PAXPR04MB918589D35F8B10307D4D430E89059@PAXPR04MB9185.eurprd04.prod.outlook.com>
 <20221116143336.3385874-1-alexandr.lobakin@intel.com>
In-Reply-To: <20221116143336.3385874-1-alexandr.lobakin@intel.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?56EgFcZvNamy1GeZSHSWtPMmawF+OBxXqP4LIXlnz9II0mNCS0JEmevJF5aL?=
 =?us-ascii?Q?6mrgebwvoncZ6VYaFzCdUuzOg6802s6Dcy7x015g09MWqCpNkk8tvQj3MN0F?=
 =?us-ascii?Q?iGGKbZubflwTTKYPrieuWKZaANnAqreMT6FJXlTnTZZTV6PbSgRImQkqxkD9?=
 =?us-ascii?Q?5idACVYzOYbq0Uuk1LG2EBh8lvrApGZxGmkArkc2+swWk0yQGJ8oDjiyIoDP?=
 =?us-ascii?Q?zlHZTF06KVD3HB4X0CzUQkJaxAk2z6BJDDimIUaKy8gNXdz8FPrXawjrdrXR?=
 =?us-ascii?Q?oRekp+/piUMvfJeKTu3L8rWNjRnMynEu6UQzavddnwixwwThSLS/W4bAAY68?=
 =?us-ascii?Q?RUKnTzdBSxzQxke9Do+GFfT7e3Ql19QsLtFbFAZxsNJ5hiQDFtGLKEbrgnf0?=
 =?us-ascii?Q?XjxbD1Y8oA2gHaJUDT7T77JwGGh3L4A7kfJJRMjixXOD1N1AeSI72K+VReOM?=
 =?us-ascii?Q?8kUM79UyL7/vi4Lgu04afw7l2nXVC3qsojYXRxAupUjnjNC83K3yJXdxF7by?=
 =?us-ascii?Q?RADmBLZqHDH6I+qV2BbyEy/jwX9wD/QaU6Q/dt1Ms6jq9qQUZZZ7kMPcRNG+?=
 =?us-ascii?Q?ktrMBS017ETTKDAt7iGhgGnYo9e7fc9gxar7zpkTapEGbkTTuP0RbU+cBTkT?=
 =?us-ascii?Q?jVL6nqZAF2cckxCKoi5ztbZzNNEtLi5Ygszut2IUEfGJaknwjkHdiuGoHVJ/?=
 =?us-ascii?Q?sg9jEJ5gu3fG/EWjlNI7zXyQU/ks7kjoeNVNXlJQd4JYRD0oKNeRhhHs9gcS?=
 =?us-ascii?Q?xXINSlXOKAS9CqO/40EzJn6V2geOwgMSYOhv6T/5tP5m9AaPWdLsdE1WAFh6?=
 =?us-ascii?Q?dU/pH/awMdwETj17cE6XzGdq5jtjeInw9Yx8ofJxgE0Vie5o4BtPWBz1wlwJ?=
 =?us-ascii?Q?2csXqX0BsNT9jbN648WmBq2+JPar46jIYaOaAOP5DoWjbCqasJBH9ncoYWbl?=
 =?us-ascii?Q?EIcD6T6bUUGHKePSlN3J35xZbCexWmQBDUNJuKn7OEuydQ6ST6mncvqUj8VC?=
 =?us-ascii?Q?EwyWYfoMIxaPN7om4tKJWkxM37b3GcNRA5ome/0YauFpAq88z9j15kt1stpI?=
 =?us-ascii?Q?YvkeA764jf9RIzqKvzs/6RcJmYtUBWTeRbYzdQi9PztTMrF04PkpU+qMJQ/y?=
 =?us-ascii?Q?FMRSbtdd3WIOSi5J68oohYVuWW8rocUCcX4A25EBWtkQAgExOoZZ2oBnj+kP?=
 =?us-ascii?Q?YZ5b7QnTaUutE8sWUxYvYO+UdNTGdJO24hA97Ruwg2k22FjqfAHGPs/5y4T3?=
 =?us-ascii?Q?+zM4lm5rT/EXgaOyJ2mX5r5k7jdxi9xAytlx5ihL/IepZhGxBAE7nJg75JCj?=
 =?us-ascii?Q?hdXefDsDsWQeQBrC1Pa5GohVTyXcs3UZerrJ94YN5GpVMyG5t5lFfdBCNBXa?=
 =?us-ascii?Q?1DoSKxY3UtXFHLisk0FGMffFjDqYYDCFQdvqeg1JMRZaHk1rF9HgkiitQkNR?=
 =?us-ascii?Q?MXsDk9F7Cf9zYYnd9l/LE4zq/VEzDyHVi+IU8S0od5fQ9tfBVRF4lgLNKXh4?=
 =?us-ascii?Q?Y7NRgAz9UtlLUQTV0mQK/0/tmKRfIPS5FaoUhlJVbLwy2/8oBsESq4fmm0kr?=
 =?us-ascii?Q?rlUIDrLvMTkntIECBO43FKHbOqm7np+HR/ncyjyp?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: nxp.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: PAXPR04MB9185.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 833faaa0-05dc-438c-174e-08dac7eba281
X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Nov 2022 16:00:08.3332
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: /7Ck35pjVfGAkdumCO10gN66V/9htPQNpPfDLPY9+52fX5YrebQQLwwLMuZ/dE1xzDf4VQf4XKXp9iggCWFUqA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR04MB8771
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


> -----Original Message-----
> From: Alexander Lobakin <alexandr.lobakin@intel.com>
> Sent: Wednesday, November 16, 2022 8:34 AM
> To: Shenwei Wang <shenwei.wang@nxp.com>
> Cc: Alexander Lobakin <alexandr.lobakin@intel.com>; Andrew Lunn
> <andrew@lunn.ch>; David S. Miller <davem@davemloft.net>; Eric Dumazet
> <edumazet@google.com>; Jakub Kicinski <kuba@kernel.org>; Paolo Abeni
> <pabeni@redhat.com>; Alexei Starovoitov <ast@kernel.org>; Daniel Borkmann
> <daniel@iogearbox.net>; Jesper Dangaard Brouer <hawk@kernel.org>; John
> Fastabend <john.fastabend@gmail.com>; Wei Fang <wei.fang@nxp.com>;
> netdev@vger.kernel.org; linux-kernel@vger.kernel.org; imx@lists.linux.dev=
;
> kernel test robot <lkp@intel.com>
> Subject: [EXT] Re: [PATCH v3 1/1] net: fec: add xdp and page pool statist=
ics
>=20
> Caution: EXT Email
>=20
> From: Shenwei Wang <shenwei.wang@nxp.com>
> Date: Mon, 14 Nov 2022 21:17:48 +0000
>=20
> > > -----Original Message-----
> > > From: Alexander Lobakin <alexandr.lobakin@intel.com>
> > > Sent: Monday, November 14, 2022 9:23 AM
> > > To: Shenwei Wang <shenwei.wang@nxp.com>
> > > Cc: Alexander Lobakin <alexandr.lobakin@intel.com>; Andrew Lunn
> > > <andrew@lunn.ch>; David S. Miller <davem@davemloft.net>; Eric
> > > Dumazet <edumazet@google.com>; Jakub Kicinski <kuba@kernel.org>;
> > > Paolo Abeni <pabeni@redhat.com>; Alexei Starovoitov
> > > <ast@kernel.org>; Daniel Borkmann <daniel@iogearbox.net>; Jesper
> > > Dangaard Brouer <hawk@kernel.org>; John Fastabend
> > > <john.fastabend@gmail.com>; Wei Fang <wei.fang@nxp.com>;
> > > netdev@vger.kernel.org; linux-kernel@vger.kernel.org;
> > > imx@lists.linux.dev; kernel test robot <lkp@intel.com>
> > > Subject: [EXT] Re: [PATCH v3 1/1] net: fec: add xdp and page pool
> > > statistics
>=20
> [...]
>=20
> > Did some testing with the atomic64_t counter, with the following codes
> > to update the u64 counter in the end of every NAPI poll cycle.
> >
> > @@ -1764,7 +1768,13 @@ fec_enet_rx_queue(struct net_device *ndev, int
> > budget, u16 queue_id)
> >
> >         if (xdp_result & FEC_ENET_XDP_REDIR)
> >                 xdp_do_flush_map();
> > +#if 1
> > +       if (xdp_prog) {
> > +               int i;
> > +               for(i =3D 0; i < XDP_STATS_TOTAL; i++)
> > +                       atomic64_add(xdp_stats[i], &rxq->stats[i]);
> > +       }
> > +#endif
> >         return pkt_received;
> >  }
> >
> > With the codes above, the testing result is below:
> > root@imx8qxpc0mek:~/bpf# ./xdpsock -i eth0
> >  sock0@eth0:0 rxdrop xdp-drv
> >                    pps            pkts           1.00
> > rx                 349399         1035008
> > tx                 0              0
> >
> >  sock0@eth0:0 rxdrop xdp-drv
> >                    pps            pkts           1.00
> > rx                 349407         1384640
> > tx                 0              0
> >
> > Without  the atomic_add codes above, the testing result is below:
> > root@imx8qxpc0mek:~/bpf# ./xdpsock -i eth0
> >  sock0@eth0:0 rxdrop xdp-drv
> >                    pps            pkts           1.00
> > rx                 350109         1989130
> > tx                 0              0
> >
> >  sock0@eth0:0 rxdrop xdp-drv
> >                    pps            pkts           1.00
> > rx                 350425         2339786
> > tx                 0              0
> >
> > And regarding the u32 counter solution, the testing result is below:
> >    root@imx8qxpc0mek:~/bpf# ./xdpsock -i eth0
> >      sock0@eth0:0 rxdrop xdp-drv
> >                        pps            pkts           1.00
> >     rx                 361347         2637796
> >     tx                 0              0
> >
> > There are about 10K pkts/s difference here. Do we really want the u64
> counters?
>=20
> Where did those atomic64_t come from? u64_stats_t use either plain
> u64 for 32-bit platforms or local64_t for 64-bit ones. Take a look at [0]=
 for the
> example of how x86_64 does this, it is far from atomic64_t.

The v5 patch has changed it to u64.=20

Thanks,
Shenwei
>=20
> >
> > Regards,
> > Shenwei
> >
> > >>
> > >> Thanks,
> > >> Shenwei
> > >>
> > >>>
> > >>>        Andrew
> > >
> > > Thanks,
> > > Olek
>=20
> [0]
> https://eur01.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%2Felixi=
r.boo
> tlin.com%2Flinux%2Fv6.1-
> rc5%2Fsource%2Farch%2Fx86%2Finclude%2Fasm%2Flocal.h%23L31&amp;data
> =3D05%7C01%7Cshenwei.wang%40nxp.com%7Caf8d6634fcfc4d77e07308dac7dfa
> 4a8%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C638042060598040
> 501%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiL
> CJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=3DsWQ4Ts
> ayZvfs4pkCpC4tWvT51Cv89c4N6D5w2J%2FDC84%3D&amp;reserved=3D0
>=20
> Thanks,
> Olek