Date:   Wed, 8 Feb 2023 18:17:52 +0100
From:   Maciej Fijalkowski <maciej.fijalkowski@intel.com>
To:     Vladimir Oltean <vladimir.oltean@nxp.com>
CC:     <netdev@vger.kernel.org>, "David S. Miller" <davem@davemloft.net>,
        "Eric Dumazet" <edumazet@google.com>,
        Jakub Kicinski <kuba@kernel.org>,
        Paolo Abeni <pabeni@redhat.com>,
        Claudiu Manoil <claudiu.manoil@nxp.com>,
        =?iso-8859-1?Q?Bj=F6rn_T=F6pel?= <bjorn@kernel.org>,
        Magnus Karlsson <magnus.karlsson@intel.com>,
        Alexei Starovoitov <ast@kernel.org>,
        "Daniel Borkmann" <daniel@iogearbox.net>,
        Jesper Dangaard Brouer <hawk@kernel.org>,
        John Fastabend <john.fastabend@gmail.com>,
        <bpf@vger.kernel.org>, <linux-kernel@vger.kernel.org>
Subject: Re: [RFC PATCH net-next 11/11] net: enetc: add TX support for
 zero-copy XDP sockets
Message-ID: <Y+PZQBWRqyNf9GlS@boxer>
References: <20230206100837.451300-1-vladimir.oltean@nxp.com>
 <20230206100837.451300-12-vladimir.oltean@nxp.com>
 <Y+PPzz4AHRxZgs9r@boxer>
 <20230208170815.nsq77mpkpf7aamhg@skbuf>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20230208170815.nsq77mpkpf7aamhg@skbuf>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?KtgZ+rjCIoISg+oCemgM7pJp4QMqVt3OpmDFokCxhcCbUFnFxMzHkp4s6INA?=
 =?us-ascii?Q?r+71j41sJM6NjJrsSSV2XKpQV/h/wkJsen9i6B89YhWu7cTVLjt5hoj9JUPt?=
 =?us-ascii?Q?PbFLBoA6FHjvnfcqM7GJzA3YCwqF7sGX3YydtkWJbOZDKN1GKr/Bep/TORPk?=
 =?us-ascii?Q?HRn+zAk5jMVw1aqMrHXBW4InTdPW6HgLhwnSsjD9Dlkw8g5N33FY2TfPu6KG?=
 =?us-ascii?Q?LvrLlLoPZqKVu7EUJeneLhbCSxl6Wa3xVDizNdP1SIVwlV4U1H4ho9MNpvmG?=
 =?us-ascii?Q?aWvJ7FfiU1EUWUFibzJ4NZILLdPNIpWRWPfMZtNRMFCUZUv5+E3TBerYKJ8D?=
 =?us-ascii?Q?5MKuJX5ER3Tl/SI/lCDA12pzRDKaVvyqbxu7SbkBLY2yz5FyUQYAO2BXPxF6?=
 =?us-ascii?Q?sOzmh8+tOdJgLEtc3PfJvnuXwqpTsv0aWZ8+x8G8GbnBe+QtAi13lsB3KelM?=
 =?us-ascii?Q?9Ncl3oVrLqCPWaDIDSjZ4q/oAXvmeyFiUrphr72PvFOI/qErDq4NnPianw2A?=
 =?us-ascii?Q?n/6o0f8QmubItiBGCiVA3sD/HzZtkXI92YOKvRoC6MMezimgOc9Z+ndj0DAA?=
 =?us-ascii?Q?sQQpI2Ie8H6+776Q8qjp0OqxjNI0HEPnI2WaHIeZvOxRvlgVhiwGWLPvNF7z?=
 =?us-ascii?Q?Hzboego7GhDhPvmP68WfjCw2AsfKiSx9GHvQvf+rzbj0At5ItLct5ZIoWUD+?=
 =?us-ascii?Q?P+MHAjaW5M6U76rxla1tsQi2L9wBO8BBvO7WEl3cZmHmIpCsfWt3AGNbVUEW?=
 =?us-ascii?Q?ywFdPqIRJGU8S8Lj8kwfkmzUFc4YvjtJW/9HImPeyTWch/lVuKC+ak+kklZO?=
 =?us-ascii?Q?ezQAI/zmakQMv/mHzCRcULanh1jpHnJwa5svFePovVdTwX+pVg7PcjnTjkR6?=
 =?us-ascii?Q?Ll8zo9xzhrpVYFFgFAPZIH0Q6X8eewlJcZHkOkUDfKGIoikwMkwuEGlqEw6m?=
 =?us-ascii?Q?8YyBOa7aS3tICbGFui2gH5wcIzzLRb/n1UT083AGAKRznHWZGfUpjDSBru57?=
 =?us-ascii?Q?FpcUuO8YS2bxCAYzN2Y17RjqCREEK78KCIvzUIzQF/AExN3LX0WdFu+owMC9?=
 =?us-ascii?Q?J/O7Ww9WFq1GG3WddE5oZtOFklm7x+ZTNWjJmUcl7RG8k+D7gNPNMHL59AVL?=
 =?us-ascii?Q?YoVKD3vzBy7gkuKuQomzZ4542vJ3qiC7FI44zBqRp6r1JeR3m0oRQk2GCsOk?=
 =?us-ascii?Q?gDO4AIFY5UPeJFQgYJYMZu7sCUReb9IyCMrpJ2VuCyOcbWYr0IL8NpjBLBhI?=
 =?us-ascii?Q?8lt0k4vebYaPgI5h/gB+ERI838i8+BSVONsWH+e7Tg2CxUF5jIh5hQroHYzP?=
 =?us-ascii?Q?rckm1hE53CJFIQIBvgpMfjO6yeOKyFc96bDBIJl05/NyB/0E+Em0jJZGPJH8?=
 =?us-ascii?Q?3sRL37795jqmomx5+WSxN29QTR6Yg4aaoLdADt2F9jKKoE5jy5i2fCPTOgnL?=
 =?us-ascii?Q?OloKWWfzrvSU+2YRuZeMi5F3rr7NgLaZjQE/WkB68jEGTZkcJ+8IQxTuxWs1?=
 =?us-ascii?Q?ni6nX4RYdwQBuaPS6NYpqXaZSDCCAyTrKdFrS09unPqKl0ibitkBipLe9GJ8?=
 =?us-ascii?Q?OUKK349CFELYIBxfQXvn05ocjh9kW14R/Hyo+/G8V0AlioKcUuzQISd12y/y?=
 =?us-ascii?Q?3DnCsSLNXx9cJRFm/5Jv8uw=3D?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 8dd87550-8557-4325-67a2-08db09f86d9a
X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB6117.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Feb 2023 17:17:59.9838
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: CK9dOBWYB6n+AlFhEv7crjiStWWNLOleRnauA14OmBvtlfZqAw6KtMDNbWLw0UtQT70lrER2JsJHUX1tUQC+0l9dz2dd4ahVaOHQ49O8Pgc=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB6214
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Wed, Feb 08, 2023 at 07:08:15PM +0200, Vladimir Oltean wrote:
> Hi Maciej,
> 
> On Wed, Feb 08, 2023 at 05:37:35PM +0100, Maciej Fijalkowski wrote:
> > On Mon, Feb 06, 2023 at 12:08:37PM +0200, Vladimir Oltean wrote:
> > 
> > Hey Vladimir,
> > 
> > > Schedule NAPI by hand from enetc_xsk_wakeup(), and send frames from the
> > > XSK TX queue from NAPI context. Add them to the completion queue from
> > > the enetc_clean_tx_ring() procedure which is common for all kinds of
> > > traffic.
> > > 
> > > We reuse one of the TX rings for XDP (XDP_TX/XDP_REDIRECT) for XSK as
> > > well. They are already cropped from the TX rings that the network stack
> > > can use when XDP is enabled (with or without AF_XDP).
> > > 
> > > As for XDP_REDIRECT calling enetc's ndo_xdp_xmit, I'm not sure if that
> > > can run simultaneously with enetc_poll() (on different CPUs, but towards
> > > the same TXQ). I guess it probably can, but idk what to do about it.
> > > The problem is that enetc_xdp_xmit() sends to
> > > priv->xdp_tx_ring[smp_processor_id()], while enetc_xsk_xmit() and XDP_TX
> > > send to priv->xdp_tx_ring[NAPI instance]. So when the NAPI instance runs
> > 
> > Why not use cpu id on the latter then?
> 
> Hmm, because I want the sendto() syscall to trigger wakeup of the NAPI
> that sends traffic to the proper queue_id, rather than to the queue_id
> affine to the CPU that the sendto() syscall was made?

Ok i was referring to the thing that using cpu id on both sides would
address concurrency. Regarding your need, what i did for ice was that i
assign xdp_ring to the rx_ring and within ndo_xsk_wakeup() i pick the
rx_ring based on queue_id that comes as an arg:

https://lore.kernel.org/bpf/20220822163257.2382487-3-anthony.l.nguyen@intel.com/