Received-SPF: pass (google.com: domain of linux-kernel+bounces-96483-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Date: Thu, 7 Mar 2024 19:36:46 -0800
From: Jakub Kicinski <kuba@kernel.org>
To: Mina Almasry <almasrymina@google.com>
Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
 linux-doc@vger.kernel.org, linux-alpha@vger.kernel.org,
 linux-mips@vger.kernel.org, linux-parisc@vger.kernel.org,
 sparclinux@vger.kernel.org, linux-trace-kernel@vger.kernel.org,
 linux-arch@vger.kernel.org, bpf@vger.kernel.org,
 linux-kselftest@vger.kernel.org, linux-media@vger.kernel.org,
 dri-devel@lists.freedesktop.org, "David S. Miller" <davem@davemloft.net>,
 Eric Dumazet <edumazet@google.com>, Paolo Abeni <pabeni@redhat.com>,
 Jonathan Corbet <corbet@lwn.net>, Richard Henderson
 <richard.henderson@linaro.org>, Ivan Kokshaysky <ink@jurassic.park.msu.ru>,
 Matt Turner <mattst88@gmail.com>, Thomas Bogendoerfer
 <tsbogend@alpha.franken.de>, "James E.J. Bottomley"
 <James.Bottomley@hansenpartnership.com>, Helge Deller <deller@gmx.de>,
 Andreas Larsson <andreas@gaisler.com>, Jesper Dangaard Brouer
 <hawk@kernel.org>, Ilias Apalodimas <ilias.apalodimas@linaro.org>, Steven
 Rostedt <rostedt@goodmis.org>, Masami Hiramatsu <mhiramat@kernel.org>,
 Mathieu Desnoyers <mathieu.desnoyers@efficios.com>, Arnd Bergmann
 <arnd@arndb.de>, Alexei Starovoitov <ast@kernel.org>, Daniel Borkmann
 <daniel@iogearbox.net>, Andrii Nakryiko <andrii@kernel.org>, Martin KaFai
 Lau <martin.lau@linux.dev>, Eduard Zingerman <eddyz87@gmail.com>, Song Liu
 <song@kernel.org>, Yonghong Song <yonghong.song@linux.dev>, John Fastabend
 <john.fastabend@gmail.com>, KP Singh <kpsingh@kernel.org>, Stanislav
 Fomichev <sdf@google.com>, Hao Luo <haoluo@google.com>, Jiri Olsa
 <jolsa@kernel.org>, David Ahern <dsahern@kernel.org>, Willem de Bruijn
 <willemdebruijn.kernel@gmail.com>, Shuah Khan <shuah@kernel.org>, Sumit
 Semwal <sumit.semwal@linaro.org>, Christian =?UTF-8?B?S8O2bmln?=
 <christian.koenig@amd.com>, Pavel Begunkov <asml.silence@gmail.com>, David
 Wei <dw@davidwei.uk>, Jason Gunthorpe <jgg@ziepe.ca>, Yunsheng Lin
 <linyunsheng@huawei.com>, Shailend Chand <shailend@google.com>, Harshitha
 Ramamurthy <hramamurthy@google.com>, Shakeel Butt <shakeelb@google.com>,
 Jeroen de Borst <jeroendb@google.com>, Praveen Kaligineedi
 <pkaligineedi@google.com>
Subject: Re: [RFC PATCH net-next v6 01/15] queue_api: define queue api
Message-ID: <20240307193646.70ef5243@kernel.org>
In-Reply-To: <CAHS8izPyxn2LsOsxL98WAHse21tq3i9MCp_Xn8AA8sx5iettNQ@mail.gmail.com>
References: <20240305020153.2787423-1-almasrymina@google.com>
	<20240305020153.2787423-2-almasrymina@google.com>
	<20240307173039.00e6fbb7@kernel.org>
	<CAHS8izPyxn2LsOsxL98WAHse21tq3i9MCp_Xn8AA8sx5iettNQ@mail.gmail.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On Thu, 7 Mar 2024 18:08:24 -0800 Mina Almasry wrote:
> On Thu, Mar 7, 2024 at 5:30=E2=80=AFPM Jakub Kicinski <kuba@kernel.org> w=
rote:
> > On Mon,  4 Mar 2024 18:01:36 -0800 Mina Almasry wrote: =20
> > > + * void *(*ndo_queue_mem_alloc)(struct net_device *dev, int idx);
> > > + *   Allocate memory for an RX queue. The memory returned in the for=
m of
> > > + *   a void * can be passed to ndo_queue_mem_free() for freeing or to
> > > + *   ndo_queue_start to create an RX queue with this memory.
> > > + *
> > > + * void      (*ndo_queue_mem_free)(struct net_device *dev, void *);
> > > + *   Free memory from an RX queue.
> > > + *
> > > + * int (*ndo_queue_start)(struct net_device *dev, int idx, void *);
> > > + *   Start an RX queue at the specified index.
> > > + *
> > > + * int (*ndo_queue_stop)(struct net_device *dev, int idx, void **);
> > > + *   Stop the RX queue at the specified index.
> > >   */
> > >  struct net_device_ops {
> > >       int                     (*ndo_init)(struct net_device *dev);
> > > @@ -1679,6 +1693,16 @@ struct net_device_ops {
> > >       int                     (*ndo_hwtstamp_set)(struct net_device *=
dev,
> > >                                                   struct kernel_hwtst=
amp_config *kernel_config,
> > >                                                   struct netlink_ext_=
ack *extack);
> > > +     void *                  (*ndo_queue_mem_alloc)(struct net_devic=
e *dev,
> > > +                                                    int idx);
> > > +     void                    (*ndo_queue_mem_free)(struct net_device=
 *dev,
> > > +                                                   void *queue_mem);
> > > +     int                     (*ndo_queue_start)(struct net_device *d=
ev,
> > > +                                                int idx,
> > > +                                                void *queue_mem);
> > > +     int                     (*ndo_queue_stop)(struct net_device *de=
v,
> > > +                                               int idx,
> > > +                                               void **out_queue_mem)=
; =20
> >
> > The queue configuration object was quite an integral part of the design,
> > I'm slightly worried that it's not here :) =20
>=20
> That was a bit of a simplification I'm making since we just want to
> restart the queue. I thought it was OK to define some minimal version
> here and extend it later with configuration? Because in this context
> all we really need is to restart the queue, yes?

Right, I think it's perfectly fine for the time being.
It works, and is internal to the kernel.

> If extending with some configuration is a must please let me know what
> configuration struct you're envisioning. Were you envisioning a stub?
> Or some real configuration struct that we just don't use at the
> moment? Or one that we use for this use case somehow?

I had some ideas about storing the configuration as rules,
instead of directly in struct netdev_rx_queue.
E.g. default queue length =3D 2000, but for select queues you may
want a different length.
But application binding to a queue would always take precedence,=20
so even if the ideas ever materialize there will be no uAPI change.

> > Also we may want to rename
> > the about-to-be-merged ops from netdev_stat_ops and netdev_queue_ops,
> > and add these there?
> >
> > https://lore.kernel.org/all/20240306195509.1502746-2-kuba@kernel.org/
>=20
> Yeah, that sounds reasonable! Thanks! We could also keep the
> netdev_stat_ops and add new netdev_queue_ops alongside them if you
> prefer.

Up to you, after some soul searching we renamed the uAPI to call these
stats qstats, I just forgot to rename the op struct. But it doesn't
matter much.

> > Very excited to hear that you made progress on this and ported GVE over=
! =20
>=20
> Actually, we're still discussing but it looks like my GVE queue API
> implementation I proposed earlier may be a no-go. Likely someone from
> the GVE team will follow up here with this piece, probably in a
> separate series.

Well, it's going to be ready when it's ready :)
Speaking of things which can be merged independently,
feel free to post patch 3, maybe it can make v6.9..

> For now I'm carrying my POC for the GVE implementation out of tree
> with the rest of the driver changes:
>=20
> https://github.com/mina/linux/commit/501b734c80186545281e9edb1bf313f5a2d8=
cbee