Received-SPF: pass (google.com: domain of linux-wireless+bounces-3455-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
From: Toke =?utf-8?Q?H=C3=B8iland-J=C3=B8rgensen?= <toke@kernel.org>
To: Felix Fietkau <nbd@nbd.name>, Johannes Berg <johannes@sipsolutions.net>,
 linux-wireless@vger.kernel.org
Subject: Re: [RFC] mac80211: add AQL support for broadcast packets
In-Reply-To: <9922ecf9-d243-40a1-809b-9739d3269177@nbd.name>
References: <20240209184730.69589-1-nbd@nbd.name> <87plx4s71y.fsf@toke.dk>
 <960efcac-0995-4a42-b90c-3e66c0f56762@nbd.name>
 <66bddf2f6362c9f39f06e06c0c35b6900917b9bf.camel@sipsolutions.net>
 <87sf1yymr2.fsf@toke.dk> <9922ecf9-d243-40a1-809b-9739d3269177@nbd.name>
Date: Mon, 12 Feb 2024 14:15:26 +0100
Message-ID: <87eddhzuvl.fsf@toke.dk>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Felix Fietkau <nbd@nbd.name> writes:

> On 12.02.24 11:56, Toke H=C3=B8iland-J=C3=B8rgensen wrote:
>> Johannes Berg <johannes@sipsolutions.net> writes:
>>=20
>>> On Sat, 2024-02-10 at 17:18 +0100, Felix Fietkau wrote:
>>>>=20
>>>> > > +++ b/include/net/cfg80211.h
>>>> > > @@ -3385,6 +3385,7 @@ enum wiphy_params_flags {
>>>> > >  /* The per TXQ device queue limit in airtime */
>>>> > >  #define IEEE80211_DEFAULT_AQL_TXQ_LIMIT_L	5000
>>>> > >  #define IEEE80211_DEFAULT_AQL_TXQ_LIMIT_H	12000
>>>> > > +#define IEEE80211_DEFAULT_AQL_TXQ_LIMIT_BC	50000
>>>> >=20
>>>> > How did you arrive at the 50 ms figure for the limit on broadcast
>>>> > traffic? Seems like quite a lot? Did you experiment with different
>>>> > values?
>>>>=20
>>>> Whenever a client is connected and in powersave mode, all multicast=20
>>>> packets are buffered and sent after the beacon. Because of that I=20
>>>> decided to use half of a default beacon interval.
>>>
>>> That makes some sense, I guess.
>>=20
>> This implies that we will allow enough data to be queued up in the
>> hardware to spend half the next beacon interval just sending that
>> broadcast data? Isn't that a bit much if the goal is to prevent
>> broadcast from killing the network? What effect did you measure of this
>> patch? :)
>
> I didn't do any real measurements with this patch yet. How much=20
> broadcast data is actually sent after the beacon is still up to the=20
> driver/hardware, so depending on that, the limit might even be less than=
=20
> 50ms. I also wanted to be conservative in limiting buffering in order to=
=20
> avoid potential regressions. While 50ms may seem like much, I believe it=
=20
> is still a significant improvement over the current state, which is=20
> unlimited.
>
>> Also, as soon as something is actually transmitted, the kernel will
>> start pushing more data into the HW from the queue in the host. So the
>> HW queue limit shouldn't be set as "this is the maximum that should be
>> transmitted in one go", but rather "this is the minimum time we need for
>> the software stack to catch up and refill the queue before it runs
>> empty". So from that perspective 50ms also seems a bit high?
>
> When broadcast buffering is enabled, the driver/hardware typically=20
> prepares the set of packets to be transmitted before the beacon is sent.=
=20
> Any packet not ready by then will be sent in the next round.
> I added the 50ms limit based on that assumption.

Ah, so even if this is being done in software it's happening in the
driver, so post-TXQ dequeue? OK, in that case I guess it makes sense;
would love to see some numbers, of course, but I guess the debugfs
additions in this patch will make it possible to actually monitor the
queue lengths seen in the wild :)

>>> It does have me wondering though if we should also consider multicast
>>> for airtime fairness in some way?
>>=20
>> Yeah, that would make sense. The virtual time-based scheduler that we
>> ended up reverting actually included airtime accounting for the
>> multicast queue as well. I don't recall if there was any problem with
>> that particular part of the change, or if it's just incidental that we
>> got rid of it as part of the revert. But it may be worth revisiting and
>> adding a similar mechanism to the round-robin scheduler...
>
> The round-robin scheduler already has some consideration of multicast -=20
> it always puts the multicast queues last in the active_txqs list.

Ah, right. Hmm, not quite clear to me how that works out in terms of
fairness, but it should at least prevent the MC queue from blocking
everything else...

-Toke