Received-SPF: pass (google.com: best guess record for domain of linux-wireless-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <20191115014846.126007-1-kyan@google.com> <CA+iem5vaeLR6v_nZ1YUZhfj32wF0DrvC2nyp8nb8qYAZLQjLdw@mail.gmail.com>
 <CAA93jw5wTbFV51oFJ6tFHLUMo=bau8fbU65k57bQjOHGJoCkkQ@mail.gmail.com>
 <CA+iem5s4ZY239Q4=Gwy3WrmVhcdhesirXph6XQoOP5w-nuWcYw@mail.gmail.com>
 <CAA93jw5t0TwBVv7_DVkJ_-NsVn0ODNHwU0orp2-+LPB45iFVoQ@mail.gmail.com>
 <CA+iem5uVJFcCYpJfhker-48XPrOf3a+NWr-nKnBtGmLX2yB_Lg@mail.gmail.com>
 <8736eiam8f.fsf@toke.dk> <CA+iem5tpfEmaWJ5Mw7xF9fb=XLceZpC1LM4Avo89Mn1fL7YZVw@mail.gmail.com>
 <87a78p8rz7.fsf@toke.dk> <CA+iem5tNz2jjEOVmbh3aPTXLLZfkRjZ60-+bon1vDEJ8D4hQJw@mail.gmail.com>
 <87muco5gv5.fsf@toke.dk>
In-Reply-To: <87muco5gv5.fsf@toke.dk>
From:   Kan Yan <kyan@google.com>
Date:   Mon, 25 Nov 2019 21:04:23 -0800
Message-ID: <CA+iem5sBPq0mfz+Qx+uJqCZ6t-Cjru+GCBcYExdu6JueUbBXyw@mail.gmail.com>
Subject: Re: [Make-wifi-fast] [PATCH v8 0/2] Implement Airtime-based Queue
 Limit (AQL)
To:     =?UTF-8?B?VG9rZSBIw7hpbGFuZC1Kw7hyZ2Vuc2Vu?= <toke@redhat.com>
Cc:     Dave Taht <dave.taht@gmail.com>,
        Rajkumar Manoharan <rmanohar@codeaurora.org>,
        Kevin Hayes <kevinhayes@google.com>,
        Make-Wifi-fast <make-wifi-fast@lists.bufferbloat.net>,
        linux-wireless <linux-wireless@vger.kernel.org>,
        Yibo Zhao <yiboz@codeaurora.org>,
        John Crispin <john@phrozen.org>,
        Johannes Berg <johannes@sipsolutions.net>,
        Lorenzo Bianconi <lorenzo@kernel.org>,
        Felix Fietkau <nbd@nbd.name>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Sender: linux-wireless-owner@vger.kernel.org
Precedence: bulk

> Yeah, bpftrace can be a bit of a pain to get running; but it may be
> worth the investment longer term as well. It really is quite useful! :)

My attempt to build bpftrace didn't work out, so I just got the
sojourn time using old fashioned trace event.
The raw trace, parsed data in csv format and plots can be found here:
https://drive.google.com/open?id=3D1Mg_wHu7elYAdkXz4u--42qGCVE1nrILV

All tests are done with 2 TCP download sessions that oversubscribed
the link bandwidth.
With AQL on, the mean sojourn time about ~20000us, matches the default
codel "target".
With AQL off, the mean sojourn time is less than 4us even the latency
is off the charts, just as we expected that fd_codel with mac80211
alone is not effective for drivers with deep firmware/hardware queues.

> Any chance you could turn ecn on and off and give it a go
> again in your next test run?

ECN on shows very similar results as with ECN off. "aqm" stats from
debugfs shows it is doing ECN marking instead of dropping packets as
expected. Flent test data also is in the same link.


On Fri, Nov 22, 2019 at 2:45 AM Toke H=C3=B8iland-J=C3=B8rgensen <toke@redh=
at.com> wrote:
>
> Kan Yan <kyan@google.com> writes:
>
> >> In theory, this ought to produce a histogram of sojourn times (in
> >> microseconds):
> >> bpftrace -e 'kretprobe:codel_skb_time_func { @sojourn =3D lhist((nsecs=
 -
> > (retval << 10))/1000, 0, 100000, 1000); }'
> >
> > Thanks for the tips!
> >
> >> Can't get the CoDel drop mechanism to trigger on my system at all,
> >> though (a laptop running on iwl). I guess because there's queue
> >> backpressure to userspace first?
> >
> > What's the tcp_congestion_control in your system? Maybe it is BBR that
> > prevents bufferbloat.
>
> It's not BBR, just plain old CUBIC. I've seen the issue before that it's
> almost impossible to build a queue in the mac80211 layer when the TCP
> session is originated on the local machine, though...
>
> >> It would be interesting to see if it works for you, assuming you can g=
et
> >> bpftrace to work on your test system :)
> >
> > I can enable required kernel configuration easily, but cross-compile
> > bpftrace for an ARM64 platform may take some time and effort.
>
> Yeah, bpftrace can be a bit of a pain to get running; but it may be
> worth the investment longer term as well. It really is quite useful! :)
>
> Some links:
>
> Install guide:
> https://github.com/iovisor/bpftrace/blob/master/INSTALL.md
>
> Tutorial by one-liners:
> https://github.com/iovisor/bpftrace/blob/master/docs/tutorial_one_liners.=
md
>
> Reference guide:
> https://github.com/iovisor/bpftrace/blob/master/docs/reference_guide.md#5=
-tracepoint-static-tracing-kernel-level
>
> -Toke
>