Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
From:   Joel Fernandes <joel@joelfernandes.org>
Date:   Mon, 7 Jun 2021 12:19:01 -0400
Message-ID: <CAEXW_YTcO=hbmdq3nOx2RJfT2yPyoFnQx5niB38R2Lzpsp38bA@mail.gmail.com>
Subject: iowait boost is broken
To:     Beata Michalska <beata.michalska@arm.com>,
        Valentin Schneider <valentin.schneider@arm.com>,
        Quentin Perret <qperret@google.com>,
        Peter Zijlstra <peterz@infradead.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Qais Yousef <qais.yousef@arm.com>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Viresh Kumar <viresh.kumar@linaro.org>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

Hi all,
Looks like iowait boost is completely broken upstream. Just
documenting my findings of iowait boost issues:

1. If a CPU requests iowait boost in a cluster, another CPU can go
ahead and reset very quickly it since it thinks there's no new request
for the iowait boosting CPU
2. If the iowait is longer than a tick, then successive iowait boost
doubling does not happen. So heavy I/O waiting code never gets a
boost.
3. update_load_avg() is triggered right after the the iowait boost
request which makes another cpufreq update request, this request is a
non-iowait boost one so it ends up resetting the iowait boost request
(in the same path!).
4. Same as #3 but due the update_blocked_averages from new idle balance path.

Here is a patch that tries to address these problems and I see better
cpufreq boosting happening, however it is just a test broken patch to
highlight the issues:
https://git.kernel.org/pub/scm/linux/kernel/git/jfern/linux.git/commit/?h=sched/5.4/iowait-boost-debug-1&id=3627d896d499d168fef9a388e5d6b3359acc3423

I think we ought to rewrite the whole mess instead of fixing it since
a lot has changed in scheduler code over time it feels. Beata is
working on rewriting the whole iowait boost infra, I am glad she has
started the work on this and looking forward to helping with the
patches.

thanks,
 - Joel