Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20230508160829.2756405-1-vineeth@bitbyteword.org>
 <20230509132534.09098acc@luca64> <CAO7JXPhrqKWfsp860rRmEenxARi8U2gNMGsOn4m+aKporWwBcg@mail.gmail.com>
 <20230509224829.2fb547fd@nowhere> <20230509225417.61d36733@nowhere>
 <CAO7JXPhk5qbz9kmiu9WuXS+gXCt9+X8pP2c37hd9ChByLmXYjA@mail.gmail.com>
 <20230510090735.68f62cd4@nowhere> <CAO7JXPg03f2YnrmzoGjfHEZZcoN55cU7uVukMw31Bw3x6nnaMw@mail.gmail.com>
 <20230511093711.2be82409@nowhere> <CAO7JXPiKLFLVpqDKoRdu8c2Yd4AJ_2B2UyXJxD3kJAyiJ=n0_Q@mail.gmail.com>
 <20230511220336.05866e80@nowhere> <CAO7JXPg9K4MA7H63Num8-BHBT-Y=GQ3BnEEvaEMVKmYsXCY7Sg@mail.gmail.com>
In-Reply-To: <CAO7JXPg9K4MA7H63Num8-BHBT-Y=GQ3BnEEvaEMVKmYsXCY7Sg@mail.gmail.com>
From:   Vineeth Remanan Pillai <vineeth@bitbyteword.org>
Date:   Sun, 14 May 2023 22:56:55 -0400
Message-ID: <CAO7JXPiYL_1ePxcoH=mvGq05S75jmA3SLjDf=oe0wU4Ozm4hrA@mail.gmail.com>
Subject: Re: [PATCH 1/2] sched/deadline: accurate reclaim bandwidth for GRUB
To:     luca abeni <luca.abeni@santannapisa.it>
Cc:     Juri Lelli <juri.lelli@redhat.com>,
        Daniel Bristot de Oliveira <bristot@redhat.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Ingo Molnar <mingo@redhat.com>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Steven Rostedt <rostedt@goodmis.org>,
        Joel Fernandes <joel@joelfernandes.org>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
        Valentin Schneider <vschneid@redhat.com>,
        Jonathan Corbet <corbet@lwn.net>, linux-kernel@vger.kernel.org,
        linux-doc@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

Hi Luca,

> Sure, I shall split the patch. Joel also suggested splitting the patch
> and I was probably wrong to think that the patch was simple to be a
> single patch :-).
>
> Since equation (3) has theoretical backing, I am perfectly fine with
> using it for our fix. Will have 2 patches as you suggested.
>
> Will get the v3 out soon..
>
Sorry for the delay. I was testing the fixes and noticed one other
issue and was working on a fix.

This is with the extra_bw for tracking the unused bandwidth in the
system. As you have shown me:
 "extra_bw = Umax - ((Sum of bw of all tasks) / #cpus)"
But I noticed that tracking the extra_bw globally while rest of the
values as per-cpu causes issues with reclaiming especially when we
have tasks with small bandwidth. This is because GRUB tries to
proportionately divide unused bandwidth based on running bw on a cpu.
So even if there is only a single task in a cpu, it reclaims less if
its bandwidth is less.

I think removing the extra_bw and tracking unused bandwidth using
"max_bw - this_bw" should be okay. Since admission control will
guarantee that we don't admit more than the capacity, we should be
good with allowing tasks on a cpu to use max_bw while reclaiming.

With the above theory, the code becomes simpler and is the same as the
v2 patch.
 max{u_i, (max_bw - Uinact - Uextra)}
 = max {u_i, (max_bw - (this_bw - running_bw) - (max_bw - this_bw))}
 = max {u_i, running_bw} = running_bw
So, "dq = -(running_bw / max_bw) dt"

v2 had passed all my tests. But having Uextra broke couple of those
when it came to multiprocessors and after identifying the root cause
and re-writing the equation, everything works now and passes all
my tests. I have more confidence in the above equation as its derived
from the SMP GRUB rule using our max_bw.

Please have a look and let me know what you think about this.

I have the v3 ready with patches split into 5(including doc patch).
I shall post v3 soon after this so that you can see the code changes
as well to have a better look.

Thanks,
Vineeth