Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) client-ip=2620:137:e000::3:4;
MIME-Version: 1.0
References: <20230531115839.089944915@infradead.org> <dlbtvvm5cewqzh5bcpl4cqhcwxmnnjb6pdle5jzywiiznlactd@cmhnpim42m3p>
 <20230906131356.GG38741@noisy.programming.kicks-ass.net> <CA+q576MS0-MV1Oy-eecvmYpvNT3tqxD8syzrpxQ-Zk310hvRbw@mail.gmail.com>
 <20231002184136.GA1539@noisy.programming.kicks-ass.net> <20231005120557.GA743@noisy.programming.kicks-ass.net>
In-Reply-To: <20231005120557.GA743@noisy.programming.kicks-ass.net>
From:   Youssef Esmat <youssefesmat@chromium.org>
Date:   Thu, 5 Oct 2023 13:23:14 -0500
Message-ID: <CA+q576MLvCwH2YQFx3V2tf3f4n6JUze7jkpqqTx97UPsOHewhg@mail.gmail.com>
Subject: Re: [PATCH 00/15] sched: EEVDF and latency-nice and/or slice-attr
To:     Peter Zijlstra <peterz@infradead.org>
Cc:     Daniel Jordan <daniel.m.jordan@oracle.com>, mingo@kernel.org,
        vincent.guittot@linaro.org, linux-kernel@vger.kernel.org,
        juri.lelli@redhat.com, dietmar.eggemann@arm.com,
        rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de,
        bristot@redhat.com, corbet@lwn.net, qyousef@layalina.io,
        chris.hyser@oracle.com, patrick.bellasi@matbug.net, pjt@google.com,
        pavel@ucw.cz, qperret@google.com, tim.c.chen@linux.intel.com,
        joshdon@google.com, timj@gnu.org, kprateek.nayak@amd.com,
        yu.c.chen@intel.com, joel@joelfernandes.org, efault@gmx.de,
        tglx@linutronix.de
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Thu, Oct 5, 2023 at 7:06=E2=80=AFAM Peter Zijlstra <peterz@infradead.org=
> wrote:
>
> On Mon, Oct 02, 2023 at 08:41:36PM +0200, Peter Zijlstra wrote:
>
> > When mixing request sizes things become a little more interesting.
> >
> > Let me ponder this a little bit more.
>
> Using the attached program (I got *REALLY* fed up trying to draw these
> diagrams by hand), let us illustrate the difference between Earliest
> *Eligible* Virtual Deadline First and the one with the Eligible test
> taken out: EVDF.
>
> Specifically, the program was used with the following argument for
> EEVDF:
>
>   ./eevdf -e "0,1024,6" -e "1,1024,2" -e "2,1024,18" -v 19
>
> and with an additional '-n' for the EVDF column.
>
>
> EEVDF                                                   EVDF
>
>
> d =3D 6                                                   d =3D 6
> d =3D 2                                                   d =3D 2
> d =3D 18                                                  d =3D 18
> q =3D 2                                                   q =3D 2
>
> t=3D0 V=3D1                                                 t=3D0 V=3D1
>  A |----<                                                A |----<
> >B  |<                                                  >B  |<
>  C   |----------------<                                  C   |-----------=
-----<
>    |*--------|---------|---------|---------|----           |*--------|---=
------|---------|---------|----
>
>
> t=3D2 V=3D1                                                 t=3D2 V=3D1
> >A |----<                                                A |----<
>  B    |<                                                >B    |<
>  C   |----------------<                                  C   |-----------=
-----<
>    |*--------|---------|---------|---------|----           |*--------|---=
------|---------|---------|----
>
>
> t=3D8 V=3D3                                                 t=3D4 V=3D2
>  A       |----<                                         >A |----<
> >B    |<                                                 B      |<
>  C   |----------------<                                  C   |-----------=
-----<
>    |--*------|---------|---------|---------|----           |-*-------|---=
------|---------|---------|----
>
>
> t=3D10 V=3D4                                                t=3D10 V=3D4
>  A       |----<                                          A       |----<
>  B      |<                                              >B      |<
> >C   |----------------<                                  C   |-----------=
-----<
>    |---*-----|---------|---------|---------|----           |---*-----|---=
------|---------|---------|----
>
>
> t=3D28 V=3D10                                               t=3D12 V=3D5
>  A       |----<                                          A       |----<
> >B      |<                                              >B        |<
>  C                     |----------------<                C   |-----------=
-----<
>    |---------*---------|---------|---------|----           |----*----|---=
------|---------|---------|----
>
>
> t=3D30 V=3D11                                               t=3D14 V=3D5
>  A       |----<                                          A       |----<
> >B        |<                                            >B          |<
>  C                     |----------------<                C   |-----------=
-----<
>    |---------|*--------|---------|---------|----           |----*----|---=
------|---------|---------|----
>
>
> t=3D32 V=3D11                                               t=3D16 V=3D6
>  A       |----<                                         >A       |----<
> >B          |<                                           B            |<
>  C                     |----------------<                C   |-----------=
-----<
>    |---------|*--------|---------|---------|----           |-----*---|---=
------|---------|---------|----
>
>
> t=3D34 V=3D12                                               t=3D22 V=3D8
> >A       |----<                                          A             |-=
---<
>  B            |<                                        >B            |<
>  C                     |----------------<                C   |-----------=
-----<
>    |---------|-*-------|---------|---------|----           |-------*-|---=
------|---------|---------|----
>
>
> t=3D40 V=3D14                                               t=3D24 V=3D9
>  A             |----<                                    A             |-=
---<
> >B            |<                                        >B              |=
<
>  C                     |----------------<                C   |-----------=
-----<
>    |---------|---*-----|---------|---------|----           |--------*|---=
------|---------|---------|----
>
>
> t=3D42 V=3D15                                               t=3D26 V=3D9
>  A             |----<                                    A             |-=
---<
> >B              |<                                      >B               =
 |<
>  C                     |----------------<                C   |-----------=
-----<
>    |---------|----*----|---------|---------|----           |--------*|---=
------|---------|---------|----
>
>
> t=3D44 V=3D15                                               t=3D28 V=3D10
>  A             |----<                                   >A             |-=
---<
> >B                |<                                     B               =
   |<
>  C                     |----------------<                C   |-----------=
-----<
>    |---------|----*----|---------|---------|----           |---------*---=
------|---------|---------|----
>
>
> t=3D46 V=3D16                                               t=3D34 V=3D12
> >A             |----<                                    A               =
    |----<
>  B                  |<                                  >B               =
   |<
>  C                     |----------------<                C   |-----------=
-----<
>    |---------|-----*---|---------|---------|----           |---------|-*-=
------|---------|---------|----
>
>
> t=3D52 V=3D18                                               t=3D36 V=3D13
>  A                   |----<                              A               =
    |----<
> >B                  |<                                   B               =
     |<
>  C                     |----------------<               >C   |-----------=
-----<
>    |---------|-------*-|---------|---------|----           |---------|--*=
------|---------|---------|----
>
>
> t=3D54 V=3D19                                               t=3D54 V=3D19
>  A                   |----<                              A               =
    |----<
> >B                    |<                                >B               =
     |<
>  C                     |----------------<                C               =
      |----------------<
>    |---------|--------*|---------|---------|----           |---------|---=
-----*|---------|---------|----
>
>
> lags: -10, 6                                            lags: -7, 11
>
> BAaaBCccccccccBBBAaaBBBAaaBB                            BBAaaBBBAaaBBBAaa=
BCccccccccB
>
>
>
> As I wrote before; EVDF has worse lag bounds, but this is not
> insurmountable. The biggest problem that I can see is that of wakeup
> preemption. Currently we allow to preempt when 'current' has reached V
> (RUN_TO_PARITY in pick_eevdf()).
>
> With these rules, when EEVDF schedules C (our large slice task) at t=3D10
> above, it is only a little behind C and can be reaily preempted after
> about 2 time units.
>
> However, EVDF will delay scheduling C until much later, see how A and B
> walk far ahead of V until t=3D36. Only when will we pick C. But this mean=
s
> that we're firmly stuck with C for at least 11 time units. A newly
> placed task will be around V and will have no chance to preempt.
>

Thank you for the detailed analysis! I am still in the process of
digesting everything.
I do have a quick question, this will only be the case if we adjust
C's runtime without adjusting nice value, correct? So it does not
currently apply to the submitted code where the only way to change the
deadline is to also change the nice value and thus how fast/slow
vruntime accumulates. In other words without the sched_runtime
patch[1] we should not run into this scenario, correct?

[1] https://lore.kernel.org/lkml/20230915124354.416936110@noisy.programming=
.kicks-ass.net/

> That said, I do have me a patch to cure some of that:
>
>   https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/commit=
/?h=3Dsched/eevdf&id=3Dd7edbe431f31762e516f2730196f41322edcc621
>
> That would allow a task with a shorter request time to preempt in spite
> of RUN_TO_PARITY.
>
> However, in this example V is only 2/3 of the way to C's deadline, but
> it we were to have many more tasks, you'll see V gets closer and closer
> to C's deadline and it will become harder and harder to place such that
> preemption becomes viable.
>
> Adding 4 more tasks:
>
>   ./eevdf -e "0,1024,6" -e "1,1024,2" -e "2,1024,18" -v 19 -n -e "3,1024,=
2" -e "4,1024,2" -e "5,1024,2" -e "6,1024,2"
>
> t=3D92 V=3D16
>  A                   |----<
>  B                    |<
> >C   |----------------<
>  D                    |<
>  E                   |<
>  F                    |<
>  G                   |<
>    |---------|-----*---|---------|---------|----
>
>
> And I worry this will create very real latency spikes.
>
> That said; I do see not having the eligibility check can help. So I'm
> not opposed to having a sched_feat for this, but I would not want to
> default to EVDF.