Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Wed, 13 May 2020 09:31:22 -0400
From:   Phil Auld <pauld@redhat.com>
To:     Vincent Guittot <vincent.guittot@linaro.org>
Cc:     Ingo Molnar <mingo@redhat.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Juri Lelli <juri.lelli@redhat.com>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
        linux-kernel <linux-kernel@vger.kernel.org>,
        Tao Zhou <ouwen210@hotmail.com>,
        Pavan Kondeti <pkondeti@codeaurora.org>
Subject: Re: [PATCH v2] sched/fair: enqueue_task_fair optimization
Message-ID: <20200513133122.GH12425@lorien.usersys.redhat.com>
References: <20200513123335.28122-1-vincent.guittot@linaro.org>
 <20200513124540.GB12425@lorien.usersys.redhat.com>
 <CAKfTPtBFP5eAV-u02x42U2cQnWA56RP+wbj78rWpzj560OS+-g@mail.gmail.com>
 <20200513131337.GF12425@lorien.usersys.redhat.com>
 <CAKfTPtDYmi9wz3r1G8baG2cM3wh6004CDT11HaAu8L7-wWv=Gw@mail.gmail.com>
 <20200513131808.GG12425@lorien.usersys.redhat.com>
 <CAKfTPtBpYWDz=ZeKKa1BGOVZ+PqM=kbbgSBQpn7msxMV_5v5uA@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAKfTPtBpYWDz=ZeKKa1BGOVZ+PqM=kbbgSBQpn7msxMV_5v5uA@mail.gmail.com>
User-Agent: Mutt/1.12.1 (2019-06-15)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Wed, May 13, 2020 at 03:25:29PM +0200 Vincent Guittot wrote:
> On Wed, 13 May 2020 at 15:18, Phil Auld <pauld@redhat.com> wrote:
> >
> > On Wed, May 13, 2020 at 03:15:53PM +0200 Vincent Guittot wrote:
> > > On Wed, 13 May 2020 at 15:13, Phil Auld <pauld@redhat.com> wrote:
> > > >
> > > > On Wed, May 13, 2020 at 03:10:28PM +0200 Vincent Guittot wrote:
> > > > > On Wed, 13 May 2020 at 14:45, Phil Auld <pauld@redhat.com> wrote:
> > > > > >
> > > > > > Hi Vincent,
> > > > > >
> > > > > > On Wed, May 13, 2020 at 02:33:35PM +0200 Vincent Guittot wrote:
> > > > > > > enqueue_task_fair jumps to enqueue_throttle label when cfs_rq_of(se) is
> > > > > > > throttled which means that se can't be NULL and we can skip the test.
> > > > > > >
> > > > > >
> > > > > > s/be NULL/be non-NULL/
> > > > > >
> > > > > > I think.
> > > > >
> > > > > This sentence refers to the move of enqueue_throttle and the fact that
> > > > > se can't be null when goto enqueue_throttle and we can jump directly
> > > > > after the if statement, which is now removed in v2 because se is
> > > > > always NULL if we don't use goto enqueue_throttle.
> > > > >
> > > > > I haven't change the commit message for the remove of if statement
> > > > >
> > > >
> > > > Fair enough, it just seems backwards from the intent of the patch now.
> > > >
> > > > There is also an extra }  after the update_overutilized_status.
> > >
> > > don't know what I did but it's crap.  sorry about that
> > >
> >
> > No worries. I didn't see it when I read it either. The compiler told me :)
> 
> Yeah, but i thought that i compiled it which is obviously not true
>

It's that "obviously" correct stuff that bites you every time ;)


> >
> >
> > > Let me prepare a v3
> > >
> > > >
> > > >
> > > > Cheers,
> > > > Phil
> > > >
> > > >
> > > >
> > > > > >
> > > > > > It's more like if it doesn't jump to the label then se must be NULL for
> > > > > > the loop to terminate.  The final loop is a NOP if se is NULL. The check
> > > > > > wasn't protecting that.
> > > > > >
> > > > > > Otherwise still
> > > > > >
> > > > > > > Reviewed-by: Phil Auld <pauld@redhat.com>
> > > > > >
> > > > > > Cheers,
> > > > > > Phil
> > > > > >
> > > > > >
> > > > > > > Signed-off-by: Vincent Guittot <vincent.guittot@linaro.org>
> > > > > > > ---
> > > > > > >
> > > > > > > v2 changes:
> > > > > > > - Remove useless if statement
> > > > > > >
> > > > > > >  kernel/sched/fair.c | 39 ++++++++++++++++++++-------------------
> > > > > > >  1 file changed, 20 insertions(+), 19 deletions(-)
> > > > > > >
> > > > > > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > > > > > > index a0c690d57430..b51b12d63c39 100644
> > > > > > > --- a/kernel/sched/fair.c
> > > > > > > +++ b/kernel/sched/fair.c
> > > > > > > @@ -5513,28 +5513,29 @@ enqueue_task_fair(struct rq *rq, struct task_struct *p, int flags)
> > > > > > >                         list_add_leaf_cfs_rq(cfs_rq);
> > > > > > >       }
> > > > > > >
> > > > > > > -enqueue_throttle:
> > > > > > > -     if (!se) {
> > > > > > > -             add_nr_running(rq, 1);
> > > > > > > -             /*
> > > > > > > -              * Since new tasks are assigned an initial util_avg equal to
> > > > > > > -              * half of the spare capacity of their CPU, tiny tasks have the
> > > > > > > -              * ability to cross the overutilized threshold, which will
> > > > > > > -              * result in the load balancer ruining all the task placement
> > > > > > > -              * done by EAS. As a way to mitigate that effect, do not account
> > > > > > > -              * for the first enqueue operation of new tasks during the
> > > > > > > -              * overutilized flag detection.
> > > > > > > -              *
> > > > > > > -              * A better way of solving this problem would be to wait for
> > > > > > > -              * the PELT signals of tasks to converge before taking them
> > > > > > > -              * into account, but that is not straightforward to implement,
> > > > > > > -              * and the following generally works well enough in practice.
> > > > > > > -              */
> > > > > > > -             if (flags & ENQUEUE_WAKEUP)
> > > > > > > -                     update_overutilized_status(rq);
> > > > > > > +     /* At this point se is NULL and we are at root level*/
> > > > > > > +     add_nr_running(rq, 1);
> > > > > > > +
> > > > > > > +     /*
> > > > > > > +      * Since new tasks are assigned an initial util_avg equal to
> > > > > > > +      * half of the spare capacity of their CPU, tiny tasks have the
> > > > > > > +      * ability to cross the overutilized threshold, which will
> > > > > > > +      * result in the load balancer ruining all the task placement
> > > > > > > +      * done by EAS. As a way to mitigate that effect, do not account
> > > > > > > +      * for the first enqueue operation of new tasks during the
> > > > > > > +      * overutilized flag detection.
> > > > > > > +      *
> > > > > > > +      * A better way of solving this problem would be to wait for
> > > > > > > +      * the PELT signals of tasks to converge before taking them
> > > > > > > +      * into account, but that is not straightforward to implement,
> > > > > > > +      * and the following generally works well enough in practice.
> > > > > > > +      */
> > > > > > > +     if (flags & ENQUEUE_WAKEUP)
> > > > > > > +             update_overutilized_status(rq);
> > > > > > >
> > > > > > >       }
> > > > > > >
> > > > > > > +enqueue_throttle:
> > > > > > >       if (cfs_bandwidth_used()) {
> > > > > > >               /*
> > > > > > >                * When bandwidth control is enabled; the cfs_rq_throttled()
> > > > > > > --
> > > > > > > 2.17.1
> > > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > >
> > > >
> > > > --
> > > >
> > >
> >
> > --
> >
> 

--