MIME-Version: 1.0
In-Reply-To: <20160406083702.GE18516@e105550-lin.cambridge.arm.com>
References: <1459528717-17339-1-git-send-email-leo.yan@linaro.org>
 <20160401194948.GN3448@twins.programming.kicks-ass.net> <56FEF621.3070404@linaro.org>
 <20160402071154.GA7046@leoy-linaro> <20160404084821.GA18516@e105550-lin.cambridge.arm.com>
 <20160404183003.GA8697@intel.com> <20160405075112.GC18516@e105550-lin.cambridge.arm.com>
 <20160405001552.GB8697@intel.com> <5703EF38.2060204@arm.com> <20160406083702.GE18516@e105550-lin.cambridge.arm.com>
From: Vincent Guittot <vincent.guittot@linaro.org>
Date: Wed, 6 Apr 2016 14:14:06 +0200
Message-ID: <CAKfTPtBjTTJK8BFfA+xdZ6HJJxOD6DtfARi9Ps8X6OU4qogN6Q@mail.gmail.com>
Subject: Re: [PATCH RFC] sched/fair: let cpu's cfs_rq to reflect task migration
To: Morten Rasmussen <morten.rasmussen@arm.com>
Cc: Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Yuyang Du <yuyang.du@intel.com>, Leo Yan <leo.yan@linaro.org>,
        Steve Muckle <steve.muckle@linaro.org>,
        Peter Zijlstra <peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>,
        linux-kernel <linux-kernel@vger.kernel.org>,
        "eas-dev@lists.linaro.org" <eas-dev@lists.linaro.org>
Content-Type: text/plain; charset=UTF-8
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 1749
Lines: 35

On 6 April 2016 at 10:37, Morten Rasmussen <morten.rasmussen@arm.com> wrote:
> On Tue, Apr 05, 2016 at 06:00:40PM +0100, Dietmar Eggemann wrote:
>> @@ -2893,8 +2906,12 @@ static void attach_entity_load_avg(struct cfs_rq *cfs_rq, struct sched_entity *s
>>         se->avg.last_update_time = cfs_rq->avg.last_update_time;
>>         cfs_rq->avg.load_avg += se->avg.load_avg;
>>         cfs_rq->avg.load_sum += se->avg.load_sum;
>> -       cfs_rq->avg.util_avg += se->avg.util_avg;
>> -       cfs_rq->avg.util_sum += se->avg.util_sum;
>> +
>> +       if (!entity_is_task(se))
>> +               return;
>> +
>> +       rq_of(cfs_rq)->cfs.avg.util_avg += se->avg.util_avg;
>> +       rq_of(cfs_rq)->cfs.avg.util_sum += se->avg.util_sum;
>
> To me it seems that you cannot be sure that the rq_of(cfs_rq)->cfs.avg
> time stamp is aligned with se->avg time stamp, which is necessary before
> you can add/subtract two geometric series without introducing an error.
>
> attach_entity_load_avg() is called (through a couple of other functions)
> from the for_each_sched_entity() loop in enqueue_task_fair() which works
> its way towards the root cfs_rq, i.e. rq_of(cfs_rq)->cfs. So in the loop
> iteration where you attach the task sched_entity, we haven't yet visited
> and updated rq_of(cfs_rq)->cfs.avg.
>
> If you just add the task contribution and discover later that there is a
> time delta when you update rq_of(cfs_rq)->cfs.avg you end up decaying
> the task contribution which was already up-to-date and its util
> contribution to rq_of(cfs_rq)->cfs.avg ends up being smaller than it
> should be.
>
> Am I missing something?

Yes I agree that se->avg and rq_of(cfs_rq)->cfs.avg have to be aligned
on the same time stamp before adding or removing se.