Date: Fri, 19 Jun 2015 15:57:24 +0800
From: Boqun Feng <boqun.feng@gmail.com>
To: Yuyang Du <yuyang.du@intel.com>
Cc: mingo@kernel.org, peterz@infradead.org, linux-kernel@vger.kernel.org,
        pjt@google.com, bsegall@google.com, morten.rasmussen@arm.com,
        vincent.guittot@linaro.org, dietmar.eggemann@arm.com,
        len.brown@intel.com, rafael.j.wysocki@intel.com,
        fengguang.wu@intel.com, srikar@linux.vnet.ibm.com
Subject: Re: [PATCH v8 2/4] sched: Rewrite runnable load and utilization
 average tracking
Message-ID: <20150619075724.GA5331@fixme-laptop.cn.ibm.com>
References: <1434396367-27979-1-git-send-email-yuyang.du@intel.com>
 <1434396367-27979-3-git-send-email-yuyang.du@intel.com>
 <20150619060038.GA1240@fixme-laptop.cn.ibm.com>
 <20150618230554.GA3436@intel.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="OXfL5xGRrasGEqWY"
Content-Disposition: inline
In-Reply-To: <20150618230554.GA3436@intel.com>
User-Agent: Mutt/1.5.23+89 (0255b37be491) (2014-03-12)
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 4243
Lines: 118


--OXfL5xGRrasGEqWY
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Hi Yuyang,

On Fri, Jun 19, 2015 at 07:05:54AM +0800, Yuyang Du wrote:
> On Fri, Jun 19, 2015 at 02:00:38PM +0800, Boqun Feng wrote:
> > However, update_cfs_rq_load_avg() only updates cfs_rq->avg, the change
> > won't be contributed or aggregated to cfs_rq's parent in the
> > for_each_leaf_cfs_rq loop, therefore that's actually not a bottom-up
> > update.
> >=20
> > To fix this, I think we can add a update_cfs_shares(cfs_rq) after
> > update_cfs_rq_load_avg(). Like:
> >=20
> >  	for_each_leaf_cfs_rq(rq, cfs_rq) {
> > -		/*
> > -		 * Note: We may want to consider periodically releasing
> > -		 * rq->lock about these updates so that creating many task
> > -		 * groups does not result in continually extending hold time.
> > -		 */
> > -		__update_blocked_averages_cpu(cfs_rq->tg, rq->cpu);
> > +		/* throttled entities do not contribute to load */
> > +		if (throttled_hierarchy(cfs_rq))
> > +			continue;
> > +
> > +		update_cfs_rq_load_avg(cfs_rq_clock_task(cfs_rq), cfs_rq);
> > +		update_cfs_share(cfs_rq);
> >  	}
> >=20
> > However, I think update_cfs_share isn't cheap, because it may do a
> > bottom-up update once called. So how about just update the root cfs_rq?
> > Like:
> >=20
> > -	/*
> > -	 * Iterates the task_group tree in a bottom up fashion, see
> > -	 * list_add_leaf_cfs_rq() for details.
> > -	 */
> > -	for_each_leaf_cfs_rq(rq, cfs_rq) {
> > -		/*
> > -		 * Note: We may want to consider periodically releasing
> > -		 * rq->lock about these updates so that creating many task
> > -		 * groups does not result in continually extending hold time.
> > -		 */
> > -		__update_blocked_averages_cpu(cfs_rq->tg, rq->cpu);
> > -	}
> > +	update_cfs_rq_load_avg(rq_clock_task(rq), rq->cfs_rq);
>=20
> Hi Boqun,
>=20
> Did I get you right:
>=20
> This rewrite patch does not NEED to aggregate entity's load to cfs_rq,
> but rather directly update the cfs_rq's load (both runnable and blocked),
> so there is NO NEED to iterate all of the cfs_rqs.

Actually, I'm not sure whether we NEED to aggregate or NOT.

>=20
> So simply updating the top cfs_rq is already equivalent to the stock.
>=20

The stock does have a bottom up update, so simply updating the top
cfs_rq is not equivalent to it. Simply updateing the top cfs_rq is
equivalent to the rewrite patch, because the rewrite patch lacks of the
aggregation.

> It is better if we iterate the cfs_rq to update the actually weight
> (update_cfs_share), because the weight may have already changed, which
> would in turn change the load. But update_cfs_share is not cheap.
>=20
> Right?

You get me right for most part ;-)

My points are:

1. We *may not* need to aggregate entity's load to cfs_rq in
update_blocked_averages(), simply updating the top cfs_rq may be just
fine, but I'm not sure, so scheduler experts' insights are needed here.

2. Whether we need to aggregate or not, the update_blocked_averages() in
the rewrite patch could be improved. If we need to aggregate, we have to
add something like update_cfs_shares(). If we don't need, we can just
replace the loop with one update_cfs_rq_load_avg() on root cfs_rq.

I think we'd better to figure out the "may not" part in point 1 first to
get a reasonable implemenation of update_blocked_averages().

Is that clear now?

Thanks and Best Regards,
Boqun

--OXfL5xGRrasGEqWY
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAABCAAGBQJVg8thAAoJEEl56MO1B/q4BEkIAKyk/C9UoNQAFwEEo9uwxHgT
Y/z12fo2cLkDZ08ISiFOklETAQb6p2IyHs72Gxvzi6BAhRoGFFALpicqlP+mSpbL
xPQliDKqAHQrUEa29aD/rYba0uVqkCedwla3tfqdUUpnSFbnxSzuDchNv4RuZw6g
sPtq4bhpQNliiXNMYUerFlSY6gGWbOx9DbPUZWmooVITFWxo2gnwBG7jY2uLM4T5
c3k+VIrb5UhN+BKoU1z2ajeCShYVnvDRAIdtDnTwfs0tfwCLqVp4KTV4VJYc3C1X
HxHd27IwwQMClgGzFqh2pbIkcQhS585JZEGfUw8fEZpwa156kW/2FcTFqdvH6A4=
=Ixul
-----END PGP SIGNATURE-----

--OXfL5xGRrasGEqWY--
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/