Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Message-ID: <461f14cafabb7e6f78556f138b6aa619eff12dee.camel@surriel.com>
Subject: Re: [PATCH 09/14] sched,fair: refactor enqueue/dequeue_entity
From:   Rik van Riel <riel@surriel.com>
To:     Peter Zijlstra <peterz@infradead.org>
Cc:     linux-kernel@vger.kernel.org, kernel-team@fb.com, pjt@google.com,
        dietmar.eggemann@arm.com, mingo@redhat.com,
        morten.rasmussen@arm.com, tglx@linutronix.de,
        mgorman@techsingularity.net, vincent.guittot@linaro.org
Date:   Wed, 31 Jul 2019 11:03:01 -0400
In-Reply-To: <20190731093525.GH31425@hirez.programming.kicks-ass.net>
References: <20190722173348.9241-1-riel@surriel.com>
         <20190722173348.9241-10-riel@surriel.com>
         <20190730093617.GV31398@hirez.programming.kicks-ass.net>
         <20190731093525.GH31425@hirez.programming.kicks-ass.net>
Content-Type: multipart/signed; micalg="pgp-sha256";
        protocol="application/pgp-signature"; boundary="=-oXa4WWI8f1hjsu/vRrww"
User-Agent: Evolution 3.30.5 (3.30.5-1.fc29) 
MIME-Version: 1.0
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk


--=-oXa4WWI8f1hjsu/vRrww
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, 2019-07-31 at 11:35 +0200, Peter Zijlstra wrote:
> On Tue, Jul 30, 2019 at 11:36:17AM +0200, Peter Zijlstra wrote:
> > On Mon, Jul 22, 2019 at 01:33:43PM -0400, Rik van Riel wrote:
> > > +static bool
> > > +enqueue_entity_groups(struct cfs_rq *cfs_rq, struct sched_entity
> > > *se, int flags)
> > > +{
> > > +	/*
> > > +	 * When enqueuing a sched_entity, we must:
> > > +	 *   - Update loads to have both entity and cfs_rq synced with
> > > now.
> > > +	 *   - Add its load to cfs_rq->runnable_avg
> > > +	 *   - For group_entity, update its weight to reflect the new
> > > share of
> > > +	 *     its group cfs_rq
> > > +	 *   - Add its new weight to cfs_rq->load.weight
> > > +	 */
> > > +	if (!update_load_avg(cfs_rq, se, UPDATE_TG | DO_ATTACH))
> > > +		return false;
> > > +
> > > +	update_cfs_group(se);
> > > +	return true;
> > > +}
> > No functional, but you did make update_cfs_group() conditional. Now
> > that
> > looks OK, but maybe you can do that part in a separate patch with a
> > little justification of its own.
>=20
> To record (and extend) our discussion from IRC yesterday; I now do
> think
> the above is in fact a problem.
>=20
> The thing is that update_cfs_group() does not soly rely on the tg
> state;
> it also contains magic to deal with ramp up; for which you later
> introduce that max_h_load thing.
>=20
> Specifically (re)read the second part of the comment describing
> calc_group_shares() where it explains the ramp up:
>=20
>  * The problem with it is that because the average is slow -- it was
> designed
>  * to be exactly that of course -- this leads to transients in
> boundary
>  * conditions. In specific, the case where the group was idle and we
> start the
>  * one task. It takes time for our CPU's grq->avg.load_avg to build
> up,
>  * yielding bad latency etc..
>=20
>  (and further)
>=20
> So by not always calling this (and not updating h_load) you fail to
> take
> advantage of this.
>=20
> So I would suggest keeping update_cfs_group() unconditional, and
> recomputing the h_load for the entire hierarchy on enqueue.

I think I understand the problem you are pointing
out, but if update_load_avg() keeps the load average
for the runqueue unchanged (because that is rate limited
to once a jiffy, and has been like that for a while),
why would calc_group_shares() result in a different value
than what it returned the last time?

What am I overlooking?

--=20
All Rights Reversed.

--=-oXa4WWI8f1hjsu/vRrww
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNATURE-----

iQEzBAABCAAdFiEEKR73pCCtJ5Xj3yADznnekoTE3oMFAl1BraUACgkQznnekoTE
3oOxoAgAwFOZGyaVohHl1N71K7Sxs/txPqDCXWskwEsXf2FP2a74KXGMDifkyUR9
otdJ/c+UvL5e3rXRdLEjoJMvvuMRSITTY8Y95bysxnFpiChBBfmEfkR0+UWzqmmH
rl+crbatn0Kh6HdENMtGfP2A2lyNApY2q42o1MB2oC0ktYFvIgtqR2z/WXpLMwEl
ltg3zt35OaMAYQZ1JbpU2aHQdqW1IAqUwbwBWdp2w+hq3fNehLMhzazTGM2OvQp5
+nChB75ZAHHJcW0Dfs9WSU1HFT3q9nlEgS7/zTvhss6nh366eww5YRIjKxrAZqxQ
Ks33MF8l9r9OXxlmjdxvygwxEvJe7g==
=l4Jy
-----END PGP SIGNATURE-----

--=-oXa4WWI8f1hjsu/vRrww--