From:   Benjamin Segall <bsegall@google.com>
To:     shrikanth hegde <sshegde@linux.vnet.ibm.com>
Cc:     mingo@redhat.com, peterz@infradead.org, vincent.guittot@linaro.org,
        dietmar.eggemann@arm.com, tglx@linutronix.de,
        srikar@linux.vnet.ibm.com, arjan@linux.intel.com,
        svaidy@linux.ibm.com, linux-kernel@vger.kernel.org
Subject: Re: [RFC PATCH] sched/fair: Interleave cfs bandwidth timers for
 improved single thread performance at low utilization
References: <20230214120502.934324-1-sshegde@linux.vnet.ibm.com>
        <xm268rh06i97.fsf@google.com>
        <cd37483e-bf11-ec74-c240-74935bb44809@linux.vnet.ibm.com>
Date:   Wed, 15 Feb 2023 13:32:12 -0800
In-Reply-To: <cd37483e-bf11-ec74-c240-74935bb44809@linux.vnet.ibm.com>
        (shrikanth hegde's message of "Wed, 15 Feb 2023 16:31:29 +0530")
Message-ID: <xm26bklu4ntv.fsf@google.com>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
Precedence: bulk

shrikanth hegde <sshegde@linux.vnet.ibm.com> writes:

>>>
>>>              6.2.rc5                           with patch
>>>         1CG    power   2CG    power   | 1CG  power     2CG        power
>>> 1Core   218     44     315      46    | 219    45    277(+12%)    47(-2%)
>>>         219     43     315      45    | 219    44    244(+22%)    48(-6%)
>>> 	                              |
>>> 2Core   108     48     158      52    | 109    50    114(+26%)    59(-13%)
>>>         109     49     157      52    | 109    49    136(+13%)    56(-7%)
>>>                                       |
>>> 4Core    60     59      89      65    |  62    58     72(+19%)    68(-5%)
>>>          61     61      90      65    |  62    60     68(+24%)    73(-12%)
>>>                                       |
>>> 8Core    33     77      48      83    |  33    77     37(+23%)    91(-10%)
>>>          33     77      48      84    |  33    77     38(+21%)    90(-7%)
>>>
>>> There is no benefit at higher utilization of 50% or more. There is no
>>> degradation also.
>>>
>>> This is RFC PATCH V2, where the code has been shifted from hrtimer to
>>> sched. This patch sets an initial value as multiple of period/10.
>>> Here timers can still align if the time started the cgroup is within the
>>> period/10 interval. On a real life workload, time gives sufficient
>>> randomness. There can be a better interleaving by being more
>>> deterministic. For example, when there are 2 cgroups, they should
>>> have initial value of 0/50ms or 10/60ms so on. When there are 3 cgroups,
>>> 0/3/6ms or 1/4/7ms etc. That is more complicated as it has to account
>>> for cgroup addition/deletion and accuracy w.r.t to period/quota.
>>> If that approach is better here, then will come up with that patch.
>> 
>> This does seem vaguely reasonable, though the power argument of
>> consolidating wakeups and such is something that we intentionally do in
>> other situations.
>> 
> Thank you Benjamin for taking a look and spending time in reviewing this.
>> How reasonable do you think it is to just say (and what do the
>> equivalent numbers look like on your particular benchmark) "put some
>> variance on your period config if you want variance"?
>>Run to run variance is expected with this patch as the patch depends
> on time upto last period/10 as the basis for interleaving. 
> What i could infer from this comment about variance. Please correct if not.

My question is what the numbers look like if you instead prepare the
cgroups with periods that are something like 97 ms and 103ms instead of
both 100ms (keeping the quota as the same proportion as the original).