Received-SPF: pass (google.com: domain of linux-kernel+bounces-97215-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Received-SPF: Pass (protection.outlook.com: domain of arm.com designates
 63.35.35.123 as permitted sender) receiver=protection.outlook.com;
 client-ip=63.35.35.123; helo=64aa7808-outbound-1.mta.getcheckrecipient.com;
 pr=C
Authentication-Results-Original: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
Message-ID: <5a32e8e1-67cf-4296-a655-f0fc35dc880a@arm.com>
Date: Fri, 8 Mar 2024 15:11:38 +0000
User-Agent: Mozilla Thunderbird
Subject: Re: [RFC] sched/eevdf: sched feature to dismiss lag on wakeup
Content-Language: en-US
To: Tobias Huschle <huschle@linux.ibm.com>, linux-kernel@vger.kernel.org
Cc: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com,
 vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org,
 bsegall@google.com, mgorman@suse.de, bristot@redhat.com,
 vschneid@redhat.com, sshegde@linux.vnet.ibm.com, srikar@linux.vnet.ibm.com,
 linuxppc-dev@lists.ozlabs.org, nd <nd@arm.com>
References: <20240228161018.14253-1-huschle@linux.ibm.com>
From: Luis Machado <luis.machado@arm.com>
In-Reply-To: <20240228161018.14253-1-huschle@linux.ibm.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
NoDisclaimer: true
X-Microsoft-Antispam-Message-Info-Original:
 /0HRHlroZ3WUl/gY3PFxlNR9tCwV7V1GUSCv9hwg7D807vGN8dcGnnLixNHFQv2GCS02bZVSB6WjoNYU6KsBwDnPk5RlHiPkL9HzFfMNr5+a9sfAhf2YE59xcURy2V4W2RjqdielNfCTRGSG3Z4XyB2tBg/v74YuqGGMXvGxFUdnJJEAOS0Ni9Oua30yz5LsymXcRvj9uLM76QWcRFZQAhUVWFem8TlSWxfc8opd5QFf3/m4lbC4uOcFf2dxHxSZmGl3TAIEbXbW0rR2n8RgXr3kUw3YKdqIqeeJ3m2lcdFJxTxDJqLHzZndIu22NTUD2ZYT+7g9KJPTQzzCPv88quBftckF8FRxLbTXiszWvcM07Vr85t9fpZL8Nh0BWUvaJO0Pft7As9Kt33PFcTZ7FQtpdqPEXhcBQmuy/WhUcIT9tbH6zB7VG/wJhlhjBvegRh9DuKbilwJHP045BPjUlIvmwck83sH7esT5ojhv5yzi0ssmLApHI+c3xTzv0ISaMmNxkbMUG9UG4PDWy0c9L4sVIl+sN1bRz9MR7AtMYqGUZgK+hVAM6PMwr4DPjfa7wPJQAbduA1VOiRL7IQXjJYu/UH2FCNGglMfNaOdfbic=
X-Forefront-Antispam-Report-Untrusted:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:VI1PR08MB3919.eurprd08.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(1800799015)(7416005)(376005);DIR:OUT;SFP:1101;
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAVPR08MB8968
Original-Authentication-Results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=arm.com;
X-EOPAttributedMessage: 0
X-MS-Exchange-Transport-CrossTenantHeadersStripped:
 DU6PEPF00009524.eurprd02.prod.outlook.com
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id-Prvs:
	5c576016-3956-419b-bce6-08dc3f821339
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info:
	LVTYQPgeJFohz/dRHCkOgHnXj+uciZ9j9HYZ5wW8FWKzAeNJQ0W1NzNgAPd3GzRmsHtMsrxqPJnUZ7O85uAspw/vLxZ8ez3ZsN2S1hH+CjP5PDkjZhnFbhUjkuriQ23+hq4W+A2djvLgnbXCQP3FuQmQLrcQ6WtV/fCuPrDc0HjWlZpHBTT32ZF3vVYg3JIrqL0/HV0crf/i2QppubfuLfOleBHz/HRPxkTgMzolPoljIMFMMwZIoP3n3lQgOUHpbEzE4jtfbpLm0MPiaTcnnwF8voIWT88bKHtl+Eg6Vhux46lVQjKgf9YCRfVGdAhevbZesufSz/GlJmv7ELfNZ0H7caPFMbzI5SZMMcBjF/okU2TsJskX5GoTShp2RNdt8Q8BKVGm9JVg9OkAN4VI5VDLF9IuL0OiWyrPvcqTkbeZKrribsQ2OpZ389BbAjnpKqLKVP3vAy7GgIpvwAuDwPlxO3QFMw02whE40xJepGlfkZDLbo71i9nKu441ksWYLwLSwTV7JP2E1O7Z/5GUQ94Lb5NADBBesnlJj3Ooji48ElPemcjXh8E/v6DThoxqtbyvUyt1QLb7Y/mdXSktotrM/IxLSPiflX3BG43+Jx+r6sZ4gqynCrXZ9phT+pQfM1zB154Ut685KzbcMWiT66CTvclp30axXGwxffz4DCKclL7BE4MWChYLqMP9BKvJvscIMCDUZ/mvfZhMkux8KQ==
X-Forefront-Antispam-Report:
	CIP:63.35.35.123;CTRY:IE;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:64aa7808-outbound-1.mta.getcheckrecipient.com;PTR:ec2-63-35-35-123.eu-west-1.compute.amazonaws.com;CAT:NONE;SFS:(13230031)(1800799015)(376005)(36860700004)(82310400014);DIR:OUT;SFP:1101;
X-OriginatorOrg: arm.com
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Mar 2024 15:12:03.3108
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: e172a55c-fa3d-42a8-935a-08dc3f821c50
X-MS-Exchange-CrossTenant-Id: f34e5979-57d9-4aaa-ad4d-b122a662184d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=f34e5979-57d9-4aaa-ad4d-b122a662184d;Ip=[63.35.35.123];Helo=[64aa7808-outbound-1.mta.getcheckrecipient.com]
X-MS-Exchange-CrossTenant-AuthSource:
	DU6PEPF00009524.eurprd02.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM9PR08MB6082

Hi Tobias,

On 2/28/24 16:10, Tobias Huschle wrote:
> The previously used CFS scheduler gave tasks that were woken up an
> enhanced chance to see runtime immediately by deducting a certain value
> from its vruntime on runqueue placement during wakeup.
> 
> This property was used by some, at least vhost, to ensure, that certain
> kworkers are scheduled immediately after being woken up. The EEVDF
> scheduler, does not support this so far. Instead, if such a woken up
> entitiy carries a negative lag from its previous execution, it will have
> to wait for the current time slice to finish, which affects the
> performance of the process expecting the immediate execution negatively.
> 
> To address this issue, implement EEVDF strategy #2 for rejoining
> entities, which dismisses the lag from previous execution and allows
> the woken up task to run immediately (if no other entities are deemed
> to be preferred for scheduling by EEVDF).
> 
> The vruntime is decremented by an additional value of 1 to make sure,
> that the woken up tasks gets to actually run. This is of course not
> following strategy #2 in an exact manner but guarantees the expected
> behavior for the scenario described above. Without the additional
> decrement, the performance goes south even more. So there are some
> side effects I could not get my head around yet.
> 
> Questions:
> 1. The kworker getting its negative lag occurs in the following scenario
>    - kworker and a cgroup are supposed to execute on the same CPU
>    - one task within the cgroup is executing and wakes up the kworker
>    - kworker with 0 lag, gets picked immediately and finishes its
>      execution within ~5000ns
>    - on dequeue, kworker gets assigned a negative lag
>    Is this expected behavior? With this short execution time, I would
>    expect the kworker to be fine.

That strikes me as a bit odd as well. Have you been able to determine how a negative lag
is assigned to the kworker after such a short runtime?

I was looking at a different thread (https://lore.kernel.org/lkml/20240226082349.302363-1-yu.c.chen@intel.com/) that
uncovers a potential overflow in the eligibility calculation. Though I don't think that is the case for this particular
vhost problem.