Received-SPF: pass (google.com: domain of linux-kernel+bounces-150545-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Message-ID: <c2c9e554-7a2c-ebbd-ab4b-659737caab92@amd.com>
Date: Thu, 18 Apr 2024 22:38:49 +0530
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.13.1
Subject: Re: [RFC][PATCH 08/10] sched/fair: Implement delayed dequeue
Content-Language: en-US
To: Mike Galbraith <efault@gmx.de>, Peter Zijlstra <peterz@infradead.org>,
 mingo@redhat.com, juri.lelli@redhat.com, vincent.guittot@linaro.org,
 dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com,
 mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com,
 linux-kernel@vger.kernel.org
Cc: wuyun.abel@bytedance.com, tglx@linutronix.de,
 Chen Yu <yu.c.chen@intel.com>, Oliver Sang <oliver.sang@intel.com>
References: <20240405102754.435410987@infradead.org>
 <20240405110010.631664251@infradead.org>
 <557be85d-e1c1-0835-eebd-f76e32456179@amd.com>
 <ec6f811b9977eeef1b3f1b3fb951fda066fd95f5.camel@gmx.de>
 <14330cf4-8d9e-1e55-7717-653b800e5cee@amd.com>
 <747627a1414f1f33d0c237f555494149d6937800.camel@gmx.de>
 <2b9f7617f2b2130bb6270504ec3858f15d463f1d.camel@gmx.de>
From: K Prateek Nayak <kprateek.nayak@amd.com>
In-Reply-To: <2b9f7617f2b2130bb6270504ec3858f15d463f1d.camel@gmx.de>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?cTZDVDAwVkpSSEthWkRhWG1RNjNHN2ovajFhUkZxT29IK0VlRUkvSjhvZDlk?=
 =?utf-8?B?YkZVREFaY2I0NTJlQkp5Sm1YeW0raE5jaWpLL3pUVC9tOWUxSzd0YnpkelhL?=
 =?utf-8?B?b05uNmxaQlAxaHU5R0JhZjI1QXY3azhPQ0pLUzdKMENkRitmZy9BWjV3S0JP?=
 =?utf-8?B?T3NWQ0cwY1VUS1gvOUU4ZFNaM0p0NVVpbXRyblB5OWlTdENsaExXT0F2QmM4?=
 =?utf-8?B?dHZXQm1EaFE1ak0vWXBIaWhOanY0L01kNTkyalRYN0FQTm9PRUQ5UEJUWjJD?=
 =?utf-8?B?VVNMazd1MGw2eEFWMXhtdzZqRU5FUlBpRHdic0JtcHBTL1BleGNibitabnFx?=
 =?utf-8?B?Wms3VHI1TFhSNXlDQ1dGT3F3K0FXV3pBbDVhaWRxN0JFVHhKWS96L1dQQm9Q?=
 =?utf-8?B?MzAwV2pZVFVlWFNXQTFxcDJlTmpHc3c3Sm1pZHNlMXp2ZmFORFBmRW0wZmhQ?=
 =?utf-8?B?a3VWN0VyOWNrdjRHdElUejBPUDFiU0RuNzJ6SWZyYVBOQVRJakxEa2xxTHVS?=
 =?utf-8?B?N0ZKTnI5b0xHZUQrUUZhZVdlc00xL2VEK3h5RkxDb1BEeHFSeWdHSnBuKzJJ?=
 =?utf-8?B?VGZEOGlaaURXcVNTdFd4TzJJTG83OXFIZ2pLak0wdDVoeDRpNW1PWEdZcG9t?=
 =?utf-8?B?Mm9GQVMwTG5qenNiTzc5UkxHUHVNZkM5YWFhQXVLT09XMXY3YTZxeHZidGxv?=
 =?utf-8?B?WVpmS2JJUGYrMERhc1pKcHIwTGdhSzh6bnE3S2xhdjNld2ZSeW1SRWhSYWtL?=
 =?utf-8?B?MjdFWWxVZmlScmlrRW5mUmJsejVXVFdkMTNGR0NjK1RYVE4rK2h0Sm1XQm1w?=
 =?utf-8?B?czNyd2VodUxzeHJ0ei9OendMMVVRQTErVkhRY2pzdFNnYnNFYjc1RnZ0bVV5?=
 =?utf-8?B?QUFNNlFtS0pTQ3JGTEh6U2R5TDZUOVdhL1BES3prNmZjV21GdkdWbURLM2x5?=
 =?utf-8?B?TkltQjAxL1ptSUZoNDBjWjQ0dnk4Ui9kay8xM0VxcHJXUFF6bndRN3MyOHZv?=
 =?utf-8?B?d01rY3JNRVlmN2VqNmRWYVRiemY3M2crTG90WU9YbUxTdUxPYVJWd2xQTUNh?=
 =?utf-8?B?SDhqZlBnTHVRSjJVR1dEUlVYSFNnVGxPZllLcTNZOTBoYjlpL2UvUDZtZVht?=
 =?utf-8?B?cDlWZE05dG5aY1l3RnVIWjdPQkZtV2hrYTc1SUxydnFCU1c1T2RGOWVaeG1M?=
 =?utf-8?B?L2lqOFJ6V1BCUlg2QkFHQmdjc2t3MWFUUTM4MTQyNVFoLzJ3TXZjYnVWQUlV?=
 =?utf-8?B?dWVCRWtycmQzT0M5SG5kcFBlOGJCV21wZUlGakJtUEpIZUpLQnJ0QVRZZFAw?=
 =?utf-8?B?ZUFTSStEdUkwUytjWDEwVEhUdUV6QmpBdEwxM2FiVzBDaDJub0thWERqaHV5?=
 =?utf-8?B?YysyZFVidlpYKzRReGlRQW90SUttWGZCWHVyRENSL3VqYXpadmNpWVB6VmhR?=
 =?utf-8?B?T05SL1hiczNaYm16S2oyUUErSG5qYjg5OWduWitab2prZGtmV3ZOMFlUTVEy?=
 =?utf-8?B?dVUxb29IYlE3Wll6aldVVTBqQUpsc3BsQk1LZnY4U3VoMkFKSXQ5aFZqUG5v?=
 =?utf-8?B?UFg1Y1BISTBLYkM3NW1XcFhSUkVhVFV4YjZBenRnbjcvNVpLbEpaS0tKM21h?=
 =?utf-8?B?YW1qaEtCM1JuZEtkTkVwSElzekltajd3UG5Oc1U2RERwRER4cnZlNWJzckcy?=
 =?utf-8?B?VWF1T3NoVkVBUFIzelBVTHFhUGhHR0JJbkJWYjlxbkJ5M1pRNnlsSTMwUGs3?=
 =?utf-8?B?UmhLbmlUek5XazlRYzh6anFxQ3ZnT2JScUVaczlveDhzTGpvTnhXcE1PNTlv?=
 =?utf-8?B?NDBIQllmY3JDU1pCTmJQZWs5MGhRQUk4TDIwS3lTaWh2SEFBNTg5R1N6Zktu?=
 =?utf-8?B?ZS9Jc3hBdGF4V3FUSktUL1Awb3ZmQ1E5ZmY5ZENuejJrVTlxUlBrbnE2QWdu?=
 =?utf-8?B?UUNIRGRRSXg4RGdKZG55WldObjJyeDdFQnVqM0N6SFVSby9YSjNnY0ZyTEZO?=
 =?utf-8?B?UUVheVowejYra3ZRUzAzTkhOU28vOEtOY0xSSFQ0SnNLWFN6WmxhY0NmNFN2?=
 =?utf-8?B?WStMQmJCR0dlR3duV3NONm1oU2wvUVhkS21ZMC9jUFQrcllNdmFrVGRFZDNI?=
 =?utf-8?Q?xleZy6kinOGDSJY7HW1HS3A1e?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c68badf7-3126-49fe-432e-08dc5fca3df1
X-MS-Exchange-CrossTenant-AuthSource: MW2PR12MB2379.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Apr 2024 17:09:01.0595
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: IpTM9Ty3a/IZhmLfOjzFXzoCDHNIv8PUbNzUoxF66jF/xKQOw6wsC3wZvFOHkVRvhcLXZd6Y33s5E8Fxn7whaA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB5607

Hello Mike,

On 4/18/2024 9:54 PM, Mike Galbraith wrote:
> Greetings,
> 
> I tossed a couple rocks at it today, and seem to have hit the little
> bugger.  The root cause seems to be doing the delay dequeue business on
> exiting tasks.  Hunk #1 of hacklet below seems to quell the explosions.
> 
> crash> bt -sx
> PID: 21722    TASK: ffff88815710ee40  CPU: 6    COMMAND: "hackbench"
>  #0 [ffff88822773bb90] machine_kexec+0x156 at ffffffff810642d6
>  #1 [ffff88822773bbe0] __crash_kexec+0xd7 at ffffffff81152a07
>  #2 [ffff88822773bc98] crash_kexec+0x23 at ffffffff81152eb3
>  #3 [ffff88822773bca8] oops_end+0xbe at ffffffff810329be
>  #4 [ffff88822773bcc8] page_fault_oops+0x81 at ffffffff81071951
>  #5 [ffff88822773bd28] exc_page_fault+0x62 at ffffffff8194e9e2
>  #6 [ffff88822773bd50] asm_exc_page_fault+0x22 at ffffffff81a00ba2
>     [exception RIP: pick_next_task_fair+178]
>     RIP: ffffffff810d8d12  RSP: ffff88822773be00  RFLAGS: 00010006
>     RAX: ffff88813cb780b8  RBX: ffff88840edb0e80  RCX: 0000000000000000
>     RDX: 0000000000000000  RSI: ffff88813cb78080  RDI: ffff88840ec30f00
>     RBP: ffff88813cb78000   R8: ffff88815710eec0   R9: 0000000000000001
>     R10: ffff88822773bdc8  R11: 0000000000000013  R12: 0000000000030e80
>     R13: ffff88815710ee40  R14: ffff88813cb78080  R15: ffff88815710ee40
>     ORIG_RAX: ffffffffffffffff  CS: 0010  SS: 0018
>  #7 [ffff88822773be28] __schedule+0x10d at ffffffff8195865d
>  #8 [ffff88822773be98] do_task_dead+0x3e at ffffffff810cc00e
>  #9 [ffff88822773beb0] do_exit+0x770 at ffffffff8108f0e0
> #10 [ffff88822773bf00] do_group_exit+0x2c at ffffffff8108f64c
> #11 [ffff88822773bf28] __x64_sys_exit_group+0x14 at ffffffff8108f6f4
> #12 [ffff88822773bf30] do_syscall_64+0x57 at ffffffff8194ac37
> #13 [ffff88822773bf50] entry_SYSCALL_64_after_hwframe+0x76 at ffffffff81a0012b
>     RIP: 00007f4f2aa76136  RSP: 00007ffcbba84748  RFLAGS: 00000246
>     RAX: ffffffffffffffda  RBX: 0000000000000000  RCX: 00007f4f2aa76136
>     RDX: 0000000000000000  RSI: 000000000000003c  RDI: 0000000000000000
>     RBP: 00007f4f2ab86970   R8: 00000000000000e7   R9: ffffffffffffff80
>     R10: 0000000000000004  R11: 0000000000000246  R12: 00007f4f2ab86970
>     R13: 0000000000000001  R14: 00007f4f2ab8a328  R15: 0000000000000000
>     ORIG_RAX: 00000000000000e7  CS: 0033  SS: 002b
> crash> task_struct ffff88815710ee40 | grep sched_delayed
>     sched_delayed = 1 '\001',
> crash>
> 

My machine too survived the spawn test with the below patch. Thank you
for looking into this. I'll resume testing with the below patch
included.

--
Thanks and Regards,
Prateek

> ---
>  kernel/sched/fair.c |    5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -5374,6 +5374,7 @@ dequeue_entity(struct cfs_rq *cfs_rq, st
>  		update_curr(cfs_rq);
> 
>  		if (sched_feat(DELAY_DEQUEUE) && sleep &&
> +		    !(entity_is_task(se) && (task_of(se)->flags & PF_EXITING)) &&
>  		    !entity_eligible(cfs_rq, se)) {
>  			if (cfs_rq->next == se)
>  				cfs_rq->next = NULL;
> @@ -5495,14 +5496,14 @@ pick_next_entity(struct rq *rq, struct c
>  	}
> 
>  	struct sched_entity *se = pick_eevdf(cfs_rq);
> -	if (se->sched_delayed) {
> +	while (se && se->sched_delayed) {
>  		dequeue_entities(rq, se, DEQUEUE_SLEEP | DEQUEUE_DELAYED);
>  		SCHED_WARN_ON(se->sched_delayed);
>  		SCHED_WARN_ON(se->on_rq);
>  		if (sched_feat(DELAY_ZERO) && se->vlag > 0)
>  			se->vlag = 0;
> 
> -		return NULL;
> +		se = pick_eevdf(cfs_rq);
>  	}
>  	return se;
>  }
>