Received: by 2002:a05:6a10:9e8c:0:0:0:0 with SMTP id y12csp1043376pxx; Tue, 27 Oct 2020 07:02:56 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx6tc05nSG933SUKgIGQagB9L7jAdc+ecgCxdWnqnC1EYCFKMw3kcYfKArYSOAd6FDofc7m X-Received: by 2002:a2e:9dd1:: with SMTP id x17mr1092962ljj.219.1603807376195; Tue, 27 Oct 2020 07:02:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1603807376; cv=none; d=google.com; s=arc-20160816; b=Sd90OU8wVFeev9NZuHU8mZSAA+8R9QP3FLim49yA5QbQx/1wqe09jJOSPnRCHRlp6a DlNZE5lK5Bq5n8ahF8SrJsEIuL8va0pSV/dYnxWVY7WeMtoAYnCvthvndYAITEhhsFjf Ez/S6wSFSTm692XOQ/WmONKtd3M9STyjgND/d9Q9Ro+VRxelzsMaXtOD4CN8m4Cf2+gf R+oRHfwLJGvViWGFauEC8Fn7uBnvBbqd4Tfq7QJWrb7fNOd0InrsXeXPe4ijDfmBZXV8 qX2YItZWVANTdPrjNSsxLsk/5qDWJTDa7Kvjdag8rTPvgYGkOtQicLQqu1vDRIZnLR2h 3Iig== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:content-language :in-reply-to:mime-version:user-agent:date:message-id:from:references :cc:to:subject:ironport-sdr:ironport-sdr; bh=jJqD1UzCFFwxJpjiW1JsT0bA/48gReSBu6+BiMjuZ2E=; b=X5wMTr4H6XhQ/dk77eCzs4aDC379Gg18K5eRv75wc0LsoayNRURFVyhLECNGC/cP+4 yKt6Hj0odO2FsxOvDzMx1JPUZD5yw/Kt+/MvvaUzvCBMyzPOL6n0lpSRjUXQcIFQve5r cG90pJ0JPRr/apUSO8MNmkDRd/aI135ZVBaHWgjc1DZfn8tHIZiV+DMaZH43s3EAkm3c /XSN9KCYa8LWo1QX2bE7I5fRCycp6v3mgtPbfezU3h8xnC3NsIFKdNylNwWuQ/kB+rm2 5SxMlHmkpG6zDfCJq36inabKhpnKsOz/sRci3OUx3/gCse1vGI9q3wrO7csKe0E4yThR casQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id j2si894408edj.194.2020.10.27.07.02.30; Tue, 27 Oct 2020 07:02:56 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2504513AbgJ0DRP (ORCPT + 99 others); Mon, 26 Oct 2020 23:17:15 -0400 Received: from mga07.intel.com ([134.134.136.100]:45687 "EHLO mga07.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2442515AbgJ0DRP (ORCPT ); Mon, 26 Oct 2020 23:17:15 -0400 IronPort-SDR: xZxNBoxQEC9cVmxyt2e22kdUlrErOq/ZCiX+oIGe5IbVAauibxHcmle0u1aoBW0Zv8xEofi4/V JfZ87dsx0cUw== X-IronPort-AV: E=McAfee;i="6000,8403,9786"; a="232208385" X-IronPort-AV: E=Sophos;i="5.77,422,1596524400"; d="scan'208";a="232208385" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 26 Oct 2020 20:17:13 -0700 IronPort-SDR: UmTG3wTrOWapmJw0go0yrBJY7ZFiP2bFASJRNgqwUXoOL59MoDTzftmnvQCCFBb01ADME4nDVd SHgs6zZ+Oj7A== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,422,1596524400"; d="scan'208";a="361226808" Received: from cli6-desk1.ccr.corp.intel.com (HELO [10.239.161.135]) ([10.239.161.135]) by orsmga007.jf.intel.com with ESMTP; 26 Oct 2020 20:17:04 -0700 Subject: Re: [PATCH v8 -tip 02/26] sched: Introduce sched_class::pick_task() To: Peter Zijlstra , Vineeth Pillai Cc: Joel Fernandes , Nishanth Aravamudan , Julien Desfossez , Tim Chen , Aaron Lu , Aubrey Li , Thomas Glexiner , LKML , Ingo Molnar , Linus Torvalds , Frederic Weisbecker , Kees Cook , Greg Kerr , Phil Auld , Valentin Schneider , Mel Gorman , Pawan Gupta , Paolo Bonzini , vineeth@bitbyteword.org, Chen Yu , Christian Brauner , Agata Gruza , Antonio Gomez Iglesias , graf@amazon.com, konrad.wilk@oracle.com, Dario Faggioli , Paul Turner , Steven Rostedt , Patrick Bellasi , =?UTF-8?B?YmVuYmppYW5nKOiSi+W9qik=?= , Alexandre Chartre , James.Bottomley@hansenpartnership.com, OWeisse@umich.edu, Dhaval Giani , Junaid Shahid , Jesse Barnes , "Hyser,Chris" , "Paul E. McKenney" , Tim Chen , "Ning, Hongyu" References: <20201020014336.2076526-1-joel@joelfernandes.org> <20201020014336.2076526-3-joel@joelfernandes.org> <8ea1aa61-4a1c-2687-9f15-1062d37606c7@linux.intel.com> <20201023214702.GA3603399@google.com> <4241e5ac-ecdf-8634-fa0d-dd6759e477e1@linux.microsoft.com> <8230ada7-839f-2335-9a55-b09f6a813e91@linux.microsoft.com> <20201026090131.GE2628@hirez.programming.kicks-ass.net> From: "Li, Aubrey" Message-ID: <3c6e2d04-ca47-299c-1131-31f003909875@linux.intel.com> Date: Tue, 27 Oct 2020 11:17:03 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.9.0 MIME-Version: 1.0 In-Reply-To: <20201026090131.GE2628@hirez.programming.kicks-ass.net> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2020/10/26 17:01, Peter Zijlstra wrote: > On Sat, Oct 24, 2020 at 08:27:16AM -0400, Vineeth Pillai wrote: >> >> >> On 10/24/20 7:10 AM, Vineeth Pillai wrote: >>> >>> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c >>> index 93a3b874077d..4cae5ac48b60 100644 >>> --- a/kernel/sched/fair.c >>> +++ b/kernel/sched/fair.c >>> @@ -4428,12 +4428,14 @@ pick_next_entity(struct cfs_rq *cfs_rq, struct >>> sched_entity *curr) >>>                         se = second; >>>         } >>> >>> -       if (cfs_rq->next && wakeup_preempt_entity(cfs_rq->next, left) < >>> 1) { >>> +       if (left && cfs_rq->next && >>> +                       wakeup_preempt_entity(cfs_rq->next, left) < 1) { >>>                 /* >>>                  * Someone really wants this to run. If it's not unfair, >>> run it. >>>                  */ >>>                 se = cfs_rq->next; >>> -       } else if (cfs_rq->last && wakeup_preempt_entity(cfs_rq->last, >>> left) < 1) { >>> +       } else if (left && cfs_rq->last && >>> +                       wakeup_preempt_entity(cfs_rq->last, left) < 1) { >>>                 /* >>>                  * Prefer last buddy, try to return the CPU to a >>> preempted task. >>> >>> >>> There reason for left being NULL needs to be investigated. This was >>> there from v1 and we did not yet get to it. I shall try to debug later >>> this week. >> >> Thinking more about it and looking at the crash, I think that >> 'left == NULL' can happen in pick_next_entity for core scheduling. >> If a cfs_rq has only one task that is running, then it will be >> dequeued and 'left = __pick_first_entity()' will be NULL as the >> cfs_rq will be empty. This would not happen outside of coresched >> because we never call pick_tack() before put_prev_task() which >> will enqueue the task back. >> >> With core scheduling, a cpu can call pick_task() for its sibling while >> the sibling is still running the active task and put_prev_task has yet >> not been called. This can result in 'left == NULL'. > > Quite correct. Hurmph.. the reason we do this is because... we do the > update_curr() the wrong way around. And I can't seem to remember why we > do that (it was in my original patches). > > Something like so seems the obvious thing to do, but I can't seem to > remember why we're not doing it :-( > > --- a/kernel/sched/fair.c > +++ b/kernel/sched/fair.c > @@ -6950,15 +6950,10 @@ static struct task_struct *pick_task_fai > do { > struct sched_entity *curr = cfs_rq->curr; > > - se = pick_next_entity(cfs_rq, NULL); > + if (curr && curr->on_rq) > + update_curr(cfs_rq); > > - if (curr) { > - if (se && curr->on_rq) > - update_curr(cfs_rq); > - > - if (!se || entity_before(curr, se)) > - se = curr; > - } > + se = pick_next_entity(cfs_rq, curr); > > cfs_rq = group_cfs_rq(se); > } while (cfs_rq); > This patch works too for my benchmark, thanks Peter!