Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 1 Apr 2022 13:46:18 +0200
From:   Peter Zijlstra <peterz@infradead.org>
To:     Joel Fernandes <joel@joelfernandes.org>
Cc:     Ingo Molnar <mingo@kernel.org>,
        Steven Rostedt <rostedt@goodmis.org>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
        Daniel Bristot de Oliveira <bristot@redhat.com>,
        Guenter Roeck <groeck@chromium.org>
Subject: Re: [PATCH] sched/core: Fix forceidle balancing
Message-ID: <YkbmCr6ojXYiWzkP@hirez.programming.kicks-ass.net>
References: <20220330160535.GN8939@worktop.programming.kicks-ass.net>
 <CAEXW_YQNi42gahbSJ1skadh_8D+Ry6ZOmMqSU5BdidfCbmOtRg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAEXW_YQNi42gahbSJ1skadh_8D+Ry6ZOmMqSU5BdidfCbmOtRg@mail.gmail.com>
Precedence: bulk

On Thu, Mar 31, 2022 at 03:00:40PM -0400, Joel Fernandes wrote:
> Hi,
> 
> By the way, might be slightly related - we still see crashes with
> pick_task_fair() in our kernel even with this change:
> https://lkml.org/lkml/2020/11/17/2137

Please as to not use lkml.org. Please use something with a MsgID in like
lore.

> Is it possible that when doing pick_task_fair() especially on a remote
> CPU, both the "cfs_rq->curr" and the rbtree's "left" be NULL with core
> scheduling? In this case, se will be NULL and can cause crashes right?
> I think the code assumes this can never happen.
> 
> +Guenter Roeck  kindly debugged pick_task_fair() in a crash as
> follows. Copying some details he mentioned in a bug report:
> 
> Assembler/source:
> 
>   25:   e8 4f 11 00 00          call   0x1179             ; se =
> pick_next_entity(cfs_rq, curr);
>   2a:*  48 8b 98 60 01 00 00    mov    0x160(%rax),%rbx   ; trapping
> instruction [cfs_rq = group_cfs_rq(se);]
>   31:   48 85 db                test   %rbx,%rbx
>   34:   75 d1                   jne    0x7
>   36:   48 89 c7                mov    %rax,%rdi
> 
> At 2a: RAX = se == NULL after pick_next_entity(). Looking closely into
> pick_next_entity(), it can indeed return NULL if curr is NULL and if
> left in pick_next_entity() is NULL. Per line 7:, curr is in %r14 and
> indeed 0.
> 
> Thoughts?

It is possible for ->curr and ->leftmost to be NULL, but then we should
also be having ->nr_running == 0 and not call pick in the first place.
Because picking a task from no tasks doesn't make much sense.