Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Sender: Tejun Heo <htejun@gmail.com>
Date:   Tue, 6 Dec 2022 11:42:27 -1000
From:   Tejun Heo <tj@kernel.org>
To:     Barret Rhoden <brho@google.com>
Cc:     torvalds@linux-foundation.org, mingo@redhat.com,
        peterz@infradead.org, juri.lelli@redhat.com,
        vincent.guittot@linaro.org, dietmar.eggemann@arm.com,
        rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de,
        bristot@redhat.com, vschneid@redhat.com, ast@kernel.org,
        daniel@iogearbox.net, andrii@kernel.org, martin.lau@kernel.org,
        joshdon@google.com, pjt@google.com, derkling@google.com,
        haoluo@google.com, dvernet@meta.com, dschatzberg@meta.com,
        dskarlat@cs.cmu.edu, riel@surriel.com,
        linux-kernel@vger.kernel.org, bpf@vger.kernel.org,
        kernel-team@meta.com
Subject: Re: [PATCH 14/31] sched_ext: Implement BPF extensible scheduler class
Message-ID: <Y4+3Q2FnG1bsGH55@slm.duckdns.org>
References: <20221130082313.3241517-1-tj@kernel.org>
 <20221130082313.3241517-15-tj@kernel.org>
 <8d146099-a12a-c5a1-4829-dec95497fdca@google.com>
 <Y4o9gV2v8eyI1arK@slm.duckdns.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Y4o9gV2v8eyI1arK@slm.duckdns.org>
Precedence: bulk

Hello, Barret.

On Fri, Dec 02, 2022 at 08:01:37AM -1000, Tejun Heo wrote:
> > you still end up grabbing both locks, but just not at the same time.
> 
> Yeah, this probably would look better than the current double lock dancing,
> especially in the finish_dispatch() path.
>
> > plus, task_rq_lock() takes the guesswork out of whether you're getting p's
> > rq lock or not.  it looks like you're using the holding_cpu to handle the
> > race where p moves cpus after you read task_rq(p) but before you lock that
> > task_rq.  maybe you can drop the whole concept of the holding_cpu?
> 
> ->holding_cpu is there to basically detect intervening dequeues, so if we
> lock them out with TASK_ON_RQ_MIGRATING, we might be able to drop it. I need
> to look into it more tho. Things get pretty subtle around there, so I could
> easily be missing something. I'll try this and let you know how it goes.

I tried both and I'm pretty ambivalent. The problem is that the
finish_dispatch() path can't use TASK_ON_RQ_MIGRATING the way the consume
path does because the dispatch path isn't starting with the task locked. The
only claim it has to the task is through p->scx.ops_state.

It can be argued that getting rid of double locking is still nice but given
that holding_cpu is needed anyway and can play the same role, I'm not sure
how attractive it is. I suppose we can go either way.

Thanks.

-- 
tejun