Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
References: <865de121-8190-5d30-ece5-3b097dc74431@kernel.dk> <jhjpnaqtztl.mognet@arm.com> <20200526200015.GG325280@hirez.programming.kicks-ass.net>
User-agent: mu4e 0.9.17; emacs 26.3
From:   Valentin Schneider <valentin.schneider@arm.com>
To:     Peter Zijlstra <peterz@infradead.org>
Cc:     Jens Axboe <axboe@kernel.dk>, Ingo Molnar <mingo@redhat.com>,
        "linux-kernel\@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Stefano Garzarella <sgarzare@redhat.com>
Subject: Re: [PATCH] sched/fair: don't NUMA balance for kthreads
In-reply-to: <20200526200015.GG325280@hirez.programming.kicks-ass.net>
Date:   Wed, 27 May 2020 00:42:40 +0100
Message-ID: <jhjlfletg9b.mognet@arm.com>
MIME-Version: 1.0
Content-Type: text/plain
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk


On 26/05/20 21:00, Peter Zijlstra wrote:
> On Tue, May 26, 2020 at 05:40:06PM +0100, Valentin Schneider wrote:
>
>> > Change the task_tick_numa() check to exclude kernel threads in general,
>> > as it doesn't make sense to attempt ot balance for kthreads anyway.
>> >
>>
>> Does it? (this isn't a rethorical question)
>>
>> Suppose a given kthread ends up doing more accesses to some pages
>> (via use_mm()) than the other threads that access them, wouldn't it make
>> sense to take that into account when it comes to NUMA balancing?
>
> Well, task_tick_numa() tries and farm off a bunch of actual work to
> task_work_add(), and there's so very little userspace for a kernel
> thread to return to... :-)

Err, true... I did say pipe dreams!

I had only really taken note of the exit / return to userspace
callbacks, but I see io_uring has its own task_work_run() calls, which
(I think) explains how we can end up with a kthread actually running
task_numa_work().

I'm also thinking we really don't want that task_numa_work() to be left
hanging on the task_work list, because that self-looping thing will not
play nice to whatever else has been queued (which AFAICT shouldn't happen
under normal conditions, i.e. !kthreads).