Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <244e207a-95d5-2ff3-d0ec-c974538032af@redhat.com>
Date:   Wed, 5 Jul 2023 10:07:34 -0400
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.7.1
Subject: Re: [PATCH] sched/core: Use empty mask to reset cpumasks in
 sched_setaffinity()
Content-Language: en-US
To:     Peter Zijlstra <peterz@infradead.org>
Cc:     Ingo Molnar <mingo@redhat.com>, Juri Lelli <juri.lelli@redhat.com>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
        Daniel Bristot de Oliveira <bristot@redhat.com>,
        Valentin Schneider <vschneid@redhat.com>,
        linux-kernel@vger.kernel.org, Phil Auld <pauld@redhat.com>,
        Brent Rowsell <browsell@redhat.com>,
        Peter Hunt <pehunt@redhat.com>
References: <20230628211637.1679348-1-longman@redhat.com>
 <20230703102604.GC4253@hirez.programming.kicks-ass.net>
 <5bc41342-5ba6-68e9-8315-9e5cef65d102@redhat.com>
 <20230705093752.GW4253@hirez.programming.kicks-ass.net>
From:   Waiman Long <longman@redhat.com>
In-Reply-To: <20230705093752.GW4253@hirez.programming.kicks-ass.net>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk


On 7/5/23 05:37, Peter Zijlstra wrote:
> On Mon, Jul 03, 2023 at 10:55:02AM -0400, Waiman Long wrote:
>
>> Our OpenShift team has actually hit a problem with the recent persistent
>> user provided cpu affinity change because they are relying on the fact that
>> moving a task to a different cpuset will reset cpu affinity to the cpuset
>> default which is no longer true. That is the main reason behind this patch
>> to provide a way to reset cpu affinity to the cpuset default.
> Where is the sched_setaffinity() in that story?
>
> So somewhere this thing did a sched_setaffinity() and then starts
> playing with cpusets. Instead of adding more sched_setaffinity() calls,
> can't it just remove some?

I don't know the full picture. From what I understand, there is a master 
control process that limit its cpu affinity to just a limited set of 
housekeeping CPUs. It then spawn child processes to be run in different 
containers. The control process doesn't need to change its cpu affinity.

In the past, putting the child processes in a different container 
(cpuset) will reset its affinity to that of the cpuset. That is not true 
anymore because user_cpus_ptr is inherited in the forked child process. 
I have thought about 2 ways to address that. Either we introduce a new 
clone flag to disable the inheritance of users_cpu_ptr or a way to reset 
the cpu affinity to the default which is what this patch does.

Cheers,
Longman