Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5;
Message-ID: <31e06652-1dbd-e32f-3123-d17e178c5c27@redhat.com>
Date:   Tue, 10 Oct 2023 16:03:41 -0400
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.14.0
Subject: Re: [PATCH] cgroup/cpuset: Change nr_deadline_tasks to an atomic_t
 value
Content-Language: en-US
From:   Waiman Long <longman@redhat.com>
To:     Juri Lelli <juri.lelli@redhat.com>
Cc:     Tejun Heo <tj@kernel.org>, Zefan Li <lizefan.x@bytedance.com>,
        Johannes Weiner <hannes@cmpxchg.org>,
        linux-kernel@vger.kernel.org, cgroups@vger.kernel.org,
        Peter Zijlstra <peterz@infradead.org>,
        Ingo Molnar <mingo@kernel.org>,
        Qais Yousef <qyousef@layalina.io>, Hao Luo <haoluo@google.com>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Xia Fukun <xiafukun@huawei.com>
References: <20231009191515.3262292-1-longman@redhat.com>
 <ZSTiULEnD7SF9n7y@localhost.localdomain>
 <6b769316-6434-5054-43f5-7933fc2bee01@redhat.com>
In-Reply-To: <6b769316-6434-5054-43f5-7933fc2bee01@redhat.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk


On 10/10/23 15:44, Waiman Long wrote:
>
> On 10/10/23 01:34, Juri Lelli wrote:
>> Hi,
>>
>> On 09/10/23 15:15, Waiman Long wrote:
>>> The nr_deadline_tasks field in cpuset structure was introduced by
>>> commit 6c24849f5515 ("sched/cpuset: Keep track of SCHED_DEADLINE task
>>> in cpusets"). Unlike nr_migrate_dl_tasks which is only modified under
>>> cpuset_mutex, nr_deadline_tasks can be updated in various contexts
>>> under different locks. As a result, data races may happen that cause
>>> incorrect value to be stored in nr_deadline_tasks leading to incorrect
>> Could you please make an example of such data races?
>
> Since update to cs->nr_deadline_tasks is not protected by a single 
> lock, it is possible that multiple CPUs may try to modify it at the 
> same time.  It is possible that nr_deadline_tasks++ and 
> nr_deadline_tasks-- can be done in a single instruction like in x86 
> and hence atomic. However, operation like "cs->nr_deadline_tasks += 
> cs->nr_migrate_dl_tasks" is likely a RMW operation and so is subjected 
> to racing. It is mostly theoretical, but probably not impossible.

Sorry, even increment and decrement operators are not atomic.

inc_dl_tasks_cs() is only called from switched_to_dl() in deadline.c 
which is protected by the rq_lock, but there are multiple rq's. 
dec_dl_tasks_cs() is called from switched_from_dl() in deadline.c and 
cgroup_exit() in cgroup.c. The later one is protected by css_set_lock. 
The other place where nr_deadline_tasks can be changed is in 
cpuset_attach() protected by cpuset_mutex.

Cheers,
Longman