Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
To:     Vincent Guittot <vincent.guittot@linaro.org>
Cc:     Benjamin Segall <bsegall@google.com>,
        Ingo Molnar <mingo@redhat.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Juri Lelli <juri.lelli@redhat.com>,
        =?UTF-8?Q?Michal_Koutn=c3=bd?= <mkoutny@suse.com>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Mel Gorman <mgorman@suse.de>,
        Daniel Bristot de Oliveira <bristot@redhat.com>,
        Valentin Schneider <Valentin.Schneider@arm.com>,
        linux-kernel@vger.kernel.org, Odin Ugedal <odin@uged.al>,
        Kevin Tanguy <kevin.tanguy@corp.ovh.com>,
        Brad Spengler <spender@grsecurity.net>
References: <b98e3434-67cd-34b7-9e81-148ea31a851c@grsecurity.net>
 <20211103190613.3595047-1-minipli@grsecurity.net>
 <xm26ilx86gmp.fsf@google.com>
 <CAKfTPtBm4vHr=svju=Qg6eZmcv8YDghtM2r_pOahZ2gC3tzTxg@mail.gmail.com>
 <a6a3c6c9-d5ea-59b6-8871-0f72bff38833@grsecurity.net>
 <CAKfTPtBxoKBRWs4Z3Pxsk8==Ka9SG7NS3LzfOV33-2UXfhSM=g@mail.gmail.com>
From:   Mathias Krause <minipli@grsecurity.net>
Subject: Re: [PATCH] sched/fair: Prevent dead task groups from regaining
 cfs_rq's
Message-ID: <cd3778d3-6980-a804-47e3-82b09dc960a4@grsecurity.net>
Date:   Thu, 4 Nov 2021 18:37:50 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <CAKfTPtBxoKBRWs4Z3Pxsk8==Ka9SG7NS3LzfOV33-2UXfhSM=g@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Precedence: bulk

Am 04.11.21 um 17:49 schrieb Vincent Guittot:
> [snip]
> 
> Ok so we must have 2 GPs:
> 
> list_del_rcu(&tg->siblings);
> GP to wait for the end of ongoing walk_tg_tree_from : synchronize_rcu
> in your patch
> list_del_leaf_cfs_rq(tg->cfs_rq[cpu]); if on_list
> remove_entity_load_avg(tg->se[cpu]);
> GP to wait for the end of ongoing for_each_leaf_cfs_rq_safe (print_cfs_stats)
> kfree everything

Basically yes, but with my patch we already have these two, as there's
at least one RCU GP between after sched_offline_group() finishes and
sched_free_group() / cpu_cgroup_css_free() starts.

So we either use my patch as-is or move unregister_fair_sched_group() to
free_fair_sched_group() and use kfree_rcu() instead of kfree(). Both
approaches have pros and cons.

Pro for my version is the early unlinking of cfs_rq's for dead task
groups, so no surprises later on. Con is the explicit synchronize_rcu().

Pro for the kfree_rcu() approach is the lack of the explicit
synchronize_rcu() call, so no explicit blocking operation. Con is that
we have cfs_rq's re-added to dead task groups which feels wrong and need
to find a suitable member to overlap with the rcu_head in each involved
data type.

Which one do you prefer?

Thanks,
Mathias