Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 2 Aug 2023 15:01:05 +0800
From:   Aaron Lu <aaron.lu@intel.com>
To:     Chen Yu <yu.c.chen@intel.com>
CC:     Peter Zijlstra <peterz@infradead.org>,
        Ingo Molnar <mingo@redhat.com>,
        Juri Lelli <juri.lelli@redhat.com>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Daniel Jordan <daniel.m.jordan@oracle.com>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
        Daniel Bristot de Oliveira <bristot@redhat.com>,
        "Valentin Schneider" <vschneid@redhat.com>,
        Tim Chen <tim.c.chen@intel.com>,
        "Nitin Tekchandani" <nitin.tekchandani@intel.com>,
        Waiman Long <longman@redhat.com>,
        <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 1/4] sched/fair: free allocated memory on error in
 alloc_fair_sched_group()
Message-ID: <20230802070105.GA26783@ziqianlu-dell>
References: <20230718134120.81199-1-aaron.lu@intel.com>
 <20230718134120.81199-2-aaron.lu@intel.com>
 <ZLasCKO4KA5Pgvdd@chenyu5-mobl2>
 <20230719021355.GA84059@ziqianlu-dell>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20230719021355.GA84059@ziqianlu-dell>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?embJWe+pjGphIwVgR/IITcuIBoHAxBBRwIYWwh6x/gSKzrN1ktHgK9RWyJk1?=
 =?us-ascii?Q?9T1ESRZs1Zef5zHamvu14N7tKhih8IB0R/+6oO0mGD36uWEhUNGWuVXnrA2A?=
 =?us-ascii?Q?drBeojC2Un7G+Q3kEM3f/4wYX+6PQ5bTMkBq44oDacIdVSPr7/CG7HhjyX2s?=
 =?us-ascii?Q?uhnwF6e6cJIREYgpiLrfE8sXixCguZTdELxV1tvCPoINUjOdXOWCgs1GcDp8?=
 =?us-ascii?Q?CkNxyo1lRXfLtmWBRn0UizwDyu1BqPeQrfegfhRXZj3emSM2wWBT4XyzQhu4?=
 =?us-ascii?Q?+KtU+cIDIk0mutoLGVDkytQpG5pe+xRQ9SGHtOH/ZPDBTVls8kCnT1o4Me7I?=
 =?us-ascii?Q?DY8w7MLQh1G1Mk+xFidzK/MbeGsPNogngNCm7zBq/yDYOIAmFsmreTaLdMg8?=
 =?us-ascii?Q?DspEAXTQGdlnHRYGdGbh7ajsYNNl1nx0nDNEXvy/aPh1JGJnww4H08g3TqKr?=
 =?us-ascii?Q?1PRbi5HsCcWyGcUHrZJDrrkknlJ7fwe+m46plae6M0GaqbYtWDlIW1+TzC40?=
 =?us-ascii?Q?HofYy3D5e9AEn0ZDSs4BDJx5qCCVE4qqzedLtfntUtGCCNKiuUXmTZChVa4k?=
 =?us-ascii?Q?tuuPfqY1jIsRXCTqwbQPldj1X/eUmrA19KvFKHN2JNZNq6leyBffaflgX1+b?=
 =?us-ascii?Q?V9VlPt4Y6UJbZZ4bV6Lj5fXy490+pcD2iMMDAsPYbPqHF+wk2Q/qrMs5aUDe?=
 =?us-ascii?Q?JH7CnWebHO23vXtuNyECOQObRmjRwskrYsEUT1olUqqq436QoqJinzy5sTGV?=
 =?us-ascii?Q?xHxrjiNOfLnduxYPZPhhzHM+hawScyfpgfJ1F5GG3E1pmqzCd+3uDbFtLiHo?=
 =?us-ascii?Q?1woIZiM8jNa0vWdwF4iR3gBTc/Q9ewi784ubdzhDyeHDSDNWYgCugddk6GXi?=
 =?us-ascii?Q?wNMQEzJkNKW27uQwGZnlh1o574Mo2zBJjgrYvFu5i7Nm+Frf9xFXNiijWIEC?=
 =?us-ascii?Q?1Wa/vj/77jQk8+kbQFMnd/Cr5kyZKvyuE7mbF1gLYi5397Ibpr/I4mQpnA3Z?=
 =?us-ascii?Q?1WSuTdvVGuDtDM0zSZOLy/Nh4XGWyVrfe0pi7x9JOi4Ej6kX6UkxCtDn/xjU?=
 =?us-ascii?Q?c2ENqVskc33I9u1bwGDx+DgDhwXUQsAy4p64cVPRjpqdOeUdGl/sS+cbajmu?=
 =?us-ascii?Q?74tjCRolhHUo6vsqARJLX+gyR7TUHzeHc7UyIpMmq5tWrN1ZN0g5PZrkcrKc?=
 =?us-ascii?Q?3WdtN1daBzS/+BYecFOH38+Q9BHHwODpJhbe3bmTzBr02OAs4VgDYAMwzSAB?=
 =?us-ascii?Q?vVgGqtBI7Ik1PCT25h8oDz7iEEsbkCuNOwBfAemFDPDjgQ0p3L8dwhmLSB+v?=
 =?us-ascii?Q?myabxCH6T2IxHDbVIBrUXiSdj7Sz7xF0fcn+UFFZ3i6yK98lc+ckKZ39nxBD?=
 =?us-ascii?Q?S4jgmyqWNq/YMSCY6/pwXo+QUQlW5uadOcgI0rfAV/UGIqsEQNJfAZ8JemeL?=
 =?us-ascii?Q?BcwSqi6igdPnWIyC0I3B9SONbi6IAb2n+uNPScF53wh3gIc8j2MPw53P7wYA?=
 =?us-ascii?Q?0L1veBsVcSHXYwX1dIKrOOxql9N8t1U32rNgrLp+He7MmF7a4p8vMY8s1n4/?=
 =?us-ascii?Q?CBEJvnCANPECgi6XOpcRN+MJvnWoZ5RLV9Yw0Eb4?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 590fa11f-51d9-4544-d5b5-08db9326448f
X-MS-Exchange-CrossTenant-AuthSource: BYAPR11MB3062.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Aug 2023 07:01:17.3415
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 2fbMzwe/8Fw7HSKFGpizRI7s6u4VzK1yEDxP5Yaqoc/PSwCdnVk5DvVx7rx5I6V87h39vVclObV77sWMFTMQyA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR11MB8086
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,
        RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE,
        URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Wed, Jul 19, 2023 at 10:13:55AM +0800, Aaron Lu wrote:
> On Tue, Jul 18, 2023 at 11:13:12PM +0800, Chen Yu wrote:
> > On 2023-07-18 at 21:41:17 +0800, Aaron Lu wrote:
> > > There is one struct cfs_rq and one struct se on each cpu for a taskgroup
> > > and when allocation for tg->cfs_rq[X] failed, the already allocated
> > > tg->cfs_rq[0]..tg->cfs_rq[X-1] should be freed. The same for tg->se.
> > > 
> > > Signed-off-by: Aaron Lu <aaron.lu@intel.com>
> > > ---
> > >  kernel/sched/fair.c | 23 ++++++++++++++++-------
> > >  1 file changed, 16 insertions(+), 7 deletions(-)
> > > 
> > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> > > index a80a73909dc2..0f913487928d 100644
> > > --- a/kernel/sched/fair.c
> > > +++ b/kernel/sched/fair.c
> > > @@ -12443,10 +12443,10 @@ int alloc_fair_sched_group(struct task_group *tg, struct task_group *parent)
> > >  
> > >  	tg->cfs_rq = kcalloc(nr_cpu_ids, sizeof(cfs_rq), GFP_KERNEL);
> > >  	if (!tg->cfs_rq)
> > > -		goto err;
> > > +		return 0;
> > >  	tg->se = kcalloc(nr_cpu_ids, sizeof(se), GFP_KERNEL);
> > >  	if (!tg->se)
> > > -		goto err;
> > > +		goto err_free_rq_pointer;
> > >  
> > >  	tg->shares = NICE_0_LOAD;
> > >  
> > > @@ -12456,12 +12456,12 @@ int alloc_fair_sched_group(struct task_group *tg, struct task_group *parent)
> > >  		cfs_rq = kzalloc_node(sizeof(struct cfs_rq),
> > >  				      GFP_KERNEL, cpu_to_node(i));
> > >  		if (!cfs_rq)
> > > -			goto err;
> > > +			goto err_free;
> > >  
> > >  		se = kzalloc_node(sizeof(struct sched_entity_stats),
> > >  				  GFP_KERNEL, cpu_to_node(i));
> > >  		if (!se)
> > > -			goto err_free_rq;
> > > +			goto err_free;
> > >  
> > >  		init_cfs_rq(cfs_rq);
> > >  		init_tg_cfs_entry(tg, cfs_rq, se, i, parent->se[i]);
> > > @@ -12470,9 +12470,18 @@ int alloc_fair_sched_group(struct task_group *tg, struct task_group *parent)
> > >  
> > >  	return 1;
> > >  
> > > -err_free_rq:
> > > -	kfree(cfs_rq);
> > > -err:
> > > +err_free:
> > > +	for_each_possible_cpu(i) {
> > > +		kfree(tg->cfs_rq[i]);
> > > +		kfree(tg->se[i]);
> > > +
> > > +		if (!tg->cfs_rq[i] && !tg->se[i])
> > > +			break;
> > > +	}
> > > +	kfree(tg->se);
> > > +err_free_rq_pointer:
> > > +	kfree(tg->cfs_rq);
> > > +
> > Not sure if I overlooked, if alloc_fair_sched_group() fails in sched_create_group(),
> > would sched_free_group()->free_fair_sched_group() do the cleanup?
> 
> You are right, I overlooked... Thanks for pointing this out.

While preparing v2, one thing still looks strange in the existing code:

int alloc_fair_sched_group(struct task_group *tg, struct task_group *parent)
{
	... ...

	for_each_possible_cpu(i) {
		cfs_rq = kzalloc_node(sizeof(struct cfs_rq),
				      GFP_KERNEL, cpu_to_node(i));
		if (!cfs_rq)
			goto err;

		se = kzalloc_node(sizeof(struct sched_entity_stats),
				  GFP_KERNEL, cpu_to_node(i));
		if (!se)
			goto err_free_rq;
			~~~~~~~~~~~~~~~~

Since free_fair_sched_group() will take care freeing these memories on
error path, it looks unnecessary to do this free for a random cfs_rq
here. Or do I miss something again...?

		init_cfs_rq(cfs_rq);
		init_tg_cfs_entry(tg, cfs_rq, se, i, parent->se[i]);
		init_entity_runnable_average(se);
	}

	return 1;

err_free_rq:
	kfree(cfs_rq);
err:
	return 0;
}

I plan to change it to this:

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index a80a73909dc2..ef2618ad26eb 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -12461,7 +12461,7 @@ int alloc_fair_sched_group(struct task_group *tg, struct task_group *parent)
 		se = kzalloc_node(sizeof(struct sched_entity_stats),
 				  GFP_KERNEL, cpu_to_node(i));
 		if (!se)
-			goto err_free_rq;
+			goto err;
 
 		init_cfs_rq(cfs_rq);
 		init_tg_cfs_entry(tg, cfs_rq, se, i, parent->se[i]);
@@ -12470,8 +12470,6 @@ int alloc_fair_sched_group(struct task_group *tg, struct task_group *parent)
 
 	return 1;
 
-err_free_rq:
-	kfree(cfs_rq);
 err:
 	return 0;
 }