Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) client-ip=2620:137:e000::1:18;
Message-ID: <f78a0d14-b0d4-b6ef-4d6b-561df5bd8a16@amd.com>
Date:   Thu, 24 Feb 2022 09:05:14 -0600
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.6.1
Subject: Re: [PATCH v2] kernel/sched: Update schedstats when migrating threads
Content-Language: en-US
To:     Phil Auld <pauld@redhat.com>, mingo@kernel.org
Cc:     Peter Zijlstra <peterz@infradead.org>, juri.lelli@redhat.com,
        vincent.guittot@linaro.org, mingo@redhat.com,
        dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com,
        mgorman@suse.de, bristot@redhat.com, linux-kernel@vger.kernel.org
References: <20220126152222.5429-1-carlos.bilbao@amd.com>
 <YhYKL4hxx4TNKHGD@hirez.programming.kicks-ass.net>
 <aac8f860-c01f-bda0-9f1b-029b234213c2@amd.com>
 <YhZSqd+d03oWUPP6@lorien.usersys.redhat.com>
 <0e42c46a-ccc4-e793-00b8-ae407e06846f@amd.com>
 <YhZXGv34YTV5omKq@lorien.usersys.redhat.com>
 <b7a28537-8bfb-7f0d-5ed3-f301983e4e30@amd.com>
 <3c6224ca-c0da-7ba9-1396-a2e74c4a40e8@amd.com>
 <YheQIU7oFDcaPoD7@lorien.usersys.redhat.com>
From:   Carlos Bilbao <carlos.bilbao@amd.com>
In-Reply-To: <YheQIU7oFDcaPoD7@lorien.usersys.redhat.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?U1d2ZWZMZzYrM2IzM1VyMDJZYWVkYW1xU2FxNzZ6OXlJV2VxaExHRm85aGFo?=
 =?utf-8?B?bUFpQVBpbWRCMEI1QlRTaG8zUjg0RWJnWmpLSkR2RDZCUnNBTGRvU1Zuc2R4?=
 =?utf-8?B?aDdXRk5Zb0l4YTdvZkREcUl3QWp3UlJaNG1OaWFyQSt1VlNwYlhNb2NhMHBv?=
 =?utf-8?B?dFc5Q3VEcXZPeHdNclNmdE9GeFVlSS9GMG5PZUNrdlpvaHByb2tsVXAzamZC?=
 =?utf-8?B?WU9GS21IRGJaNVFFWE4rdkhlakNLSTlWT05sUk5Bb21LekRqS0xNODVGL3ph?=
 =?utf-8?B?THJPSmtlRHArUUNiOHVlRHA3WWwxNS9jTERFM1d2akpXQW42bUNTdTFieFFI?=
 =?utf-8?B?M1RUazBSS1YvVTU0d21CazN3REc2VGdkaDFYdmhlaVk1M1l6dGd5dkYxaDVn?=
 =?utf-8?B?a1pzc1c5STl0RUN1bEc1MDJhcXF1dXg2Wm1TbUNqcXBJZ05qbHpHT1I1eVZO?=
 =?utf-8?B?QSs3QzdiRTQvTituT2Nmbm0vYmxqbm5jT0lvWnpZNlo0Y1F5bnh3NUs0LzVr?=
 =?utf-8?B?RG9mZnNlcG5XelBvVVlRNk1hWWc0c0M3WGhJSGh3bDRJM2ZSQzZobzErVzZw?=
 =?utf-8?B?SUVLeTVLSDNDYTBaaEY2YTJZa2pVUkZwRWg1RHVvR08xOUVEc24reUEyZXVz?=
 =?utf-8?B?STd6eWU5QnhJRGIzNTM4WTFQdDBnSnNCNFRNRVBKUitaaXVJYWtuMGFmanhm?=
 =?utf-8?B?dHp5ei81TkIxUmIyYW1EQVVHY25BdGZKbTZ5YndaOUtyV2RWdGZhWmY0UTdv?=
 =?utf-8?B?QlQ1SGM4cHcyTGVWRzFKcHlkWC9PdjExM25QYi9KSXl5dUlmYnp0OU5LV21X?=
 =?utf-8?B?SjJwNFZiZTRUTUphNG1XZHd4UkQxWThHNDNCUmVQcTFMb29NRmJ3NjVmRkJC?=
 =?utf-8?B?cEVGeldaOXpvL2pGSG43OVNyUGF4YzJ4TzlUWjNiRjVVTktEMFdPTlFMMVlh?=
 =?utf-8?B?ejFlU3VZODNzRDJMTUxqbzF1QzQ1Q0d1ZHppam5BQ3RZMjhiQjhPMFFnSlVT?=
 =?utf-8?B?bFpyMDNlRWs4MFJEcG05dFhtQ3VZWXlTQnMzQ1J0bFRLM01PdHJHRU0rMW8v?=
 =?utf-8?B?a0lnWmZNUmVhM2ptMjJOdHlubnhoYVk2TnNDWU5ldHRaRHlPUUtzU0hKdW1p?=
 =?utf-8?B?cnkzK3RIQTRHVjBxd1JSOGE1Z21DU20xWGNqVUxQODVEWEZ1ZEJSdkZWb2Qy?=
 =?utf-8?B?RzJKVEx1UWo2ZDVIMlBjK0dGZFZRaW4zZXlkL0U1M0t0TGFBZ0xNSzZtbUtL?=
 =?utf-8?B?VTRpdEtVaUxjUTI0U1VJOXl6SHdud0o2ZGpIK3JzeWw4QXpHcHFraVNzU1pW?=
 =?utf-8?B?S3FiK211SVFnOG5LQ1JkaG5mZXROTG1QbEN2KzI2UXdjTkRKbGlNcStaQ2Jk?=
 =?utf-8?B?cVEwVTNUTjhsWWR1WUt3ZGpvSjkzdVYrVUpWdlNaVlhwQnBEYjdPR1BIazVZ?=
 =?utf-8?B?T1RLVGNwWllkUnNWTFppWGUwYVROa2VNYzNYeEVqZHdwN0R1WWZCVHlzell2?=
 =?utf-8?B?MkN0Y0k3cnZZMnVwcjVCKzE2L1NCRWJuRnord010S040YlpSamZzemUzOWtG?=
 =?utf-8?B?d0hQNFVXdTdjMFlBV3RyK1BFdE1sYkZNNmJqRUQ2b0lQYVpVOXhUVHdLK0lp?=
 =?utf-8?B?REtvTXdvN2ZteGFiaElCVEVvQVFkb0g0ejN5L0Jtd3hZL041OEJUZXBFdUdY?=
 =?utf-8?B?d0FZNnU5dzJJdStFa2xlVk1xRkpoR0s5UXFDMFM0UjRoY0FZNTFTK3crMnk1?=
 =?utf-8?B?ZVAwbDFJSzZpLzFzcE9ENUFXTEtBQ3NISFZKWklnejRhWTdwWUxxUWNUeU1n?=
 =?utf-8?B?NlluenQvOGs2cnJ6M05jK3Z1RXVmWmRUK1NyWHhXckdVczNhNmxmZ0Q5TEpC?=
 =?utf-8?B?OUhNVTN5MHNMaDNjY0s4bWVTSUljZ1JIbXhsTVJxQUQ2MnNyQysyL3NvODJo?=
 =?utf-8?B?QlM0Yi93WUdBSjlkdDNDYVVXMzRvTmhwRHNpZi8zaklFVVE2M3IwVnJkTjFL?=
 =?utf-8?B?R2NlSmZ5ciswdkp2QnJrVXRwTVM0VXNnVEFTOEwxN2Vqd0gweHc5SWFENHJY?=
 =?utf-8?B?cTAyMm1Jb1p4bStadThINUpGQ3NQWkloaXRuQW1Rb0NwMU16bHZhYnMzK1JJ?=
 =?utf-8?B?K0FqWjgyVldtOTdIbEFQL3ZFWkFrNVlheWFBUVVySTdTNTFZRzRjd1Rka3h5?=
 =?utf-8?Q?ZssX7HQkt60h60WKcqIS6v8=3D?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ec508239-1768-4c77-f9ab-08d9f7a7108c
X-MS-Exchange-CrossTenant-AuthSource: BL1PR12MB5874.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Feb 2022 15:05:16.0784
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: U0prkkf63ikr7RfDvlPEnSfLd4aVYn327qXZ90reGxBkVl/QuyR51E46nKQuiL8SWz1YZ/5rZjAsJVSkTHimFA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR12MB1846
X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,NICE_REPLY_A,RDNS_NONE,SPF_HELO_NONE,
	T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
	lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 2/24/2022 8:03 AM, Phil Auld wrote:
> On Wed, Feb 23, 2022 at 10:13:50AM -0600 Carlos Bilbao wrote:
>> The kernel manages per-task scheduler statistics or schedstats. Update
>> function migrate_task_to() to increase the counter for migrations.
>>
>> Signed-off-by: Carlos Bilbao <carlos.bilbao@amd.com>
>> ---
>> Changelog:
>> 	v2: Update commit message, don't reinitialize sched fields.
>> ---
>>  kernel/sched/core.c | 2 +-
>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/kernel/sched/core.c b/kernel/sched/core.c
>> index fcf0c180617c..1360e501c737 100644
>> --- a/kernel/sched/core.c
>> +++ b/kernel/sched/core.c
>> @@ -8751,7 +8751,7 @@ int migrate_task_to(struct task_struct *p, int target_cpu)
>>  	if (!cpumask_test_cpu(target_cpu, p->cpus_ptr))
>>  		return -EINVAL;
>>  
>> -	/* TODO: This is not properly updating schedstats */
>> +	schedstat_inc(p->stats.nr_migrations_cold);
>>
> 
> I was going to give a reviewed by since I was looking at this,
> but I can't convince myself that nr_migrations_cold is right.
> This looks more like a "hot" migration (using stop_cpu to force
> it). But nr_migrations_cold is not incremented anywhere else so
> maybe it's a terminology thing.
> 
> Can you tell me why this is the right counter?
> 
>
The alternative to nr_migrations_cold is nr_forced_migrations, and since
we call this function in the context of NUMA balancing it didn't seem like
an appropriate fit at first glance. But documentation in this is scarce and 
I'm not very confident with the terminology either. Maybe Ingo Molar -the
original author of the structure- can clarify the specifics of these two 
counters so we can be sure about this? Maybe we need a new counter?

> Thanks,
> Phil
> 
> 
>>  	trace_sched_move_numa(p, curr_cpu, target_cpu);
>>  	return stop_one_cpu(curr_cpu, migration_cpu_stop, &arg);
>> -- 
>> 2.27.0
>>
> 

Thanks,
Carlos