Received: by 2002:ab2:6a05:0:b0:1f8:1780:a4ed with SMTP id w5csp3205678lqo; Wed, 15 May 2024 03:00:56 -0700 (PDT) X-Forwarded-Encrypted: i=3; AJvYcCXllJh+wlJYtGtVr0OcQsjCgcKuf3IkkPM8xA6p2XkuMV/1aUj0wK2xPe1LfJDqMFVd7IEe/y0toMq1ZPC2I0EiJABBwwkhEC9iXL2ZNQ== X-Google-Smtp-Source: AGHT+IGvJJTeUczLgFDzOwBtwNSsuMeF9+on13EygJjrd1PTrdGgCR04ywfxLxmJgnP54SwVtcEq X-Received: by 2002:a05:6a00:2ea1:b0:6ea:d004:33c9 with SMTP id d2e1a72fcca58-6f4e038a176mr16166935b3a.30.1715767256106; Wed, 15 May 2024 03:00:56 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1715767256; cv=pass; d=google.com; s=arc-20160816; b=FzfLfqJLlPPxPDGXlPlzCJzEaHcy0e5KCmRpjofWwzP+9P6+jd0AB8bEycs3cuZIuX 9S2sSMixIjafsgc2JVyXjheAK17wXOQ+1l+wgQvjQkVVK3GZZm3WiCQXevtbRs4CVUIG AsHJFmsMez5KlfNVLtFaanaStkQ+c3tI7FN2+J7+4HVVfdswTevVHpONjhE7EF+nZ1l5 u5qO2K0qoyyu9CFn8FvKEe47s5vM77ft+DZnUq0NjqSPsCHPbxci14NSH3r5SHXDfZna 1XMuVFU9oXL2LuCPd/Fbwgg0UVWtOyzWUHaS6TLCMwd9uQTvoEsTyWIKMldmGLbSYJGo TVgQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:in-reply-to:content-language:from :references:cc:to:subject:user-agent:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:date:message-id; bh=PrjIddaXMJ26Y+Cyfc9FrVG23Osjq2py2YhMPROwrqA=; fh=dmZgRLmLsGKVG83a8A/q3UjLOkr9jfunb6NG/wbMtaA=; b=g1h91DgDT+6B2XWCiLW4dLA7U8EGNP+ZimEW5zYhDsB8cuxUDk8iMpkz1MBEqNaNKn 0cj5jWf6/b8aHyqe/MH+N+L518zoqdmofDBe3cKXPq6OyaF8jkJtu4BXCL1uDKCE8zZ6 akAMNOKgOepTVjiBDQPf0DRJk6Jmpy/oxpdw/dUHwxzuwamca13qNq4+jhwbaSP1p6A5 GuPAdbCW0WLNb8WxRZlEvwkovGUeJbuzfYOQiHduL0JqsR7l+9oGWqPs9oAwcRQ0GOgL IxPcvctbU1YpQEFrb4K4577IJ8ircu9RqWJGdg9Df3M2R5kI7WiXiMHO6FWER/SQjXQs 0/qQ==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-179734-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-179734-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Return-Path: Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id 41be03b00d2f7-634103f5be9si13439916a12.338.2024.05.15.03.00.55 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 May 2024 03:00:56 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel+bounces-179734-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=arm.com dmarc=pass fromdomain=arm.com); spf=pass (google.com: domain of linux-kernel+bounces-179734-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-179734-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 7258128528E for ; Wed, 15 May 2024 10:00:55 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id E520E5A11F; Wed, 15 May 2024 10:00:47 +0000 (UTC) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A469E54645; Wed, 15 May 2024 10:00:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715767247; cv=none; b=GTJKhnKvfpaQ/GnH2LPX4PS/WuJi/fPDQVDb/GTBum0Dhp2B3YqEn44b8BJ1yQSrpDJbOeSFYBS2R8EkpSltYiPTcTe2j3jUw3XT1BVtgceuQGVbTtdrLzOvKIgv0Ok24B2gtUDFN5Xh5LaIHihJiVuBCG9J7ZmO791+Ml1riq4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715767247; c=relaxed/simple; bh=UP6ARF9IZdzS2kDPVK4Ew0G7gxoE0lxuEJNvLxI+258=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=Ikqv8lUb4tdXnS9TIFL0V87jnW2ocqhqaSZXP9y9v0G/WMYGAhuuAEV/A/2YnUJhBNbUWEojLSWKbu1UeGd6fqs/qfZEEXXvegEE+IkpZjQ7E+xI6pmwSs1HcO5b5x6mKzSIu/62bfO5P85NrkqphQVb5Y4o9ZZMY8MQ95XEo/s= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9BB0E1007; Wed, 15 May 2024 03:01:09 -0700 (PDT) Received: from [192.168.178.6] (usa-sjc-mx-foss1.foss.arm.com [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 7B6EF3F762; Wed, 15 May 2024 03:00:42 -0700 (PDT) Message-ID: <09717ad7-2a4b-486c-a4f5-e3f09a212add@arm.com> Date: Wed, 15 May 2024 12:00:40 +0200 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v3] sched: Consolidate cpufreq updates To: Qais Yousef Cc: "Rafael J. Wysocki" , Viresh Kumar , Ingo Molnar , Peter Zijlstra , Vincent Guittot , Juri Lelli , Steven Rostedt , Ben Segall , Mel Gorman , Daniel Bristot de Oliveira , Valentin Schneider , Christian Loehle , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org References: <20240512190018.531820-1-qyousef@layalina.io> <9e845146-8a31-407c-a5ee-e2e32f1655e5@arm.com> <20240513220903.no2j6zl4tk7lr6um@airbuntu> From: Dietmar Eggemann Content-Language: en-US In-Reply-To: <20240513220903.no2j6zl4tk7lr6um@airbuntu> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit On 14/05/2024 00:09, Qais Yousef wrote: > On 05/13/24 14:43, Dietmar Eggemann wrote: >> On 12/05/2024 21:00, Qais Yousef wrote: >> >> [...] >> >>> @@ -4682,7 +4659,7 @@ static void attach_entity_load_avg(struct cfs_rq *cfs_rq, struct sched_entity *s >>> >>> add_tg_cfs_propagate(cfs_rq, se->avg.load_sum); >>> >>> - cfs_rq_util_change(cfs_rq, 0); >>> + cpufreq_update_util(rq_of(cfs_rq), 0); >> >> Isn't this slighlty different now? >> >> before: >> >> if (&rq->cfs == cfs_rq) { >> cpufreq_update_util(rq, ....) >> } >> >> now: >> >> cpufreq_update_util(rq_of(cfs_rq), ...) >> >> You should get way more updates from attach/detach now. > > Yes, well spotted! > > Looking at the path more closely, I can see this is called from > enqueue_task_fair() path when a task migrates to new CPU. And when > attach_task_cfs_rq() which is called when we switch_to_fair(), which I already > cover in the policy change for the RUNNING task, or when > task_change_group_fair() which what I originally understood Vincent was > referring to. I moved the update to this function after the detach/attach > operations with better guards to avoid unnecessary update. Yeah, all !root cfs_rq attach or detach wouldn't change anything since the util_avg wouldn't have propagated to the root cfs_rq yet. So sugov_get_util() wouldn't see a difference. Yes, enqueue_entity() sets DO_ATTACH unconditionally. And dequeue_entity() sets DO_DETACH for a migrating (!wakeup migrating) task. For a wakeup migrating task we have remove_entity_load_avg() but this can't remove util_avg from the cfs_rq. This is deferred to update_cfs_rq_load_avg() in update_load_avg() or __update_blocked_fair(). And switched_{to,from}_fair() (check_class_changed()) and task_change_group_fair() are the other 2 users of {attach,detach}_entity_load_avg(). (plus online_fair_sched_group() for attach). > I understood this will lead to big change and better apply immediately vs > wait for the next context switch. But I'll ask the question again, can we drop > this and defer to context switch? Hard to say really, probably we can. All benchmarks with score numbers will create plenty of context switches so you wont see a diff. And for more lighter testcases you would have to study the differences in trace files and reason about the implications of potentially kick CPUfreq a little bit later. [...]