Received: by 2002:ac0:a582:0:0:0:0:0 with SMTP id m2-v6csp829507imm; Wed, 10 Oct 2018 05:07:11 -0700 (PDT) X-Google-Smtp-Source: ACcGV61JZ32PihG5iyFZAlpE5KsMrtd9FhLFrj4gJbaEJp3RN7o16LqmyRU5Hf7NUlJnF731Lur6 X-Received: by 2002:a62:6344:: with SMTP id x65-v6mr23347760pfb.13.1539173231736; Wed, 10 Oct 2018 05:07:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1539173231; cv=none; d=google.com; s=arc-20160816; b=DUsWgZ+eHovITByFXLr/rLZYOAPJeioLrdEofUztn8iy6ycultzVYub0BvpCdWSkk7 aQFGysTojndde9KoJT94JYD1LvuvHB+atvz5VG4/swZf7rNIk9whCDeK6bHpyWogO0Ky 3iql0VJ20CG9pjEWWTmg7GHHs/DJn+BP+wRspmM+m2Ad4OEB+50xiJSly76jksxrGJxt owo51wsIFFxRiMZDrMBnU3/vJ1BKZnoc8AK0zsjubRbBvvsXZwWChQ3Q1KAWtQ4gvS9w zvjJ2/XsH3+shw49g1cNP5b6wrD25Z2Wwv89+w+PhKxo6jGGHMMWtRSqir5U5Rrv6UqU dbFQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:dkim-signature; bh=OTlklHlMT6t/foWmubxPEEF28fE0Umj3UB/67j3ty14=; b=mUnmxO25VIh09K0ryPn/EoICtPjT8eo4uI8BVlObmuFsfC1umJalPdVUfCIXRbt1fx eAT2CwrUV3Zj3+r5PIn+ZDcokhAugyt+HuZs7Bt/w9ickcytvFlx1XUOEtbzk1o72WB1 ds54uTvPrVngvmBi4VisIJAt69gQ8MGkQ5DdoFrW4037FTh/hF0fqJIfP7w7o/iXn2ld pNVsm14ifublYqI7ozpyLY2hC1AZlkptkVMh3jAI0H4iDwfwuMfcxCfee8PheOvRxLbL ZLM6+jqJqrXerHp9KrxE3nFGAQFWeTkibd2HDayrRAKS3uNoefXAp3WAkxiGGbUbiaj6 tB2w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ifyQwNh4; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id t7-v6si18376770pfb.16.2018.10.10.05.06.56; Wed, 10 Oct 2018 05:07:11 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ifyQwNh4; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727013AbeJJT0n (ORCPT + 99 others); Wed, 10 Oct 2018 15:26:43 -0400 Received: from mail-it1-f195.google.com ([209.85.166.195]:51131 "EHLO mail-it1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726923AbeJJT0n (ORCPT ); Wed, 10 Oct 2018 15:26:43 -0400 Received: by mail-it1-f195.google.com with SMTP id k206-v6so1498368ite.0 for ; Wed, 10 Oct 2018 05:04:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=OTlklHlMT6t/foWmubxPEEF28fE0Umj3UB/67j3ty14=; b=ifyQwNh4t6USOfmnRYDHLK1nEi6xzOWh1Av5IJ3q6CbqjuMZSowADHio+glgR5XDge /pRiq0+3519ZlwmDtWbcS5amFI1ssxSxDBnke9ma9lwz9QKdlG2NRDXkrzeVdfX1w1jJ OSq0ECNvEZj0PoBr0NPR8+PM5ZglFUOZwEVPA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=OTlklHlMT6t/foWmubxPEEF28fE0Umj3UB/67j3ty14=; b=KoLimtCoid+iKnGFfblH2sCAuRyBZKcQ+BDKF2rPJbU2i0bxwooH4zNoEnhy187aP9 Ay3GYQTe8K6pNuq/43CyghqEThyAeK2uc0/usX5fXGJS/uBngxAOXTnJha8vlUxqaiPI 7Vu/dNUSGcmrSNdMyHvTgGCpZc8VtICgNFwz/QC+nKTFwpWoN3D6AT9Xs2t1lQRPU90x hLoLUuDp275FYrDaw6oPtKRWZvu0Y9rwdE78bS9LDRAwC6O6YRpUpYKTctgU7wdjapqh f9b55jRK9A8Z7pX7Nf6r4qqjjPBwRKufA9BELAmRtGDTbINz1RJ9UdYm7jGHbcFTn0K1 s9MQ== X-Gm-Message-State: ABuFfogzgMNJhkljduaNKBAPYfXszmIBfGlI9PPp/JN9OJzbKIw52RJ3 0wh/bGrXJ2UWwayZVD35yLEh/ZWsW9ptq3lwCs1xrw== X-Received: by 2002:a02:5a01:: with SMTP id v1-v6mr8054549jaa.11.1539173091456; Wed, 10 Oct 2018 05:04:51 -0700 (PDT) MIME-Version: 1.0 References: <1539102302-9057-1-git-send-email-thara.gopinath@linaro.org> <20181010061751.GA37224@gmail.com> <20181010082933.4ful4dzk7rkijcwu@queper01-lin> <20181010095459.orw2gse75klpwosx@queper01-lin> <20181010103623.ttjexasymdpi66lu@queper01-lin> In-Reply-To: <20181010103623.ttjexasymdpi66lu@queper01-lin> From: Vincent Guittot Date: Wed, 10 Oct 2018 14:04:40 +0200 Message-ID: Subject: Re: [RFC PATCH 0/7] Introduce thermal pressure To: Quentin Perret Cc: Ingo Molnar , Thara Gopinath , linux-kernel , Ingo Molnar , Peter Zijlstra , Zhang Rui , "gregkh@linuxfoundation.org" , "Rafael J. Wysocki" , Amit Kachhap , viresh kumar , Javi Merino , Eduardo Valentin , Daniel Lezcano , "open list:THERMAL" , Ionela Voinescu Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 10 Oct 2018 at 12:36, Quentin Perret wrote: > > On Wednesday 10 Oct 2018 at 12:14:32 (+0200), Vincent Guittot wrote: > > On Wed, 10 Oct 2018 at 11:55, Quentin Perret wrote: > > > > > > Hi Vincent, > > > > > > On Wednesday 10 Oct 2018 at 10:50:05 (+0200), Vincent Guittot wrote: > > > > The problem with reflecting directly the capping is that it happens > > > > far more often than the pace at which cpu_capacity_orig is updated in > > > > the scheduler. > > > > > > Hmm, how can you be so sure ? That most likely depends on the workload, > > > the platform and the thermal governor. Some platforms heat up slowly, > > > some quickly. The pace at which the thermal governor will change things > > > should depend on that I assume. > > > > > > > This means that at the moment when scheduler uses the > > > > value, it might not be correct anymore. > > > > > > And OTOH, when you remove a cap for example, it will take time before > > > the scheduler can see the newly available capacity if you need to wait > > > for the signal to decay. So you are using a wrong information too in > > > that scenario. > > > > But we stay consistant with all other metrics. The same happen when a > > task decide to stay idle for a long time after some activity... You > > will have to wait for the signal to decay > > Because you see that as a task going idle. You could also say that > removing the cap from a CPU is equivalent to migrating a task off that > CPU. In this case you should see the newly available cap pretty much > immediately. > > Also, I feel like the whole thermal capping story could interest the > Intel folks who, IIRC, were discussing how to represent their 'boosted' > OPPs in the scheduler some time ago. You can see the Intel boost thing > as a cap I think -- some OPPs can be inaccessible in some case. I'd be > interested to see what's their take on this. > > > > > Then, this value are also used > > > > when building the sched_domain and setting max_cpu_capacity which > > > > would also implies the rebuilt the sched_domain topology ... > > > > > > Wait what ? I thought the thermal cap was reflected in capacity_of, not > > > capacity_orig_of ... You need to rebuild the sched_domain in case of > > > thermal pressure ? > > I can't locate where you need to do this in the series. Do you actually > need to rebuild the sd hierarchy ? This patchset doesn't touch cpu_capacity_orig and doesn't need to as it assume that the max capacity is unchanged but some capacity is momentary stolen by thermal. If you want to reflect immediately all thermal capping change, you have to update this field and all related fields and struct around > > > > Hmm, let me have a closer look at the patches, I must have missed > > > something ... > > > > > > > The pace of changing the capping is to fast to reflect that in the > > > > whole scheduler topology > > > > > > That's probably true in some cases, but it'd be cool to have numbers to > > > back up that statement, I think. > > > > > > Now, if you do need to rebuild the sched domain topology every time you > > > update the thermal pressure, I think the PELT HL is _way_ too short for > > > that ... You can't rebuild the whole thing every 32ms or so. Or am I > > > misunderstanding something ? > > > > > > > > Thara, have you tried to experiment with a simpler implementation as > > > > > suggested by Ingo ? > > > > > > > > > > Also, assuming that we do want to average things, do we actually want to > > > > > tie the thermal ramp-up time to the PELT half life ? That provides > > > > > nice maths properties wrt the other signals, but it's not obvious to me > > > > > that this thermal 'constant' should be the same on all platforms. Or > > > > > maybe it should ? > > > > > > > > The main interest of using PELT signal is that thermal pressure will > > > > evolve at the same pace as other signals used in the scheduler. > > > > > > Right, I think this is a nice property too (assuming that we actually > > > want to average things out). > > > > > > > With > > > > thermal pressure, we have the exact same problem as with RT tasks. The > > > > thermal will cap the max frequency which will cap the utilization of > > > > the tasks running on the CPU > > > > > > Well, the nature of the signal is slightly different IMO. Yes it's > > > capacity, but you're no actually measuring time spent on the CPU. All > > > other PELT signals are based on time, this thermal thing isn't, so it is > > > kinda different in a way. And I'm still wondering if it could be helpful > > > > hmmm ... it is based on time too. > > You're not actually measuring the time spent on the CPU by the 'thermal > task'. There is no such thing as a 'thermal task'. You're just trying to > model things like that, but the thermal stuff isn't actually > interrupting running tasks to eat CPU cycles. It just makes thing run > slower, which isn't exactly the same thing IMO. > > But maybe that's a detail. > > > Both signals (current ones and thermal one) are really close. The main > > difference with other utilization signal is that instead of providing > > a running/not running boolean that is then weighted by the current > > capacity, the signal uses direclty the capped max capacity to reflect > > the amount of cycle that is stolen by thermal mitigation. > > > > > to be able to have a different HL for that thermal signal. That would > > > 'break' the nice maths properties we have, yes, but is it a problem or is > > > it actually helpful to cope with the thermal characteristics of > > > different platforms ? > > > > If you don't use the sign kind of signal with the same responsiveness, > > you will start to see some OPP toggles as an example when the thermal > > state change because one metrics will change faster than the other one > > and you can't have a correct view of the system. Same problem was > > happening with rt task. > > Well, that wasn't the problem with rt tasks. The problem with RT tasks > was that the time they spend on the CPU wasn't accounted _at all_ when > selecting frequency for CFS, not that the accounting was at a different > pace ... The problem was the same with RT, the cfs utilization was lower than reality because RT steals soem cycle to CFS So schedutil was selecting a lower frequency when cfs was running whereas the CPU was fully used. The same can happen with thermal: cap the max freq because of thermal the utilization with decrease. remove the cap the utilization is still low and you will select a low OPP because you don't take into account cycle stolen by thermal like with RT Regards, Vincent > > Thanks, > Quentin