Received: by 2002:a05:6a10:206:0:0:0:0 with SMTP id 6csp1681464pxj; Wed, 19 May 2021 11:20:16 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzJa8VpM4swi9mdjcUOcp9x/aZj3PvnV7Gc7uiiaMbPNbwye3TpDjZr8VEfC0vniFwt4xlN X-Received: by 2002:a02:2a8c:: with SMTP id w134mr267727jaw.138.1621448416453; Wed, 19 May 2021 11:20:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1621448416; cv=none; d=google.com; s=arc-20160816; b=PgDo+itPIGWrekljOMkyQ+KKxMiOWpeJPBavZAxjJZXIBO6DehZcgJmv/tomkAJZ8a JbOZH0rqZqJSe2FGhCjXQxmuC/asdPlIUcLvXN6bP8+OHNrYUMqKNpJwnZeHxH3pDolO BI63hVmGcvtCFlkPCC4QpKRzxB7AZJxuENOWT9niS2B1VtBu1ZCERfj5eGi5jWq3/r8J tOAxVucAishvq6j9Q478YQuN4m82vsqvhtYAVcGZZSz6Eqj45b77m4/HXPPltesgNlWx Irr1A/YQRz5JtJIX55K/7MRK0N+JvTgSbDqpfhRMgYBWdJKvbLnNU6UaSw8QRJD+7/2G hOBw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:subject:message-id:date:from:in-reply-to :references:mime-version:dkim-signature; bh=lpRvF+xJ/4IcCek+cggWbs10JI8lkpvOBQ+cLldKIIU=; b=Y3AoQ31QtYMiXPEzzDFjH2d6fvViEh3Ln6LkNVNVNyZaI1b0BZalDCPFHOn1aVRM5b Jad6wwtb6XXIqPoHo0KBRUQjZ8myw2v4lF6uL4nvBurmJTs72VGxhv68qRLvVnVgl2sv qkMXM2qn4BoavTtS+M0mIcBL/JcaNeH5YOXXEQ49MJJy2AP9pO6ycfk9ktUZJt2L+7+A kxz/SIY1NJ9YFgr+dpDT9yzsivETkQtnpe3Ep9sYuitdUUIf9UwF2kUvUf4kVjQWAPbQ 9xANTGULR4781nM3wGPNKuKLYBvjgWOxXLEIX6lk2NMPcdBPWmnxXpy7DisgrNdn8PES EZnA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Q1Xn+dOV; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id c13si354061ilq.55.2021.05.19.11.20.04; Wed, 19 May 2021 11:20:16 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Q1Xn+dOV; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S240624AbhERP3o (ORCPT + 99 others); Tue, 18 May 2021 11:29:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40604 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S240041AbhERP3n (ORCPT ); Tue, 18 May 2021 11:29:43 -0400 Received: from mail-lj1-x233.google.com (mail-lj1-x233.google.com [IPv6:2a00:1450:4864:20::233]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 37AD1C061573 for ; Tue, 18 May 2021 08:28:24 -0700 (PDT) Received: by mail-lj1-x233.google.com with SMTP id p20so12078558ljj.8 for ; Tue, 18 May 2021 08:28:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=lpRvF+xJ/4IcCek+cggWbs10JI8lkpvOBQ+cLldKIIU=; b=Q1Xn+dOVy2cUeeM8Q6YEGcIaBmPSenlSExzNe2VV+Ln/RroRbTMnsUCBvrEa/4S8Kp P3LjSGgIiWdAnedCndCWvTmNhyI87HcaxrqU8ZpeKW8TuQo1fM5Gnhh4pC34MYbg0TYb HbTEL+hzqTlFdDlvjpvcsnkjp5uzkmNDy5aFP8Cftwi4ZCywZK0gYCmfeGR0Kooi3v1R vnPb+61q6NDDFoPO7PIMziu5DMaTjRae5PR+l+pobPndB43tBo2pI6sVhAURhcvX/cgj HJNpSggF6E+TaIOPX2LPVNoTjGs0NJhQoY5VILW0Dma5Hye49UvG8I92oxc9Al28YWu4 sSlw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=lpRvF+xJ/4IcCek+cggWbs10JI8lkpvOBQ+cLldKIIU=; b=jaMppizaVhhDLfJyWYkc2egtV+Ha5Zn2D9fdoi9RH3RyeLpfFqq1UqdBuDmwoCYrjS cj/tED24UhVLnWJkRgh9Ci1lYITHp/Uk76P8jtKxqXnGi4128O+cwPpja6a89fubZhTM qUm2fHgJMII9RHcJdiAPttaMX6RvdLvNBsu8JIC/G6P1uTYx2xr9dV8Yg+XLk2ylwCVX kAi5psw4Jj9KITWwue4XZeab5AEUOh4sics5673paiLGSeq3dpJmLLnRHWL+rGnZOIZF vgpBpo0L4W4UpKKgsDajTZ+uUNJmmd0l+E0s8ob7Et+cHS5U6b+NK5FznDWE+VADZKEJ AGkg== X-Gm-Message-State: AOAM533DjPhnbcLlI/Jrz/1eMQl7RBbo13E1/zm+Q/1FOt4ooKpicZbl aIR68TzSmjluX5vi22pQX3FkSiZbvJrO4u0wz5ROXg== X-Received: by 2002:a2e:9211:: with SMTP id k17mr4620637ljg.284.1621351702710; Tue, 18 May 2021 08:28:22 -0700 (PDT) MIME-Version: 1.0 References: <1621239831-5870-1-git-send-email-beata.michalska@arm.com> <1621239831-5870-2-git-send-email-beata.michalska@arm.com> <20210518142746.GA3993@e120325.cambridge.arm.com> <20210518150947.GC3993@e120325.cambridge.arm.com> In-Reply-To: <20210518150947.GC3993@e120325.cambridge.arm.com> From: Vincent Guittot Date: Tue, 18 May 2021 17:28:11 +0200 Message-ID: Subject: Re: [PATCH v4 1/3] sched/core: Introduce SD_ASYM_CPUCAPACITY_FULL sched_domain flag To: Beata Michalska Cc: linux-kernel , Peter Zijlstra , Ingo Molnar , Juri Lelli , Valentin Schneider , Dietmar Eggemann , "corbet@lwn.net" , Randy Dunlap , Linux Doc Mailing List Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 18 May 2021 at 17:09, Beata Michalska wrote: > > On Tue, May 18, 2021 at 04:53:09PM +0200, Vincent Guittot wrote: > > On Tue, 18 May 2021 at 16:27, Beata Michalska wrote: > > > > > > On Tue, May 18, 2021 at 03:39:27PM +0200, Vincent Guittot wrote: > > > > On Mon, 17 May 2021 at 10:24, Beata Michalska wrote: > > > > > > > > > > Introducing new, complementary to SD_ASYM_CPUCAPACITY, sched_domain > > > > > topology flag, to distinguish between shed_domains where any CPU > > > > > capacity asymmetry is detected (SD_ASYM_CPUCAPACITY) and ones where > > > > > a full range of CPU capacities is visible to all domain members > > > > > (SD_ASYM_CPUCAPACITY_FULL). > > > > > > > > I'm not sure about what you want to detect: > > > > > > > > Is it a sched_domain level with a full range of cpu capacity, i.e. > > > > with at least 1 min capacity and 1 max capacity ? > > > > or do you want to get at least 1 cpu of each capacity ? > > > That would be at least one CPU of each available capacity within given domain, > > > so full -set- of available capacities within a domain. > > > > Would be good to add the precision. > Will do. > > > > Although I'm not sure if that's the best policy compared to only > > getting the range which would be far simpler to implement. > > Do you have some topology example ? > > An example from second patch from the series: > > DIE [ ] > MC [ ][ ] > > CPU [0] [1] [2] [3] [4] [5] [6] [7] > Capacity |.....| |.....| |.....| |.....| > L M B B The one above , which is described in your patchset, works with the range policy > > Where: > arch_scale_cpu_capacity(L) = 512 > arch_scale_cpu_capacity(M) = 871 > arch_scale_cpu_capacity(B) = 1024 > > which could also look like: > > DIE [ ] > MC [ ][ ] > > CPU [0] [1] [2] [3] [4] [5] [6] [7] [8] [9] > Capacity |.....| |.....| |.....| |.....| |.....| > L M B L B I know that that HW guys can come with crazy idea but they would probably add M instead of L with B in the 2nd cluster as a boost of performance at the cost of powering up another "cluster" in which case the range policy works as well > > Considering only range would mean loosing the 2 (M) CPUs out of sight > for feec in some cases. Is it realistic ? Considering all the code and complexity added by patch 2, will we really use it at the end ? Regards, Vincent > > --- > BR. > B > > > > > > > > > > > > > > > > > > --- > > > BR > > > B. > > > > > > > > > > > > > > > > > > With the distinction between full and partial CPU capacity asymmetry, > > > > > brought in by the newly introduced flag, the scope of the original > > > > > SD_ASYM_CPUCAPACITY flag gets shifted, still maintaining the existing > > > > > behaviour when one is detected on a given sched domain, allowing > > > > > misfit migrations within sched domains that do not observe full range > > > > > of CPU capacities but still do have members with different capacity > > > > > values. It loses though it's meaning when it comes to the lowest CPU > > > > > asymmetry sched_domain level per-cpu pointer, which is to be now > > > > > denoted by SD_ASYM_CPUCAPACITY_FULL flag. > > > > > > > > > > Signed-off-by: Beata Michalska > > > > > Reviewed-by: Valentin Schneider > > > > > --- > > > > > include/linux/sched/sd_flags.h | 10 ++++++++++ > > > > > 1 file changed, 10 insertions(+) > > > > > > > > > > diff --git a/include/linux/sched/sd_flags.h b/include/linux/sched/sd_flags.h > > > > > index 34b21e9..57bde66 100644 > > > > > --- a/include/linux/sched/sd_flags.h > > > > > +++ b/include/linux/sched/sd_flags.h > > > > > @@ -91,6 +91,16 @@ SD_FLAG(SD_WAKE_AFFINE, SDF_SHARED_CHILD) > > > > > SD_FLAG(SD_ASYM_CPUCAPACITY, SDF_SHARED_PARENT | SDF_NEEDS_GROUPS) > > > > > > > > > > /* > > > > > + * Domain members have different CPU capacities spanning all unique CPU > > > > > + * capacity values. > > > > > + * > > > > > + * SHARED_PARENT: Set from the topmost domain down to the first domain where > > > > > + * all available CPU capacities are visible > > > > > + * NEEDS_GROUPS: Per-CPU capacity is asymmetric between groups. > > > > > + */ > > > > > +SD_FLAG(SD_ASYM_CPUCAPACITY_FULL, SDF_SHARED_PARENT | SDF_NEEDS_GROUPS) > > > > > + > > > > > +/* > > > > > * Domain members share CPU capacity (i.e. SMT) > > > > > * > > > > > * SHARED_CHILD: Set from the base domain up until spanned CPUs no longer share > > > > > -- > > > > > 2.7.4 > > > > >