Received: by 2002:ad5:474a:0:0:0:0:0 with SMTP id i10csp10973277imu; Thu, 6 Dec 2018 09:29:47 -0800 (PST) X-Google-Smtp-Source: AFSGD/WaU3zNhpvNlAEBIwO5zMEaqOP5dwXr2T0yhGrmIpA6RmDMuVQ2IZELMwGpd13IO7iR6D+S X-Received: by 2002:a63:dd55:: with SMTP id g21mr24409851pgj.86.1544117387197; Thu, 06 Dec 2018 09:29:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544117387; cv=none; d=google.com; s=arc-20160816; b=L8YqE9H4SypkaRHtFTn3kJ+x6+88Mg3b1pl9OLaI55z/pYWm21rwAqgdGm0bDOl82m bo5KB6FsFdrbfZixZ/ngHhLazPvqa1PWEW8y8/iAD5l6MIdTFxQU8SpjGNQGMtHXH7HV O3DDBd8RSfO/cGHLkToWJkkykaUUzRasCTucQFn5gPMNMmFbKORg9QqFZrcBPIlF0keh DKuMtZ1Km1cVQBq6Aiasc9E8/6yK68pinmvFNtrT6EDyqcPF1QMieZGHHy3jxlSgU+Wj ctwpwEagT5HVvyfPrr0ZJ+NjES7z4z8TuFks5Ze89TJtJlgOCENPaHlNDMF7CnVeYCdx beHA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding :content-language:in-reply-to:mime-version:user-agent:date :message-id:from:references:cc:to:subject; bh=J6tTT5HVXtgbtwOd8mUhT6MT1XjOf+Nh+PJwBji4aQU=; b=a0qEZ0Rp0wcBGN2fShYixg7vUoTmYl/CbGazqdTTw3rEkZSml/sTDcPaTXKAKOGpMb ImEbN/SU140bQg+hAnsJVGwClz+Rs+Sw4Z1XsLKst/T50UBN4GLS/VAIjSfhmrEaC4G7 nHNZLuh8IWJcHQL0dkMNZw3XcFrtrnKAQIFF3duek9gx/WLdVRGXZqhaT4RRo8KPH9dB QHSJrn/lZBVb9cizQciP/4BVxO5xmQ+2/sUshnaxzs0VRnKd3cYIA3C/HIlOqk2nb5f8 AQWfrRfmHjTMgYHxh5X/R/1qnZDA4+HJIYJBrX7VUfjahJPDDn0ozqVeqXclP+MPxdNK cPUg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id i6si663690plt.290.2018.12.06.09.29.27; Thu, 06 Dec 2018 09:29:47 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725986AbeLFR2m (ORCPT + 99 others); Thu, 6 Dec 2018 12:28:42 -0500 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:56722 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725944AbeLFR2m (ORCPT ); Thu, 6 Dec 2018 12:28:42 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9B46580D; Thu, 6 Dec 2018 09:28:41 -0800 (PST) Received: from [10.1.194.37] (e113632-lin.cambridge.arm.com [10.1.194.37]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3D2653F5AF; Thu, 6 Dec 2018 09:28:39 -0800 (PST) Subject: Re: [PATCH v3 03/10] sched/topology: Provide cfs_overload_cpus bitmap To: Steven Sistare , mingo@redhat.com, peterz@infradead.org Cc: subhra.mazumdar@oracle.com, dhaval.giani@oracle.com, daniel.m.jordan@oracle.com, pavel.tatashin@microsoft.com, matt@codeblueprint.co.uk, umgwanakikbuti@gmail.com, riel@redhat.com, jbacik@fb.com, juri.lelli@redhat.com, vincent.guittot@linaro.org, quentin.perret@arm.com, linux-kernel@vger.kernel.org References: <1541767840-93588-1-git-send-email-steven.sistare@oracle.com> <1541767840-93588-4-git-send-email-steven.sistare@oracle.com> <0857925d-a24e-90ea-e28c-90d69b2f66dd@oracle.com> <7d9b6789-af17-bcab-e52d-7e05483e10ea@arm.com> <094f54a9-a6ec-3c0d-4e06-6572023963c6@arm.com> <2e1f4d52-8ac6-04a3-c453-1679ef3df0e7@oracle.com> From: Valentin Schneider Message-ID: Date: Thu, 6 Dec 2018 17:28:37 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: <2e1f4d52-8ac6-04a3-c453-1679ef3df0e7@oracle.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-GB Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Steve, On 06/12/2018 16:40, Steven Sistare wrote: > [...] >> >> Ah yes, that would work. Thing is, I had excluded having the misfit masks >> being in the sd_llc_shareds, since from a logical standpoint they don't >> really belong there. >> >> With asymmetric CPU capacities we kind of disregard the cache landscape > > Sure, but adding awareness of the cache hierarchy can only make it better, > and a per-LLC mask organization can serve both the overloaded and misfit > use cases quite naturally. > [...] >> So in truth I was envisioning separate SD_ASYM_CPUCAPACITY-based >> sparsemasks, which is why I was rambling about SD_ASYM_CPUCAPACITY siblings >> of sd_llc_*()... *But* after I had a go at it, it looked to me like that >> was a lot of duplicated code. > > I would be happy to review your code and make suggestions to reduce duplication, > and happy to continue to discuss clean and optimal handling for misfits. However, > I have a request: can we push my patches across the finish line first? Stealing > for misfits can be its own patch series. Please consider sending your reviewed-by > for the next version of my series. I will send it soon. > Sure, as things stand right now I'm fairly convinced this doesn't harm asymmetric systems. The only thing I would add (ignoring misfits) is that with EAS we would need to gate stealing with something like: !static_branch_unlikely(&sched_energy_present) || READ_ONCE(rq->rd->overutilized) And who "gets" to add this gating (or at least, when must it be added) depends on which patch-set gets in first. [...] >> Sadly I think that doesn't work as well for cfs_overload_cpus since you >> can't split a sparsemask's chunks over several NUMA nodes, so we'd be >> stuck with an allocation on a single node (but we already do that in some >> places, e.g. for nohz.idle_cpus_mask, so... Is it that bad?). > > It can be bad for high memory bandwidth workloads, as the sparsemasks will > be displaced from cache and we incur remote memory latencies on next access. > Aye, I just caught up with the LPC videos and was about to reply here to say that, all things considered, it's probably not such a good idea... > - Steve >