Received: by 2002:a05:6358:1087:b0:cb:c9d3:cd90 with SMTP id j7csp1110486rwi; Wed, 19 Oct 2022 06:53:55 -0700 (PDT) X-Google-Smtp-Source: AMsMyM7iMjLudwXN8ScCCjTfaSMZHHWhDfFZWKu/2Ok1k5qcgtxAioG1OBlcggTRrQU93uG7t5c1 X-Received: by 2002:a17:90b:38ca:b0:20d:a0e7:af33 with SMTP id nn10-20020a17090b38ca00b0020da0e7af33mr40107081pjb.154.1666187624634; Wed, 19 Oct 2022 06:53:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1666187624; cv=none; d=google.com; s=arc-20160816; b=tniseK9c0ODB8mdqfNdIkn5uGw391eC7gQUTC0vT+72nhcSGh8sSrWva28I+thRd4q oYEuD2M64EE6KWtIA7Krku2A1ZjQmMooQxE0k82MjFV2StEnLYFWHV9tfk5YvzKiWz/F DFHbEstv0Ec5Z4sP/noNKSu28DABmcZMS5S96lpDJRzCpnKVAdtk0QZ/kHibX0jwwJof N4Vs2rqCAJYDg+LIA6nfXFn0mkKYxx0mJpPsPQLtTCkbGQcmBa5Ak0uchel8VuDuu5Dc UHIwXX+iSmn4gN9hRnZzIwnpceFRRH8PEH4p7Tm93qilQ+LsrPkcHRW2n5KT7hEAgOjH gTvA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=00hqzPQ60wefKYMOhXC429UksHq3kwxJP8z9O6lm324=; b=X0IwaOKO3yju62Ix0B4gSIm/Bract6BO+zYpBBzcEPFJ3viVXZVhbt9pCE6bgxyyiH 7U5qGGnu0p4E+TSiiLBrTiDMvtDIJvLVFblv7Ieonq/JQNojwBikFRg/7NuNdVtXbvyL jtbwlirmOPU5mw9C3EZj6NSumBcdmxirSNrmv8WjRc5wlKngqSwZL0Q97rbpZbwujdsB nZFJCoFd7fYYyPH0M1tfVcFaI+IOmwEmUw6U9GNYJC0ZLNCBlG3Yvw0U9+EDZBuEsupm T7Sg+ph9O7TZ86pKhdONEd0cCWrAtsmfOLAwXTIE9ZgDUGkYJxWqSQMRULV+UCU5zVCo E8NA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=MOjgelvd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h20-20020a62b414000000b0054114c6685asi16345804pfn.129.2022.10.19.06.53.32; Wed, 19 Oct 2022 06:53:44 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@bytedance-com.20210112.gappssmtp.com header.s=20210112 header.b=MOjgelvd; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=bytedance.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230517AbiJSMsP (ORCPT + 99 others); Wed, 19 Oct 2022 08:48:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58682 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231569AbiJSMrd (ORCPT ); Wed, 19 Oct 2022 08:47:33 -0400 Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0419CF4B for ; Wed, 19 Oct 2022 05:30:24 -0700 (PDT) Received: by mail-pj1-x1032.google.com with SMTP id a5-20020a17090aa50500b002008eeb040eso737500pjq.1 for ; Wed, 19 Oct 2022 05:30:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance-com.20210112.gappssmtp.com; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=00hqzPQ60wefKYMOhXC429UksHq3kwxJP8z9O6lm324=; b=MOjgelvdOpUz3hTkdT2hLaJgmsoNf4pMmZlcPSbRLd6dvofUiGAaVZZtr5/JC0p2Q/ 1JqYzhRFPeuM3biNvdYpbwtv428NyqVc3WyFV4Stc577+nAII0OmzfDUorGoGH43Xv2L qui08vC7vApFJVsqLX8i3emFeZei0ZmcMqvEqcNQ1IcLDo/MEbeXhV9NT9MGB/B/jMQ4 7u4uyWnhd6PQAiwmytzgEyHaB1Y9kEtD25v1C8fXVGkDiczZmr2jkyqdUtczNtXudSb1 3gxcTfN3uV1Z7663OkP1TNqpKtPjHpBkvpYd8nuJLtnxASr0HfAzs01koDty3H5Djtmm q4uA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=00hqzPQ60wefKYMOhXC429UksHq3kwxJP8z9O6lm324=; b=PpkysfGe7w61TJUnKmm7PDEUWdyH6r/bZfRJgyxOwuof/kv7mtJ/qRG2o1jqLI93In Q8ae1hXJ+WeTocsJd46jXMz0kBuPKitZS5UMTih4J950d4Bk20yeBIqeHHLN0zBLY92n o3mPYB5qHdd9tpBaiOzBgvNlZTDEdVkqZ9EgCj/IhH9P3bNUlWvSMPUQKAxfXap+GFn3 H6fbaRqsZdWuwekp18sLkyDVbHTgzBn4O7hl0q/nBXVE54MQYgWP2vI9QpVO3MPta9fa 9VnXNRyNz+6BGBm6rEbidoF8hqL9LMV+1R3Ns0SWWYXrra6eBhijveJUhmCB/j8MvglM f5aA== X-Gm-Message-State: ACrzQf0WcPkLbawdK+GUzV3gMzQK6AMMCnip3rpXPEmDlE2Y5KvWppr1 z+jOMRRpQUyCrh8Y+27nJ4eR3A== X-Received: by 2002:a17:902:9b82:b0:183:fffb:1bfe with SMTP id y2-20020a1709029b8200b00183fffb1bfemr8265751plp.173.1666182569537; Wed, 19 Oct 2022 05:29:29 -0700 (PDT) Received: from C02DV8HUMD6R.bytedance.net ([139.177.225.237]) by smtp.gmail.com with ESMTPSA id c21-20020a63da15000000b00439c6a4e1ccsm9881825pgh.62.2022.10.19.05.29.22 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 19 Oct 2022 05:29:29 -0700 (PDT) From: Abel Wu To: Peter Zijlstra , Ingo Molnar , Mel Gorman , Vincent Guittot , Dietmar Eggemann , Valentin Schneider Cc: Josh Don , Chen Yu , Tim Chen , K Prateek Nayak , "Gautham R . Shenoy" , Aubrey Li , Qais Yousef , Juri Lelli , Rik van Riel , Yicong Yang , Barry Song <21cnbao@gmail.com>, linux-kernel@vger.kernel.org, Abel Wu Subject: [PATCH v6 0/4] sched/fair: Improve scan efficiency of SIS Date: Wed, 19 Oct 2022 20:28:55 +0800 Message-Id: <20221019122859.18399-1-wuyun.abel@bytedance.com> X-Mailer: git-send-email 2.37.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This patchset tries to improve SIS scan efficiency by recording idle cpus in a cpumask for each LLC which will be used as a target cpuset in the domain scan. The cpus are recorded at CORE granule to avoid tasks being stack on same core. v5 -> v6: - Rename SIS_FILTER to SIS_CORE as it can only be activated when SMT is enabled and better describes the behavior of CORE granule update & load delivery. - Removed the part of limited scan for idle cores since it might be better to open another thread to discuss the strategies such as limited or scaled depth. But keep the part of full scan for idle cores when LLC is overloaded because SIS_CORE can greatly reduce the overhead of full scan in such case. - Removed the state of sd_is_busy which indicates an LLC is fully busy and we can safely skip the SIS domain scan. I would prefer leave this to SIS_UTIL. - The filter generation mechanism is replaced by in-place updates during domain scan to better deal with partial scan failures. - Collect Reviewed-bys from Tim Chen v4 -> v5: - Add limited scan for idle cores when overloaded, suggested by Mel - Split out several patches since they are irrelevant to this scope - Add quick check on ttwu_pending before core update - Wrap the filter into SIS_FILTER feature, suggested by Chen Yu - Move the main filter logic to the idle path, because the newidle balance can bail out early if rq->avg_idle is small enough and lose chances to update the filter. v3 -> v4: - Update filter in load_balance rather than in the tick - Now the filter contains unoccupied cpus rather than overloaded ones - Added mechanisms to deal with the false positive cases v2 -> v3: - Removed sched-idle balance feature and focus on SIS - Take non-CFS tasks into consideration - Several fixes/improvement suggested by Josh Don v1 -> v2: - Several optimizations on sched-idle balancing - Ignore asym topos in can_migrate_task - Add more benchmarks including SIS efficiency - Re-organize patch as suggested by Mel Gorman Abel Wu (4): sched/fair: Skip core update if task pending sched/fair: Ignore SIS_UTIL when has_idle_core sched/fair: Introduce SIS_CORE sched/fair: Deal with SIS scan failures include/linux/sched/topology.h | 15 ++++ kernel/sched/fair.c | 122 +++++++++++++++++++++++++++++---- kernel/sched/features.h | 7 ++ kernel/sched/sched.h | 3 + kernel/sched/topology.c | 8 ++- 5 files changed, 141 insertions(+), 14 deletions(-) -- 2.37.3