Received: by 2002:a05:7412:37c9:b0:e2:908c:2ebd with SMTP id jz9csp2581618rdb; Fri, 22 Sep 2023 02:59:31 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHdxNUHytY/P1quhskfN8IIUJsGF9eTuHEmmuK2zlgYVt8cUh8Av5ZxMD1EB3q3HyPEXR5a X-Received: by 2002:a05:6358:3381:b0:135:46d9:12f7 with SMTP id i1-20020a056358338100b0013546d912f7mr9538581rwd.26.1695376771269; Fri, 22 Sep 2023 02:59:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695376771; cv=none; d=google.com; s=arc-20160816; b=kADTexfcRdQAS4rkZTNXRv5sxUCqQWwetINgnm/yC8C7jkAolp2x4VtJGVWbX55v3d wa84NgyofxbFzHD3mMza4JxJyD3t9D3SemMklhMk41YbkhW4cY+8E5L5RSNKaezc2xhf ji9Qne9mOOszjR8O4x+loyrvV81vT3pCYs0XB8HvRYK0VASl1CSY5h7xSWtOy7CBWjne jxbuwSgfBz8EGjvNIBRYC55mWehrrz5xEMxGjAE5XJfASRfmTdXpBmBh8ZVFh1XPHc0+ YGOnn7cQWBP3z104B50IfUVzPpnFL+UUouIU4ZNP7SZpgp+bTHvjGbz5vRuiQ1rQlLJj REdg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:in-reply-to :mime-version:user-agent:date:message-id:from:references:cc:to :subject; bh=YSGvuB27B+acTjA+JOgmasr8VpiWHmgqqy5Ij3wzTb0=; fh=uNF132NRfPMLTbWmCbvOnvWRzodc95M3tDwkjXnbAKk=; b=LAg0gV4YWcehcE9XCIAnrgY/wesul28+RaWfnWATKG83DS3nULL3hog/de4YGL2mRj c5O7UE6yFBJ3X/fUdQ1NCkT3oMOC9aJTv1qd8qwG8OHQS4ETSputa4QB4brddv6h/gUh xSConBgMsqXVmH0NQYA/lHzdXXZKtqDQhs1qkmw7xQOx+BHONKVu67D2mLZLAExYdYaZ kfvzEp1eQRVgCooh4lgIwnqu6fhvzYVGVHq+Sb1tXwqxJgijKSKxZvZTsn5hdU5siu9Y 3bP4FehIXlRvygSvu+wVRgS1QEZ0sI6B1oAvHk28lTjRZ3/34pQ0j3p0pJ0jWUTmBluY QHEw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Return-Path: Received: from pete.vger.email (pete.vger.email. [2620:137:e000::3:6]) by mx.google.com with ESMTPS id e10-20020a65688a000000b0055c81ab9a9fsi3441482pgt.582.2023.09.22.02.59.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Sep 2023 02:59:31 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) client-ip=2620:137:e000::3:6; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=huawei.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by pete.vger.email (Postfix) with ESMTP id B27D68026C12; Fri, 22 Sep 2023 02:47:29 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at pete.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232949AbjIVJrR (ORCPT + 99 others); Fri, 22 Sep 2023 05:47:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49610 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233276AbjIVJrJ (ORCPT ); Fri, 22 Sep 2023 05:47:09 -0400 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D5A371B3 for ; Fri, 22 Sep 2023 02:47:01 -0700 (PDT) Received: from canpemm500009.china.huawei.com (unknown [172.30.72.55]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4RsS6V59GbzrSmx; Fri, 22 Sep 2023 17:44:50 +0800 (CST) Received: from [10.67.121.177] (10.67.121.177) by canpemm500009.china.huawei.com (7.192.105.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.31; Fri, 22 Sep 2023 17:46:59 +0800 Subject: Re: [PATCH] arch_topology: Support SMT control on arm64 To: Sudeep Holla CC: , , , , , , , , , Yicong Yang , Linuxarm References: <20230919123319.23785-1-yangyicong@huawei.com> <20230921150333.c2zqigs3xxwcg4ln@bogus> From: Yicong Yang Message-ID: <0ea971ba-549f-62ed-a181-41b5572cd190@huawei.com> Date: Fri, 22 Sep 2023 17:46:59 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.5.1 MIME-Version: 1.0 In-Reply-To: <20230921150333.c2zqigs3xxwcg4ln@bogus> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-Originating-IP: [10.67.121.177] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To canpemm500009.china.huawei.com (7.192.105.203) X-CFilter-Loop: Reflected X-Spam-Status: No, score=-2.2 required=5.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Fri, 22 Sep 2023 02:47:30 -0700 (PDT) On 2023/9/21 23:03, Sudeep Holla wrote: > On Tue, Sep 19, 2023 at 08:33:19PM +0800, Yicong Yang wrote: >> From: Yicong Yang >> >> The core CPU control framework supports runtime SMT control which >> is not yet supported on arm64. Besides the general vulnerabilities >> concerns we want this runtime control on our arm64 server for: >> >> - better single CPU performance in some cases >> - saving overall power consumption >> >> This patch implements it in the following aspects: >> >> - implement the callbacks of the core >> - update the SMT status after the topology enumerated on arm64 >> - select HOTPLUG_SMT for arm64 >> >> For disabling SMT we'll offline all the secondary threads and >> only leave the primary thread. Since we don't have restriction >> for primary thread selection, the first thread is chosen as the >> primary thread in this implementation. >> >> Tests has been done on our ACPI based arm64 server and on >> ACPI/OF based QEMU VMs. >> >> Signed-off-by: Yicong Yang >> --- >> arch/arm64/Kconfig | 1 + >> drivers/base/arch_topology.c | 63 +++++++++++++++++++++++++++++++++++ >> include/linux/arch_topology.h | 11 ++++++ >> 3 files changed, 75 insertions(+) >> >> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig >> index b10515c0200b..531a71c7f499 100644 >> --- a/arch/arm64/Kconfig >> +++ b/arch/arm64/Kconfig >> @@ -233,6 +233,7 @@ config ARM64 >> select HAVE_KRETPROBES >> select HAVE_GENERIC_VDSO >> select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU >> + select HOTPLUG_SMT if SMP >> select IRQ_DOMAIN >> select IRQ_FORCED_THREADING >> select KASAN_VMALLOC if KASAN >> diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c >> index b741b5ba82bd..75a693834fff 100644 >> --- a/drivers/base/arch_topology.c >> +++ b/drivers/base/arch_topology.c >> @@ -729,6 +729,63 @@ const struct cpumask *cpu_clustergroup_mask(int cpu) >> return &cpu_topology[cpu].cluster_sibling; >> } >> >> +#ifdef CONFIG_HOTPLUG_SMT >> +static int topology_smt_num_threads = 1; >> + >> +void __init topology_smt_set_num_threads(void) >> +{ >> + int cpu, sibling, threads; >> + >> + /* >> + * Walk all the CPUs to find the largest thread number, in case we're >> + * on a heterogeneous platform with only part of the CPU cores support >> + * SMT. >> + * >> + * Get the thread number by checking the CPUs with same core id >> + * rather than checking the topology_sibling_cpumask(), since the >> + * sibling mask will not cover all the CPUs if there's CPU offline. >> + */ >> + for_each_possible_cpu(cpu) { >> + threads = 1; >> + >> + /* Invalid thread id, this CPU is not in a SMT core */ >> + if (cpu_topology[cpu].thread_id == -1) >> + continue; >> + >> + for_each_possible_cpu(sibling) { > > I would really like to avoid parsing all the cpus here(O(cpu^2)) > What if we use a temp cpumask to record each visited CPU and make it O(cpu)? I didn't use it because I don't want to see a memory allocation failure for the cpumask. In current implementation there's no way to fail. > Another random thought(just looking at DT parsing) is we can count threads > while parsing itself if we need the info early before the topology cpumasks > are setup. Need to look at ACPI parsing and how to make that generic but > thought of checking the idea here first. > The ACPI parsing is in each arch's codes (currently for arm64 and loongarch). The code will be isolated to DT, arm64 ACPI parsing, loogarch ACPI parsing and future ACPI based archs, it'll be redundant and hard to maintian, I think. So I perfer to unify the processing since the logic is common, just check the finally built cpu_topology array regardless whether we're on an ACPI/OF based system. > [...] > >> @@ -841,6 +898,12 @@ void __init init_cpu_topology(void) >> reset_cpu_topology(); >> } >> >> + /* >> + * By this stage we get to know whether we support SMT or not, update >> + * the information for the core. >> + */ >> + topology_smt_set_num_threads(); >> + > > Does this have to be done this early ? I was wondering if we can defer until > all the cpumasks are set and you can rely on the thread_sibling mask ? > You can just get the weight of that mask on all cpus and choose the max value. > We do this at this stage because we've already gotten the necessary informations. As commented in topology_smt_set_num_threads(), I didn't use sibling masks because the sibling masks will not contain all the informations, it'll only be updated when CPUs going online in secondary_start_kernel()->store_cpu_topology(). Think of the case if we boot with maxcpus=1, the SMT status won't be collected completely if we're using the sibling mask. For example, the sibling mask of the boot CPU will be correct, but not for its sibling cores. Thanks.