Received: by 2002:a05:6a10:413:0:0:0:0 with SMTP id 19csp1437808pxp; Thu, 17 Mar 2022 09:14:07 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwHHV3vnYirLEzBvktiwWtelk69RzN9jL77unmjrzamIN9jGMvbkY36nYMRhzIfo0w/h2QQ X-Received: by 2002:a17:907:6ea4:b0:6db:5921:4497 with SMTP id sh36-20020a1709076ea400b006db59214497mr5269615ejc.532.1647533646830; Thu, 17 Mar 2022 09:14:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1647533646; cv=none; d=google.com; s=arc-20160816; b=iF4xT/WzTAg4A4+QnF1MJz2LqFw4roy/lvzXy03zkd2qYdOf8fDM9bGvUPB2lmq9jU 4zKh4pQf4u/P9QWEZklJkgwDnqp78PkVZwjSXqXyHd8AoCEjPVGALbGr1hJUzlCSjTeO xbt+Soj++0Wr1ejhcJaunnsveBmoDv/JZz62J6jhUpBNWjUbANG2LmYlDpcRA7FJReFp Pck0RAh1Ud6zKvxzpOyeh4n02Kl7MEyw0RB9860XJJnLofWPOC9fY4fDc6xz22LQgTo4 7J/NuLB10+OlOmlRCYf50GZgZeYmG+hSMjNNvl+wWH+8SRO7v+8fCOT0yayeaTrJNnwI UFaQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=fqJRv56l32D1BVSoDkBaNbis0fiVofjy5I5K5492MU8=; b=mcBZGVJrwuKAqW15+Sy+I9C093SkmOiOtRvgT/bFiVYh+j1srIvyauAP7YlFIaUIjp NFvHQj4EzpArcqNPHZI3L0SFpytwI7+xZN3XhnXDo/cQHT5fyqw+LrYxxhYJoQ/ObHNX bbaX0avTiffoyzH3nj0MRXmRXQk51ycOOJ7y2FENHNRO8q2bW82ahmKIa6G5qxszQMYx gTiZLlmQ0etAC0L6ZEsM8eRLfi4UU2FfFbIKwg9h3Iw+f71jV6g/3JppeDrqvIu3BtUj Um6aRXR9peiwWJ+iQbNHmK1R5eMPXzG7qlXpk359BUw/xZvQhkRoD+6/5etUH9J5I9Iu xIAg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linuxfoundation.org header.s=korg header.b=dqPXuN9y; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id u3-20020a50c043000000b00418c2b5bd37si1883336edd.25.2022.03.17.09.13.40; Thu, 17 Mar 2022 09:14:06 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linuxfoundation.org header.s=korg header.b=dqPXuN9y; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234501AbiCQM5h (ORCPT + 99 others); Thu, 17 Mar 2022 08:57:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39672 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234922AbiCQMyE (ORCPT ); Thu, 17 Mar 2022 08:54:04 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1A0D51F125F; Thu, 17 Mar 2022 05:52:48 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A7FD961578; Thu, 17 Mar 2022 12:52:47 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AFAA5C340E9; Thu, 17 Mar 2022 12:52:46 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1647521567; bh=sy3aP4fnh8SyAGLvHPsjL4Mx6PXZBmUBSdow+IfmEWg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dqPXuN9yMjGSWYRC0U4b83vGUwtgXqlWBli7esJaV+DfIM6f4oFP/Bl7L7sQ/KQVO eQul7Jff5PA/aC/T1ggF34fY2OavoEM3WvGSyHUxBmUo4Lp2JaSIepyWooOQ9hJqUt mN+3HWy1NoWL0k8q8bIgubpUavpXnHqFY6/Taa98= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Alexander Lobakin , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Thomas Bogendoerfer , Sasha Levin Subject: [PATCH 5.16 13/28] MIPS: smp: fill in sibling and core maps earlier Date: Thu, 17 Mar 2022 13:46:04 +0100 Message-Id: <20220317124527.146347831@linuxfoundation.org> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220317124526.768423926@linuxfoundation.org> References: <20220317124526.768423926@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-8.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Alexander Lobakin [ Upstream commit f2703def339c793674010cc9f01bfe4980231808 ] After enabling CONFIG_SCHED_CORE (landed during 5.14 cycle), 2-core 2-thread-per-core interAptiv (CPS-driven) started emitting the following: [ 0.025698] CPU1 revision is: 0001a120 (MIPS interAptiv (multi)) [ 0.048183] ------------[ cut here ]------------ [ 0.048187] WARNING: CPU: 1 PID: 0 at kernel/sched/core.c:6025 sched_core_cpu_starting+0x198/0x240 [ 0.048220] Modules linked in: [ 0.048233] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.17.0-rc3+ #35 b7b319f24073fd9a3c2aa7ad15fb7993eec0b26f [ 0.048247] Stack : 817f0000 00000004 327804c8 810eb050 00000000 00000004 00000000 c314fdd1 [ 0.048278] 830cbd64 819c0000 81800000 817f0000 83070bf4 00000001 830cbd08 00000000 [ 0.048307] 00000000 00000000 815fcbc4 00000000 00000000 00000000 00000000 00000000 [ 0.048334] 00000000 00000000 00000000 00000000 817f0000 00000000 00000000 817f6f34 [ 0.048361] 817f0000 818a3c00 817f0000 00000004 00000000 00000000 4dc33260 0018c933 [ 0.048389] ... [ 0.048396] Call Trace: [ 0.048399] [<8105a7bc>] show_stack+0x3c/0x140 [ 0.048424] [<8131c2a0>] dump_stack_lvl+0x60/0x80 [ 0.048440] [<8108b5c0>] __warn+0xc0/0xf4 [ 0.048454] [<8108b658>] warn_slowpath_fmt+0x64/0x10c [ 0.048467] [<810bd418>] sched_core_cpu_starting+0x198/0x240 [ 0.048483] [<810c6514>] sched_cpu_starting+0x14/0x80 [ 0.048497] [<8108c0f8>] cpuhp_invoke_callback_range+0x78/0x140 [ 0.048510] [<8108d914>] notify_cpu_starting+0x94/0x140 [ 0.048523] [<8106593c>] start_secondary+0xbc/0x280 [ 0.048539] [ 0.048543] ---[ end trace 0000000000000000 ]--- [ 0.048636] Synchronize counters for CPU 1: done. ...for each but CPU 0/boot. Basic debug printks right before the mentioned line say: [ 0.048170] CPU: 1, smt_mask: So smt_mask, which is sibling mask obviously, is empty when entering the function. This is critical, as sched_core_cpu_starting() calculates core-scheduling parameters only once per CPU start, and it's crucial to have all the parameters filled in at that moment (at least it uses cpu_smt_mask() which in fact is `&cpu_sibling_map[cpu]` on MIPS). A bit of debugging led me to that set_cpu_sibling_map() performing the actual map calculation, was being invocated after notify_cpu_start(), and exactly the latter function starts CPU HP callback round (sched_core_cpu_starting() is basically a CPU HP callback). While the flow is same on ARM64 (maps after the notifier, although before calling set_cpu_online()), x86 started calculating sibling maps earlier than starting the CPU HP callbacks in Linux 4.14 (see [0] for the reference). Neither me nor my brief tests couldn't find any potential caveats in calculating the maps right after performing delay calibration, but the WARN splat is now gone. The very same debug prints now yield exactly what I expected from them: [ 0.048433] CPU: 1, smt_mask: 0-1 [0] https://git.kernel.org/pub/scm/linux/kernel/git/mips/linux.git/commit/?id=76ce7cfe35ef Signed-off-by: Alexander Lobakin Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Bogendoerfer Signed-off-by: Sasha Levin --- arch/mips/kernel/smp.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/mips/kernel/smp.c b/arch/mips/kernel/smp.c index d542fb7af3ba..1986d1309410 100644 --- a/arch/mips/kernel/smp.c +++ b/arch/mips/kernel/smp.c @@ -351,6 +351,9 @@ asmlinkage void start_secondary(void) cpu = smp_processor_id(); cpu_data[cpu].udelay_val = loops_per_jiffy; + set_cpu_sibling_map(cpu); + set_cpu_core_map(cpu); + cpumask_set_cpu(cpu, &cpu_coherent_mask); notify_cpu_starting(cpu); @@ -362,9 +365,6 @@ asmlinkage void start_secondary(void) /* The CPU is running and counters synchronised, now mark it online */ set_cpu_online(cpu, true); - set_cpu_sibling_map(cpu); - set_cpu_core_map(cpu); - calculate_cpu_foreign_map(); /* -- 2.34.1