Received: by 2002:a25:8b91:0:0:0:0:0 with SMTP id j17csp3884755ybl; Mon, 13 Jan 2020 04:24:31 -0800 (PST) X-Google-Smtp-Source: APXvYqw8cLAzjWuMnM+Qcxnk8p/eBd1Lt6UXOq9tl4c22wNyUXdyaCmkvkqsiLWmQTvEWToUArDz X-Received: by 2002:a05:6830:149a:: with SMTP id s26mr13308413otq.55.1578918271711; Mon, 13 Jan 2020 04:24:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1578918271; cv=none; d=google.com; s=arc-20160816; b=FwKW1YbELlbERq4wy3/3DjzkJzZxwvbxTwXM8HAk35ol1y5zAn9kNbzoO1dPtsr33v vG3SuxmfKG3ooPyvdLteApM4HoKVP9Xb+p1OrY8P3l02IApW2rotSao+O3wJLyEVFmmU v0lvGDxzbC8pdZ+/2QatEvWmhrddrG35H1Rd9o91cqYYYfVDWUfcISZnE2F47gsjOpFB SdyJ6yrSXVkSx2nGta25X6iwtBH3SRq8n2pCM7b8LKM86PLoO6xYbCK/jyp+O82shqLt SbB86zfqEp/UgLw6pS+D39vnrh/2VRrMV5ZVQQ2c7gUA6gDUggBR9PuAit4L/il/4wJR kNnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding :content-language:in-reply-to:mime-version:user-agent:date :message-id:from:references:cc:to:subject; bh=UKC7AfNtxiKQArhbU80Yn5JLex73XaUzS5WBoWkSGmM=; b=Rh1T+y+yIRKKcz1fgjhScX9Ewx9ET5BKxAqrl+PxphdXSCysXQYPSSUrCeWFwfuExM bj6T12xJkVSczuU9TF2eFIvMuIJOD52XL6xgp/gWkgmH+KLUcDyYe0nkfHHOV7gTr2ez sJrM83qpTUs+1CHmhwDjLkwwXUGQ4dPeu5qDZacgd26UwwcfiFD8lxzUQf6N4vSQ3PCx 55ZoskNfZqwyK0Z7C0pOUxxTxiX6kNL2Cc6oEvO0K3DpSfecsiL44S/Weq0N1q60+LyN kv30MLWBXzohqIp90dF7Mr7k1o66i3p0pttfsEBK9tjWalVDGTpeavk4H6RHOXZmpw53 Ig4g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id l5si5823671oil.96.2020.01.13.04.24.19; Mon, 13 Jan 2020 04:24:31 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728656AbgAMMWV (ORCPT + 99 others); Mon, 13 Jan 2020 07:22:21 -0500 Received: from foss.arm.com ([217.140.110.172]:38750 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726450AbgAMMWU (ORCPT ); Mon, 13 Jan 2020 07:22:20 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4291B13D5; Mon, 13 Jan 2020 04:22:20 -0800 (PST) Received: from [192.168.0.103] (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id F1C303F68E; Mon, 13 Jan 2020 04:22:18 -0800 (PST) Subject: Re: [PATCH] cpu-topology: warn if NUMA configurations conflicts with lower layer To: "Zengtao (B)" , Valentin Schneider , Morten Rasmussen Cc: Sudeep Holla , Linuxarm , Greg Kroah-Hartman , "Rafael J. Wysocki" , "linux-kernel@vger.kernel.org" References: <1577088979-8545-1-git-send-email-prime.zeng@hisilicon.com> <20191231164051.GA4864@bogus> <678F3D1BB717D949B966B68EAEB446ED340AE1D3@dggemm526-mbx.china.huawei.com> <20200102112955.GC4864@bogus> <678F3D1BB717D949B966B68EAEB446ED340AEB67@dggemm526-mbx.china.huawei.com> <678F3D1BB717D949B966B68EAEB446ED340AFCA0@dggemm526-mbx.china.huawei.com> <20200103114011.GB19390@bogus> <678F3D1BB717D949B966B68EAEB446ED340B31E9@dggemm526-mbx.china.huawei.com> <20200109104306.GA10914@e105550-lin.cambridge.arm.com> <678F3D1BB717D949B966B68EAEB446ED340BEDD6@dggemm526-mbx.china.huawei.com> <1a8f7963-97e9-62cc-12d2-39f816dfaf67@arm.com> <678F3D1BB717D949B966B68EAEB446ED340E2592@DGGEMM506-MBX.china.huawei.com> <15050bf2-99ec-e604-ab95-827ce86fd693@arm.com> <678F3D1BB717D949B966B68EAEB446ED340E420D@DGGEMM506-MBX.china.huawei.com> From: Dietmar Eggemann Message-ID: <06aa5fef-9b72-0f6d-9070-831a0c9b8db0@arm.com> Date: Mon, 13 Jan 2020 13:22:17 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.2.2 MIME-Version: 1.0 In-Reply-To: <678F3D1BB717D949B966B68EAEB446ED340E420D@DGGEMM506-MBX.china.huawei.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 13.01.20 13:08, Zengtao (B) wrote: >> -----Original Message----- >> From: Valentin Schneider [mailto:valentin.schneider@arm.com] >> Sent: Monday, January 13, 2020 7:17 PM >> To: Zengtao (B); Morten Rasmussen >> Cc: Sudeep Holla; Linuxarm; Greg Kroah-Hartman; Rafael J. Wysocki; >> linux-kernel@vger.kernel.org >> Subject: Re: [PATCH] cpu-topology: warn if NUMA configurations >> conflicts with lower layer >> >> On 13/01/2020 06:51, Zengtao (B) wrote: >>> I have tried both, this previous one don't work. But this one seems >> work >>> correctly with the warning message printout as expected. >>> >> >> Thanks for trying it out. >> >>> This patch is based on the fact " non-NUMA spans shouldn't overlap ", >> I am >>> not quite sure if this is always true? >>> >> >> I think this is required for get_group() to work properly. Otherwise, >> successive get_group() calls may override (and break) the sd->groups >> linking as you initially reported. >> >> In your example, for MC level we have >> >> tl->mask(3) == 3-7 >> tl->mask(4) == 4-7 >> >> Which partially overlaps, causing the relinking of '7->3' to '7->4'. Valid >> configurations would be >> >> wholly disjoint: >> tl->mask(3) == 0-3 >> tl->maks(4) == 4-7 >> >> equal: >> tl->mask(3) == 3-7 >> tl->mask(4) == 3-7 >> >>> Anyway, Could you help to raise the new patch? >>> >> >> Ideally I'd like to be able to reproduce this locally first (TBH I'd like >> to get my first suggestion to work since it's less intrusive). Could you >> share how you were able to trigger this? Dietmar's been trying to >> reproduce >> this with qemu but I don't think he's there just yet. > > Do you have got a hardware platform with clusters?what's the hardware > Cpu topology? I can test this with: sudo qemu-system-aarch64 -kernel ./Image -hda ./qemu-image-aarch64.img -append 'root=/dev/vda console=ttyAMA0 loglevel=8 sched_debug' -smp cores=8 --nographic -m 512 -cpu cortex-a53 -machine virt -numa node,cpus=0-2,nodeid=0 -numa node,cpus=3-7,nodeid=1 and diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c index 1eb81f113786..e941a402e5f1 100644 --- a/drivers/base/arch_topology.c +++ b/drivers/base/arch_topology.c @@ -465,6 +465,9 @@ void update_siblings_masks(unsigned int cpuid) if (cpuid_topo->package_id != cpu_topo->package_id) continue; + if ((cpu < 4 && cpuid > 3) || (cpu > 3 && cpuid < 4)) + continue; + cpumask_set_cpu(cpuid, &cpu_topo->core_sibling); cpumask_set_cpu(cpu, &cpuid_topo->core_sibling); on mainline qemu. I do need the hack in update_siblings_masks() since the topology detection via -smp cores=X, sockets=Y doesn't work yet.