Received: by 2002:a05:6a10:206:0:0:0:0 with SMTP id 6csp3284195pxj; Tue, 11 May 2021 00:32:47 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxUqnRlIyi0Dg1FYYXNDwpmukCCMNFE6Ve+2xLuG1vvxOa03h7cssF5nwAOYjx3+re/c3Ta X-Received: by 2002:a92:6902:: with SMTP id e2mr25735492ilc.172.1620718366971; Tue, 11 May 2021 00:32:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1620718366; cv=none; d=google.com; s=arc-20160816; b=sZ7g+1V6F7tZzl/OcDLbpKw4HcOrieSYOhOFD5VzVcELdsOWjPhpWv+neXdrx9FGBS 9hoP4ImFFodx5VY9TSCmgBVRr5WvCS9L6cMTj+XtEQE2D0/oQ87hpYG1b8bKZI6ECusE aOYjGShsdlyE1o4NzN1BFdvQ8KXOvWhmwU6YaDhlL7FVjXW0SikOZoL+ofvfya8BpbHD CoQJ96ZfHJnnS3RNWUGKtFVY63dPkq+7nOtI4RiGc7wvWbtaUCovGB4vyR0H43KSK3Th Y1ULxAkMSByU/Y9VP7g3de4kKoeXSZKiP+8p9RmaixwlT4W8d7Tc353AWWxV/SJ2/bXm Y9jw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=MIqjaZO00c7j96e3E5odnRjvjW8XfjRSMn2eBpvgxrI=; b=rq9EZBtfJiJI/LIzgFlUeqp1xz+kZxNzEmv1zEF+YwTpVqZ7F5S674yCitlSkCGxBg 0ilOTAXR0Gzg8MxY+5e306nirV5r9nrOj95mCfb71+1IHQWsKe3u21UDfAEZvJciT6fi lBdLvOOiZF99xYOg7pSdFHGfJ/0jJ5jMAoO09jMhdRzleKPZ1FRe9FVyjJjtUDEwXOKd aGreW6PGlMMfgewwq/iOm7ZBwTYeusivTmFlqBG9+ePqWqIyb1j/s4ye3Xl3bFBWx3CO ZLCegLcqrDOohdH74zpvZ5MSpriVIwr/cjSN2uru5/ryswmQNXdHQ0Yk865SlhSo60z1 bO2Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=tgbhQ4QX; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id s16si17644773jas.113.2021.05.11.00.32.34; Tue, 11 May 2021 00:32:46 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@ibm.com header.s=pp1 header.b=tgbhQ4QX; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229924AbhEKHdN (ORCPT + 99 others); Tue, 11 May 2021 03:33:13 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:45122 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S229637AbhEKHdM (ORCPT ); Tue, 11 May 2021 03:33:12 -0400 Received: from pps.filterd (m0098414.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 14B75CWW134387; Tue, 11 May 2021 03:31:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : mime-version : content-transfer-encoding; s=pp1; bh=MIqjaZO00c7j96e3E5odnRjvjW8XfjRSMn2eBpvgxrI=; b=tgbhQ4QXHblEWshgsoq+Jv1TJ2u+55ZDak+wVdyyunnmPTBuhZPKXpmN+k/6bL+3iOLK nDbEFhwqhDmRJBtNZ03UKul3/RCez04esu5jc/vjXwE0fx/AnxUNIS3H7amAvy/2mH1C SU8Eiji68W2SDnUBzSumiCasjaGqDnSD9i/mWld7IpQb9m5EXvod1uMM2j2MX1usL4yL cIyiHQkWbGvhci/P2LThmi0enb8VV0npBN4dBeBfGXFggE/mlgLfD7E1u9eJ84etYJFd K5kJ5qUDHekG9yg1FLLryV9fvuysH/0u8IUgNxyrzqNoK6X1u6Uv+BwFGb57ifDTVPaZ OA== Received: from ppma06ams.nl.ibm.com (66.31.33a9.ip4.static.sl-reverse.com [169.51.49.102]) by mx0b-001b2d01.pphosted.com with ESMTP id 38fm1tag1h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 11 May 2021 03:31:42 -0400 Received: from pps.filterd (ppma06ams.nl.ibm.com [127.0.0.1]) by ppma06ams.nl.ibm.com (8.16.0.43/8.16.0.43) with SMTP id 14B7MW0a004054; Tue, 11 May 2021 07:31:40 GMT Received: from b06cxnps4075.portsmouth.uk.ibm.com (d06relay12.portsmouth.uk.ibm.com [9.149.109.197]) by ppma06ams.nl.ibm.com with ESMTP id 38dhwh9gyt-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Tue, 11 May 2021 07:31:40 +0000 Received: from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com [9.149.105.58]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 14B7VcbM33030500 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 11 May 2021 07:31:38 GMT Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 01AED4C05C; Tue, 11 May 2021 07:31:38 +0000 (GMT) Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A71774C07E; Tue, 11 May 2021 07:31:37 +0000 (GMT) Received: from localhost.localdomain (unknown [9.145.34.120]) by d06av22.portsmouth.uk.ibm.com (Postfix) with ESMTP; Tue, 11 May 2021 07:31:37 +0000 (GMT) From: Laurent Dufour To: mpe@ellerman.id.au, benh@kernel.crashing.org, paulus@samba.org Cc: nathanl@linux.ibm.com, linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, Srikar Dronamraju Subject: [PATCH v2] ppc64/numa: consider the max numa node for migratable LPAR Date: Tue, 11 May 2021 09:31:36 +0200 Message-Id: <20210511073136.17795-1-ldufour@linux.ibm.com> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: 3N0Xa6K1QIt-ipkp-Nt-4r7gxQiD8Eoy X-Proofpoint-GUID: 3N0Xa6K1QIt-ipkp-Nt-4r7gxQiD8Eoy X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391,18.0.761 definitions=2021-05-11_02:2021-05-10,2021-05-11 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 phishscore=0 mlxscore=0 suspectscore=0 clxscore=1015 lowpriorityscore=0 mlxlogscore=999 spamscore=0 impostorscore=0 adultscore=0 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104190000 definitions=main-2105110053 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org When a LPAR is migratable, we should consider the maximum possible NUMA node instead the number of NUMA node from the actual system. The DT property 'ibm,current-associativity-domains' is defining the maximum number of nodes the LPAR can see when running on that box. But if the LPAR is being migrated on another box, it may seen up to the nodes defined by 'ibm,max-associativity-domains'. So if a LPAR is migratable, that value should be used. Unfortunately, there is no easy way to know if a LPAR is migratable or not. The hypervisor is exporting the property 'ibm,migratable-partition' in the case it set to migrate partition, but that would not mean that the current partition is migratable. Without this patch, when a LPAR is started on a 2 nodes box and then migrated to a 3 nodes box, the hypervisor may spread the LPAR's CPUs on the 3rd node. In that case if a CPU from that 3rd node is added to the LPAR, it will be wrongly assigned to the node because the kernel has been set to use up to 2 nodes (the configuration of the departure node). With this patch applies, the CPU is correctly added to the 3rd node. Fixes: f9f130ff2ec9 ("powerpc/numa: Detect support for coregroup") Reviewed-by: Srikar Dronamraju Signed-off-by: Laurent Dufour --- V2: Address Srikar's comments - Fix the commit message - Use pr_info instead printk(KERN_INFO..) --- arch/powerpc/mm/numa.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/arch/powerpc/mm/numa.c b/arch/powerpc/mm/numa.c index f2bf98bdcea2..094a1076fd1f 100644 --- a/arch/powerpc/mm/numa.c +++ b/arch/powerpc/mm/numa.c @@ -893,7 +893,7 @@ static void __init setup_node_data(int nid, u64 start_pfn, u64 end_pfn) static void __init find_possible_nodes(void) { struct device_node *rtas; - const __be32 *domains; + const __be32 *domains = NULL; int prop_length, max_nodes; u32 i; @@ -909,9 +909,14 @@ static void __init find_possible_nodes(void) * it doesn't exist, then fallback on ibm,max-associativity-domains. * Current denotes what the platform can support compared to max * which denotes what the Hypervisor can support. + * + * If the LPAR is migratable, new nodes might be activated after a LPM, + * so we should consider the max number in that case. */ - domains = of_get_property(rtas, "ibm,current-associativity-domains", - &prop_length); + if (!of_get_property(of_root, "ibm,migratable-partition", NULL)) + domains = of_get_property(rtas, + "ibm,current-associativity-domains", + &prop_length); if (!domains) { domains = of_get_property(rtas, "ibm,max-associativity-domains", &prop_length); @@ -920,6 +925,8 @@ static void __init find_possible_nodes(void) } max_nodes = of_read_number(&domains[min_common_depth], 1); + pr_info("Partition configured for %d NUMA nodes.\n", max_nodes); + for (i = 0; i < max_nodes; i++) { if (!node_possible(i)) node_set(i, node_possible_map); -- 2.31.1