Received: by 2002:a05:6a10:16a7:0:0:0:0 with SMTP id gp39csp684851pxb; Thu, 5 Nov 2020 10:11:15 -0800 (PST) X-Google-Smtp-Source: ABdhPJzIm45CGAHTcFZ34tHrBL108e6yVT4bHDB86P7t+Qs7YzjBALzxQUgYsHFo767XfZabtlHh X-Received: by 2002:a5d:424a:: with SMTP id s10mr4410711wrr.130.1604599875054; Thu, 05 Nov 2020 10:11:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1604599875; cv=none; d=google.com; s=arc-20160816; b=KHh4CGw8XADlETS3soPZXypzAyYnVR1cyN8JC3Yzpn2wLa4HwSkFg9dqaUmiaN3vXj ++f0/7RomdW1dOXPJ3mwN45cpR5InTiboY449Vkpdg/o93LQHwBD6zeYlfaeybHJ8e0F cIUacvvk9OUB2cg4SIAidUYsZVPBKjWU10xr4S7RLiPphDAVCDH1Kla+881Xx55p5njo 3Fl7SZWhjcDk0CGBwEmmWP+mZeqEq2B79uJzdl2ELYf7rwMLqWkGlRg/VJUKebawcmIa jAkhT3bqICOLjAyKY2qph4LeFaMfOLBLMRJACuuXCfv8qBwuBa8m/D7jjkO6DK3Z6bDu 8bqg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:to:from:cc:in-reply-to:subject:date:dkim-signature; bh=evErAM+3R8POckj6BoWNfzcQKezQH3REiwTi+k5Ke30=; b=Y55Q8seWoUmk21dQ8tUfgUSy06/ahWOQOd1d2rHv6iHuK9OzxQFd/kJEOb8TyPiVkI QtjcmoCAY6xKYQJvLrnQkYVRgbIZ3SlrQuBHJGio4TIhi9+8hJCc/cO6LRvn78gzNPNV LyC4vl6sgLkNyJoDSfeo7t3AbajMX0ddBt1C6krIjFoEM3d7CSZfoQtORnkN2agHwvO4 Vbyx2Vuvg5LMZ+cYfDIYXT0Bxfdl5zde/rekU1j/tUIrW5d25fJ0R1V/V/bMXfqHLIWo oAp57PdaqrfhzQ3onQFIT2Gw7HGDAJvLMqZRQ59SzAHrEklBxMVi64Iop48e+tXI2MQ9 SEzA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@dabbelt-com.20150623.gappssmtp.com header.s=20150623 header.b=cvZYUkjB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id l19si1729667edr.128.2020.11.05.10.10.49; Thu, 05 Nov 2020 10:11:15 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@dabbelt-com.20150623.gappssmtp.com header.s=20150623 header.b=cvZYUkjB; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731350AbgKESHC (ORCPT + 99 others); Thu, 5 Nov 2020 13:07:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50626 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725862AbgKESHC (ORCPT ); Thu, 5 Nov 2020 13:07:02 -0500 Received: from mail-pg1-x544.google.com (mail-pg1-x544.google.com [IPv6:2607:f8b0:4864:20::544]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E49FDC0613D2 for ; Thu, 5 Nov 2020 10:07:01 -0800 (PST) Received: by mail-pg1-x544.google.com with SMTP id g12so1878930pgm.8 for ; Thu, 05 Nov 2020 10:07:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dabbelt-com.20150623.gappssmtp.com; s=20150623; h=date:subject:in-reply-to:cc:from:to:message-id:mime-version :content-transfer-encoding; bh=evErAM+3R8POckj6BoWNfzcQKezQH3REiwTi+k5Ke30=; b=cvZYUkjBWInIl2l1mjIWf1vI7V8s6znitGQb6c5FeaYxHv0JPbh4k6w9pz9IC9AVgi gDivP8WJC+El0KFshMutO5n1YhCRHEIxrqzYwu6bCUnrciwL9E/9tGm25eHqZdW/XI91 T59XUK3EUSN6RVDl9mXW4WPshbsdGRiNI8znzVKFYGQbhx9IrGxgBTrOhued5XD/a+Rl jGfmJLxTf7oGVAKBpYdYB481ef3UnuDvjfFSwTdh0stV94i0MsDhYbrVvvmy+VsOz+Ip B26F8d2w0VLVBJD8zghUyljlnVXk+Mse1xJEVITFTDvRgVup52j0H2KEoatpkwDS97+8 tARQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:subject:in-reply-to:cc:from:to:message-id :mime-version:content-transfer-encoding; bh=evErAM+3R8POckj6BoWNfzcQKezQH3REiwTi+k5Ke30=; b=Obd4MTriKGtThMv2N+X4C23EUt943NNCqeISKFmcsTmLWVME6bS84WY1R38mag0VXU JdfCBmJSTiYUXEmvyW81FLHPgW34kRD70sAxLlLcwFRN9FKBtiRzl2hBHAlz9v6xJSU9 LxI2Xv5ek67eijNLGhmKt/LbE/MEW6tXbIzL3MOe4+PwCegJVN2E5JkHC0R3ys5y5wHL wxaYPgtXVqOQcjjrte1xhHhEQnhM2tBkrLU6nUla/XD+o80oXfdMmgmmRrzK7nhronzR WJ1Jqc3yU5yokwW3Gf1vMCjNmKpxth5VZH9Rqn222bjE/WgS3MUfNL+Tzw8x7ALpvrJW WkRw== X-Gm-Message-State: AOAM531RNxsqiq4pX/XsSi8UnFzBaoCl/KlpzSVzzMwJB3Ky4KDKmIVX 6EXagNqXTNfyX5P+G9udbH/cPFObydAJpA== X-Received: by 2002:aa7:9607:0:b029:155:2b85:93f5 with SMTP id q7-20020aa796070000b02901552b8593f5mr3642353pfg.36.1604599621026; Thu, 05 Nov 2020 10:07:01 -0800 (PST) Received: from localhost (76-210-143-223.lightspeed.sntcca.sbcglobal.net. [76.210.143.223]) by smtp.gmail.com with ESMTPSA id b16sm3180544pju.16.2020.11.05.10.07.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 05 Nov 2020 10:07:00 -0800 (PST) Date: Thu, 05 Nov 2020 10:07:00 -0800 (PST) X-Google-Original-Date: Thu, 05 Nov 2020 09:33:55 PST (-0800) Subject: Re: [PATCH v4 0/5] Unify NUMA implementation between ARM64 & RISC-V In-Reply-To: <20201006001752.248564-1-atish.patra@wdc.com> CC: linux-kernel@vger.kernel.org, Atish Patra , Jonathan.Cameron@huawei.com, aou@eecs.berkeley.edu, akpm@linux-foundation.org, anshuman.khandual@arm.com, anup@brainfault.org, Arnd Bergmann , catalin.marinas@arm.com, david@redhat.com, greentime.hu@sifive.com, Greg KH , justin.he@arm.com, wangkefeng.wang@huawei.com, linux-arch@vger.kernel.org, linux-riscv@lists.infradead.org, rppt@kernel.org, nsaenzjulienne@suse.de, Paul Walmsley , rafael@kernel.org, steven.price@arm.com, will@kernel.org, zong.li@sifive.com, linux-arm-kernel@lists.infradead.org From: Palmer Dabbelt To: Atish Patra , Will Deacon , maz@kernel.org Message-ID: Mime-Version: 1.0 (MHng) Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, 05 Oct 2020 17:17:47 PDT (-0700), Atish Patra wrote: > This series attempts to move the ARM64 numa implementation to common > code so that RISC-V can leverage that as well instead of reimplementing > it again. > > RISC-V specific bits are based on initial work done by Greentime Hu [1] but > modified to reuse the common implementation to avoid duplication. > > [1] https://lkml.org/lkml/2020/1/10/233 > > This series has been tested on qemu with numa enabled for both RISC-V & ARM64. > It would be great if somebody can test it on numa capable ARM64 hardware platforms. > This patch series doesn't modify the maintainers list for the common code (arch_numa) > as I am not sure if somebody from ARM64 community or Greg should take up the > maintainership. Ganapatrao was the original author of the arm64 version. > I would be happy to update that in the next revision once it is decided. > > # numactl --hardware > available: 2 nodes (0-1) > node 0 cpus: 0 1 2 3 > node 0 size: 486 MB > node 0 free: 470 MB > node 1 cpus: 4 5 6 7 > node 1 size: 424 MB > node 1 free: 408 MB > node distances: > node 0 1 > 0: 10 20 > 1: 20 10 > # numactl -show > policy: default > preferred node: current > physcpubind: 0 1 2 3 4 5 6 7 > cpubind: 0 1 > nodebind: 0 1 > membind: 0 1 > > The patches are also available at > https://github.com/atishp04/linux/tree/5.10_numa_unified_v4 > > For RISC-V, the following qemu series is a pre-requisite(already available in upstream) > https://patchwork.kernel.org/project/qemu-devel/list/?series=303313 > > Testing: > RISC-V: > Tested in Qemu and 2 socket OmniXtend FPGA. > > ARM64: > 2 socket kunpeng920 (4 nodes around 250G a node) > Tested-by: Jonathan Cameron > > There may be some minor conflicts with Mike's cleanup series [2] depending on the > order in which these two series are being accepted. I can rebase on top his series > if required. > > [2] https://lkml.org/lkml/2020/8/18/754 > > Changes from v3->v4: > 1. Removed redundant duplicate header. > 2. Added Reviewed-by tags. > > Changes from v2->v3: > 1. Added Acked-by/Reviewed-by tags. > 2. Replaced asm/acpi.h with linux/acpi.h > 3. Defined arch_acpi_numa_init as static. > > Changes from v1->v2: > 1. Replaced ARM64 specific compile time protection with ACPI specific ones. > 2. Dropped common pcibus_to_node changes. Added required changes in RISC-V. > 3. Fixed few typos. > > Atish Patra (4): > numa: Move numa implementation to common code > arm64, numa: Change the numa init functions name to be generic > riscv: Separate memory init from paging init > riscv: Add numa support for riscv64 platform > > Greentime Hu (1): > riscv: Add support pte_protnone and pmd_protnone if > CONFIG_NUMA_BALANCING > > arch/arm64/Kconfig | 1 + > arch/arm64/include/asm/numa.h | 45 +---------------- > arch/arm64/kernel/acpi_numa.c | 13 ----- > arch/arm64/mm/Makefile | 1 - > arch/arm64/mm/init.c | 4 +- > arch/riscv/Kconfig | 31 +++++++++++- > arch/riscv/include/asm/mmzone.h | 13 +++++ > arch/riscv/include/asm/numa.h | 8 +++ > arch/riscv/include/asm/pci.h | 14 ++++++ > arch/riscv/include/asm/pgtable.h | 21 ++++++++ > arch/riscv/kernel/setup.c | 11 ++++- > arch/riscv/kernel/smpboot.c | 12 ++++- > arch/riscv/mm/init.c | 10 +++- > drivers/base/Kconfig | 6 +++ > drivers/base/Makefile | 1 + > .../mm/numa.c => drivers/base/arch_numa.c | 30 ++++++++++-- > include/asm-generic/numa.h | 49 +++++++++++++++++++ > 17 files changed, 199 insertions(+), 71 deletions(-) > create mode 100644 arch/riscv/include/asm/mmzone.h > create mode 100644 arch/riscv/include/asm/numa.h > rename arch/arm64/mm/numa.c => drivers/base/arch_numa.c (95%) > create mode 100644 include/asm-generic/numa.h Sorry it took me a while to get around to this, I had some work stuff to deal with and have managed to get buried in email. This all looks fine to me, but the way it's structured make it kind of hard to apply -- essentially I can't take the first two without at least some Acks from the arm64 folks, and it smells to me like it'd be better to have those go through the arm64 tree. The RISC-V stuff isn't that heavywight, but I'd like it to at least land in my for-next at some point as otherwise it'll be completely untested. arm64 guys: do you want to try and do some sort of shared base tag sort of thing for these, or do you want me to refactor this such that it adds the generic stuff before removing the arm64 stuff so we can decouble that way?