Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Valentin Schneider <vschneid@redhat.com>
To:     Hao Jia <jiahao.os@bytedance.com>, mingo@redhat.com,
        peterz@infradead.org, juri.lelli@redhat.com,
        vincent.guittot@linaro.org, dietmar.eggemann@arm.com,
        rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de,
        bristot@redhat.com
Cc:     linux-kernel@vger.kernel.org, Hao Jia <jiahao.os@bytedance.com>
Subject: Re: [PATCH] sched/topology: Optimized copy default topology in
 sched_init_numa()
In-Reply-To: <20220627105349.80715-1-jiahao.os@bytedance.com>
References: <20220627105349.80715-1-jiahao.os@bytedance.com>
Date:   Mon, 04 Jul 2022 15:39:27 +0100
Message-ID: <xhsmh35fhgcww.mognet@vschneid.remote.csb>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On 27/06/22 18:53, Hao Jia wrote:
> The size of struct sched_domain_topology_level is 64 bytes.
> For NUMA platforms, almost all are multi-core (enable CONFIG_SCHED_MC),
> That is to say, the default_topology array has at least 128 bytes that
> need to be copied in sched_init_numa(). For most x86 platforms,
> CONFIG_SCHED_SMT will be enabled, so more copies will be required.
>
> And memcpy() will be optimized under different architectures.
> Fortunately, for platforms with CONFIG_NUMA enabled,
> these optimizations are likely to be used.
> So, let's use memcpy to copy default topology in sched_init_numa().
>
> Tests are done in an Intel Xeon(R) Platinum 8260 CPU@2.40GHz machine
> with 2 NUMA nodes each of which has 24 cores with SMT2 enabled, so 96
> CPUs in total.
>
> Use RDTSC to count time-consuming, and based on 5.19-rc4.
>
> Enable CONFIG_SCHED_SMT && CONFIG_SCHED_CLUSTER && CONFIG_SCHED_MC,
> So the default_topology array has 256 bytes that need to be copied
> in sched_init_numa().
>                      5.19-rc4   5.19-rc4 with patch
> average tsc ticks    516.57      85.33   =EF=BC=88-83.48%*=EF=BC=89
>
> Enable CONFIG_SCHED_MC, So the default_topology array has
> 128 bytes that need to be copied in sched_init_numa().
>                      5.19-rc4   5.19-rc4 with patch
> average tsc ticks    65.71       55.00   =EF=BC=88-16.30%*=EF=BC=89
>
> Signed-off-by: Hao Jia <jiahao.os@bytedance.com>

It's not a very hot path but I guess this lets you shave off a bit of boot
time... While you're at it, you could add an early

  if (nr_node_ids =3D=3D 1)
          return;

since !NUMA systems still go through sched_init_numa() if they have a
kernel with CONFIG_NUMA (which should be most of them nowdays) and IIRC
they end up with an unused NODE topology level.

Regardless:

Reviewed-by: Valentin Schneider <vschneid@redhat.com>