Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Content-Type: text/plain;
        charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.600.7\))
Subject: Re: [PATCH v2 0/9]  Introduce SMT level and add PowerPC support
From:   Sachin Sant <sachinp@linux.ibm.com>
In-Reply-To: <20230628100558.43482-1-ldufour@linux.ibm.com>
Date:   Wed, 28 Jun 2023 21:03:28 +0530
Cc:     linuxppc-dev <linuxppc-dev@lists.ozlabs.org>,
        linux-arch@vger.kernel.org, dave.hansen@linux.intel.com,
        open list <linux-kernel@vger.kernel.org>,
        Ingo Molnar <mingo@redhat.com>, bp@alien8.de,
        npiggin@gmail.com, tglx@linutronix.de
Content-Transfer-Encoding: quoted-printable
Message-Id: <88E208A6-F4E0-4DE9-8752-C9652B978BC6@linux.ibm.com>
References: <20230628100558.43482-1-ldufour@linux.ibm.com>
To:     Laurent Dufour <ldufour@linux.ibm.com>
Precedence: bulk


> On 28-Jun-2023, at 3:35 PM, Laurent Dufour <ldufour@linux.ibm.com> =
wrote:
>=20
> I'm taking over the series Michael sent previously [1] which is =
smartly
> reviewing the initial series I sent [2].  This series is addressing =
the
> comments sent by Thomas and me on the Michael's one.
>=20
> Here is a short introduction to the issue this series is addressing:
>=20
> When a new CPU is added, the kernel is activating all its threads. =
This
> leads to weird, but functional, result when adding CPU on a SMT 4 =
system
> for instance.
>=20
> Here the newly added CPU 1 has 8 threads while the other one has 4 =
threads
> active (system has been booted with the 'smt-enabled=3D4' kernel =
option):
>=20
> ltcden3-lp12:~ # ppc64_cpu --info
> Core   0:    0*    1*    2*    3*    4     5     6     7
> Core   1:    8*    9*   10*   11*   12*   13*   14*   15*
>=20
> This mixed SMT level may confused end users and/or some applications.
>=20

Thanks for the patches Laurent.

Is the SMT level retained even when dynamically changing SMT values?
I am observing difference in behaviour with and without smt-enabled
kernel command line option.

When smt-enabled=3D option is specified SMT level is retained across=20
cpu core remove and add.

Without this option but changing SMT level during runtime using
ppc64_cpu =E2=80=94smt=3D<level>, the SMT level is not retained after
cpu core add.

[root@ltcden8-lp8 ~]# ppc64_cpu =E2=80=94smt=3D4
[root@ltcden8-lp8 ~]# ppc64_cpu --info
Core   0:    0*    1*    2*    3*    4     5     6     7  =20
Core   1:    8*    9*   10*   11*   12    13    14    15  =20
Core   2:   16*   17*   18*   19*   20    21    22    23  =20
Core   3:   24*   25*   26*   27*   28    29    30    31  =20

Remove a core, SMT level is retained.

[root@ltcden8-lp8 ~]# ppc64_cpu --info
Core   0:    0*    1*    2*    3*    4     5     6     7 =20
Core   1:    8*    9*   10*   11*   12    13    14    15 =20
Core   2:   16*   17*   18*   19*   20    21    22    23 =20
[root@ltcden8-lp8 ~]# =20

Add 3 cores, SMT level is not retained.

[  496.600648] Fallback order for Node 1: 1 0=20
[  496.600655] Built 1 zonelists, mobility grouping on.  Total pages: =
1228159
[  496.600676] Policy zone: Normal
[  496.661173] WARNING: workqueue cpumask: online intersect > possible =
intersect
[  499.530646] Fallback order for Node 3: 3 0=20
[  499.530655] Built 2 zonelists, mobility grouping on.  Total pages: =
1228159
[  499.530675] Policy zone: Normal

ppc64_cpu --info
Core   0:    0*    1*    2*    3*    4     5     6     7 =20
Core   1:    8*    9*   10*   11*   12    13    14    15 =20
Core   2:   16*   17*   18*   19*   20    21    22    23 =20
Core   3:   24*   25*   26*   27*   28*   29*   30*   31*=20
Core   4:   32*   33*   34*   35*   36*   37*   38*   39*=20
Core   5:   40*   41*   42*   43*   44*   45*   46*   47*
 [root@ltcden8-lp8 ~]#=20

- Sachin=