Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Libo Chen <libo.chen@oracle.com>
To:     gregkh@linuxfoundation.org, masahiroy@kernel.org,
        tglx@linutronix.de, peterz@infradead.org, mingo@kernel.org,
        vbabka@suse.cz, akpm@linux-foundation.org
Cc:     linux-kernel@vger.kernel.org, linux-kbuild@vger.kernel.org,
        linux-arm-kernel@lists.infradead.org, linux-arch@vger.kernel.org
Subject: [PATCH RESEND 0/1] lib/Kconfig: remove DEBUG_PER_CPU_MAPS dependency for CPUMASK_OFFSTACK
Date:   Tue, 12 Apr 2022 16:15:07 -0700
Message-Id: <20220412231508.32629-1-libo.chen@oracle.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?mfDV4OGmyxHwiQ7fAa206QuoI5EO62SB0QUJ2xRZVPdIb1WSfz2IjbkgswZR?=
 =?us-ascii?Q?sProak+vIOC9sMIW2/AcpLw6x6E+4vA/mv5yo5Yn85d6/Cvwg9xtScYLFac/?=
 =?us-ascii?Q?OfiIUG9zDfjJpstCEfrh9xIlK8hrkHLZvoL6b+4xJcKf7HV9mvl8uBQWDcEd?=
 =?us-ascii?Q?TM46gjGx+zYj0kLM6fNQ8I11xbgEytJm/468mGnZ9neHTqcMpJtLfmbFjk4n?=
 =?us-ascii?Q?sOOqMoXw+cxLf6kKOQ4xerlMnNIZiPHiV0k4QXaUHUwPDFKADhrzJrpAbQEi?=
 =?us-ascii?Q?f/uxBwlpxVXJ/JM7pdTo/Ktspg4ZcqYvFtqYaz9xmUaumm+6BNe2sfyNSCxx?=
 =?us-ascii?Q?s0c2fJENluUDBiTqLmsL44QvCb/ZZWwLJvFiiGU4qbglLyMe1hln+eK/09dz?=
 =?us-ascii?Q?cRZmavz3GDCxomJM9rqfJ2UPZ0bopHrGx5ek2u2qj7LReEaUVwfckvyaQr47?=
 =?us-ascii?Q?ugdg07nvpAocelcw7zBWMCQo5JajkRjfw91bxt7xQsfkp4pfm7WTNYCYgPJr?=
 =?us-ascii?Q?oIGmNOO2aX7EwU3+8otm4MJxC3GnR6ryfycaT0YIdtTIhM9ADFkD6aou93oR?=
 =?us-ascii?Q?SJYGncx3YLPS8kl4m5JmKzoZvRnbrVl/oOb4Z360M6k8ZT924cr3Oxf/A2K4?=
 =?us-ascii?Q?xQQMo2GJBTi13D/K25sH9BEMqVO2BNDN62U1ncs00/kDoaS6RA+NoNYGR/Ml?=
 =?us-ascii?Q?l60aIpv2CVG3thUlsAYDUxNNpa2EMaZS3Cq+mT7+9XWQf80usRhfYDPtfaQV?=
 =?us-ascii?Q?vO1jZ/Qr1O3tL4sVPGn1X+r2OozlJgxVgE5XkTskK0bfiwKuv5HWrNBUvUyy?=
 =?us-ascii?Q?aM4cHMNNUA80BScpdtYVx0ubnAbCrlc2ZeUknH9Zv+iycHLHrW6l+NFEYzhY?=
 =?us-ascii?Q?MN0W93NJa5NCmskQaZGpqDCX7WMLRHf5wuaIW31LCke+btOdg4eAxTZi1yBp?=
 =?us-ascii?Q?gF0szrfucBWoLL6rrmqZaXphN1ZcyPoVLxoeayiPUlY9DYHNgwAfyPIWXgMH?=
 =?us-ascii?Q?1Xx+a8qN2ek0XHmsRdiRk/yzxdjSWVJ/JXO8SWNJRtfe9J+Hfpyjr5kp0e4n?=
 =?us-ascii?Q?+YmbWwgqR7YKWJBCpEWpCPEUYS4SOnWQGYo+Kpj0AxJaY4ZPU6B9ZesHEF4g?=
 =?us-ascii?Q?61gbyjj6x3LDjMbnvlOevWr2DhgxEnZKGXWxvifIV1mfdGMD8W8Cz9uUota6?=
 =?us-ascii?Q?61CMdQNQbekng21xIQL8ecdiigGDpT8h3JwyAgIoeMNH08cEvAf68dbMiGhO?=
 =?us-ascii?Q?vK4HP6WEmzV00ZgMPoKnCtoe1+f2ezbFB2k7Q9YFZTAoU3Lyygqu0OICv+BF?=
 =?us-ascii?Q?vQE/JEFBQen1VKIJLmP075S8pZXunXe0KKpt9uahiTh9wlAYnZrzYCaTBbs5?=
 =?us-ascii?Q?wPwmbu1esJ46q94ZeXFI2mAW+SDPc985A0GhUi3JemWZpZv+Fjoabnc8Ymfp?=
 =?us-ascii?Q?vAKoA0RkZQfUWp4O6laPG+dGGwVGHJxvfENixkYge7wGyi4GYdSWd2XdWhIO?=
 =?us-ascii?Q?nQP0lDdZYnIUPG/q6ENWZP/FUrE6+d0HpLJef/Xg0iRvz9CStHK8V0Y31i2i?=
 =?us-ascii?Q?jrfQsS5rxwOi5tp4+H41HrW+uRkA/kUlbSudl+lL68Q0MrF8TJEpj8vr0oxL?=
 =?us-ascii?Q?asLRLzU6jgqItKHFaqc6ubofIm8WpMzIXgqGFSIom1zjcwCBKmG8TbeloIYs?=
 =?us-ascii?Q?Xs3AazXnKykvBOojKq44Sp8VXTtHrISZzxzw3UC3jAQ+aOHqlVUzO5S0Buz0?=
 =?us-ascii?Q?9EQMdJwRJA=3D=3D?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 31cee91e-dd32-4d37-f3bd-08da1cda4e7d
X-MS-Exchange-CrossTenant-AuthSource: SA2PR10MB4569.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Apr 2022 23:15:17.3702
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: jtygYuE4TFjrJJe3rYW71oTvkF9Ki8OK89VFWS9IrKGmRIJWZKzOOhIxukCgRK31TM8dLyrWzVlokFxeUOmoNQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR10MB3588
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486,18.0.858
 definitions=2022-04-12_06:2022-04-11,2022-04-12 signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=981 malwarescore=0
 mlxscore=0 phishscore=0 suspectscore=0 spamscore=0 adultscore=0
 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2202240000 definitions=main-2204120103
X-Proofpoint-GUID: jVMKXL1QEvXyvFpkytaZo9TCVTDejuGr
X-Proofpoint-ORIG-GUID: jVMKXL1QEvXyvFpkytaZo9TCVTDejuGr
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

We encountered an issue related to NR_CPUMASK_BITS on ARM64. It turned
out CPUMASK_OFFSTACK was not enabled and NR_CPUS is set to 4096 in our
aarch64 config, so each cpumask operation is unnecessarily expensive
esp. on small VMs. There are decent number of cpumask operations in the
scheduler. Many of them are on the hot paths. The cumulative effect is
quite siginificant. With CONFIG_CPUMASK_OFFSTACKT set, we saw ~10% gain on
ApacheBench and a few percentage on redis in a 32-core ARM64 VM with 5.16
kernel.
One may argue why not just set NR_CPUS to smaller values, the thing is,
for example, we support VMs with flexible shapes. The customer can
specify the number of CPUs ranging from 1 to 160. You cannot just prepare
160 kernel images for each release with one for each shape. And kernel
with NR_CPUS=160 doesn't perform well on a 1-CPU VM. It's important that
we can set CPUMASK_OFFSTACK=y so that the kernel can dynamically allocate
cpumasks based on the actual number of CPUs in the system.

The problem I am trying to address in this patch is currently
CPUMASK_OFFSTACK depends on DEBUG_PER_CPU_MAPS except for x86 and it
doesn't even show up in kconfig menu as well as kconfig
without DEBUG_PER_CPU_MAPS=y. We should remove such outdated,
unnecessary dependency.

Of course, I am open to other ideas. And people who are more familar with
this matter can shed a light on why this dependency has been kept for so
long. My goal here is to determine if DEBUG_PER_CPU_MAPS is absoultely
necessary for CPUMASK_OFFSTACK.

RESEND:
- Correct the subject line of cover letter
- Send to more people, esp. gkh for more attention since no one seems to
  be responsible for this part of code

Libo Chen (1):
  lib/Kconfig: remove DEBUG_PER_CPU_MAPS dependency for CPUMASK_OFFSTACK

 lib/Kconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

-- 
2.27.0