Received: by 2002:ac0:e350:0:0:0:0:0 with SMTP id g16csp472188imn; Fri, 29 Jul 2022 12:43:46 -0700 (PDT) X-Google-Smtp-Source: AGRyM1u4Df+OvYY96zagl2Le1FzMd/Le2SIQ+tzfM2ode4nYQy5G0+vVk4r9F51zvVSM7yulH6Xa X-Received: by 2002:a05:6402:17dc:b0:43d:a1e:67f4 with SMTP id s28-20020a05640217dc00b0043d0a1e67f4mr5069639edy.191.1659123826288; Fri, 29 Jul 2022 12:43:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1659123826; cv=none; d=google.com; s=arc-20160816; b=FYAmB3MNzNd6Jb81DeKWM+Y9iu8Pq8job9+H0p7qdyB1HUlHDknouTfLddcuNp0KQp nRzXRLvktnKnIy0V/vglqYBipf7Y83vgjSlX9uCcRQeQDq66P/Pa8N/4Oiyom2kBjszq bUsZSVr4FvcYtveP91N3vUk7hPSVUyAT0EuPK/yvvAKMRfBS/rKDMyYSlpkruCcCH+Di wVrZZ+wcoclzHGwdubo+pl4RzarT3ExlWKAEJwuoQEOlNcBv1s8nI5htSV7eC78Blgbi xQWD1iD7nHn7bYD80wwNc6nSvJDjI25TMPJWQBGY9z4XbGGO1Y4Cwd+wKIPPFYGT5GMe nPdQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from:dkim-signature:dkim-filter; bh=NkCVjM+S9iEg/gRamTSXlL2ZBMGnYHhG/cpmn7pzdsM=; b=X/NhfEU8rToNjwvnFw84RB/EK6KgTFpJGArPrMag7ppa2G2GZ/dlKEx25I1KlB0RLp sfchZQ/Q0bRyj9/WrzCXfWOIUFjEqot0rIxTDJyOIveFSD2yJEvhVdSHYXVMo14GFOZF d5fa91Rabd/c3N5QM99tt3fL4yeTtEhvgVwVh9GrBsyb+tEFBlHxfKHGuu0G1yehalUd wvCsuZebbmOrc4xZtBrdqHfJi0nF9lxVXzsj4W0+lPMxa5IZGlJRO9FF0yQPsSBdNhrJ DVIuYujHiBBrQrPtZRsdkFs2kCh3PpiYqcthRh386TXXDwAKdnhW3EEHSgjdCJvWHOfk 405Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@efficios.com header.s=default header.b=vHXjUpf6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=efficios.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id wt9-20020a170906ee8900b006f44d992450si108119ejb.11.2022.07.29.12.43.20; Fri, 29 Jul 2022 12:43:46 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@efficios.com header.s=default header.b=vHXjUpf6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=efficios.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238511AbiG2TCr (ORCPT + 99 others); Fri, 29 Jul 2022 15:02:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50872 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238407AbiG2TCo (ORCPT ); Fri, 29 Jul 2022 15:02:44 -0400 Received: from mail.efficios.com (mail.efficios.com [167.114.26.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E88C988741; Fri, 29 Jul 2022 12:02:42 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by mail.efficios.com (Postfix) with ESMTP id 384D1318E8D; Fri, 29 Jul 2022 15:02:42 -0400 (EDT) Received: from mail.efficios.com ([127.0.0.1]) by localhost (mail03.efficios.com [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id JCEWs45x8sPA; Fri, 29 Jul 2022 15:02:41 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mail.efficios.com (Postfix) with ESMTP id D2829318B4D; Fri, 29 Jul 2022 15:02:41 -0400 (EDT) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.efficios.com D2829318B4D DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=efficios.com; s=default; t=1659121361; bh=NkCVjM+S9iEg/gRamTSXlL2ZBMGnYHhG/cpmn7pzdsM=; h=From:To:Date:Message-Id; b=vHXjUpf6ynyLVX2tJKxWcMZlrEwd080p2Xx9F4MGq2ZpFmgW7EKpmVCrOFD0Ol4bQ i6AM80OogfzmsZ7H2i2syre9K6envqnIoTr/Ezb5nNlQLx5/DUi2piYO1Y+atjkehQ 9kw+frBN9PCw7W2iFr7LBBtzuBQmwfVOyUZehj5gyNZXwC4Atm961wzsRLNPXGYgil ihscdvLpYc/ei5t0ojStR4vvDxSq/6hovty8ftV1MQlSrVr+FtAF2dlO7JolTg+qUq 60w++DbclkJxCiFBqnovxx3PZPkA56Sw22YedJFw3ZLQnJhWjOuNoTLEg4yK7VikXT 86HZjXRUzlH0A== X-Virus-Scanned: amavisd-new at efficios.com Received: from mail.efficios.com ([127.0.0.1]) by localhost (mail03.efficios.com [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id oAahqyNA6sEZ; Fri, 29 Jul 2022 15:02:41 -0400 (EDT) Received: from localhost.localdomain (192-222-180-24.qc.cable.ebox.net [192.222.180.24]) by mail.efficios.com (Postfix) with ESMTPSA id 8BCCC3187FF; Fri, 29 Jul 2022 15:02:41 -0400 (EDT) From: Mathieu Desnoyers To: Peter Zijlstra Cc: linux-kernel@vger.kernel.org, Thomas Gleixner , "Paul E . McKenney" , Boqun Feng , "H . Peter Anvin" , Paul Turner , linux-api@vger.kernel.org, Christian Brauner , Florian Weimer , David.Laight@ACULAB.COM, carlos@redhat.com, Peter Oskolkov , Mathieu Desnoyers Subject: [PATCH v3 03/23] rseq: extend struct rseq with numa node id Date: Fri, 29 Jul 2022 15:02:05 -0400 Message-Id: <20220729190225.12726-4-mathieu.desnoyers@efficios.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20220729190225.12726-1-mathieu.desnoyers@efficios.com> References: <20220729190225.12726-1-mathieu.desnoyers@efficios.com> X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Adding the NUMA node id to struct rseq is a straightforward thing to do, and a good way to figure out if anything in the user-space ecosystem prevents extending struct rseq. This NUMA node id field allows memory allocators such as tcmalloc to take advantage of fast access to the current NUMA node id to perform NUMA-aware memory allocation. It can also be useful for implementing fast-paths for NUMA-aware user-space mutexes. It also allows implementing getcpu(2) purely in user-space. Signed-off-by: Mathieu Desnoyers --- include/trace/events/rseq.h | 4 +++- include/uapi/linux/rseq.h | 8 ++++++++ kernel/rseq.c | 19 +++++++++++++------ 3 files changed, 24 insertions(+), 7 deletions(-) diff --git a/include/trace/events/rseq.h b/include/trace/events/rseq.h index a04a64bc1a00..6bd442697354 100644 --- a/include/trace/events/rseq.h +++ b/include/trace/events/rseq.h @@ -16,13 +16,15 @@ TRACE_EVENT(rseq_update, TP_STRUCT__entry( __field(s32, cpu_id) + __field(s32, node_id) ), TP_fast_assign( __entry->cpu_id = raw_smp_processor_id(); + __entry->node_id = cpu_to_node(raw_smp_processor_id()); ), - TP_printk("cpu_id=%d", __entry->cpu_id) + TP_printk("cpu_id=%d node_id=%d", __entry->cpu_id, __entry->node_id) ); TRACE_EVENT(rseq_ip_fixup, diff --git a/include/uapi/linux/rseq.h b/include/uapi/linux/rseq.h index 05d3c4cdeb40..1cb90a435c5c 100644 --- a/include/uapi/linux/rseq.h +++ b/include/uapi/linux/rseq.h @@ -131,6 +131,14 @@ struct rseq { */ __u32 flags; + /* + * Restartable sequences node_id field. Updated by the kernel. Read by + * user-space with single-copy atomicity semantics. This field should + * only be read by the thread which registered this data structure. + * Aligned on 32-bit. Contains the current NUMA node ID. + */ + __u32 node_id; + /* * Flexible array member at end of structure, after last feature field. */ diff --git a/kernel/rseq.c b/kernel/rseq.c index 46dc5c2ce2b7..cb7d8a5afc82 100644 --- a/kernel/rseq.c +++ b/kernel/rseq.c @@ -84,15 +84,17 @@ * F1. */ -static int rseq_update_cpu_id(struct task_struct *t) +static int rseq_update_cpu_node_id(struct task_struct *t) { - u32 cpu_id = raw_smp_processor_id(); struct rseq __user *rseq = t->rseq; + u32 cpu_id = raw_smp_processor_id(); + u32 node_id = cpu_to_node(cpu_id); if (!user_write_access_begin(rseq, t->rseq_len)) goto efault; unsafe_put_user(cpu_id, &rseq->cpu_id_start, efault_end); unsafe_put_user(cpu_id, &rseq->cpu_id, efault_end); + unsafe_put_user(node_id, &rseq->node_id, efault_end); /* * Additional feature fields added after ORIG_RSEQ_SIZE * need to be conditionally updated only if @@ -108,9 +110,9 @@ static int rseq_update_cpu_id(struct task_struct *t) return -EFAULT; } -static int rseq_reset_rseq_cpu_id(struct task_struct *t) +static int rseq_reset_rseq_cpu_node_id(struct task_struct *t) { - u32 cpu_id_start = 0, cpu_id = RSEQ_CPU_ID_UNINITIALIZED; + u32 cpu_id_start = 0, cpu_id = RSEQ_CPU_ID_UNINITIALIZED, node_id = 0; /* * Reset cpu_id_start to its initial state (0). @@ -124,6 +126,11 @@ static int rseq_reset_rseq_cpu_id(struct task_struct *t) */ if (put_user(cpu_id, &t->rseq->cpu_id)) return -EFAULT; + /* + * Reset node_id to its initial state (0). + */ + if (put_user(node_id, &t->rseq->node_id)) + return -EFAULT; /* * Additional feature fields added after ORIG_RSEQ_SIZE * need to be conditionally reset only if @@ -306,7 +313,7 @@ void __rseq_handle_notify_resume(struct ksignal *ksig, struct pt_regs *regs) if (unlikely(ret < 0)) goto error; } - if (unlikely(rseq_update_cpu_id(t))) + if (unlikely(rseq_update_cpu_node_id(t))) goto error; return; @@ -353,7 +360,7 @@ SYSCALL_DEFINE4(rseq, struct rseq __user *, rseq, u32, rseq_len, return -EINVAL; if (current->rseq_sig != sig) return -EPERM; - ret = rseq_reset_rseq_cpu_id(current); + ret = rseq_reset_rseq_cpu_node_id(current); if (ret) return ret; current->rseq = NULL; -- 2.17.1