Received: by 2002:a05:6358:9144:b0:117:f937:c515 with SMTP id r4csp1145199rwr; Thu, 27 Apr 2023 13:03:12 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6xPfxsFOxhtMvXjJroLK0qv+24UMfDvYY0UBMBXsteLYckOB0UIae4Px2U3FiTHgoS4hA9 X-Received: by 2002:a17:90b:198:b0:249:6086:a301 with SMTP id t24-20020a17090b019800b002496086a301mr2989776pjs.27.1682625791818; Thu, 27 Apr 2023 13:03:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1682625791; cv=none; d=google.com; s=arc-20160816; b=f3Xb+q3yZRsJlQQOTePZZIVoWpbHRcJS16DB4i1zMXNVZNUBk8kZKbOqYKXGlFQu0k DPJH6hJy6Oy21RP4cYMc3AOJFeHnbqZmRJyDTi1e019XMibczwAuYHPDRJIB2N6MInac eDjWNS8c7O5XHBveV7RAugnjENO1D6yY1RO0pCRhL5ku6RmfPS3Ym7JgAeS2SE3C+mzV vN+4m1C4sxZN1Z0YXgYZgkAL0VfmIEjcGgq0QwFANQvTgUlVrmJo28L3PLins3F+HpHJ 5u+YcRIjNdoGAVkt6S3d0nn/GaV8JcyfAKGDy9PzRRUbhdyMtO9yADb5k/PMeZfup1h9 Pvjw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:in-reply-to:content-disposition:mime-version :message-id:subject:cc:to:from:date:sender:dkim-signature; bh=989hUtGEXHxQonnOCeEb5WSkYFpgBCEOi68ZJXZiflU=; b=HmbDCFidwU/JEnAtHWCVatPiEcMgF+mH8h43imuxnR8lNkUac2U847RwJsITCP4eSO 0aY4Pp04J5ylDpKvYQhAibh/ZVyVWMcJqnBSZd1ur2J3J/BUJJgMahdguX+TwUGpHoFm sVAzzSt4pXYyPMUGx+N4LjNErMuuMfV4SktLTayo2pYodXiW123MawD+hk31YAObiue5 SVghdGygKd+egiZ30VRow4zOw3dcuA+0LtqQ66NVWzJWJBq24EY8wOwF0d6LPXuayyWH UjB/rMzh/mY90iEDNwBW9nmLLcxU6J6rSWPqz0mtDOtpmU4pnsyYreGLMZpAzV43FqF/ O1sg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=KHMtiqtI; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i13-20020a17090acf8d00b00246cad7fab9si351599pju.180.2023.04.27.13.02.55; Thu, 27 Apr 2023 13:03:11 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=KHMtiqtI; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344059AbjD0T6J (ORCPT + 99 others); Thu, 27 Apr 2023 15:58:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42342 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229750AbjD0T6I (ORCPT ); Thu, 27 Apr 2023 15:58:08 -0400 Received: from mail-ed1-x530.google.com (mail-ed1-x530.google.com [IPv6:2a00:1450:4864:20::530]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 97EE4102 for ; Thu, 27 Apr 2023 12:58:06 -0700 (PDT) Received: by mail-ed1-x530.google.com with SMTP id 4fb4d7f45d1cf-50a14564d17so10236702a12.0 for ; Thu, 27 Apr 2023 12:58:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682625485; x=1685217485; h=in-reply-to:content-disposition:mime-version:message-id:subject:cc :to:from:date:sender:from:to:cc:subject:date:message-id:reply-to; bh=989hUtGEXHxQonnOCeEb5WSkYFpgBCEOi68ZJXZiflU=; b=KHMtiqtIEQyaAaUY74wM37hOWzOYQAA7bak/Wd/KNMo4RGd8gMPh6DKQ5IrTPjFvQs uGtO2NtoV2S76tm1tCFyroXTHoF0LFidchjwiM+53DwT930pW2dpyWrQfvoVqe7Lr+m6 kmT5tAqfXurlPk9sAonWL5KimRYQShROQ387NCJYnHnt53pz5MAMX+Hq+bWtdjEE5mpp P6k/yRp90/cFCfcQhv/AJ8nRZ6O5LLfnfrn13RkuIHmxmwC9t8Pl5DnuhbyApvyR+Qqq CGsoEbTL47jmPEY+xHR18zbX0hBjvUM3Oz9V3xU7Jbi2GaJjvi77r7Mclpm3yYYvHVby R2SA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682625485; x=1685217485; h=in-reply-to:content-disposition:mime-version:message-id:subject:cc :to:from:date:sender:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=989hUtGEXHxQonnOCeEb5WSkYFpgBCEOi68ZJXZiflU=; b=ZVnizyh6eaKIxq7ZqKJvl6uIPwoYhZu0p1XM+Q++QdlwubvVvLLwnqQJHzSfT+hKWN 0AybBYo2lk4JjRrVjHLPRndHeiof9UpsBWZstmC/IyKJ4Nds8nYIENw+k6AT/GvZz/ko ST00yV4y3kFunN9RifKtJ8Ry8nsjaUpa/YDo2bict/oOPk4BJGwPGhIUECmlL808uLX6 kAqQYaF+8LYjWwoRY8XWZcGeLYDMYx9Vf8wnCSLVioUvWXann7lx5Fn7oFQyzKXGPZ1X RBElPzRVoqqE/W3/QIeTmHA9uvXQB7rur3LRF9R55K7a02T5TkR9fjaHkiQMgFvsy0X8 RZPA== X-Gm-Message-State: AC+VfDxnUWBgHcXqzzSMaM/DEaAoabIRdQVGDZe4RVQXZnp1IdCfBg7u HrRf/mmLvwWtxapcP2iqty0= X-Received: by 2002:a05:6402:42c8:b0:506:71bd:3931 with SMTP id i8-20020a05640242c800b0050671bd3931mr7911478edc.2.1682625484975; Thu, 27 Apr 2023 12:58:04 -0700 (PDT) Received: from gmail.com ([31.46.243.138]) by smtp.gmail.com with ESMTPSA id p19-20020a05640210d300b005067d089aafsm8151110edu.11.2023.04.27.12.58.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 27 Apr 2023 12:58:04 -0700 (PDT) Sender: Ingo Molnar Date: Thu, 27 Apr 2023 21:58:00 +0200 From: Ingo Molnar To: Linus Torvalds Cc: linux-kernel@vger.kernel.org, Peter Zijlstra , Will Deacon , Waiman Long , Boqun Feng , Thomas Gleixner , Andrew Morton Subject: [GIT PULL] locking changes for v6.4 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=1.6 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, FSL_HELO_FAKE,HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Level: * X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Linus, Please pull the latest locking/core git tree from: git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git locking-core-2023-04-27 # HEAD: 93eff603d6a2bc1895eeb7063dbd0661bb760b74 locking/atomic: Correct (cmp)xhcg instrumentation Locking changes in v6.4: - Introduce local{,64}_try_cmpxchg() - a slightly more optimal primitive, which will be used in perf events ring-buffer code. - Add more atomic_add_negative() variants, used by rcuref: - Add rcuref - A scalable reference count implementation for RCU managed objects - Add non-atomic __xchg() variant, use it in a couple of places - Misc cleanups and fixes. NOTE: the internal merge commit of locking/rcuref [1afa833d607b6] merging two commits from a WIP branch has no explanation. I noticed this too late, and we can re-do this tree if it's an issue. Thanks, Ingo ------------------> Andrzej Hajda (5): arch: rename all internal names __xchg to __arch_xchg linux/include: add non-atomic version of xchg arch/*/uprobes: simplify arch_uretprobe_hijack_return_addr llist: simplify __llist_del_all drm/i915/gt: use __xchg instead of internal helper Mark Rutland (1): locking/atomic: Correct (cmp)xhcg instrumentation Sebastian Andrzej Siewior (1): locking/rwbase: Mitigate indefinite writer starvation. Thomas Gleixner (2): atomics: Provide atomic_add_negative() variants atomics: Provide rcuref - scalable reference counting Uros Bizjak (4): locking/atomic: Add generic try_cmpxchg{,64}_local support locking/generic: Wire up local{,64}_try_cmpxchg locking/arch: Wire up local_try_cmpxchg locking/x86: Define arch_try_cmpxchg_local arch/alpha/include/asm/cmpxchg.h | 10 +- arch/alpha/include/asm/local.h | 12 +- arch/arc/include/asm/cmpxchg.h | 4 +- arch/arm/include/asm/cmpxchg.h | 7 +- arch/arm/probes/uprobes/core.c | 8 +- arch/arm64/include/asm/cmpxchg.h | 7 +- arch/arm64/kernel/probes/uprobes.c | 9 +- arch/csky/kernel/probes/uprobes.c | 9 +- arch/hexagon/include/asm/cmpxchg.h | 10 +- arch/ia64/include/asm/cmpxchg.h | 2 +- arch/ia64/include/uapi/asm/cmpxchg.h | 4 +- arch/loongarch/include/asm/cmpxchg.h | 4 +- arch/loongarch/include/asm/local.h | 13 +- arch/m68k/include/asm/cmpxchg.h | 6 +- arch/mips/include/asm/cmpxchg.h | 4 +- arch/mips/include/asm/local.h | 13 +- arch/mips/kernel/uprobes.c | 10 +- arch/openrisc/include/asm/cmpxchg.h | 10 +- arch/parisc/include/asm/cmpxchg.h | 4 +- arch/powerpc/include/asm/cmpxchg.h | 4 +- arch/powerpc/include/asm/local.h | 11 + arch/powerpc/kernel/uprobes.c | 10 +- arch/riscv/include/asm/atomic.h | 2 +- arch/riscv/include/asm/cmpxchg.h | 4 +- arch/riscv/kernel/probes/uprobes.c | 9 +- arch/s390/include/asm/cmpxchg.h | 8 +- arch/s390/kernel/uprobes.c | 7 +- arch/sh/include/asm/cmpxchg.h | 4 +- arch/sparc/include/asm/cmpxchg_32.h | 4 +- arch/sparc/include/asm/cmpxchg_64.h | 6 +- arch/sparc/kernel/uprobes.c | 7 +- arch/x86/include/asm/cmpxchg.h | 6 + arch/x86/include/asm/local.h | 13 +- arch/xtensa/include/asm/cmpxchg.h | 4 +- drivers/gpu/drm/i915/gt/intel_engine_cs.c | 2 +- drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c | 4 +- .../gpu/drm/i915/gt/intel_execlists_submission.c | 4 +- drivers/gpu/drm/i915/gt/intel_ggtt.c | 4 +- drivers/gpu/drm/i915/gt/intel_gsc.c | 2 +- drivers/gpu/drm/i915/gt/intel_gt.c | 4 +- drivers/gpu/drm/i915/gt/intel_gt_pm.c | 2 +- drivers/gpu/drm/i915/gt/intel_lrc.c | 6 +- drivers/gpu/drm/i915/gt/intel_migrate.c | 2 +- drivers/gpu/drm/i915/gt/intel_rc6.c | 2 +- drivers/gpu/drm/i915/gt/intel_rps.c | 2 +- drivers/gpu/drm/i915/gt/selftest_context.c | 2 +- drivers/gpu/drm/i915/gt/selftest_ring_submission.c | 2 +- drivers/gpu/drm/i915/gt/selftest_timeline.c | 2 +- drivers/gpu/drm/i915/gt/uc/intel_uc.c | 2 +- drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c | 2 +- drivers/gpu/drm/i915/i915_utils.h | 1 + include/asm-generic/local.h | 1 + include/asm-generic/local64.h | 12 +- include/linux/atomic/atomic-arch-fallback.h | 230 ++++++++++++++++- include/linux/atomic/atomic-instrumented.h | 152 ++++++++--- include/linux/atomic/atomic-long.h | 38 ++- include/linux/llist.h | 6 +- include/linux/non-atomic/xchg.h | 19 ++ include/linux/rcuref.h | 155 ++++++++++++ include/linux/types.h | 6 + kernel/locking/rwbase_rt.c | 9 - lib/Makefile | 2 +- lib/rcuref.c | 281 +++++++++++++++++++++ scripts/atomic/atomics.tbl | 2 +- scripts/atomic/fallbacks/add_negative | 11 +- scripts/atomic/gen-atomic-fallback.sh | 4 + scripts/atomic/gen-atomic-instrumented.sh | 8 +- 67 files changed, 1019 insertions(+), 207 deletions(-) create mode 100644 include/linux/non-atomic/xchg.h create mode 100644 include/linux/rcuref.h create mode 100644 lib/rcuref.c