Received: by 2002:a05:6a10:1287:0:0:0:0 with SMTP id d7csp5949571pxv; Thu, 29 Jul 2021 02:33:37 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxn+w2oV4WaRe08fOYOE+tmT2aYNQ+UA8nPYNrRH6KEoS/P7RDy45DGIDmBl62CdxBEIC0a X-Received: by 2002:a17:906:b0d4:: with SMTP id bk20mr3605621ejb.535.1627551217350; Thu, 29 Jul 2021 02:33:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1627551217; cv=none; d=google.com; s=arc-20160816; b=cQlSSO8qZjvCxQhrBF41XZCc4uoiMkJtUDw3C/eHHCmeqPTyUKDTgnjPUdOIg6ek8O BzF1lwwVwhETCZ9ZgtkOoqeiAZ9gfjmfar2dKhFMhSUzeEB6Rw9I27OW3e9kVH5Elc4Z ++9ihb/IAKWU35eIKuTDG+dxYoK5tVE9RLSbP8BG1CtroWvGkbgZzMiI7S5VcxcQVlgR wFYTyVN/ci97Z3SfRu1jePLbQxeCEBN8id1M7tup2XxgtOC6PdIaS/OhlPmZCp3PSodc o5jvk/YiqrrfXyUXveSuetKsFJAvjm9sYeB27VP6ZwW7aXJi/HRrMQqMgtgOe8CVAUOX OfHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from; bh=VoCJaaw5AKIzRCe7IVVVJM4PGWYaNS6WcQggoDKjY2k=; b=rfxKEUEeic89u/1+JQ6HwNCYLKbFWetJabuHk1F/IIPotUu7BxP83wxYiHDEl14VEA 4f6DnJ+3YF/6aU32xSQjHFEKQR90H5bqDmLEcBXGmMikr/4DLwuyJ2ef4DUoKVoGoxzl GTcDwaKkOj0evh/PnJ6KTeyFqsKqhp8nIDDwtMHifUdlYU1O81vPpQMEsZw4ZYBPCat4 AeExA4AHuEDnS+jr1H0EwCFYCRYquTPtfXqNKuSgWJNJupc6I/PSiR6xu6ScPuB4v3+L e/YZcA9OsD9YJvrAVjjcIGsuwmV4i5e40mrjiA4uwIHRYsieQFpxFwcHn0MoqpVRsHTX PYaw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id gv23si2485779ejb.534.2021.07.29.02.33.13; Thu, 29 Jul 2021 02:33:37 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235488AbhG2Jag (ORCPT + 99 others); Thu, 29 Jul 2021 05:30:36 -0400 Received: from mail.loongson.cn ([114.242.206.163]:40776 "EHLO loongson.cn" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S235141AbhG2Jaf (ORCPT ); Thu, 29 Jul 2021 05:30:35 -0400 Received: from localhost.localdomain (unknown [117.81.121.223]) by mail.loongson.cn (Coremail) with SMTP id AQAAf9Bx4OIhdQJhEdklAA--.25639S2; Thu, 29 Jul 2021 17:30:14 +0800 (CST) From: Rui Wang To: Peter Zijlstra , Ingo Molnar , Will Deacon , Arnd Bergmann Cc: Waiman Long , Boqun Feng , Guo Ren , linux-arch@vger.kernel.org, linux-kernel@vger.kernel.org, Rui Wang , Rui Wang , Xuefeng Li , Huacai Chen , Jiaxun Yang , Huacai Chen , kernel test robot Subject: [RFC PATCH v3] locking/atomic: Implement atomic{,64,_long}_{fetch_,}{andnot_or}{,_relaxed,_acquire,_release}() Date: Thu, 29 Jul 2021 17:30:03 +0800 Message-Id: <20210729093003.146166-1-wangrui@loongson.cn> X-Mailer: git-send-email 2.32.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-CM-TRANSID: AQAAf9Bx4OIhdQJhEdklAA--.25639S2 X-Coremail-Antispam: 1UD129KBjvAXoWfAFy8AF1kXr48GrWDAw1DWrg_yoW8ur1Uto Z3KFW5tr4Iqry7W395KF1Iqr4DXF1jqayrGry0vrsYvF9rZanYqwnrGr1jyr1xX34kAr4k Wr4ftw13trWqqwn5n29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7v73VFW2AGmfu7bjvjm3 AaLaJ3UjIYCTnIWjp_UUUY67AC8VAFwI0_Gr0_Xr1l1xkIjI8I6I8E6xAIw20EY4v20xva j40_Wr0E3s1l1IIY67AEw4v_Jr0_Jr4l8cAvFVAK0II2c7xJM28CjxkF64kEwVA0rcxSw2 x7M28EF7xvwVC0I7IYx2IY67AKxVW5JVW7JwA2z4x0Y4vE2Ix0cI8IcVCY1x0267AKxVW8 Jr0_Cr1UM28EF7xvwVC2z280aVAFwI0_GcCE3s1l84ACjcxK6I8E87Iv6xkF7I0E14v26r xl6s0DM2AIxVAIcxkEcVAq07x20xvEncxIr21l5I8CrVACY4xI64kE6c02F40Ex7xfMcIj 6xIIjxv20xvE14v26r1j6r18McIj6I8E87Iv67AKxVWUJVW8JwAm72CE4IkC6x0Yz7v_Jr 0_Gr1lF7xvr2IYc2Ij64vIr41lF7I21c0EjII2zVCS5cI20VAGYxC7M4IIrI8v6xkF7I0E 8cxan2IY04v7MxkIecxEwVAFwVW8uwCF04k20xvY0x0EwIxGrwCFx2IqxVCFs4IE7xkEbV WUJVW8JwC20s026c02F40E14v26r1j6r18MI8I3I0E7480Y4vE14v26r106r1rMI8E67AF 67kF1VAFwI0_GFv_WrylIxkGc2Ij64vIr41lIxAIcVC0I7IYx2IY67AKxVWUJVWUCwCI42 IY6xIIjxv20xvEc7CjxVAFwI0_Gr0_Cr1lIxAIcVCF04k26cxKx2IYs7xG6rW3Jr0E3s1l IxAIcVC2z280aVAFwI0_Jr0_Gr1lIxAIcVC2z280aVCY1x0267AKxVW8JVW8JrUvcSsGvf C2KfnxnUUI43ZEXa7VUjMmh5UUUUU== X-CM-SenderInfo: pzdqw2txl6z05rqj20fqof0/ Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This patch introduce a new atomic primitive andnot_or: * atomic_andnot_or * atomic_fetch_andnot_or * atomic_fetch_andnot_or_relaxed * atomic_fetch_andnot_or_acquire * atomic_fetch_andnot_or_release * atomic64_andnot_or * atomic64_fetch_andnot_or * atomic64_fetch_andnot_or_relaxed * atomic64_fetch_andnot_or_acquire * atomic64_fetch_andnot_or_release * atomic_long_andnot_or * atomic_long_fetch_andnot_or * atomic_long_fetch_andnot_or_relaxed * atomic_long_fetch_andnot_or_acquire * atomic_long_fetch_andnot_or_release Reported-by: kernel test robot Signed-off-by: Rui Wang --- include/asm-generic/atomic-instrumented.h | 72 +++++- include/asm-generic/atomic-long.h | 62 ++++- include/linux/atomic-arch-fallback.h | 262 +++++++++++++++++++++- lib/atomic64_test.c | 92 ++++---- scripts/atomic/atomics.tbl | 1 + scripts/atomic/fallbacks/andnot_or | 25 +++ 6 files changed, 471 insertions(+), 43 deletions(-) create mode 100755 scripts/atomic/fallbacks/andnot_or diff --git a/include/asm-generic/atomic-instrumented.h b/include/asm-generic/atomic-instrumented.h index bc45af52c93b..8f5efade88b7 100644 --- a/include/asm-generic/atomic-instrumented.h +++ b/include/asm-generic/atomic-instrumented.h @@ -599,6 +599,41 @@ atomic_dec_if_positive(atomic_t *v) return arch_atomic_dec_if_positive(v); } +static __always_inline void +atomic_andnot_or(int m, int o, atomic_t *v) +{ + instrument_atomic_read_write(v, sizeof(*v)); + arch_atomic_andnot_or(m, o, v); +} + +static __always_inline int +atomic_fetch_andnot_or(int m, int o, atomic_t *v) +{ + instrument_atomic_read_write(v, sizeof(*v)); + return arch_atomic_fetch_andnot_or(m, o, v); +} + +static __always_inline int +atomic_fetch_andnot_or_acquire(int m, int o, atomic_t *v) +{ + instrument_atomic_read_write(v, sizeof(*v)); + return arch_atomic_fetch_andnot_or_acquire(m, o, v); +} + +static __always_inline int +atomic_fetch_andnot_or_release(int m, int o, atomic_t *v) +{ + instrument_atomic_read_write(v, sizeof(*v)); + return arch_atomic_fetch_andnot_or_release(m, o, v); +} + +static __always_inline int +atomic_fetch_andnot_or_relaxed(int m, int o, atomic_t *v) +{ + instrument_atomic_read_write(v, sizeof(*v)); + return arch_atomic_fetch_andnot_or_relaxed(m, o, v); +} + static __always_inline s64 atomic64_read(const atomic64_t *v) { @@ -1177,6 +1212,41 @@ atomic64_dec_if_positive(atomic64_t *v) return arch_atomic64_dec_if_positive(v); } +static __always_inline void +atomic64_andnot_or(s64 m, s64 o, atomic64_t *v) +{ + instrument_atomic_read_write(v, sizeof(*v)); + arch_atomic64_andnot_or(m, o, v); +} + +static __always_inline s64 +atomic64_fetch_andnot_or(s64 m, s64 o, atomic64_t *v) +{ + instrument_atomic_read_write(v, sizeof(*v)); + return arch_atomic64_fetch_andnot_or(m, o, v); +} + +static __always_inline s64 +atomic64_fetch_andnot_or_acquire(s64 m, s64 o, atomic64_t *v) +{ + instrument_atomic_read_write(v, sizeof(*v)); + return arch_atomic64_fetch_andnot_or_acquire(m, o, v); +} + +static __always_inline s64 +atomic64_fetch_andnot_or_release(s64 m, s64 o, atomic64_t *v) +{ + instrument_atomic_read_write(v, sizeof(*v)); + return arch_atomic64_fetch_andnot_or_release(m, o, v); +} + +static __always_inline s64 +atomic64_fetch_andnot_or_relaxed(s64 m, s64 o, atomic64_t *v) +{ + instrument_atomic_read_write(v, sizeof(*v)); + return arch_atomic64_fetch_andnot_or_relaxed(m, o, v); +} + #define xchg(ptr, ...) \ ({ \ typeof(ptr) __ai_ptr = (ptr); \ @@ -1334,4 +1404,4 @@ atomic64_dec_if_positive(atomic64_t *v) }) #endif /* _ASM_GENERIC_ATOMIC_INSTRUMENTED_H */ -// 1d7c3a25aca5c7fb031c307be4c3d24c7b48fcd5 +// 9c9792d0dcd1fb3de8eeda1225ebbd0d811fb941 diff --git a/include/asm-generic/atomic-long.h b/include/asm-generic/atomic-long.h index 073cf40f431b..0c61626b42d2 100644 --- a/include/asm-generic/atomic-long.h +++ b/include/asm-generic/atomic-long.h @@ -515,6 +515,36 @@ atomic_long_dec_if_positive(atomic_long_t *v) return atomic64_dec_if_positive(v); } +static __always_inline void +atomic_long_andnot_or(long m, long o, atomic_long_t *v) +{ + atomic64_andnot_or(m, o, v); +} + +static __always_inline long +atomic_long_fetch_andnot_or(long m, long o, atomic_long_t *v) +{ + return atomic64_fetch_andnot_or(m, o, v); +} + +static __always_inline long +atomic_long_fetch_andnot_or_acquire(long m, long o, atomic_long_t *v) +{ + return atomic64_fetch_andnot_or_acquire(m, o, v); +} + +static __always_inline long +atomic_long_fetch_andnot_or_release(long m, long o, atomic_long_t *v) +{ + return atomic64_fetch_andnot_or_release(m, o, v); +} + +static __always_inline long +atomic_long_fetch_andnot_or_relaxed(long m, long o, atomic_long_t *v) +{ + return atomic64_fetch_andnot_or_relaxed(m, o, v); +} + #else /* CONFIG_64BIT */ static __always_inline long @@ -1009,6 +1039,36 @@ atomic_long_dec_if_positive(atomic_long_t *v) return atomic_dec_if_positive(v); } +static __always_inline void +atomic_long_andnot_or(long m, long o, atomic_long_t *v) +{ + atomic_andnot_or(m, o, v); +} + +static __always_inline long +atomic_long_fetch_andnot_or(long m, long o, atomic_long_t *v) +{ + return atomic_fetch_andnot_or(m, o, v); +} + +static __always_inline long +atomic_long_fetch_andnot_or_acquire(long m, long o, atomic_long_t *v) +{ + return atomic_fetch_andnot_or_acquire(m, o, v); +} + +static __always_inline long +atomic_long_fetch_andnot_or_release(long m, long o, atomic_long_t *v) +{ + return atomic_fetch_andnot_or_release(m, o, v); +} + +static __always_inline long +atomic_long_fetch_andnot_or_relaxed(long m, long o, atomic_long_t *v) +{ + return atomic_fetch_andnot_or_relaxed(m, o, v); +} + #endif /* CONFIG_64BIT */ #endif /* _ASM_GENERIC_ATOMIC_LONG_H */ -// a624200981f552b2c6be4f32fe44da8289f30d87 +// 3ab842342b36b655b902481be793ba7a04c5a88d diff --git a/include/linux/atomic-arch-fallback.h b/include/linux/atomic-arch-fallback.h index a3dba31df01e..93a68face24f 100644 --- a/include/linux/atomic-arch-fallback.h +++ b/include/linux/atomic-arch-fallback.h @@ -1250,6 +1250,136 @@ arch_atomic_dec_if_positive(atomic_t *v) #define arch_atomic_dec_if_positive arch_atomic_dec_if_positive #endif +#ifndef arch_atomic_andnot_or +static __always_inline void +arch_atomic_andnot_or(int m, int o, atomic_t *v) +{ + (void)({ + int N, O = arch_atomic_read(v); + do { + N = O; + N &= ~m; + N |= o; + } while (!arch_atomic_try_cmpxchg_relaxed(v, &O, N)); + O; + }); +} +#define arch_atomic_andnot_or arch_atomic_andnot_or +#endif + +#ifndef arch_atomic_fetch_andnot_or_relaxed +#ifdef arch_atomic_fetch_andnot_or +#define arch_atomic_fetch_andnot_or_acquire arch_atomic_fetch_andnot_or +#define arch_atomic_fetch_andnot_or_release arch_atomic_fetch_andnot_or +#define arch_atomic_fetch_andnot_or_relaxed arch_atomic_fetch_andnot_or +#endif /* arch_atomic_fetch_andnot_or */ + +#ifndef arch_atomic_fetch_andnot_or +static __always_inline int +arch_atomic_fetch_andnot_or(int m, int o, atomic_t *v) +{ + return ({ + int N, O = arch_atomic_read(v); + do { + N = O; + N &= ~m; + N |= o; + } while (!arch_atomic_try_cmpxchg(v, &O, N)); + O; + }); +} +#define arch_atomic_fetch_andnot_or arch_atomic_fetch_andnot_or +#endif + +#ifndef arch_atomic_fetch_andnot_or_acquire +static __always_inline int +arch_atomic_fetch_andnot_or_acquire(int m, int o, atomic_t *v) +{ + return ({ + int N, O = arch_atomic_read(v); + do { + N = O; + N &= ~m; + N |= o; + } while (!arch_atomic_try_cmpxchg_acquire(v, &O, N)); + O; + }); +} +#define arch_atomic_fetch_andnot_or_acquire arch_atomic_fetch_andnot_or_acquire +#endif + +#ifndef arch_atomic_fetch_andnot_or_release +static __always_inline int +arch_atomic_fetch_andnot_or_release(int m, int o, atomic_t *v) +{ + return ({ + int N, O = arch_atomic_read(v); + do { + N = O; + N &= ~m; + N |= o; + } while (!arch_atomic_try_cmpxchg_release(v, &O, N)); + O; + }); +} +#define arch_atomic_fetch_andnot_or_release arch_atomic_fetch_andnot_or_release +#endif + +#ifndef arch_atomic_fetch_andnot_or_relaxed +static __always_inline int +arch_atomic_fetch_andnot_or_relaxed(int m, int o, atomic_t *v) +{ + return ({ + int N, O = arch_atomic_read(v); + do { + N = O; + N &= ~m; + N |= o; + } while (!arch_atomic_try_cmpxchg_relaxed(v, &O, N)); + O; + }); +} +#define arch_atomic_fetch_andnot_or_relaxed arch_atomic_fetch_andnot_or_relaxed +#endif + +#else /* arch_atomic_fetch_andnot_or_relaxed */ + +#ifndef arch_atomic_fetch_andnot_or_acquire +static __always_inline int +arch_atomic_fetch_andnot_or_acquire(int m, int o, atomic_t *v) +{ + int ret = arch_atomic_fetch_andnot_or_relaxed(m, o, v); + __atomic_acquire_fence(); + return ret; +} +#define arch_atomic_fetch_andnot_or_acquire arch_atomic_fetch_andnot_or_acquire +#endif + +#ifndef arch_atomic_fetch_andnot_or_release +static __always_inline int +arch_atomic_fetch_andnot_or_release(int m, int o, atomic_t *v) +{ + __atomic_release_fence(); + return arch_atomic_fetch_andnot_or_relaxed(m, o, v); +} +#define arch_atomic_fetch_andnot_or_release arch_atomic_fetch_andnot_or_release +#endif + +#ifndef arch_atomic_fetch_andnot_or +static __always_inline int +arch_atomic_fetch_andnot_or(int m, int o, atomic_t *v) +{ + int ret; + __atomic_pre_full_fence(); + ret = arch_atomic_fetch_andnot_or_relaxed(m, o, v); + __atomic_post_full_fence(); + return ret; +} +#define arch_atomic_fetch_andnot_or arch_atomic_fetch_andnot_or +#endif + +#endif /* arch_atomic_fetch_andnot_or_relaxed */ + #ifdef CONFIG_GENERIC_ATOMIC64 #include #endif @@ -2357,5 +2487,135 @@ arch_atomic64_dec_if_positive(atomic64_t *v) #define arch_atomic64_dec_if_positive arch_atomic64_dec_if_positive #endif +#ifndef arch_atomic64_andnot_or +static __always_inline void +arch_atomic64_andnot_or(s64 m, s64 o, atomic64_t *v) +{ + (void)({ + s64 N, O = arch_atomic64_read(v); + do { + N = O; + N &= ~m; + N |= o; + } while (!arch_atomic64_try_cmpxchg_relaxed(v, &O, N)); + O; + }); +} +#define arch_atomic64_andnot_or arch_atomic64_andnot_or +#endif + +#ifndef arch_atomic64_fetch_andnot_or_relaxed +#ifdef arch_atomic64_fetch_andnot_or +#define arch_atomic64_fetch_andnot_or_acquire arch_atomic64_fetch_andnot_or +#define arch_atomic64_fetch_andnot_or_release arch_atomic64_fetch_andnot_or +#define arch_atomic64_fetch_andnot_or_relaxed arch_atomic64_fetch_andnot_or +#endif /* arch_atomic64_fetch_andnot_or */ + +#ifndef arch_atomic64_fetch_andnot_or +static __always_inline s64 +arch_atomic64_fetch_andnot_or(s64 m, s64 o, atomic64_t *v) +{ + return ({ + s64 N, O = arch_atomic64_read(v); + do { + N = O; + N &= ~m; + N |= o; + } while (!arch_atomic64_try_cmpxchg(v, &O, N)); + O; + }); +} +#define arch_atomic64_fetch_andnot_or arch_atomic64_fetch_andnot_or +#endif + +#ifndef arch_atomic64_fetch_andnot_or_acquire +static __always_inline s64 +arch_atomic64_fetch_andnot_or_acquire(s64 m, s64 o, atomic64_t *v) +{ + return ({ + s64 N, O = arch_atomic64_read(v); + do { + N = O; + N &= ~m; + N |= o; + } while (!arch_atomic64_try_cmpxchg_acquire(v, &O, N)); + O; + }); +} +#define arch_atomic64_fetch_andnot_or_acquire arch_atomic64_fetch_andnot_or_acquire +#endif + +#ifndef arch_atomic64_fetch_andnot_or_release +static __always_inline s64 +arch_atomic64_fetch_andnot_or_release(s64 m, s64 o, atomic64_t *v) +{ + return ({ + s64 N, O = arch_atomic64_read(v); + do { + N = O; + N &= ~m; + N |= o; + } while (!arch_atomic64_try_cmpxchg_release(v, &O, N)); + O; + }); +} +#define arch_atomic64_fetch_andnot_or_release arch_atomic64_fetch_andnot_or_release +#endif + +#ifndef arch_atomic64_fetch_andnot_or_relaxed +static __always_inline s64 +arch_atomic64_fetch_andnot_or_relaxed(s64 m, s64 o, atomic64_t *v) +{ + return ({ + s64 N, O = arch_atomic64_read(v); + do { + N = O; + N &= ~m; + N |= o; + } while (!arch_atomic64_try_cmpxchg_relaxed(v, &O, N)); + O; + }); +} +#define arch_atomic64_fetch_andnot_or_relaxed arch_atomic64_fetch_andnot_or_relaxed +#endif + +#else /* arch_atomic64_fetch_andnot_or_relaxed */ + +#ifndef arch_atomic64_fetch_andnot_or_acquire +static __always_inline s64 +arch_atomic64_fetch_andnot_or_acquire(s64 m, s64 o, atomic64_t *v) +{ + s64 ret = arch_atomic64_fetch_andnot_or_relaxed(m, o, v); + __atomic_acquire_fence(); + return ret; +} +#define arch_atomic64_fetch_andnot_or_acquire arch_atomic64_fetch_andnot_or_acquire +#endif + +#ifndef arch_atomic64_fetch_andnot_or_release +static __always_inline s64 +arch_atomic64_fetch_andnot_or_release(s64 m, s64 o, atomic64_t *v) +{ + __atomic_release_fence(); + return arch_atomic64_fetch_andnot_or_relaxed(m, o, v); +} +#define arch_atomic64_fetch_andnot_or_release arch_atomic64_fetch_andnot_or_release +#endif + +#ifndef arch_atomic64_fetch_andnot_or +static __always_inline s64 +arch_atomic64_fetch_andnot_or(s64 m, s64 o, atomic64_t *v) +{ + s64 ret; + __atomic_pre_full_fence(); + ret = arch_atomic64_fetch_andnot_or_relaxed(m, o, v); + __atomic_post_full_fence(); + return ret; +} +#define arch_atomic64_fetch_andnot_or arch_atomic64_fetch_andnot_or +#endif + +#endif /* arch_atomic64_fetch_andnot_or_relaxed */ + #endif /* _LINUX_ATOMIC_FALLBACK_H */ -// cca554917d7ea73d5e3e7397dd70c484cad9b2c4 +// 74f7ec8a3bee44a12678be13ca294c61c4a77941 diff --git a/lib/atomic64_test.c b/lib/atomic64_test.c index d9d170238165..fedc83118a29 100644 --- a/lib/atomic64_test.c +++ b/lib/atomic64_test.c @@ -17,12 +17,18 @@ #include /* for boot_cpu_has below */ #endif -#define TEST(bit, op, c_op, val) \ +#define COP(c_op1, c_op2, val1, val2...) \ +do { \ + (void)(r c_op1 val1); \ + (void)(r c_op2 val2); \ +} while (0) + +#define TEST(bit, op, c_op1, c_op2, args...) \ do { \ atomic##bit##_set(&v, v0); \ r = v0; \ - atomic##bit##_##op(val, &v); \ - r c_op val; \ + atomic##bit##_##op(args, &v); \ + COP(c_op1, c_op2, args); \ WARN(atomic##bit##_read(&v) != r, "%Lx != %Lx\n", \ (unsigned long long)atomic##bit##_read(&v), \ (unsigned long long)r); \ @@ -50,12 +56,12 @@ do { \ BUG_ON(atomic##bit##_read(&v) != r); \ } while (0) -#define TEST_FETCH(bit, op, c_op, val) \ +#define TEST_FETCH(bit, op, c_op1, c_op2, args...) \ do { \ atomic##bit##_set(&v, v0); \ r = v0; \ - r c_op val; \ - BUG_ON(atomic##bit##_##op(val, &v) != v0); \ + COP(c_op1, c_op2, args); \ + BUG_ON(atomic##bit##_##op(args, &v) != v0); \ BUG_ON(atomic##bit##_read(&v) != r); \ } while (0) @@ -64,9 +70,9 @@ do { \ FAMILY_TEST(TEST_RETURN, bit, op, c_op, val); \ } while (0) -#define FETCH_FAMILY_TEST(bit, op, c_op, val) \ +#define FETCH_FAMILY_TEST(bit, op, args...) \ do { \ - FAMILY_TEST(TEST_FETCH, bit, op, c_op, val); \ + FAMILY_TEST(TEST_FETCH, bit, op, args); \ } while (0) #define TEST_ARGS(bit, op, init, ret, expect, args...) \ @@ -105,35 +111,38 @@ static __init void test_atomic(void) { int v0 = 0xaaa31337; int v1 = 0xdeadbeef; + int mask = 0x0000ffff; int onestwos = 0x11112222; int one = 1; atomic_t v; int r; - TEST(, add, +=, onestwos); - TEST(, add, +=, -one); - TEST(, sub, -=, onestwos); - TEST(, sub, -=, -one); - TEST(, or, |=, v1); - TEST(, and, &=, v1); - TEST(, xor, ^=, v1); - TEST(, andnot, &= ~, v1); + TEST(, add, +=, , onestwos); + TEST(, add, +=, , -one); + TEST(, sub, -=, , onestwos); + TEST(, sub, -=, , -one); + TEST(, or, |=, , v1); + TEST(, and, &=, , v1); + TEST(, xor, ^=, , v1); + TEST(, andnot, &= ~, , v1); + TEST(, andnot_or, &= ~, |=, mask, one); RETURN_FAMILY_TEST(, add_return, +=, onestwos); RETURN_FAMILY_TEST(, add_return, +=, -one); RETURN_FAMILY_TEST(, sub_return, -=, onestwos); RETURN_FAMILY_TEST(, sub_return, -=, -one); - FETCH_FAMILY_TEST(, fetch_add, +=, onestwos); - FETCH_FAMILY_TEST(, fetch_add, +=, -one); - FETCH_FAMILY_TEST(, fetch_sub, -=, onestwos); - FETCH_FAMILY_TEST(, fetch_sub, -=, -one); + FETCH_FAMILY_TEST(, fetch_add, +=, , onestwos); + FETCH_FAMILY_TEST(, fetch_add, +=, , -one); + FETCH_FAMILY_TEST(, fetch_sub, -=, , onestwos); + FETCH_FAMILY_TEST(, fetch_sub, -=, , -one); - FETCH_FAMILY_TEST(, fetch_or, |=, v1); - FETCH_FAMILY_TEST(, fetch_and, &=, v1); - FETCH_FAMILY_TEST(, fetch_andnot, &= ~, v1); - FETCH_FAMILY_TEST(, fetch_xor, ^=, v1); + FETCH_FAMILY_TEST(, fetch_or, |=, , v1); + FETCH_FAMILY_TEST(, fetch_and, &=, , v1); + FETCH_FAMILY_TEST(, fetch_andnot, &= ~, , v1); + FETCH_FAMILY_TEST(, fetch_xor, ^=, , v1); + FETCH_FAMILY_TEST(, fetch_andnot_or, &= ~, |=, mask, one); INC_RETURN_FAMILY_TEST(, v0); DEC_RETURN_FAMILY_TEST(, v0); @@ -150,6 +159,7 @@ static __init void test_atomic64(void) long long v1 = 0xdeadbeefdeafcafeLL; long long v2 = 0xfaceabadf00df001LL; long long v3 = 0x8000000000000000LL; + long long mask = 0x00000000ffffffffLL; long long onestwos = 0x1111111122222222LL; long long one = 1LL; int r_int; @@ -163,29 +173,31 @@ static __init void test_atomic64(void) BUG_ON(v.counter != r); BUG_ON(atomic64_read(&v) != r); - TEST(64, add, +=, onestwos); - TEST(64, add, +=, -one); - TEST(64, sub, -=, onestwos); - TEST(64, sub, -=, -one); - TEST(64, or, |=, v1); - TEST(64, and, &=, v1); - TEST(64, xor, ^=, v1); - TEST(64, andnot, &= ~, v1); + TEST(64, add, +=, , onestwos); + TEST(64, add, +=, , -one); + TEST(64, sub, -=, , onestwos); + TEST(64, sub, -=, , -one); + TEST(64, or, |=, , v1); + TEST(64, and, &=, , v1); + TEST(64, xor, ^=, , v1); + TEST(64, andnot, &= ~, , v1); + TEST(64, andnot_or, &= ~, |=, mask, one); RETURN_FAMILY_TEST(64, add_return, +=, onestwos); RETURN_FAMILY_TEST(64, add_return, +=, -one); RETURN_FAMILY_TEST(64, sub_return, -=, onestwos); RETURN_FAMILY_TEST(64, sub_return, -=, -one); - FETCH_FAMILY_TEST(64, fetch_add, +=, onestwos); - FETCH_FAMILY_TEST(64, fetch_add, +=, -one); - FETCH_FAMILY_TEST(64, fetch_sub, -=, onestwos); - FETCH_FAMILY_TEST(64, fetch_sub, -=, -one); + FETCH_FAMILY_TEST(64, fetch_add, +=, , onestwos); + FETCH_FAMILY_TEST(64, fetch_add, +=, , -one); + FETCH_FAMILY_TEST(64, fetch_sub, -=, , onestwos); + FETCH_FAMILY_TEST(64, fetch_sub, -=, , -one); - FETCH_FAMILY_TEST(64, fetch_or, |=, v1); - FETCH_FAMILY_TEST(64, fetch_and, &=, v1); - FETCH_FAMILY_TEST(64, fetch_andnot, &= ~, v1); - FETCH_FAMILY_TEST(64, fetch_xor, ^=, v1); + FETCH_FAMILY_TEST(64, fetch_or, |=, , v1); + FETCH_FAMILY_TEST(64, fetch_and, &=, , v1); + FETCH_FAMILY_TEST(64, fetch_andnot, &= ~, , v1); + FETCH_FAMILY_TEST(64, fetch_xor, ^=, , v1); + FETCH_FAMILY_TEST(64, fetch_andnot_or, &= ~, |=, mask, one); INIT(v0); atomic64_inc(&v); diff --git a/scripts/atomic/atomics.tbl b/scripts/atomic/atomics.tbl index fbee2f6190d9..db6fe1dfcdb4 100755 --- a/scripts/atomic/atomics.tbl +++ b/scripts/atomic/atomics.tbl @@ -39,3 +39,4 @@ inc_not_zero b v inc_unless_negative b v dec_unless_positive b v dec_if_positive i v +andnot_or vF i:m i:o v diff --git a/scripts/atomic/fallbacks/andnot_or b/scripts/atomic/fallbacks/andnot_or new file mode 100755 index 000000000000..a835045208de --- /dev/null +++ b/scripts/atomic/fallbacks/andnot_or @@ -0,0 +1,25 @@ +local try_order=${order} + +# +# non-value returning atomics are implicity relaxed +# +if [ -z "${retstmt}" ]; then + try_order="_relaxed" + retstmt="(void)" +fi + +cat <