Received: by 2002:a25:4158:0:0:0:0:0 with SMTP id o85csp6375153yba; Thu, 11 Apr 2019 18:53:45 -0700 (PDT) X-Google-Smtp-Source: APXvYqx6+D+qGv2zN+Ix66DqswiSSXT/evxdPMvVwp5ogG+LyGcPnVtSDsKUJ0ingjydPIXRL+pf X-Received: by 2002:a63:ff18:: with SMTP id k24mr49482629pgi.140.1555034025892; Thu, 11 Apr 2019 18:53:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1555034025; cv=none; d=google.com; s=arc-20160816; b=PmirEq1tAFTzJiA1jOgJ1Fd2d9/+htAjdK/fa9IDe6zAmMxKm/mGXiiNF5hKIc77EU xiMk4XvVtdpElDqJUo5Y49YPNDKcyOWADkC21PdIZoFkH+Nwj229xDjpKbV587TyQkb7 Pdfjm/UwecC6pYuzXKPnkuR2uvhUYb3iK3UUpjwO+oF7C9GuFcFXSyr9VqcrQHMJZh1X wyxD+EFQZHJScUiqdLZtetb+SesEfSPBw3XTp8hTBw+X8AvBVaH0cRBgDy5YICopHkPD wpaK2xK8zlY2vn9bfISBK390Qq2tz5bDxsKj/wHo0Yb8G+blGBtEeWxIsqkJiRoM9098 zzAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:cc:subject:date:to :from; bh=gvKhdquoVQQRjEBqR/76/zkyeG8+BgD416IKZUcT+b4=; b=KkWs8KLGtshnJkFFWHNvf0ptkwsQS/pBBiOG+aQcSEIv6LgX54Qs7gCLzpKpCERcQN uoUaI0Z0FMzTB+MHHfAaOmghBlxiM9Rs3e3i+wOwefGMKtdRGOQVlO4ZsabKBMBEDgDH LKEHhZ5Q/dgLzK5ooDp3JtqDY/rJFe0qI+xyIDIRQfx0x9U56nDT0x7C99AwezRy5xnS MN+5XoxJfa7pGoM/6/XxWXTArJao1xmeh34gPZoWUt32qoB+QBWS0QdszLz7EFBx4zoo HDaIL/snRJiw0m3/IksLpFbi8u56D9MCS6iYCqStxKYTOXWOM6LHE6H8PuumwRqqjpKj 8/Nw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id z2si34765057pgu.437.2019.04.11.18.53.30; Thu, 11 Apr 2019 18:53:45 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726847AbfDLBwn (ORCPT + 99 others); Thu, 11 Apr 2019 21:52:43 -0400 Received: from mx2.suse.de ([195.135.220.15]:48896 "EHLO mx1.suse.de" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726661AbfDLBwm (ORCPT ); Thu, 11 Apr 2019 21:52:42 -0400 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id D5DFAAD0A; Fri, 12 Apr 2019 01:52:39 +0000 (UTC) From: NeilBrown To: Thomas Graf , Herbert Xu , David Miller Date: Fri, 12 Apr 2019 11:52:08 +1000 Subject: [PATCH 2/5] rhashtable: reorder some inline functions and macros. Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Guenter Roeck Message-ID: <155503392797.17793.15780367123758287135.stgit@noble.brown> In-Reply-To: <155503371949.17793.8266195008003399968.stgit@noble.brown> References: <155503371949.17793.8266195008003399968.stgit@noble.brown> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This patch only moves some code around, it doesn't change the code at all. A subsequent patch will benefit from this as it needs to add calls to functions which are now defined before the call-site, but weren't before. Signed-off-by: NeilBrown --- include/linux/rhashtable.h | 142 ++++++++++++++++++++++---------------------- 1 file changed, 71 insertions(+), 71 deletions(-) diff --git a/include/linux/rhashtable.h b/include/linux/rhashtable.h index 2711cbf01b64..c504cd820736 100644 --- a/include/linux/rhashtable.h +++ b/include/linux/rhashtable.h @@ -87,77 +87,6 @@ struct bucket_table { struct rhash_lock_head __rcu *buckets[] ____cacheline_aligned_in_smp; }; -/* - * We lock a bucket by setting BIT(1) in the pointer - this is always - * zero in real pointers and in the nulls marker. - * bit_spin_locks do not handle contention well, but the whole point - * of the hashtable design is to achieve minimum per-bucket contention. - * A nested hash table might not have a bucket pointer. In that case - * we cannot get a lock. For remove and replace the bucket cannot be - * interesting and doesn't need locking. - * For insert we allocate the bucket if this is the last bucket_table, - * and then take the lock. - * Sometimes we unlock a bucket by writing a new pointer there. In that - * case we don't need to unlock, but we do need to reset state such as - * local_bh. For that we have rht_assign_unlock(). As rcu_assign_pointer() - * provides the same release semantics that bit_spin_unlock() provides, - * this is safe. - */ - -static inline void rht_lock(struct bucket_table *tbl, - struct rhash_lock_head **bkt) -{ - local_bh_disable(); - bit_spin_lock(1, (unsigned long *)bkt); - lock_map_acquire(&tbl->dep_map); -} - -static inline void rht_lock_nested(struct bucket_table *tbl, - struct rhash_lock_head **bucket, - unsigned int subclass) -{ - local_bh_disable(); - bit_spin_lock(1, (unsigned long *)bucket); - lock_acquire_exclusive(&tbl->dep_map, subclass, 0, NULL, _THIS_IP_); -} - -static inline void rht_unlock(struct bucket_table *tbl, - struct rhash_lock_head **bkt) -{ - lock_map_release(&tbl->dep_map); - bit_spin_unlock(1, (unsigned long *)bkt); - local_bh_enable(); -} - -static inline void rht_assign_unlock(struct bucket_table *tbl, - struct rhash_lock_head __rcu **bkt, - struct rhash_head *obj) -{ - struct rhash_head __rcu **p = (struct rhash_head __rcu **)bkt; - - lock_map_release(&tbl->dep_map); - rcu_assign_pointer(*p, obj); - preempt_enable(); - __release(bitlock); - local_bh_enable(); -} - -/* - * If 'p' is a bucket head and might be locked: - * rht_ptr() returns the address without the lock bit. - * rht_ptr_locked() returns the address WITH the lock bit. - */ -static inline struct rhash_head __rcu *rht_ptr(const struct rhash_lock_head *p) -{ - return (void *)(((unsigned long)p) & ~BIT(1)); -} - -static inline struct rhash_lock_head __rcu *rht_ptr_locked(const - struct rhash_head *p) -{ - return (void *)(((unsigned long)p) | BIT(1)); -} - /* * NULLS_MARKER() expects a hash value with the low * bits mostly likely to be significant, and it discards @@ -372,6 +301,77 @@ static inline struct rhash_lock_head __rcu **rht_bucket_insert( &tbl->buckets[hash]; } +/* + * We lock a bucket by setting BIT(1) in the pointer - this is always + * zero in real pointers and in the nulls marker. + * bit_spin_locks do not handle contention well, but the whole point + * of the hashtable design is to achieve minimum per-bucket contention. + * A nested hash table might not have a bucket pointer. In that case + * we cannot get a lock. For remove and replace the bucket cannot be + * interesting and doesn't need locking. + * For insert we allocate the bucket if this is the last bucket_table, + * and then take the lock. + * Sometimes we unlock a bucket by writing a new pointer there. In that + * case we don't need to unlock, but we do need to reset state such as + * local_bh. For that we have rht_assign_unlock(). As rcu_assign_pointer() + * provides the same release semantics that bit_spin_unlock() provides, + * this is safe. + */ + +static inline void rht_lock(struct bucket_table *tbl, + struct rhash_lock_head **bkt) +{ + local_bh_disable(); + bit_spin_lock(1, (unsigned long *)bkt); + lock_map_acquire(&tbl->dep_map); +} + +static inline void rht_lock_nested(struct bucket_table *tbl, + struct rhash_lock_head **bucket, + unsigned int subclass) +{ + local_bh_disable(); + bit_spin_lock(1, (unsigned long *)bucket); + lock_acquire_exclusive(&tbl->dep_map, subclass, 0, NULL, _THIS_IP_); +} + +static inline void rht_unlock(struct bucket_table *tbl, + struct rhash_lock_head **bkt) +{ + lock_map_release(&tbl->dep_map); + bit_spin_unlock(1, (unsigned long *)bkt); + local_bh_enable(); +} + +/* + * If 'p' is a bucket head and might be locked: + * rht_ptr() returns the address without the lock bit. + * rht_ptr_locked() returns the address WITH the lock bit. + */ +static inline struct rhash_head __rcu *rht_ptr(const struct rhash_lock_head *p) +{ + return (void *)(((unsigned long)p) & ~BIT(1)); +} + +static inline struct rhash_lock_head __rcu *rht_ptr_locked(const + struct rhash_head *p) +{ + return (void *)(((unsigned long)p) | BIT(1)); +} + +static inline void rht_assign_unlock(struct bucket_table *tbl, + struct rhash_lock_head __rcu **bkt, + struct rhash_head *obj) +{ + struct rhash_head __rcu **p = (struct rhash_head __rcu **)bkt; + + lock_map_release(&tbl->dep_map); + rcu_assign_pointer(*p, obj); + preempt_enable(); + __release(bitlock); + local_bh_enable(); +} + /** * rht_for_each_from - iterate over hash chain from given head * @pos: the &struct rhash_head to use as a loop cursor.