Received: by 2002:a05:7412:d1aa:b0:fc:a2b0:25d7 with SMTP id ba42csp739886rdb; Mon, 29 Jan 2024 17:44:54 -0800 (PST) X-Google-Smtp-Source: AGHT+IFCuWbXWjsojkmhpXHKEncnxQtREWqh+Qnb43wnPEXnw5H9TCPsurW/jcJcyqiNMaP+vxJW X-Received: by 2002:a05:620a:4788:b0:783:3ad9:c9ce with SMTP id dt8-20020a05620a478800b007833ad9c9cemr6566094qkb.50.1706579094613; Mon, 29 Jan 2024 17:44:54 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706579094; cv=pass; d=google.com; s=arc-20160816; b=LGXoy+x/507KvIBAn81FnX89dj/BadWVUMsrjGWyYgCmc+Dxlu5rmQPwV4HmOeA1xU Gbn6U2vG5NeARBDdsWZ85tn1vNY92UUw2K0EEqGXtOzNT2+MT1rOJbFDLkkJ+fOC8Nsc 87mN1mribbVP1renMOBAtkax15XMS2jL2ZLBokt5ZnwasrK9xigkshxiMfyLKgsIX07B y1Yxp7pJiq2HDJE8h/tB7Boqx9jSy8mXlqYVPmnYJ4gUdW4lzffsZG/iIJJ9D98xkHsM KEP/pYeVwaAHOVRqv+KrG0oskysAyKCyx9YCkSOOz8l2UcWwJpu9SmcoU6msAVDvS3lo QZzg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature; bh=nW+93M2Q9nWxSPgR21ybfgSivHJtX7f76WUEXvkAumg=; fh=pEdtrChUTDmvuLILGSavwskaPZjBcM3e1cB9O4e7o4U=; b=LZoGWTQn+fNpDoYbaARUfWcFt56On3IWM5YSUqJtAQCCDxjIO712bD2icUA4Ly+XXN uZuAXFkk6YysWveULq44i3qH9i38xdsJFymD3gNKVnMceeZeH/seukcP+ztWVa6NwpSx 4yYJVwxCIfyy/A3MBIJ43N0NMDx6pWfcGenSFmTCK45uF/FK9Ih8WvBSrds+KFty83TZ Z/Jm/83BH4ovI+AffU2u48dyCiynL/yHsWt9aJ1jQrp6ACivmtj43XKI7xtybstdQNm4 hxtGvY/OWdvJt6eXcSbj+06cKs6o1i3zhQolsu0SLQG8ZOC+o+U9FYus+jLf9KEuvSJM 0JAQ== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=0e5cKCip; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43750-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43750-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Return-Path: Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [2604:1380:45d1:ec00::1]) by mx.google.com with ESMTPS id f18-20020a05620a12f200b00783f6341992si4864430qkl.288.2024.01.29.17.44.54 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:44:54 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-43750-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1; Authentication-Results: mx.google.com; dkim=pass header.i=@cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=0e5cKCip; arc=pass (i=1 spf=pass spfdomain=cmpxchg.org dkim=pass dkdomain=cmpxchg-org.20230601.gappssmtp.com dmarc=pass fromdomain=cmpxchg.org); spf=pass (google.com: domain of linux-kernel+bounces-43750-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-43750-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=cmpxchg.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 522281C23ECE for ; Tue, 30 Jan 2024 01:44:54 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id EBD79364A3; Tue, 30 Jan 2024 01:42:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b="0e5cKCip" Received: from mail-qv1-f50.google.com (mail-qv1-f50.google.com [209.85.219.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5510738393 for ; Tue, 30 Jan 2024 01:42:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.50 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578950; cv=none; b=unjdyYhAkgRuywN/p8AHdh0FrPJ4KZfNBOGGNr7KeR5xbn0M9OHVSVJk903QYmuIxivMv2hJtdqLIuCTQtBx2ipnMgVWe6EfC7Bb6XjQ/QRlZSKQ6MQ6CNy4hSCmj0akj+wcYN4BaXPt938T+Wd1nMy7IfTFR2HCPvwtl8vG1xk= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706578950; c=relaxed/simple; bh=9xw/W03ukpr/L6i7AIbF53T7wElpUaGJf2hRLuUR5ws=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GzOpBQoc9ArjqXpZeh4BRLC2qL3sCX9ULwGRuLCOERl9nn95VFraiE9N5xHkHuY8SRD0LIeZsbjkDVmfQZfLHejn1DseopksO9zLlm4hiwd8DfT1h+cR1SCOcY3RDUGJCuz/OVLwWRveQfhQMcECZ+FcpCj+ETzNrO7EuY8gQWw= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org; spf=pass smtp.mailfrom=cmpxchg.org; dkim=pass (2048-bit key) header.d=cmpxchg-org.20230601.gappssmtp.com header.i=@cmpxchg-org.20230601.gappssmtp.com header.b=0e5cKCip; arc=none smtp.client-ip=209.85.219.50 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=cmpxchg.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=cmpxchg.org Received: by mail-qv1-f50.google.com with SMTP id 6a1803df08f44-68c444f9272so14319546d6.3 for ; Mon, 29 Jan 2024 17:42:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1706578947; x=1707183747; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=nW+93M2Q9nWxSPgR21ybfgSivHJtX7f76WUEXvkAumg=; b=0e5cKCipbmAbU9orMkEXr3CPe36KKB1C+HdJ0pnKiwUkkDGfvKsujIvPdVx4UIRCfp xN3glyGGe6wLu9Ww0AzCkrY+ebAEBEGTVcQG177fl/OoqIIJuQ4HB2zPQRrj+WLh/96s RQB0QwuNO5fbojE8HAdOousipxB6IZPBTvXeboIRhlFdYb4gsOKVugoiH+fmRwetgfFZ O7IFFSvIrlgvrUu6Gi9fTm4RbDNEmvbJb2zsE2r6B5i4Uiu4/SCrND5DZ2105HPfOoFm fLrJQOSh5pmr5NI6cvQ3TTSR1YXHMcqNgPd85v+SL8OEGV6PTLCOrb/QqSFX4s0CpVqv nsjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706578947; x=1707183747; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nW+93M2Q9nWxSPgR21ybfgSivHJtX7f76WUEXvkAumg=; b=AM+L5tTCzS4LyYRikW55QS1fjsJVDC3EfSulBfY6IVQZLLe0pfbe/08nh610xr9OTj dCycLbZ8aaLra2HbxUG3Zsx6b/sj30snq9b0nWBJQi+21Eu9676TMAOa3jXsbrXwtauV sfyrk9H5hSxAWfl0HjPxOt/Qu8ZdlG9xoTmL7KjeoVpcRbLRHLnVN3IOy7MeGshtCteS PkR7lTQk88KLvCfOnaNRsaQzaSYE0zHvWWc3tjqoFeVeaFtuQJsfFO0pFCA/NhZbKaPG cI6PGAr1uKJA8vf7qbB3uyNcvm5CK6qBMagzrkIXpAW0TxDXFxcYO26fL0J+wYrWMTVU 1TDQ== X-Gm-Message-State: AOJu0Yx4r6VCDCYtgd6Q0MUltSWeESH/yLt2uty4icq3pIqg9xXEDtlZ TKfP/xQocnTBMJDAB5TJu8FuCagoU5s2Hbi+j+MFC/vJZKl/CRMlDFyjzs1V3II= X-Received: by 2002:a05:6214:cab:b0:686:9441:ac30 with SMTP id s11-20020a0562140cab00b006869441ac30mr8699046qvs.22.1706578947264; Mon, 29 Jan 2024 17:42:27 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id lr9-20020a0562145bc900b006819bb31533sm3993392qvb.99.2024.01.29.17.42.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 29 Jan 2024 17:42:26 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Nhat Pham , Yosry Ahmed , Chengming Zhou , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 11/20] mm: zswap: function ordering: pool refcounting Date: Mon, 29 Jan 2024 20:36:47 -0500 Message-ID: <20240130014208.565554-12-hannes@cmpxchg.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240130014208.565554-1-hannes@cmpxchg.org> References: <20240130014208.565554-1-hannes@cmpxchg.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Move pool refcounting functions into the pool section. First the destroy functions, then the get and put which uses them. __zswap_pool_empty() has an upward reference to the global zswap_pools, to sanity check it's not the currently active pool that's being freed. That gets the forward decl for zswap_pool_cuyrrent(). This puts the get and put function above all callers, so kill the forward decls as well. Signed-off-by: Johannes Weiner --- mm/zswap.c | 94 +++++++++++++++++++++++++++--------------------------- 1 file changed, 47 insertions(+), 47 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 805d9a35f633..33775f2224b7 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -278,8 +278,6 @@ static inline struct zswap_tree *swap_zswap_tree(swp_entry_t swp) static int zswap_writeback_entry(struct zswap_entry *entry, swp_entry_t swpentry); -static int zswap_pool_get(struct zswap_pool *pool); -static void zswap_pool_put(struct zswap_pool *pool); static bool zswap_is_full(void) { @@ -472,6 +470,53 @@ static void zswap_pool_destroy(struct zswap_pool *pool) kfree(pool); } +static void __zswap_pool_release(struct work_struct *work) +{ + struct zswap_pool *pool = container_of(work, typeof(*pool), + release_work); + + synchronize_rcu(); + + /* nobody should have been able to get a kref... */ + WARN_ON(kref_get_unless_zero(&pool->kref)); + + /* pool is now off zswap_pools list and has no references. */ + zswap_pool_destroy(pool); +} + +static struct zswap_pool *zswap_pool_current(void); + +static void __zswap_pool_empty(struct kref *kref) +{ + struct zswap_pool *pool; + + pool = container_of(kref, typeof(*pool), kref); + + spin_lock(&zswap_pools_lock); + + WARN_ON(pool == zswap_pool_current()); + + list_del_rcu(&pool->list); + + INIT_WORK(&pool->release_work, __zswap_pool_release); + schedule_work(&pool->release_work); + + spin_unlock(&zswap_pools_lock); +} + +static int __must_check zswap_pool_get(struct zswap_pool *pool) +{ + if (!pool) + return 0; + + return kref_get_unless_zero(&pool->kref); +} + +static void zswap_pool_put(struct zswap_pool *pool) +{ + kref_put(&pool->kref, __zswap_pool_empty); +} + /* should be called under RCU */ #ifdef CONFIG_MEMCG static inline struct mem_cgroup *mem_cgroup_from_entry(struct zswap_entry *entry) @@ -1122,51 +1167,6 @@ static void shrink_worker(struct work_struct *w) zswap_pool_put(pool); } -static int __must_check zswap_pool_get(struct zswap_pool *pool) -{ - if (!pool) - return 0; - - return kref_get_unless_zero(&pool->kref); -} - -static void __zswap_pool_release(struct work_struct *work) -{ - struct zswap_pool *pool = container_of(work, typeof(*pool), - release_work); - - synchronize_rcu(); - - /* nobody should have been able to get a kref... */ - WARN_ON(kref_get_unless_zero(&pool->kref)); - - /* pool is now off zswap_pools list and has no references. */ - zswap_pool_destroy(pool); -} - -static void __zswap_pool_empty(struct kref *kref) -{ - struct zswap_pool *pool; - - pool = container_of(kref, typeof(*pool), kref); - - spin_lock(&zswap_pools_lock); - - WARN_ON(pool == zswap_pool_current()); - - list_del_rcu(&pool->list); - - INIT_WORK(&pool->release_work, __zswap_pool_release); - schedule_work(&pool->release_work); - - spin_unlock(&zswap_pools_lock); -} - -static void zswap_pool_put(struct zswap_pool *pool) -{ - kref_put(&pool->kref, __zswap_pool_empty); -} - /********************************* * param callbacks **********************************/ -- 2.43.0