Received: by 2002:a05:6358:d09b:b0:dc:cd0c:909e with SMTP id jc27csp4073759rwb; Tue, 8 Nov 2022 11:43:15 -0800 (PST) X-Google-Smtp-Source: AMsMyM7Sr6hI0DoHtVLZcqcc5olp8eazbIBgCf32yejhwC8s9jI9aKaQA7PMSIs95FveF5zaB37E X-Received: by 2002:a05:6a00:18a8:b0:56c:702a:5f49 with SMTP id x40-20020a056a0018a800b0056c702a5f49mr58028496pfh.1.1667936594872; Tue, 08 Nov 2022 11:43:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1667936594; cv=none; d=google.com; s=arc-20160816; b=tSMfbroiePlAD95xAWRqzaDqCaKtG3Hj3tnizOKvoqDsdg5NIshgAcI/J26j8nqxqk Ytv9f8MSDVI3j21o/LZ7He9m4hvY4DhdBhhMpRHR7gwdp/QT6VPPozKYD5UntntVoXEA 5aOw8fdG8qMISM0W7HhZf/emQqC3cPjIHKnXIaVYU0OySZ/6nH83owUfdewQe6GHfMcY CkGVYS8tpvHknvGBS7VNjxa/aAwN1uisCt4XQGxfxfLuBPtUWtBEyFWnh7pLKsoB/+aA lRoRFxQjY8aGrE3F4X6YMMm8/Zb1VnBbLNh4Hd8O7ZBSIeAbyTGWsS4LPFnfQqvqOywu H8vA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=TdXYXXujE5c6Z8uTdZ5KokARMRQa3FrJU+bkNzs2RLY=; b=cdR+zz51d43osJcFtmh1eCXg8h2Qiif8Z+m5nEUjsxdHzfHdH/pyxndQba/NuJ0WBG rDGe3WJHpTkZkdQ/eRWfraHx4jg/ffeI/prjl5Kxyc9XPYFFa2tErtwhzpOePnhK0r5K 9bFEZNkn//DvA3WH+pfqX++Kv31bAMkOnkY1CbQlHp5+zyxkXok7uVWamD2wonmigceT uzVtI5IabUSWzpnRulwzZ0U9upU/rF6sYkCsFm/cVdWpFW5mwOsnKXjZhjeHjpQZesD7 KJGe5t/ynkfvFG7fib+0L5ldQ3kQZuf5mt6288Ipj6+K4T070PCttLGGrdIFA9j/SXRf igtQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=HSThM6hF; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id g9-20020a170902c38900b00178b95aa01fsi14069513plg.614.2022.11.08.11.43.03; Tue, 08 Nov 2022 11:43:14 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=HSThM6hF; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229647AbiKHTcN (ORCPT + 91 others); Tue, 8 Nov 2022 14:32:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34280 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229485AbiKHTcK (ORCPT ); Tue, 8 Nov 2022 14:32:10 -0500 Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 066F5682B0 for ; Tue, 8 Nov 2022 11:32:10 -0800 (PST) Received: by mail-pf1-x429.google.com with SMTP id v28so14663783pfi.12 for ; Tue, 08 Nov 2022 11:32:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TdXYXXujE5c6Z8uTdZ5KokARMRQa3FrJU+bkNzs2RLY=; b=HSThM6hFhhVtQ9XEuNUbTWYT7N7QyCqsOWJmCZ+Yc1HgTC5dVqVT5bIrBAjLJjYN4H DtZ4Od1NEvWEM7Q8mFdZTbfTJE3GcYDBXEa/wKhZksZQdEXvopPYjROUxytN+JRYnCAm PRXAIiGoz9m1Ca7Sa/Y36uQ24ak+Yw9WNj0uVQtUJNufX6MAtOUZYBw9F8TlnQraD6t7 okXhYKtevjEUpjKI3L3YStKt6jPXNUGnPKtTSWoAEANPeEcaIj34eCx8kbh+b8JlXl30 /fwr4LcBJksre4uH1ZMCiTg/wkn14O5eKPDZeoJmyIHSZ6IxnevkEL92goAY6m45eBPZ Y4KQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TdXYXXujE5c6Z8uTdZ5KokARMRQa3FrJU+bkNzs2RLY=; b=VYelQZu3aLOC2reqzM1ibsdlNiSTlTYKQ0JOaqOPvq3hShEaFE/4IQwg4MGeHrwvv2 E8yDbIvYIuLyWGSfXxIlCsp74vUKOiObFodjd/WfVDlWgNaZj6SfNqIhKnpD3i/qjaBU lzvqQ/9KTtBa7asO/cY/AmvXPMQqy0RGo7qzHHgiT+CxHhtO+Ifi4Kj3QylfqW0tL6jS YTmTscHef+l7oATUzaK2sGgbqf54a6olzr+GoAgmzpyS8W4vrHnI5eH6kU05aQWysMHR MB1WCqmntOf40KVeL7EJX932Sk/26nqeu5500ZHBOIn1Q+GxPvgp24DH8qV6WWXNWe4M a1UA== X-Gm-Message-State: ACrzQf1cQO/T1zbcq4hZR3LHimJ5f1upqh/UL24MPjFQc9JeLzIFCW/z V/tRY04BwyoiobuZzkGpw52ZekohrGAYCg== X-Received: by 2002:a63:5a05:0:b0:434:23a5:a5ca with SMTP id o5-20020a635a05000000b0043423a5a5camr49494580pgb.515.1667935929482; Tue, 08 Nov 2022 11:32:09 -0800 (PST) Received: from localhost (fwdproxy-prn-007.fbsv.net. [2a03:2880:ff:7::face:b00c]) by smtp.gmail.com with ESMTPSA id a13-20020a63e84d000000b0046ae5cfc3d5sm6054230pgk.61.2022.11.08.11.32.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 08 Nov 2022 11:32:09 -0800 (PST) From: Nhat Pham To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, minchan@kernel.org, ngupta@vflare.org, senozhatsky@chromium.org, sjenning@redhat.com, ddstreet@ieee.org, vitaly.wool@konsulko.com Subject: [PATCH v3 1/5] zswap: fix writeback lock ordering for zsmalloc Date: Tue, 8 Nov 2022 11:32:03 -0800 Message-Id: <20221108193207.3297327-2-nphamcs@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221108193207.3297327-1-nphamcs@gmail.com> References: <20221108193207.3297327-1-nphamcs@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Johannes Weiner zswap's customary lock order is tree->lock before pool->lock, because the tree->lock protects the entries' refcount, and the free callbacks in the backends acquire their respective pool locks to dispatch the backing object. zsmalloc's map callback takes the pool lock, so zswap must not grab the tree->lock while a handle is mapped. This currently only happens during writeback, which isn't implemented for zsmalloc. In preparation for it, move the tree->lock section out of the mapped entry section Signed-off-by: Johannes Weiner Signed-off-by: Nhat Pham --- mm/zswap.c | 37 ++++++++++++++++++++----------------- 1 file changed, 20 insertions(+), 17 deletions(-) diff --git a/mm/zswap.c b/mm/zswap.c index 2d48fd59cc7a..2d69c1d678fe 100644 --- a/mm/zswap.c +++ b/mm/zswap.c @@ -958,7 +958,7 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) }; if (!zpool_can_sleep_mapped(pool)) { - tmp = kmalloc(PAGE_SIZE, GFP_ATOMIC); + tmp = kmalloc(PAGE_SIZE, GFP_KERNEL); if (!tmp) return -ENOMEM; } @@ -968,6 +968,7 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) swpentry = zhdr->swpentry; /* here */ tree = zswap_trees[swp_type(swpentry)]; offset = swp_offset(swpentry); + zpool_unmap_handle(pool, handle); /* find and ref zswap entry */ spin_lock(&tree->lock); @@ -975,20 +976,12 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) if (!entry) { /* entry was invalidated */ spin_unlock(&tree->lock); - zpool_unmap_handle(pool, handle); kfree(tmp); return 0; } spin_unlock(&tree->lock); BUG_ON(offset != entry->offset); - src = (u8 *)zhdr + sizeof(struct zswap_header); - if (!zpool_can_sleep_mapped(pool)) { - memcpy(tmp, src, entry->length); - src = tmp; - zpool_unmap_handle(pool, handle); - } - /* try to allocate swap cache page */ switch (zswap_get_swap_cache_page(swpentry, &page)) { case ZSWAP_SWAPCACHE_FAIL: /* no memory or invalidate happened */ @@ -1006,6 +999,14 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) acomp_ctx = raw_cpu_ptr(entry->pool->acomp_ctx); dlen = PAGE_SIZE; + zhdr = zpool_map_handle(pool, handle, ZPOOL_MM_RO); + src = (u8 *)zhdr + sizeof(struct zswap_header); + if (!zpool_can_sleep_mapped(pool)) { + memcpy(tmp, src, entry->length); + src = tmp; + zpool_unmap_handle(pool, handle); + } + mutex_lock(acomp_ctx->mutex); sg_init_one(&input, src, entry->length); sg_init_table(&output, 1); @@ -1015,6 +1016,11 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) dlen = acomp_ctx->req->dlen; mutex_unlock(acomp_ctx->mutex); + if (!zpool_can_sleep_mapped(pool)) + kfree(tmp); + else + zpool_unmap_handle(pool, handle); + BUG_ON(ret); BUG_ON(dlen != PAGE_SIZE); @@ -1045,7 +1051,11 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) zswap_entry_put(tree, entry); spin_unlock(&tree->lock); - goto end; + return ret; + +fail: + if (!zpool_can_sleep_mapped(pool)) + kfree(tmp); /* * if we get here due to ZSWAP_SWAPCACHE_EXIST @@ -1054,17 +1064,10 @@ static int zswap_writeback_entry(struct zpool *pool, unsigned long handle) * if we free the entry in the following put * it is also okay to return !0 */ -fail: spin_lock(&tree->lock); zswap_entry_put(tree, entry); spin_unlock(&tree->lock); -end: - if (zpool_can_sleep_mapped(pool)) - zpool_unmap_handle(pool, handle); - else - kfree(tmp); - return ret; } -- 2.30.2