Received: by 2002:a05:6622:f08:0:0:0:0 with SMTP id l8csp4481993ivc; Tue, 3 Nov 2020 13:11:14 -0800 (PST) X-Google-Smtp-Source: ABdhPJx7CeRfWieNZmMfJDAcxGxDoZmTG7PXFkZtnRB4bNjarKJLkVyuY7zH1qBQxtaR3qYRvtBQ X-Received: by 2002:a17:906:d1ce:: with SMTP id bs14mr21343014ejb.548.1604437873824; Tue, 03 Nov 2020 13:11:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1604437873; cv=none; d=google.com; s=arc-20160816; b=nctFgcfafSVcitc4/zQcCthA16jGMyONmWHahwZdSBWViggUGRb0iYycLYwCsX/M0u GX//9QGMZB+eMrFbHRAaMzp/7gunZB/zW0jiUw74PwZFRZEYw50NK8Sln8UHbESkgy3v I0xTkcXc0ozJydketHTpJs0MM7xi+XA0J2KZfN5zpNY35JNDqR/RqiOOk39A7DQO8x3w 3PwN1uVxnxyMPOiWpfKevpwV/84x09OTxl4O287e0FdOtZfxARua+n+Dcq4v9YROxIM/ nFxYFXbWkrsHpN/MKKGdhzxQe9QuwzPJJgqCjbVWQFIujEQRwcGNq7VTtgxUgXwXUJs4 AuuA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=aJ7DwNhcHvdJWfAMybhTQz2DbbWZRZFYLlWJiI6ODgI=; b=eS+tORflLlmA7Vm/yheyg4mI5PpkEbtdgF9J62IWRXmahBcAU5ws45zopPTeIXXoNR njzKKZY39zRNeSiar/uM4sjaJk6ihN+chBt+HDZXEuc7/2r7MEFWdrWfauAq2ZiXVGf0 nOkh4rul9fXVrHP3ldRIYkXcVK22c1f+NryL904O4kx+a7Lq25t1SLMVJ397Nu1Y7SaY 31IPWZbFUWKGMPyD9ZaQZ3EoGUQ3IecxuAIsP7waDb3+QaqLnSOMADaEvguKkBDmFWNo IaRNPHneHmcfvD3otOhako3tGQbkt93bC1ZXycOrssTXeI/sx2Ls3fR/McFRqsFs2FPm tr/w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=XTKzdEjT; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id bk9si13636ejb.42.2020.11.03.13.10.50; Tue, 03 Nov 2020 13:11:13 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=XTKzdEjT; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388305AbgKCVHf (ORCPT + 99 others); Tue, 3 Nov 2020 16:07:35 -0500 Received: from mail.kernel.org ([198.145.29.99]:46752 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2388311AbgKCVHc (ORCPT ); Tue, 3 Nov 2020 16:07:32 -0500 Received: from localhost (83-86-74-64.cable.dynamic.v4.ziggo.nl [83.86.74.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 64E47206B5; Tue, 3 Nov 2020 21:07:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1604437650; bh=BiZiOeKvlUn6tEbv4dJv2UOkGFZBeLzLrp5rIQ07QQo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XTKzdEjTGfnKXp/NAd+gemqzfV4P0Z7S2MWCp7wSAY7+3GYEG1VblGr4PDH0Ythfi pvpr8SPesMSIoqd06cOIHtCANwhNAAsbVSMR5Qn4meHaa5ky4s93GxSPAyzVm938gJ t6tRcyBWitVO0R3UOewLRdOLR4kd4P1M7RTQdTcM= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Johannes Thumshirn , Filipe Manana , David Sterba Subject: [PATCH 4.19 130/191] btrfs: reschedule if necessary when logging directory items Date: Tue, 3 Nov 2020 21:37:02 +0100 Message-Id: <20201103203245.276698912@linuxfoundation.org> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201103203232.656475008@linuxfoundation.org> References: <20201103203232.656475008@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Filipe Manana commit bb56f02f26fe23798edb1b2175707419b28c752a upstream. Logging directories with many entries can take a significant amount of time, and in some cases monopolize a cpu/core for a long time if the logging task doesn't happen to block often enough. Johannes and Lu Fengqi reported test case generic/041 triggering a soft lockup when the kernel has CONFIG_SOFTLOCKUP_DETECTOR=y. For this test case we log an inode with 3002 hard links, and because the test removed one hard link before fsyncing the file, the inode logging causes the parent directory do be logged as well, which has 6004 directory items to log (3002 BTRFS_DIR_ITEM_KEY items plus 3002 BTRFS_DIR_INDEX_KEY items), so it can take a significant amount of time and trigger the soft lockup. So just make tree-log.c:log_dir_items() reschedule when necessary, releasing the current search path before doing so and then resume from where it was before the reschedule. The stack trace produced when the soft lockup happens is the following: [10480.277653] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [xfs_io:28172] [10480.279418] Modules linked in: dm_thin_pool dm_persistent_data (...) [10480.284915] irq event stamp: 29646366 [10480.285987] hardirqs last enabled at (29646365): [] __slab_alloc.constprop.0+0x56/0x60 [10480.288482] hardirqs last disabled at (29646366): [] irqentry_enter+0x1d/0x50 [10480.290856] softirqs last enabled at (4612): [] __do_softirq+0x323/0x56c [10480.293615] softirqs last disabled at (4483): [] asm_call_on_stack+0xf/0x20 [10480.296428] CPU: 2 PID: 28172 Comm: xfs_io Not tainted 5.9.0-rc4-default+ #1248 [10480.298948] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.12.0-59-gc9ba527-rebuilt.opensuse.org 04/01/2014 [10480.302455] RIP: 0010:__slab_alloc.constprop.0+0x19/0x60 [10480.304151] Code: 86 e8 31 75 21 00 66 66 2e 0f 1f 84 00 00 00 (...) [10480.309558] RSP: 0018:ffffadbe09397a58 EFLAGS: 00000282 [10480.311179] RAX: ffff8a495ab92840 RBX: 0000000000000282 RCX: 0000000000000006 [10480.313242] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffff85249b66 [10480.315260] RBP: ffff8a497d04b740 R08: 0000000000000001 R09: 0000000000000001 [10480.317229] R10: ffff8a497d044800 R11: ffff8a495ab93c40 R12: 0000000000000000 [10480.319169] R13: 0000000000000000 R14: 0000000000000c40 R15: ffffffffc01daf70 [10480.321104] FS: 00007fa1dc5c0e40(0000) GS:ffff8a497da00000(0000) knlGS:0000000000000000 [10480.323559] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [10480.325235] CR2: 00007fa1dc5befb8 CR3: 0000000004f8a006 CR4: 0000000000170ea0 [10480.327259] Call Trace: [10480.328286] ? overwrite_item+0x1f0/0x5a0 [btrfs] [10480.329784] __kmalloc+0x831/0xa20 [10480.331009] ? btrfs_get_32+0xb0/0x1d0 [btrfs] [10480.332464] overwrite_item+0x1f0/0x5a0 [btrfs] [10480.333948] log_dir_items+0x2ee/0x570 [btrfs] [10480.335413] log_directory_changes+0x82/0xd0 [btrfs] [10480.336926] btrfs_log_inode+0xc9b/0xda0 [btrfs] [10480.338374] ? init_once+0x20/0x20 [btrfs] [10480.339711] btrfs_log_inode_parent+0x8d3/0xd10 [btrfs] [10480.341257] ? dget_parent+0x97/0x2e0 [10480.342480] btrfs_log_dentry_safe+0x3a/0x50 [btrfs] [10480.343977] btrfs_sync_file+0x24b/0x5e0 [btrfs] [10480.345381] do_fsync+0x38/0x70 [10480.346483] __x64_sys_fsync+0x10/0x20 [10480.347703] do_syscall_64+0x2d/0x70 [10480.348891] entry_SYSCALL_64_after_hwframe+0x44/0xa9 [10480.350444] RIP: 0033:0x7fa1dc80970b [10480.351642] Code: 0f 05 48 3d 00 f0 ff ff 77 45 c3 0f 1f 40 00 48 (...) [10480.356952] RSP: 002b:00007fffb3d081d0 EFLAGS: 00000293 ORIG_RAX: 000000000000004a [10480.359458] RAX: ffffffffffffffda RBX: 0000562d93d45e40 RCX: 00007fa1dc80970b [10480.361426] RDX: 0000562d93d44ab0 RSI: 0000562d93d45e60 RDI: 0000000000000003 [10480.363367] RBP: 0000000000000001 R08: 0000000000000000 R09: 00007fa1dc7b2a40 [10480.365317] R10: 0000562d93d0e366 R11: 0000000000000293 R12: 0000000000000001 [10480.367299] R13: 0000562d93d45290 R14: 0000562d93d45e40 R15: 0000562d93d45e60 Link: https://lore.kernel.org/linux-btrfs/20180713090216.GC575@fnst.localdomain/ Reported-by: Johannes Thumshirn CC: stable@vger.kernel.org # 4.4+ Tested-by: Johannes Thumshirn Reviewed-by: Johannes Thumshirn Signed-off-by: Filipe Manana Reviewed-by: David Sterba Signed-off-by: David Sterba Signed-off-by: Greg Kroah-Hartman --- fs/btrfs/tree-log.c | 8 ++++++++ 1 file changed, 8 insertions(+) --- a/fs/btrfs/tree-log.c +++ b/fs/btrfs/tree-log.c @@ -3589,6 +3589,7 @@ static noinline int log_dir_items(struct * search and this search we'll not find the key again and can just * bail. */ +search: ret = btrfs_search_slot(NULL, root, &min_key, path, 0, 0); if (ret != 0) goto done; @@ -3608,6 +3609,13 @@ static noinline int log_dir_items(struct if (min_key.objectid != ino || min_key.type != key_type) goto done; + + if (need_resched()) { + btrfs_release_path(path); + cond_resched(); + goto search; + } + ret = overwrite_item(trans, log, dst_path, src, i, &min_key); if (ret) {