Received: by 2002:ac0:a5b6:0:0:0:0:0 with SMTP id m51-v6csp476020imm; Fri, 1 Jun 2018 04:29:05 -0700 (PDT) X-Google-Smtp-Source: ADUXVKKtDRvRIyTn7msoEs5FJEEFsldPfASpdy+7kplmx5BoWHubvmNmBxlDKQw8jZvWSq4hVt/7 X-Received: by 2002:a63:7c04:: with SMTP id x4-v6mr8512801pgc.67.1527852545672; Fri, 01 Jun 2018 04:29:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1527852545; cv=none; d=google.com; s=arc-20160816; b=RHtUt7ALDGetltWuEMG6PNdnzZQqCkUyzDUSqpV+/8N9tg3UT/znkenLwZEDGP4PzM QvIC6dVAdEfUXiKuTbdHqlS/FocT3KWtaYp8gB47wktvZhL8tR8O6ResOW1tRsOnxgbB EtVg3XoWeXcG9uU5eLpAGPqLERWD4rrPP27m+2PKBBrXwPEIYhldTueKzwRSBtSRg3zd c1a8b2nyQbUDc+okx2Lo2zki+eDRrZdij/509FVNN6cfKpAFK8pjsir8ayoR14ozpNG6 oc487d5Mk5RhqF4485W9fEVVVX25gQOMnLvtP9tqNN3S4+saoJgFAIhcP7k7FMdMcWDM H+nQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:user-agent:in-reply-to :content-disposition:mime-version:references:message-id:subject:cc :to:from:date:arc-authentication-results; bh=JQnnqan87FtcS++gqHaxjIdoT7SW5Fd41VgxjKC4K8k=; b=npAN9IyWnWCzr8eMFYluIBJxHhNO1pGAHf5F4b8Md0vKxtGKvb7mMxdCIb7X8m1WZN Ir9myzCYArtB8jJyW8+MiSjjInDyUVvYFYK+FrteuG8juUcG56fWk0LfKCt3KojUWhY/ 5CdJz+9/9E2+mKyYEW7zYzPlF94tphSH//2ISqAwHlSsQaPlav7/Pe3L1HqR7M2P8SK2 GPts6T2m7T7tNGaTklvQLrMMKYaiSXUMxqlKaa0FWGRQM2omR4NmoUgcXIKjOcDpRnQJ A74bBfczirYW/H5Kbm5aUbxZ0BurEzw3J9bIhzZr4sGib69MW7zQfj5ZkTQhw+yUF1FB qFbg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id h3-v6si4616411plb.100.2018.06.01.04.28.50; Fri, 01 Jun 2018 04:29:05 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752282AbeFAL1q (ORCPT + 99 others); Fri, 1 Jun 2018 07:27:46 -0400 Received: from mx2.suse.de ([195.135.220.15]:54440 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752089AbeFAL1l (ORCPT ); Fri, 1 Jun 2018 07:27:41 -0400 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay1.suse.de (charybdis-ext-too.suse.de [195.135.220.254]) by mx2.suse.de (Postfix) with ESMTP id 49AE0AC81; Fri, 1 Jun 2018 11:27:40 +0000 (UTC) Date: Fri, 1 Jun 2018 13:27:39 +0200 From: Michal Hocko To: Sahitya Tummala Cc: linux-f2fs-devel@lists.sourceforge.net, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, Jaegeuk Kim , Chao Yu , vinmenon@codeaurora.org Subject: Re: deadlock during writeback when using f2fs filesystem Message-ID: <20180601112739.GA15278@dhcp22.suse.cz> References: <20180601093235.GA12489@codeaurora.org> <20180601102609.GZ15278@dhcp22.suse.cz> <20180601112050.GB12489@codeaurora.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20180601112050.GB12489@codeaurora.org> User-Agent: Mutt/1.9.5 (2018-04-13) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri 01-06-18 16:50:50, Sahitya Tummala wrote: > On Fri, Jun 01, 2018 at 12:26:09PM +0200, Michal Hocko wrote: > > On Fri 01-06-18 15:02:35, Sahitya Tummala wrote: > > > Hi, > > > > > > We are observing a deadlock scenario during FS writeback under low-memory > > > condition with F2FS filesystem. > > > > > > Here is the callstack of this scenario - > > > > > > shrink_inactive_list() > > > shrink_node_memcg.isra.74() > > > shrink_node() > > > shrink_zones(inline) > > > do_try_to_free_pages(inline) > > > try_to_free_pages() > > > __perform_reclaim(inline) > > > __alloc_pages_direct_reclaim(inline) > > > __alloc_pages_slowpath(inline) > > > no_zone() > > > __alloc_pages(inline) > > > __alloc_pages_node(inline) > > > alloc_pages_node(inline) > > > __page_cache_alloc(inline) > > > pagecache_get_page() > > > find_or_create_page(inline) > > > grab_cache_page(inline) > > > f2fs_grab_cache_page(inline) > > > __get_node_page.part.32() > > > __get_node_page(inline) > > > get_node_page() > > > update_inode_page() > > > f2fs_write_inode() > > > write_inode(inline) > > > __writeback_single_inode() > > > writeback_sb_inodes() > > > __writeback_inodes_wb() > > > wb_writeback() > > > wb_do_writeback(inline) > > > wb_workfn() > > > > > > The writeback thread is entering into the direct reclaim path due to low-memory and is > > > getting stuck in shrink_inactive_list(), as shrink_inactive_list() is inturn waiting for > > > writeback to happen for the dirty pages present in the inactive list. > > > > shrink_page_list waits only for writeback pages when we are in the memcg > > reclaim. The above seems to be the global reclaim though. Moreover > > GFP_F2FS_ZERO is GFP_NOFS so we are not waiting for writeback pages at > > all. Are you sure the above is really a deadlock? > > > > Let me correct my statement. It could be more of a livelock scenario. > > The direct reclaim path is not doing any writeback here, so the GFP_NOFS doesn't > make any difference. In this case, the direct reclaim has to reclaim ~32 pages, > which it picks up from the tail of the list. All of those tail pages are dirty > and since direct reclaim path can't do any writeback, it just loops picking and > skipping them. But there are surely other pages on the LRU list, aren't they? -- Michal Hocko SUSE Labs