Received: by 2002:a05:6902:102b:0:0:0:0 with SMTP id x11csp370528ybt; Wed, 24 Jun 2020 01:04:43 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzfOc4QMaQFUwyfIOjo9eLBpF+idq7YC1pfxz+ayoz9p12qi0/dioaz9kHq2FyESk9LztFO X-Received: by 2002:a17:906:1ed2:: with SMTP id m18mr22995339ejj.529.1592985883348; Wed, 24 Jun 2020 01:04:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1592985883; cv=none; d=google.com; s=arc-20160816; b=Gda+Cfy4Z8vD6TWNsTTL0uw/82FioCoGwucYMwuGYxzLPnhPUNGygH3BBG+9p65YZm LysCMY2WUSEGK+fYQRlN62dX/Jcw1kCR+XIqsz+Y4Nsputoc8IdqHYCuLsvnqK1bUUcG U8IdOGiUYqTEdxLNdwF2wvYkFCHrvWQW3Eb+C0uARRNzdhAAn+hC01lGrWeO82ZH26+K D/g8fXSO59kY2A2dPBZRMtA8OXQhXEIH7dQMN5DLGxeZsYdoWW+hp1++iMrTPXgto6fw BYyU6l/YIE2Mv/phuuSispX7/0riGzM4DCWhzesyVkxlevBFo5Ge3mLRmsCQIwjLBgWH k4BA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from; bh=lGkcxwLxxTD16f60/p6iIsq/QBSxIbVrX2RdjOv+SKc=; b=s0sM7cJdWJUTDKpWOSG3m9NK93TDVAi5WGKgE7VU/bxKr9gNby7DZE0xeHTM3vHZyI XDSB8Ivp3X6czkCEgrk6oI09a2i1h1TRfkj+iT17SMtV83ll+JTDNrgCfTBjQ3WX8e60 3KqrUppj7HBVrsVeDceLOt2MSEJCzGwGUsDgmTZl1aReG4+90SivRECzqOETQ80v5FDF 9qdRNGpmH9NJNXTIGbQUer3QryhYnOXiSgYZm+omHhHb831XGTABZzlVlPTUwEQZXCBV iSsjkGGNng6i7mGUwqETXHYEQScE5WRbb4EL3LWFQnNRNR5gJX9v6DpDiGSIYTpz/5Ds b82Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id mh8si3193925ejb.245.2020.06.24.01.04.19; Wed, 24 Jun 2020 01:04:43 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390272AbgFXIDb (ORCPT + 99 others); Wed, 24 Jun 2020 04:03:31 -0400 Received: from mail.fireflyinternet.com ([109.228.58.192]:62455 "EHLO fireflyinternet.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2387606AbgFXID3 (ORCPT ); Wed, 24 Jun 2020 04:03:29 -0400 X-Default-Received-SPF: pass (skip=forwardok (res=PASS)) x-ip-name=78.156.65.138; Received: from build.alporthouse.com (unverified [78.156.65.138]) by fireflyinternet.com (Firefly Internet (M1)) with ESMTP id 21599198-1500050 for multiple; Wed, 24 Jun 2020 09:02:53 +0100 From: Chris Wilson To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, intel-gfx@lists.freedesktop.org, Chris Wilson , Andrew Morton , Jason Gunthorpe Subject: [PATCH 1/2] mm/mmu_notifier: Mark up direct reclaim paths with MAYFAIL Date: Wed, 24 Jun 2020 09:02:47 +0100 Message-Id: <20200624080248.3701-1-chris@chris-wilson.co.uk> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org When direct reclaim enters the shrinker and tries to reclaim pages, it has to opportunitically unmap them [try_to_unmap_one]. For direct reclaim, the calling context is unknown and may include attempts to unmap one page of a dma object while attempting to allocate more pages for that object. Pass the information along that we are inside an opportunistic unmap that can allow that page to remain referenced and mapped, and let the callback opt in to avoiding a recursive wait. Signed-off-by: Chris Wilson Cc: Andrew Morton CC: Jason Gunthorpe --- include/linux/mmu_notifier.h | 15 ++++++++++++++- mm/mmu_notifier.c | 3 +++ mm/rmap.c | 5 +++-- 3 files changed, 20 insertions(+), 3 deletions(-) diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index fc68f3570e19..ee1ad008951c 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -48,7 +48,8 @@ enum mmu_notifier_event { MMU_NOTIFY_RELEASE, }; -#define MMU_NOTIFIER_RANGE_BLOCKABLE (1 << 0) +#define MMU_NOTIFIER_RANGE_BLOCKABLE BIT(0) +#define MMU_NOTIFIER_RANGE_MAYFAIL BIT(1) struct mmu_notifier_ops { /* @@ -169,6 +170,12 @@ struct mmu_notifier_ops { * a non-blocking behavior then the same applies to * invalidate_range_end. * + * If mayfail is set then the callback may return -EAGAIN while still + * holding its page references. This flag is set inside direct + * reclaim paths that are opportunistically trying to unmap pages + * from unknown contexts. The callback must be prepared to handle + * the matching invalidate_range_end even after failing the + * invalidate_range_start. */ int (*invalidate_range_start)(struct mmu_notifier *subscription, const struct mmu_notifier_range *range); @@ -397,6 +404,12 @@ mmu_notifier_range_blockable(const struct mmu_notifier_range *range) return (range->flags & MMU_NOTIFIER_RANGE_BLOCKABLE); } +static inline bool +mmu_notifier_range_mayfail(const struct mmu_notifier_range *range) +{ + return (range->flags & MMU_NOTIFIER_RANGE_MAYFAIL); +} + static inline void mmu_notifier_release(struct mm_struct *mm) { if (mm_has_notifiers(mm)) diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c index 352bb9f3ecc0..95b89cee7af4 100644 --- a/mm/mmu_notifier.c +++ b/mm/mmu_notifier.c @@ -493,6 +493,9 @@ static int mn_hlist_invalidate_range_start( _ret = ops->invalidate_range_start(subscription, range); if (!mmu_notifier_range_blockable(range)) non_block_end(); + if (_ret == -EAGAIN && + mmu_notifier_range_mayfail(range)) + _ret = 0; if (_ret) { pr_info("%pS callback failed with %d in %sblockable context.\n", ops->invalidate_range_start, _ret, diff --git a/mm/rmap.c b/mm/rmap.c index 5fe2dedce1fc..912b737a3353 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1406,8 +1406,9 @@ static bool try_to_unmap_one(struct page *page, struct vm_area_struct *vma, * Note that the page can not be free in this function as call of * try_to_unmap() must hold a reference on the page. */ - mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, vma, vma->vm_mm, - address, + mmu_notifier_range_init(&range, + MMU_NOTIFY_CLEAR, MMU_NOTIFIER_RANGE_MAYFAIL, + vma, vma->vm_mm, address, min(vma->vm_end, address + page_size(page))); if (PageHuge(page)) { /* -- 2.20.1