Received: by 2002:a05:6a10:1d13:0:0:0:0 with SMTP id pp19csp1783270pxb; Fri, 27 Aug 2021 18:01:17 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxLu710JILY4d2JvNPCeLtoEJhLGXnLKfTNWQAr+bzvdIvpoOwisgQEBN2pyET0TDc5SE52 X-Received: by 2002:a17:906:802:: with SMTP id e2mr12672069ejd.133.1630112477736; Fri, 27 Aug 2021 18:01:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1630112477; cv=none; d=google.com; s=arc-20160816; b=AcudOVqZfYYk++g8i7n5O+w6P+GlomkDOCH1qzZYfWU57EPUZNIIgUY1tVdVdJP4tJ kDCmR2dL3/QQ0myXUWF1ifQtrFglce7OAqmMWHCsJ7MGI3kcKjNaFAvy/zxFX6vcspvc jma4ezPQ/PnwaxJcj52kfuHdWNM0cFPfOT+pWJ66C3whw5xhtiJP0W/4BjI2h0tNpgPl deAs9V0h1yl+PWeIO0fmIULLjsTzylILqUa6+YymWyCvx6k0goUHBUCPln7BW0xZSzzY Uu+Yg/bBoONYCSHrNqFblHQDchysubZH8ZodrwOaXBHfARmX5fV5YkYTsrjo47Rmp46w PmAQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:subject:message-id:date:from:in-reply-to :references:mime-version:dkim-signature; bh=j766wmeIUROd83oO6m7AIU2PDzlQyY/Fee/X27JUyiM=; b=qgSJEaV0nRgi2Nc7JdlGcXRQGiFi+HEyqGrGS8RGcf/s61xuF91iXXmME3nPz3+I7k cf0VXGwct3S2apNl2AdKMhGN1joFhl5WN4j54aiw95zXr2fQ2kteAI/GczeINS/cPMij GZDJnrj9BCLJ8FsnqKhLuOPX31n/QZiW21L4gARc+k+bFFFZfooWg9Ah7tFlRFGhmydU 5jF53AN+C4xr9JoAEYXBTcPheQ9+1rYVTshKbMA4aTv4jUFvIP38PJYBe8mBW8Aiz5Bm SvijNWjepBY4YkqzkO+4JKYC5aEVefAV/KUXJklhTBiGySh3MNpKi3m42SVXr/0mhUHI xVNw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=AVu0ai1W; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id j3si7483265ejj.516.2021.08.27.18.00.51; Fri, 27 Aug 2021 18:01:17 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=AVu0ai1W; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232879AbhH1A7s (ORCPT + 99 others); Fri, 27 Aug 2021 20:59:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51524 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232763AbhH1A7s (ORCPT ); Fri, 27 Aug 2021 20:59:48 -0400 Received: from mail-yb1-xb36.google.com (mail-yb1-xb36.google.com [IPv6:2607:f8b0:4864:20::b36]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AB345C061796 for ; Fri, 27 Aug 2021 17:58:58 -0700 (PDT) Received: by mail-yb1-xb36.google.com with SMTP id e129so15810578yba.5 for ; Fri, 27 Aug 2021 17:58:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=j766wmeIUROd83oO6m7AIU2PDzlQyY/Fee/X27JUyiM=; b=AVu0ai1WZqJz4CZ4+Y/E1JdjJHR1v5sZBn9OND3+xYm/iRLshjkUConMoR9vkEO7gj VbJsTZYOjpjffGoi/BTtPSf+lf4fXtUFK8v5Wg65BxCOWvwM3MmDWZuTLRjJBG7S5tCE NPRL8o8pfOK6ovjRyk8/KEm8FmWqCdXOL8O4pkYyqp90gs5vM0lQHBXYb6O8Z2gzmjuG TgWS6NLuE/ALP+l+dyfdxVWy2oeRqXizb2pmN20s6e5LInI1hzajoKQFkv9ZoqIcG7QA mKQKEFXxctnYU7rxzJ4T5jRQH3x/EvZDFZh5EkNHAxNdk3BlELS/h0cmpsnIvDYuK/9N nbgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=j766wmeIUROd83oO6m7AIU2PDzlQyY/Fee/X27JUyiM=; b=TWsVEAvxSYXX2Y8RBXCgsrbvqsTmf4VJ+vLCGZl2lggnAeiaHehk7ZcrCuc4d1zr7u s216YhM2yNHz1di/WCA43SFXBd1+d73v6TzZq5uKEzqQ3C1lT8DMifoSQcpXzOhI3L80 mo88Qc8mW3IOZVIzwBmicZgF2txETuMNjWrYKfiuUMHHH71SwQewc1oyNTzmPfEncBEM /LqDoXbVLqLNeuOMvGX3uAWVtDupJjJce+uUPq4pkKp04nD1ZXfqiPG7dnczGo3fbOo/ OIeHmeq7uFfWgi/nTbeo9xHp2/le05zCY4FDoBcd/TI1g85+y9bHbgXtHAFSgG7p2SlK PxoQ== X-Gm-Message-State: AOAM533Jja0tIQ3xCSsBZ7btg/1aOIOlzPO6kKeah5d0yS947Qt5qMq/ KFOWgaM6zAzglul/MfC9cL94DaKObHDhOtWLUtKzfg== X-Received: by 2002:a25:21c5:: with SMTP id h188mr8398398ybh.23.1630112336370; Fri, 27 Aug 2021 17:58:56 -0700 (PDT) MIME-Version: 1.0 References: <20210827191858.2037087-1-surenb@google.com> <20210827191858.2037087-2-surenb@google.com> <202108271712.1E99CCF4F@keescook> In-Reply-To: <202108271712.1E99CCF4F@keescook> From: Suren Baghdasaryan Date: Fri, 27 Aug 2021 17:58:45 -0700 Message-ID: Subject: Re: [PATCH v8 1/3] mm: rearrange madvise code to allow for reuse To: Kees Cook Cc: Andrew Morton , Colin Cross , Sumit Semwal , Michal Hocko , Dave Hansen , Matthew Wilcox , "Kirill A . Shutemov" , Vlastimil Babka , Johannes Weiner , Jonathan Corbet , Al Viro , Randy Dunlap , Kalesh Singh , Peter Xu , rppt@kernel.org, Peter Zijlstra , Catalin Marinas , vincenzo.frascino@arm.com, =?UTF-8?B?Q2hpbndlbiBDaGFuZyAo5by16Yym5paHKQ==?= , Axel Rasmussen , Andrea Arcangeli , Jann Horn , apopple@nvidia.com, John Hubbard , Yu Zhao , Will Deacon , fenghua.yu@intel.com, thunder.leizhen@huawei.com, Hugh Dickins , feng.tang@intel.com, Jason Gunthorpe , Roman Gushchin , Thomas Gleixner , krisman@collabora.com, chris.hyser@oracle.com, Peter Collingbourne , "Eric W. Biederman" , Jens Axboe , legion@kernel.org, eb@emlix.com, Muchun Song , Viresh Kumar , thomascedeno@google.com, sashal@kernel.org, cxfcosmos@gmail.com, linux@rasmusvillemoes.dk, LKML , linux-fsdevel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm , kernel-team , Pekka Enberg , Ingo Molnar , Oleg Nesterov , Jan Glauber , John Stultz , Rob Landley , Cyrill Gorcunov , "Serge E. Hallyn" , David Rientjes , Rik van Riel , Mel Gorman , Michel Lespinasse , Tang Chen , Robin Holt , Shaohua Li , Sasha Levin , Minchan Kim Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Aug 27, 2021 at 5:14 PM Kees Cook wrote: > > On Fri, Aug 27, 2021 at 12:18:56PM -0700, Suren Baghdasaryan wrote: > > From: Colin Cross > > > > Refactor the madvise syscall to allow for parts of it to be reused by a > > prctl syscall that affects vmas. > > > > Move the code that walks vmas in a virtual address range into a function > > that takes a function pointer as a parameter. The only caller for now is > > sys_madvise, which uses it to call madvise_vma_behavior on each vma, but > > the next patch will add an additional caller. > > > > Move handling all vma behaviors inside madvise_behavior, and rename it to > > madvise_vma_behavior. > > > > Move the code that updates the flags on a vma, including splitting or > > merging the vma as necessary, into a new function called > > madvise_update_vma. The next patch will add support for updating a new > > anon_name field as well. > > > > Signed-off-by: Colin Cross > > Cc: Pekka Enberg > > Cc: Dave Hansen > > Cc: Peter Zijlstra > > Cc: Ingo Molnar > > Cc: Oleg Nesterov > > Cc: "Eric W. Biederman" > > Cc: Jan Glauber > > Cc: John Stultz > > Cc: Rob Landley > > Cc: Cyrill Gorcunov > > Cc: Kees Cook > > Cc: "Serge E. Hallyn" > > Cc: David Rientjes > > Cc: Al Viro > > Cc: Hugh Dickins > > Cc: Rik van Riel > > Cc: Mel Gorman > > Cc: Michel Lespinasse > > Cc: Tang Chen > > Cc: Robin Holt > > Cc: Shaohua Li > > Cc: Sasha Levin > > Cc: Johannes Weiner > > Cc: Minchan Kim > > Signed-off-by: Andrew Morton > > [sumits: rebased over v5.9-rc3] > > Signed-off-by: Sumit Semwal > > [surenb: rebased over v5.14-rc7] > > Signed-off-by: Suren Baghdasaryan > > Other folks have already reviewed this, and it does look okay to me, > too, but I find it a bit hard to review. There are at least 3 things > happening in this patch: > - moving to the walker > - merging two behavior routines > - extracting flag setting from behavior checking > > It seems like those could be separate patches, but I'm probably overly > picky. :) Thank you for taking a look. I wanted to keep the patch as close to the original as possible, but if more people find it hard to review then I'll break it up into smaller pieces. Thanks, Suren. > > -Kees > > > --- > > mm/madvise.c | 319 +++++++++++++++++++++++++++------------------------ > > 1 file changed, 172 insertions(+), 147 deletions(-) > > > > diff --git a/mm/madvise.c b/mm/madvise.c > > index 5c065bc8b5f6..359cd3fa612c 100644 > > --- a/mm/madvise.c > > +++ b/mm/madvise.c > > @@ -63,76 +63,20 @@ static int madvise_need_mmap_write(int behavior) > > } > > > > /* > > - * We can potentially split a vm area into separate > > - * areas, each area with its own behavior. > > + * Update the vm_flags on regiion of a vma, splitting it or merging it as > > + * necessary. Must be called with mmap_sem held for writing; > > */ > > -static long madvise_behavior(struct vm_area_struct *vma, > > - struct vm_area_struct **prev, > > - unsigned long start, unsigned long end, int behavior) > > +static int madvise_update_vma(struct vm_area_struct *vma, > > + struct vm_area_struct **prev, unsigned long start, > > + unsigned long end, unsigned long new_flags) > > { > > struct mm_struct *mm = vma->vm_mm; > > - int error = 0; > > + int error; > > pgoff_t pgoff; > > - unsigned long new_flags = vma->vm_flags; > > - > > - switch (behavior) { > > - case MADV_NORMAL: > > - new_flags = new_flags & ~VM_RAND_READ & ~VM_SEQ_READ; > > - break; > > - case MADV_SEQUENTIAL: > > - new_flags = (new_flags & ~VM_RAND_READ) | VM_SEQ_READ; > > - break; > > - case MADV_RANDOM: > > - new_flags = (new_flags & ~VM_SEQ_READ) | VM_RAND_READ; > > - break; > > - case MADV_DONTFORK: > > - new_flags |= VM_DONTCOPY; > > - break; > > - case MADV_DOFORK: > > - if (vma->vm_flags & VM_IO) { > > - error = -EINVAL; > > - goto out; > > - } > > - new_flags &= ~VM_DONTCOPY; > > - break; > > - case MADV_WIPEONFORK: > > - /* MADV_WIPEONFORK is only supported on anonymous memory. */ > > - if (vma->vm_file || vma->vm_flags & VM_SHARED) { > > - error = -EINVAL; > > - goto out; > > - } > > - new_flags |= VM_WIPEONFORK; > > - break; > > - case MADV_KEEPONFORK: > > - new_flags &= ~VM_WIPEONFORK; > > - break; > > - case MADV_DONTDUMP: > > - new_flags |= VM_DONTDUMP; > > - break; > > - case MADV_DODUMP: > > - if (!is_vm_hugetlb_page(vma) && new_flags & VM_SPECIAL) { > > - error = -EINVAL; > > - goto out; > > - } > > - new_flags &= ~VM_DONTDUMP; > > - break; > > - case MADV_MERGEABLE: > > - case MADV_UNMERGEABLE: > > - error = ksm_madvise(vma, start, end, behavior, &new_flags); > > - if (error) > > - goto out_convert_errno; > > - break; > > - case MADV_HUGEPAGE: > > - case MADV_NOHUGEPAGE: > > - error = hugepage_madvise(vma, &new_flags, behavior); > > - if (error) > > - goto out_convert_errno; > > - break; > > - } > > > > if (new_flags == vma->vm_flags) { > > *prev = vma; > > - goto out; > > + return 0; > > } > > > > pgoff = vma->vm_pgoff + ((start - vma->vm_start) >> PAGE_SHIFT); > > @@ -149,21 +93,21 @@ static long madvise_behavior(struct vm_area_struct *vma, > > if (start != vma->vm_start) { > > if (unlikely(mm->map_count >= sysctl_max_map_count)) { > > error = -ENOMEM; > > - goto out; > > + return error; > > } > > error = __split_vma(mm, vma, start, 1); > > if (error) > > - goto out_convert_errno; > > + return error; > > } > > > > if (end != vma->vm_end) { > > if (unlikely(mm->map_count >= sysctl_max_map_count)) { > > error = -ENOMEM; > > - goto out; > > + return error; > > } > > error = __split_vma(mm, vma, end, 0); > > if (error) > > - goto out_convert_errno; > > + return error; > > } > > > > success: > > @@ -172,15 +116,7 @@ static long madvise_behavior(struct vm_area_struct *vma, > > */ > > vma->vm_flags = new_flags; > > > > -out_convert_errno: > > - /* > > - * madvise() returns EAGAIN if kernel resources, such as > > - * slab, are temporarily unavailable. > > - */ > > - if (error == -ENOMEM) > > - error = -EAGAIN; > > -out: > > - return error; > > + return 0; > > } > > > > #ifdef CONFIG_SWAP > > @@ -930,6 +866,96 @@ static long madvise_remove(struct vm_area_struct *vma, > > return error; > > } > > > > +/* > > + * Apply an madvise behavior to a region of a vma. madvise_update_vma > > + * will handle splitting a vm area into separate areas, each area with its own > > + * behavior. > > + */ > > +static int madvise_vma_behavior(struct vm_area_struct *vma, > > + struct vm_area_struct **prev, > > + unsigned long start, unsigned long end, > > + unsigned long behavior) > > +{ > > + int error = 0; > > + unsigned long new_flags = vma->vm_flags; > > + > > + switch (behavior) { > > + case MADV_REMOVE: > > + return madvise_remove(vma, prev, start, end); > > + case MADV_WILLNEED: > > + return madvise_willneed(vma, prev, start, end); > > + case MADV_COLD: > > + return madvise_cold(vma, prev, start, end); > > + case MADV_PAGEOUT: > > + return madvise_pageout(vma, prev, start, end); > > + case MADV_FREE: > > + case MADV_DONTNEED: > > + return madvise_dontneed_free(vma, prev, start, end, behavior); > > + case MADV_POPULATE_READ: > > + case MADV_POPULATE_WRITE: > > + return madvise_populate(vma, prev, start, end, behavior); > > + case MADV_NORMAL: > > + new_flags = new_flags & ~VM_RAND_READ & ~VM_SEQ_READ; > > + break; > > + case MADV_SEQUENTIAL: > > + new_flags = (new_flags & ~VM_RAND_READ) | VM_SEQ_READ; > > + break; > > + case MADV_RANDOM: > > + new_flags = (new_flags & ~VM_SEQ_READ) | VM_RAND_READ; > > + break; > > + case MADV_DONTFORK: > > + new_flags |= VM_DONTCOPY; > > + break; > > + case MADV_DOFORK: > > + if (vma->vm_flags & VM_IO) { > > + error = -EINVAL; > > + goto out; > > + } > > + new_flags &= ~VM_DONTCOPY; > > + break; > > + case MADV_WIPEONFORK: > > + /* MADV_WIPEONFORK is only supported on anonymous memory. */ > > + if (vma->vm_file || vma->vm_flags & VM_SHARED) { > > + error = -EINVAL; > > + goto out; > > + } > > + new_flags |= VM_WIPEONFORK; > > + break; > > + case MADV_KEEPONFORK: > > + new_flags &= ~VM_WIPEONFORK; > > + break; > > + case MADV_DONTDUMP: > > + new_flags |= VM_DONTDUMP; > > + break; > > + case MADV_DODUMP: > > + if (!is_vm_hugetlb_page(vma) && new_flags & VM_SPECIAL) { > > + error = -EINVAL; > > + goto out; > > + } > > + new_flags &= ~VM_DONTDUMP; > > + break; > > + case MADV_MERGEABLE: > > + case MADV_UNMERGEABLE: > > + error = ksm_madvise(vma, start, end, behavior, &new_flags); > > + if (error) > > + goto out; > > + break; > > + case MADV_HUGEPAGE: > > + case MADV_NOHUGEPAGE: > > + error = hugepage_madvise(vma, &new_flags, behavior); > > + if (error) > > + goto out; > > + break; > > + } > > + > > + error = madvise_update_vma(vma, prev, start, end, new_flags); > > + > > +out: > > + if (error == -ENOMEM) > > + error = -EAGAIN; > > + return error; > > +} > > + > > #ifdef CONFIG_MEMORY_FAILURE > > /* > > * Error injection support for memory error handling. > > @@ -978,30 +1004,6 @@ static int madvise_inject_error(int behavior, > > } > > #endif > > > > -static long > > -madvise_vma(struct vm_area_struct *vma, struct vm_area_struct **prev, > > - unsigned long start, unsigned long end, int behavior) > > -{ > > - switch (behavior) { > > - case MADV_REMOVE: > > - return madvise_remove(vma, prev, start, end); > > - case MADV_WILLNEED: > > - return madvise_willneed(vma, prev, start, end); > > - case MADV_COLD: > > - return madvise_cold(vma, prev, start, end); > > - case MADV_PAGEOUT: > > - return madvise_pageout(vma, prev, start, end); > > - case MADV_FREE: > > - case MADV_DONTNEED: > > - return madvise_dontneed_free(vma, prev, start, end, behavior); > > - case MADV_POPULATE_READ: > > - case MADV_POPULATE_WRITE: > > - return madvise_populate(vma, prev, start, end, behavior); > > - default: > > - return madvise_behavior(vma, prev, start, end, behavior); > > - } > > -} > > - > > static bool > > madvise_behavior_valid(int behavior) > > { > > @@ -1054,6 +1056,73 @@ process_madvise_behavior_valid(int behavior) > > } > > } > > > > +/* > > + * Walk the vmas in range [start,end), and call the visit function on each one. > > + * The visit function will get start and end parameters that cover the overlap > > + * between the current vma and the original range. Any unmapped regions in the > > + * original range will result in this function returning -ENOMEM while still > > + * calling the visit function on all of the existing vmas in the range. > > + * Must be called with the mmap_lock held for reading or writing. > > + */ > > +static > > +int madvise_walk_vmas(struct mm_struct *mm, unsigned long start, > > + unsigned long end, unsigned long arg, > > + int (*visit)(struct vm_area_struct *vma, > > + struct vm_area_struct **prev, unsigned long start, > > + unsigned long end, unsigned long arg)) > > +{ > > + struct vm_area_struct *vma; > > + struct vm_area_struct *prev; > > + unsigned long tmp; > > + int unmapped_error = 0; > > + > > + /* > > + * If the interval [start,end) covers some unmapped address > > + * ranges, just ignore them, but return -ENOMEM at the end. > > + * - different from the way of handling in mlock etc. > > + */ > > + vma = find_vma_prev(mm, start, &prev); > > + if (vma && start > vma->vm_start) > > + prev = vma; > > + > > + for (;;) { > > + int error; > > + > > + /* Still start < end. */ > > + if (!vma) > > + return -ENOMEM; > > + > > + /* Here start < (end|vma->vm_end). */ > > + if (start < vma->vm_start) { > > + unmapped_error = -ENOMEM; > > + start = vma->vm_start; > > + if (start >= end) > > + break; > > + } > > + > > + /* Here vma->vm_start <= start < (end|vma->vm_end) */ > > + tmp = vma->vm_end; > > + if (end < tmp) > > + tmp = end; > > + > > + /* Here vma->vm_start <= start < tmp <= (end|vma->vm_end). */ > > + error = visit(vma, &prev, start, tmp, arg); > > + if (error) > > + return error; > > + start = tmp; > > + if (prev && start < prev->vm_end) > > + start = prev->vm_end; > > + if (start >= end) > > + break; > > + if (prev) > > + vma = prev->vm_next; > > + else /* madvise_remove dropped mmap_lock */ > > + vma = find_vma(mm, start); > > + } > > + > > + return unmapped_error; > > +} > > + > > /* > > * The madvise(2) system call. > > * > > @@ -1126,9 +1195,7 @@ process_madvise_behavior_valid(int behavior) > > */ > > int do_madvise(struct mm_struct *mm, unsigned long start, size_t len_in, int behavior) > > { > > - unsigned long end, tmp; > > - struct vm_area_struct *vma, *prev; > > - int unmapped_error = 0; > > + unsigned long end; > > int error = -EINVAL; > > int write; > > size_t len; > > @@ -1168,51 +1235,9 @@ int do_madvise(struct mm_struct *mm, unsigned long start, size_t len_in, int beh > > mmap_read_lock(mm); > > } > > > > - /* > > - * If the interval [start,end) covers some unmapped address > > - * ranges, just ignore them, but return -ENOMEM at the end. > > - * - different from the way of handling in mlock etc. > > - */ > > - vma = find_vma_prev(mm, start, &prev); > > - if (vma && start > vma->vm_start) > > - prev = vma; > > - > > blk_start_plug(&plug); > > - for (;;) { > > - /* Still start < end. */ > > - error = -ENOMEM; > > - if (!vma) > > - goto out; > > - > > - /* Here start < (end|vma->vm_end). */ > > - if (start < vma->vm_start) { > > - unmapped_error = -ENOMEM; > > - start = vma->vm_start; > > - if (start >= end) > > - goto out; > > - } > > - > > - /* Here vma->vm_start <= start < (end|vma->vm_end) */ > > - tmp = vma->vm_end; > > - if (end < tmp) > > - tmp = end; > > - > > - /* Here vma->vm_start <= start < tmp <= (end|vma->vm_end). */ > > - error = madvise_vma(vma, &prev, start, tmp, behavior); > > - if (error) > > - goto out; > > - start = tmp; > > - if (prev && start < prev->vm_end) > > - start = prev->vm_end; > > - error = unmapped_error; > > - if (start >= end) > > - goto out; > > - if (prev) > > - vma = prev->vm_next; > > - else /* madvise_remove dropped mmap_lock */ > > - vma = find_vma(mm, start); > > - } > > -out: > > + error = madvise_walk_vmas(mm, start, end, behavior, > > + madvise_vma_behavior); > > blk_finish_plug(&plug); > > if (write) > > mmap_write_unlock(mm); > > -- > > 2.33.0.259.gc128427fd7-goog > > > > -- > Kees Cook