Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753667Ab3G2Jn2 (ORCPT ); Mon, 29 Jul 2013 05:43:28 -0400 Received: from mail-ob0-f174.google.com ([209.85.214.174]:50206 "EHLO mail-ob0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751074Ab3G2JnN (ORCPT ); Mon, 29 Jul 2013 05:43:13 -0400 MIME-Version: 1.0 In-Reply-To: <1375075701-5998-10-git-send-email-iamjoonsoo.kim@lge.com> References: <1375075701-5998-1-git-send-email-iamjoonsoo.kim@lge.com> <1375075701-5998-10-git-send-email-iamjoonsoo.kim@lge.com> Date: Mon, 29 Jul 2013 17:43:13 +0800 Message-ID: Subject: Re: [PATCH v3 9/9] mm, hugetlb: decrement reserve count if VM_NORESERVE alloc page cache From: Hillf Danton To: Joonsoo Kim Cc: Andrew Morton , Rik van Riel , Mel Gorman , Michal Hocko , "Aneesh Kumar K.V" , KAMEZAWA Hiroyuki , Hugh Dickins , Davidlohr Bueso , David Gibson , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Joonsoo Kim , Wanpeng Li , Naoya Horiguchi Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 5636 Lines: 127 On Mon, Jul 29, 2013 at 1:28 PM, Joonsoo Kim wrote: > If a vma with VM_NORESERVE allocate a new page for page cache, we should > check whether this area is reserved or not. If this address is > already reserved by other process(in case of chg == 0), we should > decrement reserve count, because this allocated page will go into page > cache and currently, there is no way to know that this page comes from > reserved pool or not when releasing inode. This may introduce > over-counting problem to reserved count. With following example code, > you can easily reproduce this situation. > > Assume 2MB, nr_hugepages = 100 > > size = 20 * MB; > flag = MAP_SHARED; > p = mmap(NULL, size, PROT_READ|PROT_WRITE, flag, fd, 0); > if (p == MAP_FAILED) { > fprintf(stderr, "mmap() failed: %s\n", strerror(errno)); > return -1; > } > > flag = MAP_SHARED | MAP_NORESERVE; > q = mmap(NULL, size, PROT_READ|PROT_WRITE, flag, fd, 0); > if (q == MAP_FAILED) { > fprintf(stderr, "mmap() failed: %s\n", strerror(errno)); > } > q[0] = 'c'; > > After finish the program, run 'cat /proc/meminfo'. > You can see below result. > > HugePages_Free: 100 > HugePages_Rsvd: 1 > > To fix this, we should check our mapping type and tracked region. > If our mapping is VM_NORESERVE, VM_MAYSHARE and chg is 0, this imply > that current allocated page will go into page cache which is already > reserved region when mapping is created. In this case, we should decrease > reserve count. As implementing above, this patch solve the problem. > > Reviewed-by: Wanpeng Li > Reviewed-by: Aneesh Kumar K.V > Signed-off-by: Joonsoo Kim > Acked-by: Hillf Danton > diff --git a/mm/hugetlb.c b/mm/hugetlb.c > index 4b1b043..b3b8252 100644 > --- a/mm/hugetlb.c > +++ b/mm/hugetlb.c > @@ -443,10 +443,23 @@ void reset_vma_resv_huge_pages(struct vm_area_struct *vma) > } > > /* Returns true if the VMA has associated reserve pages */ > -static int vma_has_reserves(struct vm_area_struct *vma) > +static int vma_has_reserves(struct vm_area_struct *vma, long chg) > { > - if (vma->vm_flags & VM_NORESERVE) > - return 0; > + if (vma->vm_flags & VM_NORESERVE) { > + /* > + * This address is already reserved by other process(chg == 0), > + * so, we should decreament reserved count. Without > + * decreamenting, reserve count is remained after releasing > + * inode, because this allocated page will go into page cache > + * and is regarded as coming from reserved pool in releasing > + * step. Currently, we don't have any other solution to deal > + * with this situation properly, so add work-around here. > + */ > + if (vma->vm_flags & VM_MAYSHARE && chg == 0) > + return 1; > + else > + return 0; > + } > > /* Shared mappings always use reserves */ > if (vma->vm_flags & VM_MAYSHARE) > @@ -520,7 +533,8 @@ static struct page *dequeue_huge_page_node(struct hstate *h, int nid) > > static struct page *dequeue_huge_page_vma(struct hstate *h, > struct vm_area_struct *vma, > - unsigned long address, int avoid_reserve) > + unsigned long address, int avoid_reserve, > + long chg) > { > struct page *page = NULL; > struct mempolicy *mpol; > @@ -535,7 +549,7 @@ static struct page *dequeue_huge_page_vma(struct hstate *h, > * have no page reserves. This check ensures that reservations are > * not "stolen". The child may still get SIGKILLed > */ > - if (!vma_has_reserves(vma) && > + if (!vma_has_reserves(vma, chg) && > h->free_huge_pages - h->resv_huge_pages == 0) > goto err; > > @@ -553,8 +567,12 @@ retry_cpuset: > if (cpuset_zone_allowed_softwall(zone, htlb_alloc_mask)) { > page = dequeue_huge_page_node(h, zone_to_nid(zone)); > if (page) { > - if (!avoid_reserve && vma_has_reserves(vma)) > - h->resv_huge_pages--; > + if (avoid_reserve) > + break; > + if (!vma_has_reserves(vma, chg)) > + break; > + > + h->resv_huge_pages--; > break; > } > } > @@ -1138,7 +1156,7 @@ static struct page *alloc_huge_page(struct vm_area_struct *vma, > return ERR_PTR(-ENOSPC); > } > spin_lock(&hugetlb_lock); > - page = dequeue_huge_page_vma(h, vma, addr, avoid_reserve); > + page = dequeue_huge_page_vma(h, vma, addr, avoid_reserve, chg); > if (!page) { > spin_unlock(&hugetlb_lock); > page = alloc_buddy_huge_page(h, NUMA_NO_NODE); > -- > 1.7.9.5 > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/