Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932167AbbG1I7x (ORCPT ); Tue, 28 Jul 2015 04:59:53 -0400 Received: from mx2.suse.de ([195.135.220.15]:52508 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754865AbbG1I7s (ORCPT ); Tue, 28 Jul 2015 04:59:48 -0400 Subject: Re: [RFC v3 3/3] mm: make swapin readahead to improve thp collapse rate To: Ebru Akagunduz , linux-mm@kvack.org References: <1436819284-3964-1-git-send-email-ebru.akagunduz@gmail.com> <1436819284-3964-4-git-send-email-ebru.akagunduz@gmail.com> Cc: akpm@linux-foundation.org, kirill.shutemov@linux.intel.com, n-horiguchi@ah.jp.nec.com, aarcange@redhat.com, riel@redhat.com, iamjoonsoo.kim@lge.com, xiexiuqi@huawei.com, gorcunov@openvz.org, linux-kernel@vger.kernel.org, mgorman@suse.de, rientjes@google.com, aneesh.kumar@linux.vnet.ibm.com, hughd@google.com, hannes@cmpxchg.org, mhocko@suse.cz, boaz@plexistor.com, raindel@mellanox.com From: Vlastimil Babka Message-ID: <55B7447E.70202@suse.cz> Date: Tue, 28 Jul 2015 10:59:42 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.1.0 MIME-Version: 1.0 In-Reply-To: <1436819284-3964-4-git-send-email-ebru.akagunduz@gmail.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 6954 Lines: 202 On 07/13/2015 10:28 PM, Ebru Akagunduz wrote: > This patch makes swapin readahead to improve thp collapse rate. > When khugepaged scanned pages, there can be a few of the pages > in swap area. > > With the patch THP can collapse 4kB pages into a THP when > there are up to max_ptes_swap swap ptes in a 2MB range. > > The patch was tested with a test program that allocates > 800MB of memory, writes to it, and then sleeps. I force > the system to swap out all. Afterwards, the test program > touches the area by writing, it skips a page in each > 20 pages of the area. > > Without the patch, system did not swap in readahead. > THP rate was %47 of the program of the memory, it > did not change over time. > > With this patch, after 10 minutes of waiting khugepaged had > collapsed %99 of the program's memory. > > Signed-off-by: Ebru Akagunduz > Acked-by: Rik van Riel > --- > Changes in v2: > - Use FAULT_FLAG_ALLOW_RETRY|FAULT_FLAG_RETRY_NOWAIT flag > instead of 0x0 when called do_swap_page from > __collapse_huge_page_swapin (Rik van Riel) > > Changes in v3: > - Catch VM_FAULT_HWPOISON and VM_FAULT_OOM return cases > in __collapse_huge_page_swapin (Kirill A. Shutemov) > > Test results: > > After swapped out > ------------------------------------------------------------------- > | Anonymous | AnonHugePages | Swap | Fraction | > ------------------------------------------------------------------- > With patch | 267128 kB | 266240 kB | 532876 kB | %99 | > ------------------------------------------------------------------- > Without patch | 238160 kB | 235520 kB | 561844 kB | %98 | > ------------------------------------------------------------------- > > After swapped in > ------------------------------------------------------------------- > | Anonymous | AnonHugePages | Swap | Fraction | > ------------------------------------------------------------------- > With patch | 533876 kB | 530432 kB | 266128 kB | %99 | > ------------------------------------------------------------------- > Without patch | 499956 kB | 235520 kB | 300048 kB | %47 | > ------------------------------------------------------------------- > > include/linux/mm.h | 4 ++++ > include/trace/events/huge_memory.h | 24 ++++++++++++++++++++++ > mm/huge_memory.c | 41 ++++++++++++++++++++++++++++++++++++++ > mm/memory.c | 2 +- > 4 files changed, 70 insertions(+), 1 deletion(-) > > diff --git a/include/linux/mm.h b/include/linux/mm.h > index eacf348..603f3ba 100644 > --- a/include/linux/mm.h > +++ b/include/linux/mm.h > @@ -40,6 +40,10 @@ > #define MM_COLLAPSE_ISOLATE_FAIL 5 > #define MM_EXCEED_SWAP_PTE 2 > > +extern int do_swap_page(struct mm_struct *mm, struct vm_area_struct *vma, > + unsigned long address, pte_t *page_table, pmd_t *pmd, > + unsigned int flags, pte_t orig_pte); > + > struct mempolicy; > struct anon_vma; > struct anon_vma_chain; > diff --git a/include/trace/events/huge_memory.h b/include/trace/events/huge_memory.h > index b6bdcc4..8d34086 100644 > --- a/include/trace/events/huge_memory.h > +++ b/include/trace/events/huge_memory.h > @@ -98,6 +98,30 @@ TRACE_EVENT(mm_collapse_huge_page_isolate, > __entry->ret) > ); > > +TRACE_EVENT(mm_collapse_huge_page_swapin, > + > + TP_PROTO(struct mm_struct *mm, int swap_pte, int ret), > + > + TP_ARGS(mm, swap_pte, ret), > + > + TP_STRUCT__entry( > + __field(struct mm_struct *, mm) > + __field(int, swap_pte) > + __field(int, ret) > + ), > + > + TP_fast_assign( > + __entry->mm = mm; > + __entry->swap_pte = swap_pte; > + __entry->ret = ret; > + ), > + > + TP_printk("mm=%p, swap_pte=%d, ret=%d", > + __entry->mm, > + __entry->swap_pte, > + __entry->ret) > +); swap_pte is weird name for the number of swapped-in pages, how about "swapped_in" ? > + > #endif /* __HUGE_MEMORY_H */ > #include > > diff --git a/mm/huge_memory.c b/mm/huge_memory.c > index b4cef9d..b372b40 100644 > --- a/mm/huge_memory.c > +++ b/mm/huge_memory.c > @@ -2511,6 +2511,45 @@ static bool hugepage_vma_check(struct vm_area_struct *vma) > return true; > } > > +/* > + * Bring missing pages in from swap, to complete THP collapse. > + * Only done if khugepaged_scan_pmd believes it is worthwhile. > + * > + * Called and returns without pte mapped or spinlocks held, > + * but with mmap_sem held to protect against vma changes. > + */ > + > +static void __collapse_huge_page_swapin(struct mm_struct *mm, > + struct vm_area_struct *vma, > + unsigned long address, pmd_t *pmd, > + pte_t *pte) > +{ > + unsigned long _address; > + pte_t pteval = *pte; > + int swap_pte = 0, ret = 0; Same concern about swap_pte name. > + > + pte = pte_offset_map(pmd, address); > + for (_address = address; _address < address + HPAGE_PMD_NR*PAGE_SIZE; > + pte++, _address += PAGE_SIZE) { > + pteval = *pte; > + if (is_swap_pte(pteval)) { > + swap_pte++; Move the increment towards the end of the "if" and then it counts what was successfully swapped in :) > + ret = do_swap_page(mm, vma, _address, pte, pmd, > + FAULT_FLAG_ALLOW_RETRY|FAULT_FLAG_RETRY_NOWAIT, > + pteval); > + if (ret == VM_FAULT_HWPOISON || ret == VM_FAULT_OOM) { > + trace_mm_collapse_huge_page_swapin(mm, vma->vm_start, swap_pte, 0); The vma->vm_start should be removed otherwise this won't compile. > + return; > + } > + /* pte is unmapped now, we need to map it */ > + pte = pte_offset_map(pmd, _address); > + } > + } > + pte--; > + pte_unmap(pte); > + trace_mm_collapse_huge_page_swapin(mm, swap_pte, 1); > +} > + > static void collapse_huge_page(struct mm_struct *mm, > unsigned long address, > struct page **hpage, > @@ -2584,6 +2623,8 @@ static void collapse_huge_page(struct mm_struct *mm, > > anon_vma_lock_write(vma->anon_vma); > > + __collapse_huge_page_swapin(mm, vma, address, pmd, pte); > + > pte = pte_offset_map(pmd, address); > pte_ptl = pte_lockptr(mm, pmd); > > diff --git a/mm/memory.c b/mm/memory.c > index 67afe75..eec23a2 100644 > --- a/mm/memory.c > +++ b/mm/memory.c > @@ -2443,7 +2443,7 @@ EXPORT_SYMBOL(unmap_mapping_range); > * We return with the mmap_sem locked or unlocked in the same cases > * as does filemap_fault(). > */ > -static int do_swap_page(struct mm_struct *mm, struct vm_area_struct *vma, > +int do_swap_page(struct mm_struct *mm, struct vm_area_struct *vma, > unsigned long address, pte_t *page_table, pmd_t *pmd, > unsigned int flags, pte_t orig_pte) > { > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/