Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932690AbaJWCmo (ORCPT ); Wed, 22 Oct 2014 22:42:44 -0400 Received: from relay3.sgi.com ([192.48.152.1]:42994 "EHLO relay.sgi.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S932381AbaJWCmn (ORCPT ); Wed, 22 Oct 2014 22:42:43 -0400 Date: Wed, 22 Oct 2014 21:43:08 -0500 From: Alex Thorlton To: athorlton@sgi.com Cc: Andrew Morton , Bob Liu , David Rientjes , "Eric W. Biederman" , Hugh Dickins , Ingo Molnar , Kees Cook , "Kirill A. Shutemov" , Mel Gorman , Oleg Nesterov , Peter Zijlstra , Rik van Riel , Thomas Gleixner , Vladimir Davydov , linux-kernel@vger.kernel.org Subject: Re: [PATCH 0/4] Convert khugepaged to a task_work function Message-ID: <20141023024308.GA14217@sgi.com> References: <1414031706-101613-1-git-send-email-athorlton@sgi.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1414031706-101613-1-git-send-email-athorlton@sgi.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Please ignore! Screwed up my git send-email... Sending the proper version here in a bit... On Wed, Oct 22, 2014 at 09:35:02PM -0500, Alex Thorlton wrote: > Hey everyone, > > Last week, while discussing possible fixes for some unexpected/unwanted behavior > from khugepaged (see: https://lkml.org/lkml/2014/10/8/515) several people > mentioned possibly changing changing khugepaged to work as a task_work function > instead of a kernel thread. This will give us finer grained control over the > page collapse scans, eliminate some unnecessary scans since tasks that are > relatively inactive will not be scanned often, and eliminate the unwanted > behavior described in the email thread I mentioned. > > This initial patch is fully functional, but there are quite a few areas that > will need to be polished up before it's ready to be considered for a merge. I > wanted to get this initial version out with some basic test results quickly, so > that people can give their opinions and let me know if there's anything they'd > like to see done differently (and there probably is :). I'll give details on > the code in the individual patches. > > I gathered some pretty rudimentary test data using a 48-thread NAMD simulation > pinned to a cpuset with 8 cpus and about 60g of memory. I'm checking to see if > I'm allowed to publish the input data so that others can replicate the test. In > the meantime, if somebody knows of a publicly available benchmark that stresses > khugepaged, that would be helpful. > > The only data point I gathered was the number of pages collapsed, sampled every > ten seconds, for the lifetime of the job. This one statistic gives a pretty > decent illustration of the difference in behavior between the two kernels, but I > intend to add some other counters to measure fully completed scans, failed > allocations, and possibly scans skipped due to timer constraints. > > The data for the standard kernel (with a very small patch to add the stat > counter that I used to the task_struct) is available here: > > http://oss.sgi.com/projects/memtests/pgcollapse/output-khpd > > This was a fairly recent kernel (last Tuesday). Commit ID: > 2d65a9f48fcdf7866aab6457bc707ca233e0c791. I'll send the patches I used for that > kernel as a reply to this message shortly. > > The output from the modified kernel is stored here: > > http://oss.sgi.com/projects/memtests/pgcollapse/output-pgcollapse > > The output is stored in a pretty dumb format (*really* wide). Best viewed in a > simple text editor with word wrap off, just fyi. > > Quick summary of what I found: Both kernels performed about the same when it > comes to overall runtime, my kernel was 22 seconds faster with a total runtime > of 4:13:07. Not a significant difference, but important to note that there was > no apparent performance degradation. The most interesting result is that my > kernel completed the majority of the necessary page collapses for this job in > 2:04, whereas the mainline kernel took 29:05 to get to the same point. > > Let me know what you think. Any suggestions are appreciated! > > - Alex > > Cc: Andrew Morton > Cc: Bob Liu > Cc: David Rientjes > Cc: Eric W. Biederman > Cc: Hugh Dickins > Cc: Ingo Molnar > Cc: Kees Cook > Cc: Kirill A. Shutemov > Cc: Mel Gorman > Cc: Oleg Nesterov > Cc: Peter Zijlstra > Cc: Rik van Riel > Cc: Thomas Gleixner > Cc: Vladimir Davydov > Cc: linux-kernel@vger.kernel.org > > Alex Thorlton (4): > Disable khugepaged thread > Add pgcollapse controls to task_struct > Convert khugepaged scan functions to work with task_work > Add /proc files to expose per-mm pgcollapse stats > > fs/proc/base.c | 23 +++++++ > include/linux/khugepaged.h | 10 ++- > include/linux/sched.h | 16 +++++ > kernel/fork.c | 7 ++ > kernel/sched/fair.c | 18 +++++ > mm/huge_memory.c | 162 +++++++++++++++------------------------------ > 6 files changed, 123 insertions(+), 113 deletions(-) > > -- > 1.7.12.4 > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/