Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754035AbaA0WXk (ORCPT ); Mon, 27 Jan 2014 17:23:40 -0500 Received: from mail-ob0-f169.google.com ([209.85.214.169]:33416 "EHLO mail-ob0-f169.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753484AbaA0WXi (ORCPT ); Mon, 27 Jan 2014 17:23:38 -0500 MIME-Version: 1.0 In-Reply-To: <1388646744-15608-1-git-send-email-minchan@kernel.org> References: <1388646744-15608-1-git-send-email-minchan@kernel.org> From: KOSAKI Motohiro Date: Mon, 27 Jan 2014 17:23:17 -0500 Message-ID: Subject: Re: [PATCH v10 00/16] Volatile Ranges v10 To: Minchan Kim Cc: "linux-mm@kvack.org" , LKML , Andrew Morton , Mel Gorman , Hugh Dickins , Dave Hansen , Rik van Riel , Michel Lespinasse , Johannes Weiner , John Stultz , Dhaval Giani , "H. Peter Anvin" , Android Kernel Team , Robert Love , Mel Gorman , Dmitry Adamushko , Dave Chinner , Neil Brown , Andrea Righi , Andrea Arcangeli , "Aneesh Kumar K.V" , Mike Hommey , Taras Glek , Jan Kara , Rob Clark , Jason Evans Content-Type: text/plain; charset=ISO-8859-1 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Minchan, On Thu, Jan 2, 2014 at 2:12 AM, Minchan Kim wrote: > Hey all, > > Happy New Year! > > I know it's bad timing to send this unfamiliar large patchset for > review but hope there are some guys with freshed-brain in new year > all over the world. :) > And most important thing is that before I dive into lots of testing, > I'd like to make an agreement on design issues and others > > o Syscall interface > o Not bind with vma split/merge logic to prevent mmap_sem cost and > o Not bind with vma split/merge logic to avoid vm_area_struct memory > footprint. > o Purging logic - when we trigger purging volatile pages to prevent > working set and stop to prevent too excessive purging of volatile > pages > o How to test > Currently, we have a patched jemalloc allocator by Jason's help > although it's not perfect and more rooms to be enhanced but IMO, > it's enough to prove vrange-anonymous. The problem is that > lack of benchmark for testing vrange-file side. I hope that > Mozilla folks can help. > > So its been a while since the last release of the volatile ranges > patches, again. I and John have been busy with other things. > Still, we have been slowly chipping away at issues and differences > trying to get a patchset that we both agree on. > > There's still a few issues, but we figured any further polishing of > the patch series in private would be unproductive and it would be much > better to send the patches out for review and comment and get some wider > opinions. > > You could get full patchset by git > > git clone -b vrange-v10-rc5 --single-branch git://git.kernel.org/pub/scm/linux/kernel/git/minchan/linux.git Brief comments. - You should provide jemalloc patch too. Otherwise we cannot understand what the your mesurement mean. - Your number only claimed the effectiveness anon vrange, but not file vrange. - Still, Nobody likes file vrange. At least nobody said explicitly on the list. I don't ack file vrange part until I fully convinced Pros/Cons. You need to persuade other MM guys if you really think anon vrange is not sufficient. (Maybe LSF is the best place) - I wrote you need to put a mesurement current implementation vs VMA-based implementation at several previous iteration. Because You claimed fast, but no number and you haven't yet. I guess the reason is you don't have any access to large machine. If so, I'll offer it. Plz collaborate with us. Unfortunately, I'm very busy and I didn't have a chance to review your latest patch yet. But I'll finish it until mm summit. And, I'll show you guys how much this patch improve glibc malloc too. I and glibc folks agreed we push vrange into glibc malloc. https://sourceware.org/ml/libc-alpha/2013-12/msg00343.html Even though, I still dislike some aspect of this patch. I'd like to discuss and make better design decision with you. Thanks. > > In v10, there are some notable changes following as > > Whats new in v10: > * Fix several bugs and build break > * Add shmem_purge_page to correct purging shmem/tmpfs > * Replace slab shrinker with direct hooked reclaim path > * Optimize pte scanning by caching previous place > * Reorder patch and tidy up Cc-list > * Rebased on v3.12 > * Add vrange-anon test with jemalloc in Dhaval's test suite > - https://github.com/volatile-ranges-test/vranges-test > so, you could test any application with vrange-patched jemalloc by > LD_PRELOAD but please keep in mind that it's just a prototype to > prove vrange syscall concept so it has more rooms to optimize. > So, please do not compare it with another allocator. > > Whats new in v9: > * Updated to v3.11 > * Added vrange purging logic to purge anonymous pages on > swapless systems > * Added logic to allocate the vroot structure dynamically > to avoid added overhead to mm and address_space structures > * Lots of minor tweaks, changes and cleanups > > Still TODO: > * Sort out better solution for clearing volatility on new mmaps > - Minchan has a different approach here > * Agreement of systemcall interface > * Better discarding trigger policy to prevent working set evction > * Review, Review, Review.. Comment. > * A ton of test > > Feedback or thoughts here would be particularly helpful! > > Also, thanks to Dhaval for his maintaining and vastly improving > the volatile ranges test suite, which can be found here: > [1] https://github.com/volatile-ranges-test/vranges-test > > These patches can also be pulled from git here: > git://git.linaro.org/people/jstultz/android-dev.git dev/vrange-v9 > > We'd really welcome any feedback and comments on the patch series. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/