Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753433AbcCYPEX (ORCPT ); Fri, 25 Mar 2016 11:04:23 -0400 Received: from mail-wm0-f41.google.com ([74.125.82.41]:38890 "EHLO mail-wm0-f41.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752974AbcCYPEV (ORCPT ); Fri, 25 Mar 2016 11:04:21 -0400 Date: Fri, 25 Mar 2016 18:04:17 +0300 From: "Kirill A. Shutemov" To: Hugh Dickins Cc: "Kirill A. Shutemov" , Andrea Arcangeli , Andrew Morton , Dave Hansen , Vlastimil Babka , Christoph Lameter , Naoya Horiguchi , Jerome Marchand , Yang Shi , Sasha Levin , Ning Qu , linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Subject: Re: [PATCHv4 00/25] THP-enabled tmpfs/shmem Message-ID: <20160325150417.GA1851@node.shutemov.name> References: <1457737157-38573-1-git-send-email-kirill.shutemov@linux.intel.com> <20160324091727.GA26796@node.shutemov.name> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23.1 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2243 Lines: 63 On Thu, Mar 24, 2016 at 12:08:55PM -0700, Hugh Dickins wrote: > On Thu, 24 Mar 2016, Kirill A. Shutemov wrote: > > On Wed, Mar 23, 2016 at 01:09:05PM -0700, Hugh Dickins wrote: > > > The small files thing formed my first impression. My second > > > impression was similar, when I tried mmap(NULL, size_of_RAM, > > > PROT_READ|PROT_WRITE, MAP_ANONYMOUS|MAP_SHARED, -1, 0) and > > > cycled around the arena touching all the pages (which of > > > course has to push a little into swap): that soon OOMed. > > > > > > But there I think you probably just have some minor bug to be fixed: > > > I spent a little while trying to debug it, but then decided I'd > > > better get back to writing to you. I didn't really understand what > > > I was seeing, but when I hacked some stats into shrink_page_list(), > > > converting !is_page_cache_freeable(page) to page_cache_references(page) > > > to return the difference instead of the bool, a large proportion of > > > huge tmpfs pages seemed to have count 1 too high to be freeable at > > > that point (and one huge tmpfs page had a count of 3477). > > > > I'll reply to your other points later, but first I wanted to address this > > obvious bug. > > Thanks. That works better, but is not yet right: memory isn't freed > as it should be, so when I exit then try to run a second time, the > mmap() just gets ENOMEM (with /proc/sys/vm/overcommit_memory 0): > MemFree is low. No rush to fix, I've other stuff to do. > > I don't get as far as that on the laptop, since the first run is OOM > killed while swapping; but I can't vouch for the OOM-kill-correctness > of the base tree I'm using, and this laptop has a history of OOMing > rather too easily if all's not right. Hm. I don't see the issue. I tried to reproduce it in my VM with following script: #!/bin/sh -efu swapon -a ram="$(grep MemTotal /proc/meminfo | sed 's,[^0-9\]\+,,; s, kB,k,')" usemem -w -f /dev/zero "$ram" swapoff -a swapon -a usemem -w -f /dev/zero "$ram" cat /proc/meminfo grep thp /proc/vmstat ----- usemem is a tool from this archive: http://www.spinics.net/lists/linux-mm/attachments/gtarazbJaHPaAT.gtar It works fine even if would double size of mapping. Do you have a reproducer? -- Kirill A. Shutemov