Date: Wed, 18 Aug 2010 15:46:59 +0100
From: Chris Webb <chris@arachsys.com>
To: Wu Fengguang <fengguang.wu@intel.com>
Cc: Minchan Kim <minchan.kim@gmail.com>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
        Pekka Enberg <penberg@cs.helsinki.fi>
Subject: Re: Over-eager swapping
Message-ID: <20100818144655.GX2370@arachsys.com>
References: <AANLkTinjmZOOaq7FgwJOZ=UNGS8x8KtQWZg6nv7fqJMe@mail.gmail.com>
 <20100803042835.GA17377@localhost>
 <20100803214945.GA2326@arachsys.com>
 <20100804022148.GA5922@localhost>
 <AANLkTi=wRPXY9BTuoCe_sDCwhnRjmmwtAf_bjDKG3kXQ@mail.gmail.com>
 <20100804032400.GA14141@localhost>
 <20100804095811.GC2326@arachsys.com>
 <20100804114933.GA13527@localhost>
 <20100804120430.GB23551@arachsys.com>
 <20100818143801.GA9086@localhost>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20100818143801.GA9086@localhost>
User-Agent: Mutt/1.5.20 (2009-06-14)
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 1914
Lines: 49

Wu Fengguang <fengguang.wu@intel.com> writes:

> Did you enable any NUMA policy? That could start swapping even if
> there are lots of free pages in some nodes.

Hi. Thanks for the follow-up. We haven't done any configuration or tuning of
NUMA behaviour, but NUMA support is definitely compiled into the kernel:

  # zgrep NUMA /proc/config.gz 
  CONFIG_NUMA_IRQ_DESC=y
  CONFIG_NUMA=y
  CONFIG_K8_NUMA=y
  CONFIG_X86_64_ACPI_NUMA=y
  # CONFIG_NUMA_EMU is not set
  CONFIG_ACPI_NUMA=y
  # grep -i numa /var/log/dmesg.boot 
  NUMe: Allocated memnodemap from b000 - 1b540
  NUMA: Using 20 for the hash shift.

> Are your free pages equally distributed over the nodes? Or limited to
> some of the nodes? Try this command:
> 
>         grep MemFree /sys/devices/system/node/node*/meminfo

My worst-case machines current have swap completely turned off to make them
usable for clients, but I have one machine which is about 3GB into swap with
8GB of buffers and 3GB free. This shows

  # grep MemFree /sys/devices/system/node/node*/meminfo
  /sys/devices/system/node/node0/meminfo:Node 0 MemFree:          954500 kB
  /sys/devices/system/node/node1/meminfo:Node 1 MemFree:         2374528 kB

I could definitely imagine that one of the nodes could have dipped down to
zero in the past. I'll try enabling swap on one of our machines with the bad
problem late tonight and repeat the experiment. The node meminfo on this box
currently looks like

  # grep MemFree /sys/devices/system/node/node*/meminfo
  /sys/devices/system/node/node0/meminfo:Node 0 MemFree:           82732 kB
  /sys/devices/system/node/node1/meminfo:Node 1 MemFree:         1723896 kB

Best wishes,

Chris.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/