Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932142Ab1EKQOC (ORCPT ); Wed, 11 May 2011 12:14:02 -0400 Received: from cantor.suse.de ([195.135.220.2]:51684 "EHLO mx1.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755157Ab1EKPsV (ORCPT ); Wed, 11 May 2011 11:48:21 -0400 From: Mel Gorman To: Andrew Morton Cc: James Bottomley , Colin King , Raghavendra D Prabhu , Jan Kara , Chris Mason , Christoph Lameter , Pekka Enberg , Rik van Riel , Johannes Weiner , linux-fsdevel , linux-mm , linux-kernel , linux-ext4 , Mel Gorman Subject: [PATCH 0/3] Reduce impact to overall system of SLUB using high-order allocations Date: Wed, 11 May 2011 16:29:30 +0100 Message-Id: <1305127773-10570-1-git-send-email-mgorman@suse.de> X-Mailer: git-send-email 1.7.3.4 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3475 Lines: 71 Debian (and probably Ubuntu) have recently have changed to the default option of SLUB. There are a few reports of people experiencing hangs when copying large amounts of data with kswapd using a large amount of CPU. It appears this is down to SLUB using high orders by default and the page allocator and reclaim struggling to keep up. The following three patches reduce the cost of using those high orders. Patch 1 prevents kswapd waking up in response to SLUBs speculative use of high orders. This eliminates the hangs and while the system can still stall for long periods, it recovers. Patch 2 further reduces the cost by prevent SLUB entering direct compaction or reclaim paths on the grounds that falling back to order-0 should be cheaper. Patch 3 defaults SLUB to using order-0 on the grounds that the systems that heavily benefit from using high-order are also sized to fit in physical memory. On such systems, they should manually tune slub_max_order=3. My own data on this is not great. I haven't really been able to reproduce the same problem locally but a significant failing is that the tests weren't stressing X but I couldn't make meaningful comparisons by just randomly clicking on things (working on fixing this problem). The test case is simple. "download tar" wgets a large tar file and stores it locally. "unpack" is expanding it (15 times physical RAM in this case) and "delete source dirs" is the tarfile being deleted again. I also experimented with having the tar copied numerous times and into deeper directories to increase the size but the results were not particularly interesting so I left it as one tar. Test server, 4 CPU threads (AMD Phenom), x86_64, 2G of RAM, no X running - nowake largecopy-vanilla kswapd-v1r1 noexstep-v1r1 default0-v1r1 download tar 94 ( 0.00%) 94 ( 0.00%) 94 ( 0.00%) 93 ( 1.08%) unpack tar 521 ( 0.00%) 551 (-5.44%) 482 ( 8.09%) 488 ( 6.76%) delete source dirs 208 ( 0.00%) 218 (-4.59%) 194 ( 7.22%) 194 ( 7.22%) MMTests Statistics: duration User/Sys Time Running Test (seconds) 740.82 777.73 739.98 747.47 Total Elapsed Time (seconds) 1046.66 1273.91 962.47 936.17 Disabling kswapd alone hurts performance slightly even though testers report it fixes hangs. I would guess it's because SLUB callers are calling direct reclaim more frequently (I belatedly noticed that compaction was disabled so it's not a factor) but haven't confirmed it. However, preventing kswapd waking or entering direct reclaim and having SLUB falling back to order-0 performed noticeably faster. Just using order-0 in the first place was fastest of all. I tried running the same test on a test laptop but unfortunately due to a misconfiguration the results were lost. It would take a few hours to rerun so am posting without them. If the testers verify this series help and we agree the patches are appropriate, they should be considered a stable candidate for 2.6.38. Documentation/vm/slub.txt | 2 +- mm/page_alloc.c | 3 ++- mm/slub.c | 5 +++-- 3 files changed, 6 insertions(+), 4 deletions(-) -- 1.7.3.4 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/