Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753332AbcD1I73 (ORCPT ); Thu, 28 Apr 2016 04:59:29 -0400 Received: from mx2.suse.de ([195.135.220.15]:32940 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752430AbcD1I70 (ORCPT ); Thu, 28 Apr 2016 04:59:26 -0400 Subject: Re: [PATCH 14/14] mm, oom, compaction: prevent from should_compact_retry looping for ever for costly orders To: Michal Hocko , Andrew Morton References: <1461181647-8039-1-git-send-email-mhocko@kernel.org> <1461181647-8039-15-git-send-email-mhocko@kernel.org> Cc: Linus Torvalds , Johannes Weiner , Mel Gorman , David Rientjes , Tetsuo Handa , Joonsoo Kim , Hillf Danton , linux-mm@kvack.org, LKML , Michal Hocko From: Vlastimil Babka Message-ID: <5721D0EA.3020205@suse.cz> Date: Thu, 28 Apr 2016 10:59:22 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.7.2 MIME-Version: 1.0 In-Reply-To: <1461181647-8039-15-git-send-email-mhocko@kernel.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3168 Lines: 58 On 04/20/2016 09:47 PM, Michal Hocko wrote: > From: Michal Hocko > > "mm: consider compaction feedback also for costly allocation" has > removed the upper bound for the reclaim/compaction retries based on the > number of reclaimed pages for costly orders. While this is desirable > the patch did miss a mis interaction between reclaim, compaction and the > retry logic. Hmm perhaps reversing the order of patches 13 and 14 would be a bit safer wrt future bisections then? Add compaction_zonelist_suitable() first with the reasoning, and then immediately use it in the other patch. > The direct reclaim tries to get zones over min watermark > while compaction backs off and returns COMPACT_SKIPPED when all zones > are below low watermark + 1< to OOM then __compaction_suitable can keep returning COMPACT_SKIPPED a > high order request (e.g. hugetlb order-9) while the reclaim is not able > to release enough pages to get us over low watermark. The reclaim is > still able to make some progress (usually trashing over few remaining > pages) so we are not able to break out from the loop. > > I have seen this happening with the same test described in "mm: consider > compaction feedback also for costly allocation" on a swapless system. > The original problem got resolved by "vmscan: consider classzone_idx in > compaction_ready" but it shows how things might go wrong when we > approach the oom event horizont. > > The reason why compaction requires being over low rather than min > watermark is not clear to me. This check was there essentially since > 56de7263fcf3 ("mm: compaction: direct compact when a high-order > allocation fails"). It is clearly an implementation detail though and we > shouldn't pull it into the generic retry logic while we should be able > to cope with such eventuality. The only place in should_compact_retry > where we retry without any upper bound is for compaction_withdrawn() > case. > > Introduce compaction_zonelist_suitable function which checks the given > zonelist and returns true only if there is at least one zone which would > would unblock __compaction_suitable if more memory got reclaimed. In > this implementation it checks __compaction_suitable with NR_FREE_PAGES > plus part of the reclaimable memory as the target for the watermark check. > The reclaimable memory is reduced linearly by the allocation order. The > idea is that we do not want to reclaim all the remaining memory for a > single allocation request just unblock __compaction_suitable which > doesn't guarantee we will make a further progress. > > The new helper is then used if compaction_withdrawn() feedback was > provided so we do not retry if there is no outlook for a further > progress. !costly requests shouldn't be affected much - e.g. order-2 > pages would require to have at least 64kB on the reclaimable LRUs while > order-9 would need at least 32M which should be enough to not lock up. > > [vbabka@suse.cz: fix classzone_idx vs. high_zoneidx usage in > compaction_zonelist_suitable] > Signed-off-by: Michal Hocko Acked-by: Vlastimil Babka