Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752252AbbEPTjS (ORCPT ); Sat, 16 May 2015 15:39:18 -0400 Received: from mail-qg0-f52.google.com ([209.85.192.52]:34621 "EHLO mail-qg0-f52.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750832AbbEPTjQ (ORCPT ); Sat, 16 May 2015 15:39:16 -0400 Message-ID: <55579CE0.5060801@gmail.com> Date: Sat, 16 May 2015 15:39:12 -0400 From: Sasha Levin User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: Frederic Weisbecker , Ingo Molnar CC: LKML , Chris Metcalf , "Rafael J . Wysocki" , Peter Zijlstra , Mike Galbraith , Dave Jones , Thomas Gleixner , Oleg Nesterov , "Paul E . McKenney" , Ingo Molnar , Rik van Riel , Martin Schwidefsky Subject: Re: [PATCH 4/4] nohz: Set isolcpus when nohz_full is set References: <1430928266-24888-1-git-send-email-fweisbec@gmail.com> <1430928266-24888-5-git-send-email-fweisbec@gmail.com> In-Reply-To: <1430928266-24888-5-git-send-email-fweisbec@gmail.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2117 Lines: 50 On 05/06/2015 12:04 PM, Frederic Weisbecker wrote: > From: Chris Metcalf > > nohz_full is only useful with isolcpus also set, since otherwise the > scheduler has to run periodically to try to determine whether to steal > work from other cores. > > Accordingly, when booting with nohz_full=xxx on the command line, we > should act as if isolcpus=xxx was also set, and set (or extend) the > isolcpus set to include the nohz_full cpus. > > Acked-by: Mike Galbraith ["thumbs up!"] > Acked-by: Rik van Riel > Acked-by: Peter Zijlstra (Intel) > Signed-off-by: Chris Metcalf > Cc: Peter Zijlstra (Intel) > Cc: Paul E. McKenney > Cc: Rafael J. Wysocki > Cc: Martin Schwidefsky > Cc: Mike Galbraith > Cc: Ingo Molnar > Cc: Rik van Riel > Signed-off-by: Frederic Weisbecker Hi folks, I've noticed a regression in my testing a few days ago and bisected it down to this patch. I was seeing frequent soft lockups/RCU lockups and the load of the testing VMs would go beyond 400-500 (on 32 VCPU guests) - note I'm booting them with nohz_full=1-27. This patch sort of explains the behaviour I was seeing now: most of the cores are no longer being used by the scheduler, and the remaining cores can't deal with the load imposed on them which results in "lockups" which are really just the CPUs being unable to keep up. I always thought that nohz_full without isolcpus meant that the cores would be available to the scheduler, but it won't interfere if there is one task running on them. It seems that this patch changed that behaviour. Did I misunderstand that? Thanks, Sasha -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/