Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754913AbdIGJ0Z (ORCPT ); Thu, 7 Sep 2017 05:26:25 -0400 Received: from bombadil.infradead.org ([65.50.211.133]:37908 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754336AbdIGJ0Y (ORCPT ); Thu, 7 Sep 2017 05:26:24 -0400 Date: Thu, 7 Sep 2017 11:26:16 +0200 From: Peter Zijlstra To: Mike Galbraith Cc: Andy Lutomirski , Andy Lutomirski , Ingo Molnar , "linux-kernel@vger.kernel.org" , Tejun Heo , "Rafael J. Wysocki" , Thomas Gleixner Subject: Re: [PATCH] sched/cpuset/pm: Fix cpuset vs suspend-resume Message-ID: <20170907092616.thsuyqklit4463wj@hirez.programming.kicks-ass.net> References: <20170906082520.xgvo3hewje7jvdyo@hirez.programming.kicks-ass.net> <877A43A3-AC8F-4D7C-88E4-8E3D36B1DAFA@amacapital.net> <20170906090333.7v627vdyvhe3x2cs@hirez.programming.kicks-ass.net> <20170906161422.kvqe5y6b3ng6ouyu@hirez.programming.kicks-ass.net> <1504764929.6355.5.camel@gmx.de> <20170907091338.orwxrqkbfkki3c24@hirez.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170907091338.orwxrqkbfkki3c24@hirez.programming.kicks-ass.net> User-Agent: NeoMutt/20170609 (1.8.3) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2071 Lines: 48 On Thu, Sep 07, 2017 at 11:13:38AM +0200, Peter Zijlstra wrote: > Subject: sched/cpuset/pm: Fix cpuset vs suspend-resume > > Cpusets vs suspend-resume is _completely_ broken. And it got noticed > because it now resulted in non-cpuset usage breaking too. > > On suspend cpuset_cpu_inactive() doesn't call into > cpuset_update_active_cpus() because it doesn't want to move tasks about, > there is no need, all tasks are frozen and won't run again until after > we've resumed everything. > > But this means that when we finally do call into > cpuset_update_active_cpus() after resuming the last frozen cpu in > cpuset_cpu_active(), the top_cpuset will not have any difference with > the cpu_active_mask and this it will not in fact do _anything_. > > So the cpuset configuration will not be restored. This was largely > hidden because we would unconditionally create identity domains and > mobile users would not in fact use cpusets much. And servers what do use > cpusets tend to not suspend-resume much. > > An addition problem is that we'd not in fact wait for the cpuset work to > finish before resuming the tasks, allowing spurious migrations outside > of the specified domains. > > Fix the rebuild by introducing cpuset_force_rebuild() and fix the > ordering with cpuset_wait_for_hotplug(). > > Cc: tj@kernel.org > Cc: rjw@rjwysocki.net > Cc: efault@gmx.de > Reported-by: Andy Lutomirski > Signed-off-by: Peter Zijlstra (Intel) TJ, I _think_ it was commit: deb7aa308ea2 ("cpuset: reorganize CPU / memory hotplug handling") That wrecked things, but there's been so much changes in this area it is really hard to tell. Note how before that commit it would unconditionally rebuild the domains, and you 'optimized' that ;-) That commit also introduced the work to do the async rebuild and failed to do that flush on resume. In any case, I think we should put a fixes tag on this commit such that it gets picked up into stable kernels. Not sure anybody will try and backport it into 4 year old kernels, but who knows.