Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752189AbdGDMKw (ORCPT ); Tue, 4 Jul 2017 08:10:52 -0400 Received: from mx2.suse.de ([195.135.220.15]:54110 "EHLO mx1.suse.de" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1752038AbdGDMKv (ORCPT ); Tue, 4 Jul 2017 08:10:51 -0400 Subject: Re: [patch V2 2/2] mm/memory-hotplug: Switch locking to a percpu rwsem To: Thomas Gleixner , LKML Cc: linux-mm@kvack.org, Andrey Ryabinin , Michal Hocko , Andrew Morton , Vladimir Davydov , Peter Zijlstra References: <20170704093232.995040438@linutronix.de> <20170704093421.506836322@linutronix.de> From: Vlastimil Babka Message-ID: Date: Tue, 4 Jul 2017 14:10:49 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.0 MIME-Version: 1.0 In-Reply-To: <20170704093421.506836322@linutronix.de> Content-Type: text/plain; charset=iso-8859-15 Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 1852 Lines: 46 On 07/04/2017 11:32 AM, Thomas Gleixner wrote: > Andrey reported a potential deadlock with the memory hotplug lock and the > cpu hotplug lock. > > The reason is that memory hotplug takes the memory hotplug lock and then > calls stop_machine() which calls get_online_cpus(). That's the reverse lock > order to get_online_cpus(); get_online_mems(); in mm/slub_common.c > > The problem has been there forever. The reason why this was never reported > is that the cpu hotplug locking had this homebrewn recursive reader writer > semaphore construct which due to the recursion evaded the full lock dep > coverage. The memory hotplug code copied that construct verbatim and > therefor has similar issues. > > Three steps to fix this: > > 1) Convert the memory hotplug locking to a per cpu rwsem so the potential > issues get reported proper by lockdep. > > 2) Lock the online cpus in mem_hotplug_begin() before taking the memory > hotplug rwsem and use stop_machine_cpuslocked() in the page_alloc code > and use to avoid recursive locking. ^ s/and use // ? > > 3) The cpu hotpluck locking in #2 causes a recursive locking of the cpu > hotplug lock via __offline_pages() -> lru_add_drain_all(). Solve this by > invoking lru_add_drain_all_cpuslocked() instead. > > Reported-by: Andrey Ryabinin > Signed-off-by: Thomas Gleixner > Cc: Michal Hocko > Cc: linux-mm@kvack.org > Cc: Andrew Morton > Cc: Vlastimil Babka > Cc: Vladimir Davydov Acked-by: Vlastimil Babka > --- > mm/memory_hotplug.c | 89 ++++++++-------------------------------------------- > mm/page_alloc.c | 2 - > 2 files changed, 16 insertions(+), 75 deletions(-) Nice! Glad to see the crazy code go.