Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757069AbcLUSMj (ORCPT ); Wed, 21 Dec 2016 13:12:39 -0500 Received: from mail-it0-f66.google.com ([209.85.214.66]:35827 "EHLO mail-it0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752867AbcLUSMh (ORCPT ); Wed, 21 Dec 2016 13:12:37 -0500 MIME-Version: 1.0 In-Reply-To: <20161221223056.17c37dd6@roar.ozlabs.ibm.com> References: <20161219225826.F8CB356F@viggo.jf.intel.com> <20161221223056.17c37dd6@roar.ozlabs.ibm.com> From: Linus Torvalds Date: Wed, 21 Dec 2016 10:12:36 -0800 X-Google-Sender-Auth: _Gtend_iQVaA9GfeYYbnkejogdk Message-ID: Subject: Re: [RFC][PATCH] make global bitlock waitqueues per-node To: Nicholas Piggin Cc: Dave Hansen , Linux Kernel Mailing List , linux-mm , Andreas Gruenbacher , Bob Peterson , Mel Gorman , Peter Zijlstra , Andrew Lutomirski , Steven Whitehouse Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 1257 Lines: 30 On Wed, Dec 21, 2016 at 4:30 AM, Nicholas Piggin wrote: > > I've been doing a bit of testing, and I don't know why you're seeing > this. > > I don't think I've been able to trigger any actual page lock contention > so nothing gets put on the waitqueue to really bounce cache lines around > that I can see. The "test is the waitqueue is empty" is going to cause cache misses even if there is no contention. In fact, that's why I want the contention bit in the struct page - not because of any NUMA issues, but simply due to cache misses. And yes, with no contention the bit waiting should hopefully be able to cache things shared - which should make the bouncing much less - but there's going to be a shitload of false sharing with any actual IO, so you will get bouncing due to that. And then regular bouncing due simply to capacity misses (rather than the CPU's wanting exclusive access). With the contention bit in place, the only people actually looking at the wait queues are the ones doing IO. At which point false sharing is going to go down dramatically, but even if it were to happen it goes from a "big issue" to "who cares, the cachemiss is not noticeable compared to the IO, even with a fast SSD". Linus