Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753343AbbLCTlI (ORCPT ); Thu, 3 Dec 2015 14:41:08 -0500 Received: from mail-pf0-f174.google.com ([209.85.192.174]:36556 "EHLO mail-pf0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753186AbbLCTlG (ORCPT ); Thu, 3 Dec 2015 14:41:06 -0500 From: bsegall@google.com To: Waiman Long Cc: Ingo Molnar , Peter Zijlstra , linux-kernel@vger.kernel.org, Yuyang Du , Paul Turner , Morten Rasmussen , Scott J Norton , Douglas Hatch Subject: Re: [PATCH v2 2/3] sched/fair: Move hot load_avg into its own cacheline References: <1449081710-20185-1-git-send-email-Waiman.Long@hpe.com> <1449081710-20185-3-git-send-email-Waiman.Long@hpe.com> <5660974D.7050006@hpe.com> Date: Thu, 03 Dec 2015 11:41:03 -0800 In-Reply-To: <5660974D.7050006@hpe.com> (Waiman Long's message of "Thu, 03 Dec 2015 14:26:05 -0500") Message-ID: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 1930 Lines: 44 Waiman Long writes: > On 12/02/2015 03:02 PM, bsegall@google.com wrote: >> Waiman Long writes: >>> diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h >>> index efd3bfc..e679895 100644 >>> --- a/kernel/sched/sched.h >>> +++ b/kernel/sched/sched.h >>> @@ -248,7 +248,12 @@ struct task_group { >>> unsigned long shares; >>> >>> #ifdef CONFIG_SMP >>> - atomic_long_t load_avg; >>> + /* >>> + * load_avg can be heavily contended at clock tick time, so put >>> + * it in its own cacheline separated from the fields above which >>> + * will also be accessed at each tick. >>> + */ >>> + atomic_long_t load_avg ____cacheline_aligned; >>> #endif >>> #endif >> I suppose the question is if it would be better to just move this to >> wind up on a separate cacheline without the extra empty space, though it >> would likely be more fragile and unclear. > > I have been thinking about that too. The problem is anything that will be in the > same cacheline as load_avg and have to be accessed at clock click time will > cause the same contention problem. In the current layout, the fields after > load_avg are the rt stuff as well some list head structure and pointers. The rt > stuff should be kind of mutually exclusive of the CFS load_avg in term of usage. > The list head structure and pointers don't seem to be that frequently accessed. > So it is the right place to start a new cacheline boundary. > > Cheers, > Longman Yeah, this is a good place to start a new boundary, I was just saying you could probably remove the empty space by reordering fields, but that would be a less logical ordering in terms of programmer clarity. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/