Message-ID: <565D1B63.3070908@hpe.com>
Date: Mon, 30 Nov 2015 23:00:35 -0500
From: Waiman Long <waiman.long@hpe.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0.12) Gecko/20130109 Thunderbird/10.0.12
MIME-Version: 1.0
To: Peter Zijlstra <peterz@infradead.org>
CC: Ingo Molnar <mingo@redhat.com>, linux-kernel@vger.kernel.org,
        Scott J Norton <scott.norton@hpe.com>,
        Douglas Hatch <doug.hatch@hpe.com>, Paul Turner <pjt@google.com>,
        Ben Segall <bsegall@google.com>,
        Morten Rasmussen <morten.rasmussen@arm.com>,
        Yuyang Du <yuyang.du@intel.com>
Subject: Re: [RFC PATCH 3/3] sched/fair: Use different cachelines for readers
 and writers of load_avg
References: <1448478580-26467-1-git-send-email-Waiman.Long@hpe.com> <1448478580-26467-4-git-send-email-Waiman.Long@hpe.com> <20151130102240.GH17308@twins.programming.kicks-ass.net> <565C9FDC.9020603@hpe.com> <20151130222936.GF3816@twins.programming.kicks-ass.net>
In-Reply-To: <20151130222936.GF3816@twins.programming.kicks-ass.net>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 1620
Lines: 29

On 11/30/2015 05:29 PM, Peter Zijlstra wrote:
> On Mon, Nov 30, 2015 at 02:13:32PM -0500, Waiman Long wrote:
>>> This would only work if the structure itself is allocated with cacheline
>>> alignment, and looking at sched_create_group(), we use a plain kzalloc()
>>> for this, which doesn't guarantee any sort of alignment beyond machine
>>> word size IIRC.
>> With a RHEL 6 derived .config file, the size of the task_group structure was
>> 460 bytes on a 32-bit x86 kernel. Adding a ____cacheline_aligned tag
>> increase the size to 512 bytes. So it did make the structure a multiple of
>> the cacheline size. With both slub and slab, the allocated task group
>> pointers from kzalloc() in sched_create_group() were all multiples of 0x200.
>> So they were properly aligned for the ____cacheline_aligned tag to work.
> Not sure we should rely on sl*b doing the right thing here.
> KMALLOC_MIN_ALIGN is explicitly set to sizeof(long long). If you want
> explicit alignment, one should use KMEM_CACHE().

I think the current kernel use power-of-2 kmemcaches to satisfy kalloc() 
requests except when the size is less than or equal to 192 where there 
are some non-power-of-2 kmemcaches available. Given that the task_group 
structure is large enough with FAIR_GROUP_SCHED enabled, we shouldn't 
hit the case that the allocated buffer is not cacheline aligned.

Cheers,
Longman
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/