Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753566AbbD2GjQ (ORCPT ); Wed, 29 Apr 2015 02:39:16 -0400 Received: from mail113-249.mail.alibaba.com ([205.204.113.249]:55175 "EHLO us-alimail-mta1.hst.scl.en.alidc.net" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1753392AbbD2GjP (ORCPT ); Wed, 29 Apr 2015 02:39:15 -0400 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R231e4;FP=0|-1|-1|-1|0|-1|-1|-1;HT=r41f05025;MF=hillf.zj@alibaba-inc.com;PH=DS;RN=2;RT=2;SR=0; Reply-To: "Hillf Danton" From: "Hillf Danton" To: "'Jason Low'" Cc: "linux-kernel" References: <016401d08246$0917f130$1b47d390$@alibaba-inc.com> In-Reply-To: <016401d08246$0917f130$1b47d390$@alibaba-inc.com> Subject: Re: [PATCH v2 3/5] sched, timer: Use atomics in thread_group_cputimer to improve scalability Date: Wed, 29 Apr 2015 14:38:26 +0800 Message-ID: <016501d08247$197dfb40$4c79f1c0$@alibaba-inc.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQFe3fowMC7FR8RfBuXIK6SgwkQlh55G0CQg Content-Language: zh-cn Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 1598 Lines: 32 > > While running a database workload, we found a scalability issue with itimers. > > Much of the problem was caused by the thread_group_cputimer spinlock. > Each time we account for group system/user time, we need to obtain a > thread_group_cputimer's spinlock to update the timers. On larger systems > (such as a 16 socket machine), this caused more than 30% of total time > spent trying to obtain this kernel lock to update these group timer stats. > FYI, another cache line problem encountered by Mel, a368ab67aa mm: move zone lock to a different cache line than order-0 free page lists > This patch converts the timers to 64 bit atomic variables and use > atomic add to update them without a lock. With this patch, the percent > of total time spent updating thread group cputimer timers was reduced > from 30% down to less than 1%. > > Note: On 32 bit systems using the generic 64 bit atomics, this causes > sample_group_cputimer() to take locks 3 times instead of just 1 time. > However, we tested this patch on a 32 bit system ARM system using the > generic atomics and did not find the overhead to be much of an issue. > An explanation for why this isn't an issue is that 32 bit systems usually > have small numbers of CPUs, and cacheline contention from extra spinlocks > called periodically is not really apparent on smaller systems. > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/