Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S936436AbcJRBKJ convert rfc822-to-8bit (ORCPT ); Mon, 17 Oct 2016 21:10:09 -0400 Received: from mx2.suse.de ([195.135.220.15]:55825 "EHLO mx2.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S934873AbcJRBKA (ORCPT ); Mon, 17 Oct 2016 21:10:00 -0400 Date: Mon, 17 Oct 2016 18:09:49 -0700 From: Davidlohr Bueso To: Sebastian Andrzej Siewior Cc: Arnaldo Carvalho de Melo , Peter Zijlstra , Ingo Molnar , linux-kernel@vger.kernel.org, Davidlohr Bueso Subject: Re: [PATCH 1/2] perf bench futex: cache align the worer struct Message-ID: <20161018010949.GD29373@linux-80c1.suse> References: <20161016190803.3392-1-bigeasy@linutronix.de> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1; format=flowed Content-Disposition: inline Content-Transfer-Encoding: 8BIT In-Reply-To: <20161016190803.3392-1-bigeasy@linutronix.de> User-Agent: Mutt/1.5.24 (2015-08-30) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2490 Lines: 76 On Sun, 16 Oct 2016, Sebastian Andrzej Siewior wrote: >It popped up in perf testing that the worker consumes some amount of >CPU. It boils down to the increment of `ops` which causes cache line >bouncing between the individual threads. Are you referring to this? │ for (i = 0; i < nfutexes; i++, w->ops++) { │ be: add $0x1,%ebx 65.87 │ addq $0x1,0x18(%r12) (which is like 65% of 13% on my box with a default futex-hash run). Even better, could we get rid entirely of the ops increments and just use a local variable, then update the worker at the end of the function. The following makes 'perf' pretty much disappear in the profile. Thanks, Davidlohr ----8<-------------- From: Davidlohr Bueso Subject: [PATCH] perf/bench-futex: Avoid worker cacheline bouncing Sebastian noted that overhead for worker thread ops (throughput) accounting was producing 'perf' to appear in the profiles, consuming a non-trivial (ie 13%) amount of CPU. This is due to cacheline bouncing due to the increment of w->ops. We can easily fix this by just working on a local copy and updating the actual worker once done running, and ready to show the program summary. There is no danger of the worker being concurrent, so we can trust that no stale value is being seen by another thread. Reported-by: Sebastian Andrzej Siewior Signed-off-by: Davidlohr Bueso --- tools/perf/bench/futex-hash.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/tools/perf/bench/futex-hash.c b/tools/perf/bench/futex-hash.c index 8024cd5febd2..0b9583164da0 100644 --- a/tools/perf/bench/futex-hash.c +++ b/tools/perf/bench/futex-hash.c @@ -63,8 +63,10 @@ static const char * const bench_futex_hash_usage[] = { static void *workerfn(void *arg) { int ret; - unsigned int i; struct worker *w = (struct worker *) arg; + unsigned int i; + unsigned long ops = w->ops; /* avoid cacheline-bouncing */ + pthread_mutex_lock(&thread_lock); threads_starting--; @@ -74,7 +76,7 @@ static void *workerfn(void *arg) pthread_mutex_unlock(&thread_lock); do { - for (i = 0; i < nfutexes; i++, w->ops++) { + for (i = 0; i < nfutexes; i++, ops++) { /* * We want the futex calls to fail in order to stress * the hashing of uaddr and not measure other steps, @@ -88,6 +90,8 @@ static void *workerfn(void *arg) } } while (!done); + w->ops = ops; + return NULL; } -- 2.6.6