Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932145AbcCaJ14 (ORCPT ); Thu, 31 Mar 2016 05:27:56 -0400 Received: from terminus.zytor.com ([198.137.202.10]:38258 "EHLO terminus.zytor.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751953AbcCaJ1t (ORCPT ); Thu, 31 Mar 2016 05:27:49 -0400 Date: Thu, 31 Mar 2016 02:26:52 -0700 From: tip-bot for Wang Nan Message-ID: Cc: namhyung@kernel.org, pi3orama@163.com, vincent.weaver@maine.edu, peterz@infradead.org, hpa@zytor.com, torvalds@linux-foundation.org, linux-kernel@vger.kernel.org, hekuang@huawei.com, jolsa@kernel.org, brendan.d.gregg@gmail.com, masami.hiramatsu.pt@hitachi.com, mingo@kernel.org, alexander.shishkin@linux.intel.com, jolsa@redhat.com, ast@kernel.org, acme@redhat.com, tglx@linutronix.de, eranian@google.com, wangnan0@huawei.com, lizefan@huawei.com Reply-To: alexander.shishkin@linux.intel.com, mingo@kernel.org, jolsa@redhat.com, ast@kernel.org, acme@redhat.com, lizefan@huawei.com, eranian@google.com, tglx@linutronix.de, wangnan0@huawei.com, namhyung@kernel.org, pi3orama@163.com, peterz@infradead.org, hpa@zytor.com, torvalds@linux-foundation.org, vincent.weaver@maine.edu, masami.hiramatsu.pt@hitachi.com, linux-kernel@vger.kernel.org, jolsa@kernel.org, brendan.d.gregg@gmail.com, hekuang@huawei.com In-Reply-To: <1459147292-239310-4-git-send-email-wangnan0@huawei.com> References: <1459147292-239310-4-git-send-email-wangnan0@huawei.com> To: linux-tip-commits@vger.kernel.org Subject: [tip:perf/core] perf/ring_buffer: Prepare writing into the ring-buffer from the end Git-Commit-ID: d1b26c70246bc72922ae61d9f972d5c2588409e7 X-Mailer: tip-git-log-daemon Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8 Content-Disposition: inline Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 5381 Lines: 162 Commit-ID: d1b26c70246bc72922ae61d9f972d5c2588409e7 Gitweb: http://git.kernel.org/tip/d1b26c70246bc72922ae61d9f972d5c2588409e7 Author: Wang Nan AuthorDate: Mon, 28 Mar 2016 06:41:31 +0000 Committer: Ingo Molnar CommitDate: Thu, 31 Mar 2016 10:30:49 +0200 perf/ring_buffer: Prepare writing into the ring-buffer from the end Convert perf_output_begin() to __perf_output_begin() and make the later function able to write records from the end of the ring-buffer. Following commits will utilize the 'backward' flag. This is the core patch to support writing to the ring-buffer backwards, which will be introduced by upcoming patches to support reading from overwritable ring-buffers. In theory, this patch should not introduce any extra performance overhead since we use always_inline, but it does not hurt to double check that assumption: When CONFIG_OPTIMIZE_INLINING is disabled, the output object is nearly identical to original one. See: http://lkml.kernel.org/g/56F52E83.70409@huawei.com When CONFIG_OPTIMIZE_INLINING is enabled, the resuling object file becomes smaller: $ size kernel/events/ring_buffer.o* text data bss dec hex filename 4641 4 8 4653 122d kernel/events/ring_buffer.o.old 4545 4 8 4557 11cd kernel/events/ring_buffer.o.new Performance testing results: Calling 3000000 times of 'close(-1)', use gettimeofday() to check duration. Use 'perf record -o /dev/null -e raw_syscalls:*' to capture system calls. In ns. Testing environment: CPU : Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz Kernel : v4.5.0 MEAN STDVAR BASE 800214.950 2853.083 PRE 2253846.700 9997.014 POST 2257495.540 8516.293 Where 'BASE' is pure performance without capturing. 'PRE' is test result of pure 'v4.5.0' kernel. 'POST' is test result after this patch. Considering the stdvar, this patch doesn't hurt performance, within noise margin. For testing details, see: http://lkml.kernel.org/g/56F89DCD.1040202@huawei.com Signed-off-by: Wang Nan Signed-off-by: Peter Zijlstra (Intel) Cc: Cc: Alexander Shishkin Cc: Alexei Starovoitov Cc: Arnaldo Carvalho de Melo Cc: Brendan Gregg Cc: He Kuang Cc: Jiri Olsa Cc: Jiri Olsa Cc: Linus Torvalds Cc: Masami Hiramatsu Cc: Namhyung Kim Cc: Peter Zijlstra Cc: Stephane Eranian Cc: Thomas Gleixner Cc: Vince Weaver Cc: Zefan Li Link: http://lkml.kernel.org/r/1459147292-239310-4-git-send-email-wangnan0@huawei.com Signed-off-by: Ingo Molnar --- kernel/events/ring_buffer.c | 42 ++++++++++++++++++++++++++++++++++++------ 1 file changed, 36 insertions(+), 6 deletions(-) diff --git a/kernel/events/ring_buffer.c b/kernel/events/ring_buffer.c index 72d8127..60be55a 100644 --- a/kernel/events/ring_buffer.c +++ b/kernel/events/ring_buffer.c @@ -102,8 +102,21 @@ out: preempt_enable(); } -int perf_output_begin(struct perf_output_handle *handle, - struct perf_event *event, unsigned int size) +static bool __always_inline +ring_buffer_has_space(unsigned long head, unsigned long tail, + unsigned long data_size, unsigned int size, + bool backward) +{ + if (!backward) + return CIRC_SPACE(head, tail, data_size) >= size; + else + return CIRC_SPACE(tail, head, data_size) >= size; +} + +static int __always_inline +__perf_output_begin(struct perf_output_handle *handle, + struct perf_event *event, unsigned int size, + bool backward) { struct ring_buffer *rb; unsigned long tail, offset, head; @@ -146,9 +159,12 @@ int perf_output_begin(struct perf_output_handle *handle, do { tail = READ_ONCE(rb->user_page->data_tail); offset = head = local_read(&rb->head); - if (!rb->overwrite && - unlikely(CIRC_SPACE(head, tail, perf_data_size(rb)) < size)) - goto fail; + if (!rb->overwrite) { + if (unlikely(!ring_buffer_has_space(head, tail, + perf_data_size(rb), + size, backward))) + goto fail; + } /* * The above forms a control dependency barrier separating the @@ -162,9 +178,17 @@ int perf_output_begin(struct perf_output_handle *handle, * See perf_output_put_handle(). */ - head += size; + if (!backward) + head += size; + else + head -= size; } while (local_cmpxchg(&rb->head, offset, head) != offset); + if (backward) { + offset = head; + head = (u64)(-head); + } + /* * We rely on the implied barrier() by local_cmpxchg() to ensure * none of the data stores below can be lifted up by the compiler. @@ -206,6 +230,12 @@ out: return -ENOSPC; } +int perf_output_begin(struct perf_output_handle *handle, + struct perf_event *event, unsigned int size) +{ + return __perf_output_begin(handle, event, size, false); +} + unsigned int perf_output_copy(struct perf_output_handle *handle, const void *buf, unsigned int len) {