Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752868AbbBWPBP (ORCPT ); Mon, 23 Feb 2015 10:01:15 -0500 Received: from mga11.intel.com ([192.55.52.93]:19513 "EHLO mga11.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752716AbbBWPAP (ORCPT ); Mon, 23 Feb 2015 10:00:15 -0500 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.09,631,1418112000"; d="scan'208";a="689460030" From: Kan Liang To: a.p.zijlstra@chello.nl, linux-kernel@vger.kernel.org Cc: mingo@kernel.org, acme@infradead.org, eranian@google.com, andi@firstfloor.org, Kan Liang Subject: [PATCH V5 4/6] perf, x86: handle multiple records in PEBS buffer Date: Mon, 23 Feb 2015 09:25:54 -0500 Message-Id: <1424701556-28270-5-git-send-email-kan.liang@intel.com> X-Mailer: git-send-email 1.8.3.2 In-Reply-To: <1424701556-28270-1-git-send-email-kan.liang@intel.com> References: <1424701556-28270-1-git-send-email-kan.liang@intel.com> Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 6571 Lines: 207 From: Yan, Zheng When PEBS interrupt threshold is larger than one, the PEBS buffer may include multiple records for each PEBS event. This patch makes the code first count how many records each PEBS event has, then output the samples in batch. One corner case needs to mention is that the PEBS hardware doesn't deal well with collisions, when PEBS events happen near to each other. The records for the events can be collapsed into a single one, and it's not possible to reconstruct all events that caused the PEBS record, However in practice collisions are extremely rare, as long as different events are used. The periods are typically very large, so any collision is unlikely. When collision happens, we drop the PEBS record. Here are some numbers about collisions. Four frequently occurring events (cycles:p,instructions:p,branches:p,mem-stores:p) are tested Test events which are sampled together collision rate cycles:p,instructions:p 0.25% cycles:p,instructions:p,branches:p 0.30% cycles:p,instructions:p,branches:p,mem-stores:p 0.35% cycles:p,cycles:p 98.52% collisions are extremely rare as long as different events are used. The only way you can get a lot of collision is when you count the same thing multiple times. But it is not a useful configuration. Signed-off-by: Yan, Zheng Signed-off-by: Kan Liang --- arch/x86/kernel/cpu/perf_event_intel_ds.c | 114 ++++++++++++++++++++---------- 1 file changed, 77 insertions(+), 37 deletions(-) diff --git a/arch/x86/kernel/cpu/perf_event_intel_ds.c b/arch/x86/kernel/cpu/perf_event_intel_ds.c index 16fdb18..cf74a52 100644 --- a/arch/x86/kernel/cpu/perf_event_intel_ds.c +++ b/arch/x86/kernel/cpu/perf_event_intel_ds.c @@ -984,18 +984,52 @@ static void setup_pebs_sample_data(struct perf_event *event, } static void __intel_pmu_pebs_event(struct perf_event *event, - struct pt_regs *iregs, void *__pebs) + struct pt_regs *iregs, + void *at, void *top, int count) { + struct perf_output_handle handle; + struct perf_event_header header; struct perf_sample_data data; struct pt_regs regs; - if (!intel_pmu_save_and_restart(event)) + if (!intel_pmu_save_and_restart(event) && + !(event->hw.flags & PERF_X86_EVENT_AUTO_RELOAD)) return; - setup_pebs_sample_data(event, iregs, __pebs, &data, ®s); + setup_pebs_sample_data(event, iregs, at, &data, ®s); - if (perf_event_overflow(event, &data, ®s)) + if (perf_event_overflow(event, &data, ®s)) { x86_pmu_stop(event, 0); + return; + } + + if (count <= 1) + return; + + at += x86_pmu.pebs_record_size; + count--; + + perf_sample_data_init(&data, 0, event->hw.last_period); + perf_prepare_sample(&header, &data, event, ®s); + + if (perf_output_begin(&handle, event, header.size * count)) + return; + + for (; at < top; at += x86_pmu.pebs_record_size) { + struct pebs_record_nhm *p = at; + + if (p->status != (1 << event->hw.idx)) + continue; + + setup_pebs_sample_data(event, iregs, at, &data, ®s); + perf_output_sample(&handle, &header, &data, event); + + count--; + if (count == 0) + break; + } + + perf_output_end(&handle); } static void intel_pmu_drain_pebs_core(struct pt_regs *iregs) @@ -1036,61 +1070,67 @@ static void intel_pmu_drain_pebs_core(struct pt_regs *iregs) WARN_ONCE(n > 1, "bad leftover pebs %d\n", n); at += n - 1; - __intel_pmu_pebs_event(event, iregs, at); + __intel_pmu_pebs_event(event, iregs, at, top, 1); } static void intel_pmu_drain_pebs_nhm(struct pt_regs *iregs) { struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events); struct debug_store *ds = cpuc->ds; - struct perf_event *event = NULL; - void *at, *top; - u64 status = 0; + struct perf_event *event; + void *base, *at, *top; int bit; + int counts[MAX_PEBS_EVENTS] = {}; if (!x86_pmu.pebs_active) return; - at = (struct pebs_record_nhm *)(unsigned long)ds->pebs_buffer_base; + base = (struct pebs_record_nhm *)(unsigned long)ds->pebs_buffer_base; top = (struct pebs_record_nhm *)(unsigned long)ds->pebs_index; ds->pebs_index = ds->pebs_buffer_base; - if (unlikely(at > top)) + if (unlikely(base >= top)) return; - /* - * Should not happen, we program the threshold at 1 and do not - * set a reset value. - */ - WARN_ONCE(top - at > x86_pmu.max_pebs_events * x86_pmu.pebs_record_size, - "Unexpected number of pebs records %ld\n", - (long)(top - at) / x86_pmu.pebs_record_size); - - for (; at < top; at += x86_pmu.pebs_record_size) { + for (at = base; at < top; at += x86_pmu.pebs_record_size) { struct pebs_record_nhm *p = at; - for_each_set_bit(bit, (unsigned long *)&p->status, - x86_pmu.max_pebs_events) { - event = cpuc->events[bit]; - if (!test_bit(bit, cpuc->active_mask)) - continue; - - WARN_ON_ONCE(!event); - - if (!event->attr.precise_ip) - continue; - - if (__test_and_set_bit(bit, (unsigned long *)&status)) - continue; - - break; - } + bit = find_first_bit((unsigned long *)&p->status, + x86_pmu.max_pebs_events); + if (bit >= x86_pmu.max_pebs_events) + continue; + /* + * The PEBS hardware does not deal well with collisions, + * when the same event happens near to each other. The + * records for the events can be collapsed into a single + * one, and it's not possible to reconstruct all events + * that caused the PEBS record. However in practice, the + * collisions are extremely rare. If collision happened, + * we drop the record. its the safest choice. + */ + if (p->status != (1 << bit)) + continue; + if (!test_bit(bit, cpuc->active_mask)) + continue; + event = cpuc->events[bit]; + WARN_ON_ONCE(!event); + if (!event->attr.precise_ip) + continue; + counts[bit]++; + } - if (!event || bit >= x86_pmu.max_pebs_events) + for (bit = 0; bit < x86_pmu.max_pebs_events; bit++) { + if (counts[bit] == 0) continue; + event = cpuc->events[bit]; + for (at = base; at < top; at += x86_pmu.pebs_record_size) { + struct pebs_record_nhm *p = at; - __intel_pmu_pebs_event(event, iregs, at); + if (p->status == (1 << bit)) + break; + } + __intel_pmu_pebs_event(event, iregs, at, top, counts[bit]); } } -- 1.8.3.2 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/