Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754957AbaJMNwM (ORCPT ); Mon, 13 Oct 2014 09:52:12 -0400 Received: from mga01.intel.com ([192.55.52.88]:20096 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754258AbaJMNrm (ORCPT ); Mon, 13 Oct 2014 09:47:42 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.04,710,1406617200"; d="scan'208";a="613638845" From: Alexander Shishkin To: Peter Zijlstra Cc: Ingo Molnar , linux-kernel@vger.kernel.org, Robert Richter , Frederic Weisbecker , Mike Galbraith , Paul Mackerras , Stephane Eranian , Andi Kleen , kan.liang@intel.com, adrian.hunter@intel.com, acme@infradead.org, Alexander Shishkin Subject: [PATCH v5 03/20] perf: Support high-order allocations for AUX space Date: Mon, 13 Oct 2014 16:45:31 +0300 Message-Id: <1413207948-28202-4-git-send-email-alexander.shishkin@linux.intel.com> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1413207948-28202-1-git-send-email-alexander.shishkin@linux.intel.com> References: <1413207948-28202-1-git-send-email-alexander.shishkin@linux.intel.com> Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Some pmus (such as BTS or Intel PT without multiple-entry ToPA capability) don't support scatter-gather and will prefer larger contiguous areas for their output regions. This patch adds a new pmu capability to request higher order allocations. Signed-off-by: Alexander Shishkin --- include/linux/perf_event.h | 1 + kernel/events/ring_buffer.c | 51 +++++++++++++++++++++++++++++++++++++++------ 2 files changed, 46 insertions(+), 6 deletions(-) diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index 344058c71d..71df948b4d 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -171,6 +171,7 @@ struct perf_event; * pmu::capabilities flags */ #define PERF_PMU_CAP_NO_INTERRUPT 0x01 +#define PERF_PMU_CAP_AUX_NO_SG 0x02 /** * struct pmu - generic performance monitoring unit diff --git a/kernel/events/ring_buffer.c b/kernel/events/ring_buffer.c index 2ebb6c5d3b..b620e94ea7 100644 --- a/kernel/events/ring_buffer.c +++ b/kernel/events/ring_buffer.c @@ -242,30 +242,69 @@ ring_buffer_init(struct ring_buffer *rb, long watermark, int flags) spin_lock_init(&rb->event_lock); } +#define PERF_AUX_GFP (GFP_KERNEL | __GFP_ZERO | __GFP_NOWARN | __GFP_NORETRY) + +static struct page *rb_alloc_aux_page(int node, int order) +{ + struct page *page; + + if (order > MAX_ORDER) + order = MAX_ORDER; + + do { + page = alloc_pages_node(node, PERF_AUX_GFP, order); + } while (!page && order--); + + if (page && order) { + /* + * Communicate the allocation size to the driver + */ + split_page(page, order); + SetPagePrivate(page); + set_page_private(page, order); + } + + return page; +} + +static void rb_free_aux_page(struct ring_buffer *rb, int idx) +{ + struct page *page = virt_to_page(rb->aux_pages[idx]); + + ClearPagePrivate(page); + page->mapping = NULL; + __free_page(page); +} + int rb_alloc_aux(struct ring_buffer *rb, struct perf_event *event, pgoff_t pgoff, int nr_pages, int flags) { bool overwrite = !(flags & RING_BUFFER_WRITABLE); int node = (event->cpu == -1) ? -1 : cpu_to_node(event->cpu); - int ret = -ENOMEM; + int ret = -ENOMEM, order = 0; if (!has_aux(event)) return -ENOTSUPP; + if (event->pmu->capabilities & PERF_PMU_CAP_AUX_NO_SG) + order = get_order(nr_pages * PAGE_SIZE); + rb->aux_pages = kzalloc_node(nr_pages * sizeof(void *), GFP_KERNEL, node); if (!rb->aux_pages) return -ENOMEM; rb->free_aux = event->pmu->free_aux; - for (rb->aux_nr_pages = 0; rb->aux_nr_pages < nr_pages; - rb->aux_nr_pages++) { + for (rb->aux_nr_pages = 0; rb->aux_nr_pages < nr_pages;) { struct page *page; + int last; - page = alloc_pages_node(node, GFP_KERNEL | __GFP_ZERO, 0); + page = rb_alloc_aux_page(node, order); if (!page) goto out; - rb->aux_pages[rb->aux_nr_pages] = page_address(page); + for (last = rb->aux_nr_pages + (1 << page_private(page)); + last > rb->aux_nr_pages; rb->aux_nr_pages++) + rb->aux_pages[rb->aux_nr_pages] = page_address(page++); } rb->aux_priv = event->pmu->setup_aux(event->cpu, rb->aux_pages, nr_pages, @@ -304,7 +343,7 @@ static void __rb_free_aux(struct kref *kref) } for (pg = 0; pg < rb->aux_nr_pages; pg++) - free_page((unsigned long)rb->aux_pages[pg]); + rb_free_aux_page(rb, pg); kfree(rb->aux_pages); rb->aux_nr_pages = 0; -- 2.1.0 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/