Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933067AbdGXXGI (ORCPT ); Mon, 24 Jul 2017 19:06:08 -0400 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:52145 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932996AbdGXXC4 (ORCPT ); Mon, 24 Jul 2017 19:02:56 -0400 From: Dennis Zhou To: Tejun Heo , Christoph Lameter , Josef Bacik CC: , , , Dennis Zhou Subject: [PATCH v2 17/23] percpu: skip chunks if the alloc does not fit in the contig hint Date: Mon, 24 Jul 2017 19:02:14 -0400 Message-ID: <20170724230220.21774-18-dennisz@fb.com> X-Mailer: git-send-email 2.13.3 In-Reply-To: <20170724230220.21774-1-dennisz@fb.com> References: <20170724230220.21774-1-dennisz@fb.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [192.168.52.123] X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-07-24_14:,, signatures=0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2755 Lines: 77 From: "Dennis Zhou (Facebook)" This patch adds chunk->contig_bits_start to keep track of the contig hint's offset and the check to skip the chunk if it does not fit. If the chunk's contig hint starting offset cannot satisfy an allocation, the allocator assumes there is enough memory pressure in this chunk to either use a different chunk or create a new one. This accepts a less tight packing for a smoother latency curve. Signed-off-by: Dennis Zhou --- mm/percpu-internal.h | 2 ++ mm/percpu.c | 18 ++++++++++++++++-- 2 files changed, 18 insertions(+), 2 deletions(-) diff --git a/mm/percpu-internal.h b/mm/percpu-internal.h index e60e049..7065faf 100644 --- a/mm/percpu-internal.h +++ b/mm/percpu-internal.h @@ -29,6 +29,8 @@ struct pcpu_chunk { struct list_head list; /* linked to pcpu_slot lists */ int free_bytes; /* free bytes in the chunk */ int contig_bits; /* max contiguous size hint */ + int contig_bits_start; /* contig_bits starting + offset */ void *base_addr; /* base address of this chunk */ unsigned long *alloc_map; /* allocation map */ diff --git a/mm/percpu.c b/mm/percpu.c index ad70c67..3732373 100644 --- a/mm/percpu.c +++ b/mm/percpu.c @@ -393,12 +393,14 @@ static inline int pcpu_cnt_pop_pages(struct pcpu_chunk *chunk, int bit_off, * @bit_off: chunk offset * @bits: size of free area * - * This updates the chunk's contig hint given a free area. + * This updates the chunk's contig hint and starting offset given a free area. */ static void pcpu_chunk_update(struct pcpu_chunk *chunk, int bit_off, int bits) { - if (bits > chunk->contig_bits) + if (bits > chunk->contig_bits) { + chunk->contig_bits_start = bit_off; chunk->contig_bits = bits; + } } /** @@ -409,6 +411,7 @@ static void pcpu_chunk_update(struct pcpu_chunk *chunk, int bit_off, int bits) * * Updates: * chunk->contig_bits + * chunk->contig_bits_start * nr_empty_pop_pages */ static void pcpu_chunk_refresh_hint(struct pcpu_chunk *chunk) @@ -639,6 +642,17 @@ static int pcpu_find_block_fit(struct pcpu_chunk *chunk, int alloc_bits, int bit_off, bits; int re; /* region end */ + /* + * Check to see if the allocation can fit in the chunk's contig hint. + * This is an optimization to prevent scanning by assuming if it + * cannot fit in the global hint, there is memory pressure and creating + * a new chunk would happen soon. + */ + bit_off = ALIGN(chunk->contig_bits_start, align) - + chunk->contig_bits_start; + if (bit_off + alloc_bits > chunk->contig_bits) + return -1; + pcpu_for_each_unpop_region(chunk->alloc_map, bit_off, re, chunk->first_bit, pcpu_chunk_map_bits(chunk)) { -- 2.9.3