Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1759619AbZKFSzD (ORCPT ); Fri, 6 Nov 2009 13:55:03 -0500 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1759544AbZKFSzD (ORCPT ); Fri, 6 Nov 2009 13:55:03 -0500 Received: from smtp.ultrahosting.com ([74.213.175.253]:52326 "EHLO smtp.ultrahosting.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1759052AbZKFSzC (ORCPT ); Fri, 6 Nov 2009 13:55:02 -0500 Date: Fri, 6 Nov 2009 13:53:35 -0500 (EST) From: Christoph Lameter X-X-Sender: cl@V090114053VZO-1 To: Andi Kleen cc: npiggin@suse.de, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Tejun Heo , Ingo Molnar , KAMEZAWA Hiroyuki , "hugh.dickins@tiscali.co.uk" Subject: [RFC MM] mmap_sem scaling: only scan cpus used by an mm In-Reply-To: <20091106073946.GV31511@one.firstfloor.org> Message-ID: References: <87r5sc7kst.fsf@basil.nowhere.org> <20091106073946.GV31511@one.firstfloor.org> User-Agent: Alpine 1.10 (DEB 962 2008-03-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2819 Lines: 73 One way to reduce the cost of the writer lock is to track the cpus used and loop over the processors in that bitmap. --- arch/x86/include/asm/mmu_context.h | 1 + include/linux/mm_types.h | 3 ++- kernel/fork.c | 2 ++ mm/init-mm.c | 1 + 4 files changed, 6 insertions(+), 1 deletion(-) Index: linux-2.6/arch/x86/include/asm/mmu_context.h =================================================================== --- linux-2.6.orig/arch/x86/include/asm/mmu_context.h 2009-11-06 12:26:24.000000000 -0600 +++ linux-2.6/arch/x86/include/asm/mmu_context.h 2009-11-06 12:26:36.000000000 -0600 @@ -43,6 +43,7 @@ static inline void switch_mm(struct mm_s percpu_write(cpu_tlbstate.active_mm, next); #endif cpumask_set_cpu(cpu, mm_cpumask(next)); + cpumask_set_cpu(cpu, &next->cpus_used); /* Re-load page tables */ load_cr3(next->pgd); Index: linux-2.6/include/linux/mm_types.h =================================================================== --- linux-2.6.orig/include/linux/mm_types.h 2009-11-06 12:26:35.000000000 -0600 +++ linux-2.6/include/linux/mm_types.h 2009-11-06 12:26:36.000000000 -0600 @@ -241,6 +241,7 @@ struct mm_struct { struct linux_binfmt *binfmt; cpumask_t cpu_vm_mask; + cpumask_t cpus_used; /* Architecture-specific MM context */ mm_context_t context; @@ -291,7 +292,7 @@ static inline int mm_has_reader(struct m { int cpu; - for_each_possible_cpu(cpu) + for_each_cpu(cpu, &mm->cpus_used) if (per_cpu(mm->rss->readers, cpu)) return 1; Index: linux-2.6/mm/init-mm.c =================================================================== --- linux-2.6.orig/mm/init-mm.c 2009-11-06 12:26:35.000000000 -0600 +++ linux-2.6/mm/init-mm.c 2009-11-06 12:26:36.000000000 -0600 @@ -19,5 +19,6 @@ struct mm_struct init_mm = { .page_table_lock = __SPIN_LOCK_UNLOCKED(init_mm.page_table_lock), .mmlist = LIST_HEAD_INIT(init_mm.mmlist), .cpu_vm_mask = CPU_MASK_ALL, + .cpus_used = CPU_MASK_ALL, .rss = &init_mm_counters, }; Index: linux-2.6/kernel/fork.c =================================================================== --- linux-2.6.orig/kernel/fork.c 2009-11-06 12:26:35.000000000 -0600 +++ linux-2.6/kernel/fork.c 2009-11-06 12:26:40.000000000 -0600 @@ -297,6 +297,8 @@ static int dup_mmap(struct mm_struct *mm mm->cached_hole_size = ~0UL; mm->map_count = 0; cpumask_clear(mm_cpumask(mm)); + cpumask_clear(&mm->cpus_used); + cpumask_set_cpu(smp_processor_id(), &mm->cpus_used); mm->mm_rb = RB_ROOT; rb_link = &mm->mm_rb.rb_node; rb_parent = NULL; -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/