Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932155AbbFMJvi (ORCPT ); Sat, 13 Jun 2015 05:51:38 -0400 Received: from mail-wi0-f176.google.com ([209.85.212.176]:37642 "EHLO mail-wi0-f176.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752405AbbFMJtu (ORCPT ); Sat, 13 Jun 2015 05:49:50 -0400 From: Ingo Molnar To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, Andy Lutomirski , Andrew Morton , Denys Vlasenko , Brian Gerst , Peter Zijlstra , Borislav Petkov , "H. Peter Anvin" , Linus Torvalds , Oleg Nesterov , Thomas Gleixner , Waiman Long Subject: [PATCH 10/12] x86/mm: Make pgd_alloc()/pgd_free() lockless Date: Sat, 13 Jun 2015 11:49:13 +0200 Message-Id: <1434188955-31397-11-git-send-email-mingo@kernel.org> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1434188955-31397-1-git-send-email-mingo@kernel.org> References: <1434188955-31397-1-git-send-email-mingo@kernel.org> Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2849 Lines: 96 The fork()/exit() code uses pgd_alloc()/pgd_free() to allocate/deallocate the PGD, with platform specific code setting up kernel pagetables. The x86 code uses a global pgd_list with an associated lock to update all PGDs of all tasks in the system synchronously. The lock is still kept to synchronize updates to all PGDs in the system, but all users of the list have been migrated to use the task list. So we can remove the pgd_list addition/removal from this code. The new PGD is private while constructed, so it needs no extra locking. Cc: Andrew Morton Cc: Andy Lutomirski Cc: Borislav Petkov Cc: Brian Gerst Cc: Denys Vlasenko Cc: H. Peter Anvin Cc: Linus Torvalds Cc: Oleg Nesterov Cc: Peter Zijlstra Cc: Rik van Riel Cc: Thomas Gleixner Cc: Waiman Long Cc: linux-mm@kvack.org Signed-off-by: Ingo Molnar --- arch/x86/mm/pgtable.c | 27 +++------------------------ 1 file changed, 3 insertions(+), 24 deletions(-) diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index 7a561b7cc01c..0ab56d13f24d 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -125,22 +125,6 @@ static void pgd_ctor(struct mm_struct *mm, pgd_t *pgd) swapper_pg_dir + KERNEL_PGD_BOUNDARY, KERNEL_PGD_PTRS); } - - /* list required to sync kernel mapping updates */ - if (!SHARED_KERNEL_PMD) { - pgd_set_mm(pgd, mm); - pgd_list_add(pgd); - } -} - -static void pgd_dtor(pgd_t *pgd) -{ - if (SHARED_KERNEL_PMD) - return; - - spin_lock(&pgd_lock); - pgd_list_del(pgd); - spin_unlock(&pgd_lock); } /* @@ -370,17 +354,13 @@ pgd_t *pgd_alloc(struct mm_struct *mm) goto out_free_pmds; /* - * Make sure that pre-populating the pmds is atomic with - * respect to anything walking the pgd_list, so that they - * never see a partially populated pgd. + * No locking is needed here, as the PGD is still private, + * so no code walking the task list and looking at mm->pgd + * will be able to see it before it's fully constructed: */ - spin_lock(&pgd_lock); - pgd_ctor(mm, pgd); pgd_prepopulate_pmd(mm, pgd, pmds); - spin_unlock(&pgd_lock); - return pgd; out_free_pmds: @@ -453,7 +433,6 @@ void arch_pgd_init_late(struct mm_struct *mm) void pgd_free(struct mm_struct *mm, pgd_t *pgd) { pgd_mop_up_pmds(mm, pgd); - pgd_dtor(pgd); paravirt_pgd_free(mm, pgd); _pgd_free(pgd); } -- 2.1.4 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/