Received: by 10.213.65.68 with SMTP id h4csp1069395imn; Fri, 6 Apr 2018 14:03:09 -0700 (PDT) X-Google-Smtp-Source: AIpwx48Nduq43dlcZ2/DUwWuOg218o1+LzqegU1OB0SiQYZAeK2z9tmeYiseaSIXD+LBGstdrmAY X-Received: by 10.98.31.216 with SMTP id l85mr21702881pfj.80.1523048589808; Fri, 06 Apr 2018 14:03:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1523048589; cv=none; d=google.com; s=arc-20160816; b=fzapkV+lAIWphzLUXP7L+DVBV8sOG5OQKY2s7KWSa6ODUYjsk5Gk9bAgH7LGGnoDRD 7lYMLaUQcTTrTHhZ7hqlx8eJsSftBzsX48Zx6QCc4zQoG2KoAIVATtXQOhDUcBgddiR8 8vpzhZVfuZCC+Y2Erg0PkNic1cOavpqlQo+ovGoaaQyHJo+zBBj/KA3G0CKYvKcPZi73 utz+xT5oRLbcdgf6opO5jDaP3REXOuo0vi75mYkFLPmxxUeULqAoj36rp5D2dmMFdp4o PFW5E4i96QBi/wOXOYFXnPEZnG5TUsR0cx2V6tFx6WNMpp8i7VjQWeNQywxd6e6Hu5au pFlQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:message-id:in-reply-to:references:date :from:cc:to:subject:arc-authentication-results; bh=AqwVaIUGV5q0S9yVIDSZnzGvAe8KYI41yMzuOEQ+03I=; b=p62Jq3pWOtwip9ktxTIZ1aInkZ50KCez9wFBj+KsEW6ecGjm/kjeo51kSvi94dS4PE Nq+cEjeEa8XepUwvzg9pu6AKFe6jnK1g2S17019W032dRKvbpmR2nCQYVYlp12YWeBwg is2IxLq5SjvBQHQ6xaVwpydpH+Aki88EFU6JOU7OID+yjU5Eqm/nhTHvyAJ0AE3wzsWY uw2S4zLCA2yonUbJHX8gN8oQvrdRWSlEszf1bs+FmPnTfW6N63fQJWUsqKPKHETpJiG0 7DTu89cvr+XHvxa59Ver3WTCJEoHfnbBeTVluVeL42veVAkmoc43MoeCrxSuUTaHx3A1 2+EQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id m39-v6si882410plg.570.2018.04.06.14.02.32; Fri, 06 Apr 2018 14:03:09 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752517AbeDFU67 (ORCPT + 99 others); Fri, 6 Apr 2018 16:58:59 -0400 Received: from mga07.intel.com ([134.134.136.100]:32372 "EHLO mga07.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752205AbeDFU6M (ORCPT ); Fri, 6 Apr 2018 16:58:12 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga006.jf.intel.com ([10.7.209.51]) by orsmga105.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 06 Apr 2018 13:58:12 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.48,416,1517904000"; d="scan'208";a="32220631" Received: from viggo.jf.intel.com (HELO localhost.localdomain) ([10.54.39.119]) by orsmga006.jf.intel.com with ESMTP; 06 Apr 2018 13:58:12 -0700 Subject: [PATCH 09/11] x86/pti: enable global pages for shared areas To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, Dave Hansen , aarcange@redhat.com, luto@kernel.org, torvalds@linux-foundation.org, keescook@google.com, hughd@google.com, jgross@suse.com, x86@kernel.org, namit@vmware.com From: Dave Hansen Date: Fri, 06 Apr 2018 13:55:15 -0700 References: <20180406205501.24A1A4E7@viggo.jf.intel.com> In-Reply-To: <20180406205501.24A1A4E7@viggo.jf.intel.com> Message-Id: <20180406205515.2977EE7D@viggo.jf.intel.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Dave Hansen The entry/exit text and cpu_entry_area are mapped into userspace and the kernel. But, they are not _PAGE_GLOBAL. This creates unnecessary TLB misses. Add the _PAGE_GLOBAL flag for these areas. Signed-off-by: Dave Hansen Cc: Andrea Arcangeli Cc: Andy Lutomirski Cc: Linus Torvalds Cc: Kees Cook Cc: Hugh Dickins Cc: Juergen Gross Cc: x86@kernel.org Cc: Nadav Amit --- b/arch/x86/mm/cpu_entry_area.c | 14 +++++++++++++- b/arch/x86/mm/pti.c | 23 ++++++++++++++++++++++- 2 files changed, 35 insertions(+), 2 deletions(-) diff -puN arch/x86/mm/cpu_entry_area.c~kpti-why-no-global arch/x86/mm/cpu_entry_area.c --- a/arch/x86/mm/cpu_entry_area.c~kpti-why-no-global 2018-04-06 10:47:58.246796118 -0700 +++ b/arch/x86/mm/cpu_entry_area.c 2018-04-06 10:47:58.252796118 -0700 @@ -27,8 +27,20 @@ EXPORT_SYMBOL(get_cpu_entry_area); void cea_set_pte(void *cea_vaddr, phys_addr_t pa, pgprot_t flags) { unsigned long va = (unsigned long) cea_vaddr; + pte_t pte = pfn_pte(pa >> PAGE_SHIFT, flags); - set_pte_vaddr(va, pfn_pte(pa >> PAGE_SHIFT, flags)); + /* + * The cpu_entry_area is shared between the user and kernel + * page tables. All of its ptes can safely be global. + * _PAGE_GLOBAL gets reused to help indicate PROT_NONE for + * non-present PTEs, so be careful not to set it in that + * case to avoid confusion. + */ + if (boot_cpu_has(X86_FEATURE_PGE) && + (pgprot_val(flags) & _PAGE_PRESENT)) + pte = pte_set_flags(pte, _PAGE_GLOBAL); + + set_pte_vaddr(va, pte); } static void __init diff -puN arch/x86/mm/pti.c~kpti-why-no-global arch/x86/mm/pti.c --- a/arch/x86/mm/pti.c~kpti-why-no-global 2018-04-06 10:47:58.248796118 -0700 +++ b/arch/x86/mm/pti.c 2018-04-06 10:47:58.252796118 -0700 @@ -300,6 +300,27 @@ pti_clone_pmds(unsigned long start, unsi return; /* + * Only clone present PMDs. This ensures only setting + * _PAGE_GLOBAL on present PMDs. This should only be + * called on well-known addresses anyway, so a non- + * present PMD would be a surprise. + */ + if (WARN_ON(!(pmd_flags(*pmd) & _PAGE_PRESENT))) + return; + + /* + * Setting 'target_pmd' below creates a mapping in both + * the user and kernel page tables. It is effectively + * global, so set it as global in both copies. Note: + * the X86_FEATURE_PGE check is not _required_ because + * the CPU ignores _PAGE_GLOBAL when PGE is not + * supported. The check keeps consistentency with + * code that only set this bit when supported. + */ + if (boot_cpu_has(X86_FEATURE_PGE)) + *pmd = pmd_set_flags(*pmd, _PAGE_GLOBAL); + + /* * Copy the PMD. That is, the kernelmode and usermode * tables will share the last-level page tables of this * address range @@ -348,7 +369,7 @@ static void __init pti_clone_entry_text( { pti_clone_pmds((unsigned long) __entry_text_start, (unsigned long) __irqentry_text_end, - _PAGE_RW | _PAGE_GLOBAL); + _PAGE_RW); } /* _