Received: by 10.213.65.68 with SMTP id h4csp651458imn; Tue, 27 Mar 2018 06:23:04 -0700 (PDT) X-Google-Smtp-Source: AG47ELveQcUsEfmlZvYNFC0b6S6IJgNLUkMA8fNVw9TwF7yn6mDkRSquA9sbCkH+VeBLInyo3I+k X-Received: by 2002:a17:902:209:: with SMTP id 9-v6mr44292745plc.403.1522156983957; Tue, 27 Mar 2018 06:23:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1522156983; cv=none; d=google.com; s=arc-20160816; b=0+83s08biHljfWPQZIiim27u+jVCxD+gC9mcyf7NmtSXvQGUryHDuBxSf0MdkfVetd zpH0ceA0ZtQTwORBzo69NDPBVoPuZsBW2HPx6SxR4dj+HF4Y+g2W0TGQdRIMg6IWr+vH 1fLT7omz0TzJEXaGATjTk0UWW3VJog88cGNbLGCu1gCBuQKL4Xu5U2wjKsok/jmYJEjO JyiGb2ezP3NpuBKZUi488NgnlJWG2Zxs4602zIQxTKPulwLbc+4yI4LiKVra9x9jkklb gfn0TBgv9hATMi3VJf+GwhrJBn8k53Cz/WFPGZQKhIqyIhXShVm4KuKJgd0EWotUNOUM lS2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=k/nMmTb+UlE5G6b8gFCn6dV3Y1OY0ZF5dhQ/D7Uat+8=; b=EhA7yhIzLypci65QVtfogDNpQT5NKc+l6OWpUCrXWsK0Qute6DriO7gduZYiCKMKyc h8Ez2L92nTybkGWlR3Hk+rf3qliPQhTS0oKtTVSM1vDaPhqv08W8iNAK3LAI30STz6iW v6HVq/6vK/WZCXdqz9muMDmV/CjHS1SNDLPJ7K+QHWX1s/raDdPTpE89/hU/hBKxfxM5 7DBiRw9HdTdgiASVUuHQvvSBzNFuoJj6pJCcmcObZGG4uqGB3B5WPjhnZJE7Fl4qrEDX ZfP+jycDP5XK9/ACdB95xIAUeqhehD9qmsJzwmVF8FmDwe6vFrUqBCy1jDCgX3d2RXi3 Ruvg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id u13si897431pgq.232.2018.03.27.06.22.49; Tue, 27 Mar 2018 06:23:03 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753153AbeC0NVF (ORCPT + 99 others); Tue, 27 Mar 2018 09:21:05 -0400 Received: from foss.arm.com ([217.140.101.70]:55098 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752800AbeC0NQa (ORCPT ); Tue, 27 Mar 2018 09:16:30 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7E8211688; Tue, 27 Mar 2018 06:16:30 -0700 (PDT) Received: from en101.cambridge.arm.com (en101.cambridge.arm.com [10.1.206.73]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D55AA3F24A; Tue, 27 Mar 2018 06:16:27 -0700 (PDT) From: Suzuki K Poulose To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, cdall@kernel.org, marc.zyngier@arm.com, punit.agrawal@arm.com, will.deacon@arm.com, catalin.marinas@arm.com, pbonzini@redhat.com, rkrcmar@redhat.com, ard.biesheuvel@linaro.org, peter.maydell@linaro.org, kristina.martsenko@arm.com, mark.rutland@arm.com, Suzuki K Poulose Subject: [PATCH v2 10/17] kvm: arm64: Dynamic configuration of VTCR and VTTBR mask Date: Tue, 27 Mar 2018 14:15:20 +0100 Message-Id: <1522156531-28348-11-git-send-email-suzuki.poulose@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1522156531-28348-1-git-send-email-suzuki.poulose@arm.com> References: <1522156531-28348-1-git-send-email-suzuki.poulose@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org VTCR_EL2 holds the following key stage2 translation table parameters: SL0 - Entry level in the page table lookup. T0SZ - Denotes the size of the memory addressed by the table. We have been using fixed values for the SL0 depending on the page size as we have a fixed IPA size. But since we are about to make it dynamic, we need to calculate the SL0 at runtime per VM. Also the VTTBR:BADDR holds the base address for the stage2 translation table and the ARM ARM mandates that bits BADDR[x-1:0] should be 0, where 'x' defined using some magical constant, which depends on the page size, T0SZ and the entry level of lookup (Since the entry level page tables can be concatenated at stage2, a given T0SZ could possibly start at 2 different levels). We need a way to calculate this magical value per VM, depending on the IPA size. Luckily there is a magic formula for finding the "magic" number to find "x". See the patch for more details. This patch adds helpers to figure out the VTCR_SL0 and the magic "X" for a configuration of stage2. The other advantage we have with this change is switching the entry level for a given IPA size, depending on if we are able to get contiguous block of memory for the entry level page table. (e.g, With 64KB page size and 46bit IPA starting at level 2, finding 16 * 64KB contiguous block on a loaded system could be tricky. So we could decide to rather enter at level 1, with a single page). Cc: Marc Zyngier Cc: Christoffer Dall Signed-off-by: Suzuki K Poulose --- arch/arm64/include/asm/kvm_arm.h | 96 +++++++++++++++++++++++++++++++++++++--- arch/arm64/include/asm/kvm_mmu.h | 20 ++++++++- 2 files changed, 110 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h index b0c8417..176551c 100644 --- a/arch/arm64/include/asm/kvm_arm.h +++ b/arch/arm64/include/asm/kvm_arm.h @@ -124,6 +124,8 @@ #define VTCR_EL2_VS_8BIT (0 << VTCR_EL2_VS_SHIFT) #define VTCR_EL2_VS_16BIT (1 << VTCR_EL2_VS_SHIFT) +#define VTCR_EL2_T0SZ(x) TCR_T0SZ((x)) + /* * We configure the Stage-2 page tables to always restrict the IPA space to be * 40 bits wide (T0SZ = 24). Systems with a PARange smaller than 40 bits are @@ -150,7 +152,8 @@ * 2 level page tables (SL = 1) */ #define VTCR_EL2_TGRAN_FLAGS (VTCR_EL2_TG0_64K | VTCR_EL2_SL0_LVL1) -#define VTTBR_X_TGRAN_MAGIC 38 +#define VTCR_EL2_TGRAN_SL0_BASE 3UL + #elif defined(CONFIG_ARM64_16K_PAGES) /* * Stage2 translation configuration: @@ -158,7 +161,7 @@ * 2 level page tables (SL = 1) */ #define VTCR_EL2_TGRAN_FLAGS (VTCR_EL2_TG0_16K | VTCR_EL2_SL0_LVL1) -#define VTTBR_X_TGRAN_MAGIC 42 +#define VTCR_EL2_TGRAN_SL0_BASE 3UL #else /* 4K */ /* * Stage2 translation configuration: @@ -166,13 +169,96 @@ * 3 level page tables (SL = 1) */ #define VTCR_EL2_TGRAN_FLAGS (VTCR_EL2_TG0_4K | VTCR_EL2_SL0_LVL1) -#define VTTBR_X_TGRAN_MAGIC 37 +#define VTCR_EL2_TGRAN_SL0_BASE 2UL #endif #define VTCR_EL2_FLAGS (VTCR_EL2_COMMON_BITS | VTCR_EL2_TGRAN_FLAGS) -#define VTTBR_X (VTTBR_X_TGRAN_MAGIC - VTCR_EL2_T0SZ_IPA) +/* + * VTCR_EL2:SL0 indicates the entry level for Stage2 translation. + * Interestingly, it depends on the page size. + * See D.10.2.110, VTCR_EL2, in ARM DDI 0487B.b + * + * ----------------------------------------- + * | Entry level | 4K | 16K/64K | + * ------------------------------------------ + * | Level: 0 | 2 | - | + * ------------------------------------------ + * | Level: 1 | 1 | 2 | + * ------------------------------------------ + * | Level: 2 | 0 | 1 | + * ------------------------------------------ + * | Level: 3 | - | 0 | + * ------------------------------------------ + * + * That table roughly translates to : + * + * SL0(PAGE_SIZE, Entry_level) = SL0_BASE(PAGE_SIZE) - Entry_Level + * + * Where SL0_BASE(4K) = 2 and SL0_BASE(16K) = 3, SL0_BASE(64K) = 3, provided + * we take care of ruling out the unsupported cases and + * Entry_Level = 4 - Number_of_levels. + * + */ +#define VTCR_EL2_SL0(levels) \ + ((VTCR_EL2_TGRAN_SL0_BASE - (4 - (levels))) << VTCR_EL2_SL0_SHIFT) +/* + * ARM VMSAv8-64 defines an algorithm for finding the translation table + * descriptors in section D4.2.8 in ARM DDI 0487B.b. + * + * The algorithm defines the expectaions on the BaseAddress (for the page + * table) bits resolved at each level based on the page size, entry level + * and T0SZ. The variable "x" in the algorithm also affects the VTTBR:BADDR + * for stage2 page table. + * + * The value of "x" is calculated as : + * x = Magic_N - T0SZ + * + * where Magic_N is an integer depending on the page size and the entry + * level of the page table as below: + * + * -------------------------------------------- + * | Entry level | 4K 16K 64K | + * -------------------------------------------- + * | Level: 0 (4 levels) | 28 | - | - | + * -------------------------------------------- + * | Level: 1 (3 levels) | 37 | 31 | 25 | + * -------------------------------------------- + * | Level: 2 (2 levels) | 46 | 42 | 38 | + * -------------------------------------------- + * | Level: 3 (1 level) | - | 53 | 51 | + * -------------------------------------------- + * + * We have a magic formula for the Magic_N below. + * Which can also be expressed as: + * + * Magic_N(PAGE_SIZE, Entry_Level) = 64 - ((PAGE_SHIFT - 3) * Number of levels) + * + * where number of levels = (4 - Entry_Level). + * + * So, given that T0SZ = (64 - PA_SHIFT), we can compute 'x' as follows: + * + * x = (64 - ((PAGE_SHIFT - 3) * Number_of_levels)) - (64 - PA_SHIFT) + * = PA_SHIFT - ((PAGE_SHIFT - 3) * Number of levels) + * + * Here is one way to explain the Magic Formula: + * + * x = log2(Size_of_Entry_Level_Table) + * + * Since, we can resolve (PAGE_SHIFT - 3) bits at each level, and another + * PAGE_SHIFT bits in the PTE, we have : + * + * Bits_Entry_level = PA_SHIFT - ((PAGE_SHIFT - 3) * (n - 1) + PAGE_SHIFT) + * = PA_SHIFT - (PAGE_SHIFT - 3) * n - 3 + * where n = number of levels, and since each pointer is 8bytes, we have: + * + * x = Bits_Entry_Level + 3 + * = PA_SHIFT - (PAGE_SHIFT - 3) * n + * + * The only constraint here is that, we have to find the number of page table + * levels for a given IPA size (which we do, see STAGE2_PGTABLE_LEVELS). + */ +#define ARM64_VTTBR_X(ipa, levels) ((ipa) - ((levels) * (PAGE_SHIFT - 3))) -#define VTTBR_BADDR_MASK (((UL(1) << (PHYS_MASK_SHIFT - VTTBR_X)) - 1) << VTTBR_X) #define VTTBR_VMID_SHIFT (UL(48)) #define VTTBR_VMID_MASK(size) (_AT(u64, (1 << size) - 1) << VTTBR_VMID_SHIFT) diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index bc133ce..447bc10 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -139,7 +139,6 @@ static inline unsigned long __kern_hyp_va(unsigned long v) #define kvm_phys_shift(kvm) KVM_PHYS_SHIFT #define kvm_phys_size(kvm) (_AC(1, ULL) << kvm_phys_shift(kvm)) #define kvm_phys_mask(kvm) (kvm_phys_size(kvm) - _AC(1, ULL)) -#define kvm_vttbr_baddr_mask(kvm) VTTBR_BADDR_MASK static inline bool kvm_page_empty(void *ptr) { @@ -393,5 +392,24 @@ static inline int kvm_map_vectors(void) #define kvm_phys_to_vttbr(addr) phys_to_ttbr(addr) +/* + * Get the magic number 'x' for VTTBR:BADDR of this KVM instance. + * With v8.2 LVA extensions, 'x' (rather 'z') should be a minimum + * of 6 with 52bit IPS. + */ +static inline int kvm_vttbr_x(struct kvm *kvm) +{ + int x = ARM64_VTTBR_X(kvm_phys_shift(kvm), kvm_stage2_levels(kvm)); + + return (IS_ENABLED(CONFIG_ARM64_PA_BITS_52) && x < 6) ? 6 : x; +} + +static inline u64 kvm_vttbr_baddr_mask(struct kvm *kvm) +{ + unsigned int x = kvm_vttbr_x(kvm); + + return GENMASK_ULL(PHYS_MASK_SHIFT - 1, x); +} + #endif /* __ASSEMBLY__ */ #endif /* __ARM64_KVM_MMU_H__ */ -- 2.7.4