Received: by 2002:a05:6a10:8c0a:0:0:0:0 with SMTP id go10csp3927093pxb; Mon, 8 Feb 2021 03:49:25 -0800 (PST) X-Google-Smtp-Source: ABdhPJyajUM2iHQnL0Tp3rlLBz2/FHi2eiEyS5tUHZruvbJHIPAgdePUwU4SzMAs4aPSPSnbK3jr X-Received: by 2002:a17:906:2ed1:: with SMTP id s17mr8852913eji.153.1612784964991; Mon, 08 Feb 2021 03:49:24 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612784964; cv=none; d=google.com; s=arc-20160816; b=yRgmbYzPao81LYmiESV63ZtFEhnXufhb04+pEG9lMdWLhV4xCoD56E/eo6m70P1cOc leiBQDuZU4HcRJZrpFLdKl2WjAP6+MRikwAnlUaYgSKH3vyhhqjRCVzlYolWLSG0lcgT IQxMPxiGcON4FKqwjR/KoAdVZeBtXKjenlNBO+3ewEewH48HtvuBrM/epQVHwjPpmCcL aryqQRM0gSGnAVEvilyVrc4nno4r+HTxU+aG3L0+VdKQM73cJhkECm1pPXBPJyTT9ZNU qwvcTHo139asnE05VZUwj6zXSuSUpBYTKavnkFfvr+8b3x0MN79C+S3qOQtM6wUP2HXO iTRA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from; bh=KRLuQgxBXN94Fpz8TR4dgXABA83t8rCJE8Br8kL2lNA=; b=0VMTN2S0peIlHCW/v2nixmFjbKNt/yAnJ8ZtdS4G7IjpEno8I5Yhu4pz26XOc4XABD H5G7zHfJNqwvo6QrpCCT/0rFTHnFuVP6tAKaYD6NH1ARQqH4XfVfM4rlsIew2leW6a/F MS2ZWH2+wZFWRip8ACBPadvoY+ireZhYBkomXAmzAAKe1uz4aaG8OptG4vlYmLS88FfQ Y8xqWUIsCj9n6/EHRlPbyt6+j8xjzemqog7518IZ1PTcg1/baAepVcCSp7O8br6HEBIv dfKuGCOa4MMSlvdWPIbovHImEZhB52T9o0Jn192TWRZLQA/n2IcGZK7611Fsyqzz0ONt 7hvw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id e13si11275673edz.214.2021.02.08.03.49.00; Mon, 08 Feb 2021 03:49:24 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233201AbhBHLrP (ORCPT + 99 others); Mon, 8 Feb 2021 06:47:15 -0500 Received: from szxga06-in.huawei.com ([45.249.212.32]:12462 "EHLO szxga06-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233043AbhBHLXo (ORCPT ); Mon, 8 Feb 2021 06:23:44 -0500 Received: from DGGEMS411-HUB.china.huawei.com (unknown [172.30.72.60]) by szxga06-in.huawei.com (SkyGuard) with ESMTP id 4DZ3Wj6nB9zjK1c; Mon, 8 Feb 2021 19:21:53 +0800 (CST) Received: from DESKTOP-TMVL5KK.china.huawei.com (10.174.187.128) by DGGEMS411-HUB.china.huawei.com (10.3.19.211) with Microsoft SMTP Server id 14.3.498.0; Mon, 8 Feb 2021 19:22:53 +0800 From: Yanan Wang To: Marc Zyngier , Will Deacon , "Catalin Marinas" , James Morse , "Julien Thierry" , Suzuki K Poulose , Gavin Shan , Quentin Perret , , , , CC: , , , Yanan Wang Subject: [RFC PATCH 1/4] KVM: arm64: Move the clean of dcache to the map handler Date: Mon, 8 Feb 2021 19:22:47 +0800 Message-ID: <20210208112250.163568-2-wangyanan55@huawei.com> X-Mailer: git-send-email 2.8.4.windows.1 In-Reply-To: <20210208112250.163568-1-wangyanan55@huawei.com> References: <20210208112250.163568-1-wangyanan55@huawei.com> MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [10.174.187.128] X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org We currently uniformly clean dcache in user_mem_abort() before calling the fault handlers, if we take a translation fault and the pfn is cacheable. But if there are concurrent translation faults on the same page or block, clean of dcache for the first time is necessary while the others are not. By moving clean of dcache to the map handler, we can easily identify the conditions where CMOs are really needed and avoid the unnecessary ones. As it's a time consuming process to perform CMOs especially when flushing a block range, so this solution reduces much load of kvm and improve the efficiency of creating mappings. Signed-off-by: Yanan Wang --- arch/arm64/include/asm/kvm_mmu.h | 16 -------------- arch/arm64/kvm/hyp/pgtable.c | 38 ++++++++++++++++++++------------ arch/arm64/kvm/mmu.c | 14 +++--------- 3 files changed, 27 insertions(+), 41 deletions(-) diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h index e52d82aeadca..4ec9879e82ed 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -204,22 +204,6 @@ static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu) return (vcpu_read_sys_reg(vcpu, SCTLR_EL1) & 0b101) == 0b101; } -static inline void __clean_dcache_guest_page(kvm_pfn_t pfn, unsigned long size) -{ - void *va = page_address(pfn_to_page(pfn)); - - /* - * With FWB, we ensure that the guest always accesses memory using - * cacheable attributes, and we don't have to clean to PoC when - * faulting in pages. Furthermore, FWB implies IDC, so cleaning to - * PoU is not required either in this case. - */ - if (cpus_have_const_cap(ARM64_HAS_STAGE2_FWB)) - return; - - kvm_flush_dcache_to_poc(va, size); -} - static inline void __invalidate_icache_guest_page(kvm_pfn_t pfn, unsigned long size) { diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index 4d177ce1d536..2f4f87021980 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -464,6 +464,26 @@ static int stage2_map_set_prot_attr(enum kvm_pgtable_prot prot, return 0; } +static bool stage2_pte_cacheable(kvm_pte_t pte) +{ + u64 memattr = pte & KVM_PTE_LEAF_ATTR_LO_S2_MEMATTR; + return memattr == PAGE_S2_MEMATTR(NORMAL); +} + +static void stage2_flush_dcache(void *addr, u64 size) +{ + /* + * With FWB, we ensure that the guest always accesses memory using + * cacheable attributes, and we don't have to clean to PoC when + * faulting in pages. Furthermore, FWB implies IDC, so cleaning to + * PoU is not required either in this case. + */ + if (cpus_have_const_cap(ARM64_HAS_STAGE2_FWB)) + return; + + __flush_dcache_area(addr, size); +} + static int stage2_map_walker_try_leaf(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, struct stage2_map_data *data) @@ -495,6 +515,10 @@ static int stage2_map_walker_try_leaf(u64 addr, u64 end, u32 level, put_page(page); } + /* Flush data cache before installation of the new PTE */ + if (stage2_pte_cacheable(new)) + stage2_flush_dcache(__va(phys), granule); + smp_store_release(ptep, new); get_page(page); data->phys += granule; @@ -651,20 +675,6 @@ int kvm_pgtable_stage2_map(struct kvm_pgtable *pgt, u64 addr, u64 size, return ret; } -static void stage2_flush_dcache(void *addr, u64 size) -{ - if (cpus_have_const_cap(ARM64_HAS_STAGE2_FWB)) - return; - - __flush_dcache_area(addr, size); -} - -static bool stage2_pte_cacheable(kvm_pte_t pte) -{ - u64 memattr = pte & KVM_PTE_LEAF_ATTR_LO_S2_MEMATTR; - return memattr == PAGE_S2_MEMATTR(NORMAL); -} - static int stage2_unmap_walker(u64 addr, u64 end, u32 level, kvm_pte_t *ptep, enum kvm_pgtable_walk_flags flag, void * const arg) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 77cb2d28f2a4..d151927a7d62 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -609,11 +609,6 @@ void kvm_arch_mmu_enable_log_dirty_pt_masked(struct kvm *kvm, kvm_mmu_write_protect_pt_masked(kvm, slot, gfn_offset, mask); } -static void clean_dcache_guest_page(kvm_pfn_t pfn, unsigned long size) -{ - __clean_dcache_guest_page(pfn, size); -} - static void invalidate_icache_guest_page(kvm_pfn_t pfn, unsigned long size) { __invalidate_icache_guest_page(pfn, size); @@ -882,9 +877,6 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa, if (writable) prot |= KVM_PGTABLE_PROT_W; - if (fault_status != FSC_PERM && !device) - clean_dcache_guest_page(pfn, vma_pagesize); - if (exec_fault) { prot |= KVM_PGTABLE_PROT_X; invalidate_icache_guest_page(pfn, vma_pagesize); @@ -1144,10 +1136,10 @@ int kvm_set_spte_hva(struct kvm *kvm, unsigned long hva, pte_t pte) trace_kvm_set_spte_hva(hva); /* - * We've moved a page around, probably through CoW, so let's treat it - * just like a translation fault and clean the cache to the PoC. + * We've moved a page around, probably through CoW, so let's treat + * it just like a translation fault and the map handler will clean + * the cache to the PoC. */ - clean_dcache_guest_page(pfn, PAGE_SIZE); handle_hva_to_gpa(kvm, hva, end, &kvm_set_spte_handler, &pfn); return 0; } -- 2.23.0