Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <b30aa10f-7abc-11a0-552d-56fc88d534f7@bytedance.com>
Date:   Tue, 16 May 2023 11:16:18 +0800
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0)
 Gecko/20100101 Thunderbird/102.10.1
From:   Gang Li <ligang.bdlg@bytedance.com>
Subject: Re: [QUESTION FOR ARM64 TLB] performance issue and implementation
 difference of TLB flush
To:     x86@kernel.org, Thomas Gleixner <tglx@linutronix.de>
Cc:     Will Deacon <will@kernel.org>,
        Tomasz Nowicki <tomasz.nowicki@linaro.org>,
        Laura Abbott <lauraa@codeaurora.org>,
        Catalin Marinas <catalin.marinas@arm.com>,
        Ard Biesheuvel <ardb@kernel.org>,
        Anshuman Khandual <anshuman.khandual@arm.com>,
        Kefeng Wang <wangkefeng.wang@huawei.com>,
        Feiyang Chen <chenfeiyang@loongson.cn>,
        linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org
References: <2eb026b8-9e13-2b60-9e14-06417b142ac9@bytedance.com>
 <ZEokfJSM9a4ZZvQv@FVFF77S0Q05N>
 <369d1be2-d418-1bfb-bfc2-b25e4e542d76@bytedance.com>
 <8d8c2ed5-c29b-8ea6-84b3-3335d0682d0d@bytedance.com>
Content-Language: en-US
In-Reply-To: <8d8c2ed5-c29b-8ea6-84b3-3335d0682d0d@bytedance.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk

Hi all!

On 2023/5/5 20:28, Gang Li wrote:
> Hi,
> 
> I found that in `ghes_unmap` protected by spinlock, arm64 and x86 have
> different strategies for flushing tlb.
> 
> # arm64 call trace:
> ```
> holding a spin lock
> ghes_unmap
>   clear_fixmap
>    __set_fixmap
>     flush_tlb_kernel_range
> ```
> 
> # x86 call trace:
> ```
> holding a spin lock
> ghes_unmap
>   clear_fixmap
>    __set_fixmap
>     mmu.set_fixmap
>      native_set_fixmap
>       __native_set_fixmap
>        set_pte_vaddr
>         set_pte_vaddr_p4d
>          __set_pte_vaddr
>           flush_tlb_one_kernel
> ```
>
> arm64 broadcast TLB invalidation in ghes_unmap, because TLB entry can be
> allocated regardless of whether the CPU explicitly accesses memory.
> 
> Why doesn't x86 broadcast TLB invalidation in ghes_unmap? Is there any
> difference between x86 and arm64 in TLB allocation and invalidation 
> strategy?
> 

I found this in Intel® 64 and IA-32 Architectures Software Developer
Manuals:

> 4.10.2.3 Details of TLB Use
> Subject to the limitations given in the previous paragraph, the
> processor may cache a translation for any linear address, even if that
> address is not used to access memory. For example, the processor may
> cache translations required for prefetches and for accesses that result
> from speculative execution that would never actually occur in the
> executed code path.

Both x86 and arm64 can cache TLB for prefetches and speculative
execution. Then why are their flush policies different?

Thanks,
Gang Li