Received-SPF: pass (google.com: domain of linux-kernel+bounces-101043-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Message-ID: <tencent_DDDFB09AF2B2DC73F9570E5A02D929004609@qq.com>
Date: Wed, 13 Mar 2024 10:01:14 +0800
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] KVM: x86/mmu: treat WC memory as MMIO
Content-Language: en-US
To: Sean Christopherson <seanjc@google.com>
Cc: dave.hansen@linux.intel.com, luto@kernel.org, peterz@infradead.org,
 tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, x86@kernel.org,
 hpa@zytor.com, rdunlap@infradead.org, akpm@linux-foundation.org,
 bhelgaas@google.com, mawupeng1@huawei.com, linux-kernel@vger.kernel.org,
 pbonzini@redhat.com, kvm@vger.kernel.org
References: <tencent_4B50D08D2E6211E4F9B867F0531F2C05BA0A@qq.com>
 <Ze8vM6HcU4vnXVSS@google.com>
 <tencent_AA5D14EAA36D58807959EE9AFC9E07548108@qq.com>
 <ZfBzBUbxpF9MpII-@google.com>
From: francisco flynn <francisco_flynn@foxmail.com>
In-Reply-To: <ZfBzBUbxpF9MpII-@google.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 2024/3/12 23:21, Sean Christopherson wrote:
> On Tue, Mar 12, 2024, francisco flynn wrote:
>> On 2024/3/12 00:20, Sean Christopherson wrote:
>>> On Mon, Mar 11, 2024, francisco_flynn wrote:
>>>> when doing kvm_tdp_mmu_map for WC memory, such as pages
>>>> allocated by amdgpu ttm driver for ttm_write_combined
>>>> caching mode(e.g. host coherent in vulkan),
>>>> the spte would be set to WB, in this case, vcpu write
>>>> to these pages would goes to cache first, and never
>>>> be write-combined and host-coherent anymore. so
>>>> WC memory should be treated as MMIO, and the effective
>>>> memory type is depending on guest PAT.
>>>
>>> No, the effective memtype is not fully guest controlled.  By forcing the EPT memtype
>>> to UC, the guest can only use UC or WC.  I don't know if there's a use case for
>>
>> Well,it's actually the host mapping memory WC and guest uses WC,
> 
> No, when the guest is running, the host, i.e. KVM, sets the EPT memory type to UC
> 
>   static u8 vmx_get_mt_mask(struct kvm_vcpu *vcpu, gfn_t gfn, bool is_mmio)
>   {
> 	if (is_mmio)
> 		return MTRR_TYPE_UNCACHABLE << VMX_EPT_MT_EPTE_SHIFT;
> 
> which effectively makes the guest "MTRR" memtype UC, and thus restricts the guest
> to using UC or WC.
> 
> Your use case wants to map the memory as WC in the guest, but there are zero
> guarantees that *every* use case wants to access such memory as WC (or UC),
> i.e. forcing UC could cause performance regressions for existing use cases.
> 

yes, this may cause performance regressions in some cases.

> Ideally, KVM would force the EPT memtype to match the host PAT memtype while still
> honoring guest PAT, but if we wanted to go that route, then KVM should (a) stuff
> the exact memtype, (b) stuff the memtype for all of guest memory, and (c) do so
> for all flavors of KVM on x86, not just EPT on VMX.
> 

it's true.

> Unfortunatley, making that change now risks breaking 15+ years of KVM ABI.  And
> there's also the whole "unsafe on Intel CPUs without self-snoop" problem.
> 
>> one use case is virtio-gpu host blob, which is to map physical GPU buffers into guest
>>
>>> the host mapping memory WC while the guest uses WB, but it should be a moot point,
>>> because this this series should do what you want (allow guest to map GPU buffers
>>> as WC).
>>>
>>> https://lore.kernel.org/all/20240309010929.1403984-1-seanjc@google.com
>>>
>>
>> yes, this is what i want, but for virtio-gpu device, if we mapping WC typed 
>> GPU buffer into guest, kvm_arch_has_noncoherent_dma would return false, 
>> so on cpu without self-snoop support, guest PAT will be ignored, the effective
>> memory type would be set to WB, causing data inconsistency.
> 
> My understanding is that every Intel CPU released in the last 8+ years supports
> self-snoop.  See check_memory_type_self_snoop_errata().
> 
> IMO, that's a perfectly reasonable line to draw: if you want virtio-gpu support,
> upgrade to Ivy Bridge or later.

it make sense.