Received-SPF: pass (google.com: domain of linux-kernel+bounces-103119-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Message-ID: <73c670d1-0301-49bf-472c-97ae8d1b6c7c@oracle.com>
Date: Thu, 14 Mar 2024 03:31:10 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.15.1
Subject: Re: [PATCH 1/5] KVM: x86: Remove VMX support for virtualizing guest
 MTRR memtypes
To: Sean Christopherson <seanjc@google.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>,
        Lai Jiangshan <jiangshanlai@gmail.com>,
        "Paul E. McKenney"
 <paulmck@kernel.org>,
        Josh Triplett <josh@joshtriplett.org>, kvm@vger.kernel.org,
        rcu@vger.kernel.org, linux-kernel@vger.kernel.org,
        Kevin Tian <kevin.tian@intel.com>, Yan Zhao <yan.y.zhao@intel.com>,
        Yiwei Zhang <zzyiwei@google.com>
References: <20240309010929.1403984-1-seanjc@google.com>
 <20240309010929.1403984-2-seanjc@google.com>
 <5ee34382-b45b-2069-ea33-ef58acacaa79@oracle.com>
 <ZfCL8mCmmEx5wGwv@google.com>
Content-Language: en-US
From: Dongli Zhang <dongli.zhang@oracle.com>
In-Reply-To: <ZfCL8mCmmEx5wGwv@google.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?Qlo4Um9WZW1RQ01xZWNYa1B0S21oeFpya2JpaWVFQmNvVHZ6dTVkSUkyZjZX?=
 =?utf-8?B?b1VESUZtL1dVcjYzY1VOMDdZYTJDR0xhb0FXb1E0N2xtckxlRW5zRlZrdEJ6?=
 =?utf-8?B?MlhoR2xyMXJGVWEwZmhnUXZyZXJ0b1B2RzU1TTJ0UXdHMTh4Mlh0VnFWWkVG?=
 =?utf-8?B?V3l6OGRiMFY1d0FQNXhaNkFQL1JrNkxLNkpXM2FHTG9OVnFiTnZZcENNMkk1?=
 =?utf-8?B?ZzkxOVROc0lkTGI2YjJmeDlrZEJoejlhZk1iamhsckZIL1ExbnhtWk5iVTdY?=
 =?utf-8?B?cEJoUENJVG1JVmxuQ1pqSEg0Qkx0SXFaS1lOb2lzV0phYmFCT09ZM3N2dVRC?=
 =?utf-8?B?c3FzeGs3OWdGV3pCQnBESFdza3FtYzRiUXdkbm5sd1BpOGJkTGRVN0xOdVVo?=
 =?utf-8?B?a3JYbC9RUW5xMWt1WU1UL0RyK3docGRJZkNYdUh0RUFUUzNRdElNR1ZIUjFZ?=
 =?utf-8?B?VmVVdnMwQmtSSlNYZWdLTmYzaW5vTGs0TnBSRENNeGRKbkFXa2Jqc1hrNCtE?=
 =?utf-8?B?YkZyQ0NGMHdxc2t4dWtyM09MSUMzZFF5MzZSRUtDdmtUQ2xXdmpISjJ1UTdS?=
 =?utf-8?B?NVNPYXhQNFBZVlF6RnduV2N0ODg3RDkrYWt0VGJtNkRYRXNPR3hKMGI4K2Z1?=
 =?utf-8?B?Wnkydzlpb2trMlpncllJK3dmSXR0eUpPemdidmJoWFZmOEpWbTNRM0w2ZGRz?=
 =?utf-8?B?WFBYOFlTaWtDRFNiK0pNYUhDdmVwZ1VxMERZMkdiRndGSDR6MGZ2V01OZ1Zn?=
 =?utf-8?B?cFV4TXZrTzJGMUVEYzloODgyU2thOHVsTmp5NUExS2t1Z3ZZOUJaSmhpREd0?=
 =?utf-8?B?SURGZ0lUekY2Zm1NTXhwLy9OL2xzeWIxUFJKRUVIRUtEc0JVT1QydkF4SDk4?=
 =?utf-8?B?bngvVzJvcERxb3JCMnlYSVcxR0E4UitzV0p3T1lobnJ4MmwyTDJsK3c4cEg0?=
 =?utf-8?B?QWFlUGFtQ1prSUMxd1U5WUdmaWlnS1hQZHBIV3NvOElIWXpNb0ZFcnZOd29n?=
 =?utf-8?B?RHphNWhBbG9Db0RBcC9KZnI1QzFucllhUklqMkFHbEMwdmY3dndOMkNpUUNJ?=
 =?utf-8?B?TWFhTVMxei9aZDV3dFg4UTZvQjFUaFZ5YWIzYUtSVXc0Tk1qb2QvOHZUdkI3?=
 =?utf-8?B?TGFnRHhBYzU1dkV3TTQ0TnRUZ3dWeDJoNXhKdE5yR003MmdueEtoMW1tcVN2?=
 =?utf-8?B?WTFoUEQzaHpxRU85LzhIYmk3TmhmbUVCQi91V2dwbmc2OFB1UFdSOHFod2lu?=
 =?utf-8?B?ZGpEUURtYTNHWlM0amhzU0NoYUlEOFMvVzQ0aEM1OStkT01XRWxjdnVCQ0hC?=
 =?utf-8?B?WkJvQzhHUFBXRGhZbXhmYUxSVXIrRnZwK2pXQm9sNUhiajJBaDNPSHpZZzh4?=
 =?utf-8?B?anptbmVuS2pXbmNiT3BzQXJMZHIwNkJpYTMzWHRHSXB5WVhZbGNveUtBYmVX?=
 =?utf-8?B?VmpKMkdSWitYbXZvaTBxSEtueXJlQ09MSmRlMW1jWUU2MmhFUzFyUnh0Ykgz?=
 =?utf-8?B?YkswOEhYNFZwb1U0OWNLVTF4MlZxMWRjTHlxWVVSNVJYeHpIWVNSTkRIYWd0?=
 =?utf-8?B?TFp4cUVpNTlOOW1zTWZESkMrZ2FTMVN1bDB1SEtVVWp0UVFRTnV6U3oraWhw?=
 =?utf-8?B?eXdkdU9iMzdmRVlybEpYZ2lSRGhYN1ZCU21aSHhMeFlXdE1pRDdHbGk2RjlD?=
 =?utf-8?B?eERWeDRZdDU2MnVOVmpWa1QxN0dlL3BqYlljTzcybmtTOFRwMU40ZzN6anZh?=
 =?utf-8?B?blRRV21UelNzSmxvS1Mya1NwK1N1d3R4Tk5oZVRkU0tFL3BWNitBcUREZy8v?=
 =?utf-8?B?NjRnMHBiOXpLL2F5RktTVFZhWU1UcXpvZTRvVFJNaTFYOTV4MDE0V05OZWtn?=
 =?utf-8?B?bGV3WkE1UlJQWjhDVWRFd2hzaXFmR2pkOXZmNkQxa1hqQytyMXlITks5S2NJ?=
 =?utf-8?B?aWNxVDZudDlIUHN5ZW9qOE8rY2ZHc0hFSmJXOWFkeStCaHJLSUJoWnFsN2NG?=
 =?utf-8?B?bEdNSGpyWmJFdVZManhOUDhHalpDSnhuYW55RnN5cVE2bFNtWjQxMmFvYlBS?=
 =?utf-8?B?UTF2blNnUjdJdElkQWN5aFFBc3IxbkhqMDRaM052eS9yTVBRN1pVeERSY0xP?=
 =?utf-8?Q?12U2UYygX7DR2SjwlBOGLUzMM?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	4UJS+yTOlA6nxsg7tu2BBsaycte53eoIe0CaxRvSxzWCxu/M5ZFOpSrsONGnXOMosH0hZuAHUmze3lheqkU3Hx/ClNgOSC43Qml6rVeHT6xKNsB+zbQM1JZ+JpxcMGE7eM8V6F1UPPSd0y+CWgjspwKbwmsAnHerCDhOHOmUUtzwspz3BJpJLDC0rkskmps5VnsDWUSwhL7bMSXrZDG5s3uB4mgJFElVMCQi1Cqi9H+eyp3a+yWJ1pM0ge6WAOC/CTIWlTQLUuAzlxSogn1bPExFLpHuDz01oHbXRsq7YGVG3TueVMmWKtETm8wS0/Pl8tlLYcWRJpdERoBBMhaZy4UqHhGxVxxzjyX+uretkCsxzryjGM/HctmXhIxkdLIAPDboLvAcSG1BhBtVB+2MBFLj5ZckNiF0WB/QKx1cxMWtS1IjlkLjZZjYeImItJqZ/4QkGaMxQakXsDr9sg/8HaG7qi04HnmK0IozI9l+IzpgMNF4EdHnN9tjYvmgONGQHI6O0ybbXecE8ITvY0eJ3Ykk/Xu8zr67/x1wRmuJEvi+QH1Im3wp8YEIaXWxnJo3NlChYkt38tiOSKRudQvrg8V2hRl6JrBUmveFxvZNO5U=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b2c5715b-8f1a-48f4-f8aa-08dc4411e041
X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB2663.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Mar 2024 10:31:14.9211
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: wA5NbcVdjEbtwRovkiBwMMUYgrbOZmVZwuXQguuRBZHA5bnrp9X4VA4Y5W/hIh/IF/WCAWbAFHJnOoD9RyzIng==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR10MB4260
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2024-03-14_08,2024-03-13_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 spamscore=0
 mlxlogscore=999 bulkscore=0 phishscore=0 malwarescore=0 mlxscore=0
 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2311290000 definitions=main-2403140074
X-Proofpoint-GUID: LqbD3E7HzrP6IgqhLrsx3r8Qx9OEG5XK
X-Proofpoint-ORIG-GUID: LqbD3E7HzrP6IgqhLrsx3r8Qx9OEG5XK


On 3/12/24 10:08, Sean Christopherson wrote:
> On Mon, Mar 11, 2024, Dongli Zhang wrote:
>>
>>
>> On 3/8/24 17:09, Sean Christopherson wrote:
>>> Remove KVM's support for virtualizing guest MTRR memtypes, as full MTRR
>>> adds no value, negatively impacts guest performance, and is a maintenance
>>> burden due to it's complexity and oddities.
>>>
>>> KVM's approach to virtualizating MTRRs make no sense, at all.  KVM *only*
>>> honors guest MTRR memtypes if EPT is enabled *and* the guest has a device
>>> that may perform non-coherent DMA access.  From a hardware virtualization
>>> perspective of guest MTRRs, there is _nothing_ special about EPT.  Legacy
>>> shadowing paging doesn't magically account for guest MTRRs, nor does NPT.
>>
>> [snip]
>>
>>>  
>>> -bool __kvm_mmu_honors_guest_mtrrs(bool vm_has_noncoherent_dma)
>>> +bool kvm_mmu_may_ignore_guest_pat(void)
>>>  {
>>>  	/*
>>> -	 * If host MTRRs are ignored (shadow_memtype_mask is non-zero), and the
>>> -	 * VM has non-coherent DMA (DMA doesn't snoop CPU caches), KVM's ABI is
>>> -	 * to honor the memtype from the guest's MTRRs so that guest accesses
>>> -	 * to memory that is DMA'd aren't cached against the guest's wishes.
>>> -	 *
>>> -	 * Note, KVM may still ultimately ignore guest MTRRs for certain PFNs,
>>> -	 * e.g. KVM will force UC memtype for host MMIO.
>>> +	 * When EPT is enabled (shadow_memtype_mask is non-zero), and the VM
>>> +	 * has non-coherent DMA (DMA doesn't snoop CPU caches), KVM's ABI is to
>>> +	 * honor the memtype from the guest's PAT so that guest accesses to
>>> +	 * memory that is DMA'd aren't cached against the guest's wishes.  As a
>>> +	 * result, KVM _may_ ignore guest PAT, whereas without non-coherent DMA,
>>> +	 * KVM _always_ ignores guest PAT (when EPT is enabled).
>>>  	 */
>>> -	return vm_has_noncoherent_dma && shadow_memtype_mask;
>>> +	return shadow_memtype_mask;
>>>  }
>>>  
>>
>> Any special reason to use the naming 'may_ignore_guest_pat', but not
>> 'may_honor_guest_pat'?
> 
> Because which (after this series) is would either be misleading or outright wrong.
> If KVM returns true from the helper based solely on shadow_memtype_mask, then it's
> misleading because KVM will *always* honors guest PAT for such CPUs.  I.e. that
> name would yield this misleading statement.
> 
>   If the CPU supports self-snoop, KVM may honor guest PAT.
> 
> If KVM returns true iff self-snoop is NOT available (as proposed in this series),
> then it's outright wrong as KVM would return false, i.e. would make this incorrect
> statement:
> 
>   If the CPU supports self-snoop, KVM never honors guest PAT.
> 
> As saying that KVM may not or cannot do something is saying that KVM will never
> do that thing.
> 
> And because the EPT flag is "ignore guest PAT", not "honor guest PAT", but that's
> as much coincidence as it is anything else.
> 
>> Since it is also controlled by other cases, e.g., kvm_arch_has_noncoherent_dma()
>> at vmx_get_mt_mask(), it can be 'may_honor_guest_pat' too?
>>
>> Therefore, why not directly use 'shadow_memtype_mask' (without the API), or some
>> naming like "ept_enabled_for_hardware".
> 
> Again, after this series, KVM will *always* honor guest PAT for CPUs with self-snoop,
> i.e. KVM will *never* ignore guest PAT.  But for CPUs without self-snoop (or with
> errata), KVM conditionally honors/ignores guest PAT.
> 
>> Even with the code from PATCH 5/5, we still have high chance that VM has
>> non-coherent DMA?
> 
> I don't follow.  On CPUs with self-snoop, whether or not the VM has non-coherent
> DMA (from VFIO!) is irrelevant.  If the CPU has self-snoop, then KVM can safely
> honor guest PAT at all times.


Thank you very much for the explanation.

According to my understanding of the explanation (after this series):

1. When static_cpu_has(X86_FEATURE_SELFSNOOP) == true, it is 100% to "honor
guest PAT".

2. When static_cpu_has(X86_FEATURE_SELFSNOOP) == false (and
shadow_memtype_mask), although only 50% chance (depending on where there is
non-coherent DMA), at least now it is NOT 100% (to honor guest PAT) any longer.

Due to the fact it is not 100% (to honor guest PAT) any longer, there starts the
trend (from 100% to 50%) to "ignore guest PAT", that is:
kvm_mmu_may_ignore_guest_pat().

Dongli Zhang