Received-SPF: pass (google.com: domain of linux-kernel+bounces-89315-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Subject: Re: [PATCH v5 3/6] LoongArch: KVM: Add cpucfg area for kvm hypervisor
To: WANG Xuerui <kernel@xen0n.name>, Jiaxun Yang <jiaxun.yang@flygoat.com>,
 Huacai Chen <chenhuacai@kernel.org>
Cc: Tianrui Zhao <zhaotianrui@loongson.cn>, Juergen Gross <jgross@suse.com>,
 Paolo Bonzini <pbonzini@redhat.com>, loongarch@lists.linux.dev,
 linux-kernel@vger.kernel.org, virtualization@lists.linux.dev,
 kvm@vger.kernel.org
References: <20240222032803.2177856-1-maobibo@loongson.cn>
 <20240222032803.2177856-4-maobibo@loongson.cn>
 <CAAhV-H5eqXMqTYVb6cAVqOsDNcEDeP9HzaMKw69KFQeVaAYEdA@mail.gmail.com>
 <d1a6c424-b710-74d6-29f6-e0d8e597e1fb@loongson.cn>
 <CAAhV-H7p114hWUVrYRfKiBX3teG8sG7xmEW-Q-QT3i+xdLqDEA@mail.gmail.com>
 <06647e4a-0027-9c9f-f3bd-cd525d37b6d8@loongson.cn>
 <85781278-f3e9-4755-8715-3b9ff714fb20@app.fastmail.com>
 <0d428e30-07a8-5a91-a20c-c2469adbf613@loongson.cn>
 <327808dd-ac34-4c61-9992-38642acc9419@xen0n.name>
 <62cc24fd-025a-53c6-1c8e-2d20de54d297@loongson.cn>
 <431111f3-d84a-4311-986d-eebd91559cd3@xen0n.name>
From: maobibo <maobibo@loongson.cn>
Message-ID: <9881fa84-74c4-b7e8-63fb-607a22b1eee2@loongson.cn>
Date: Sat, 2 Mar 2024 10:57:36 +0800
User-Agent: Mozilla/5.0 (X11; Linux loongarch64; rv:68.0) Gecko/20100101
 Thunderbird/68.7.0
Precedence: bulk
MIME-Version: 1.0
In-Reply-To: <431111f3-d84a-4311-986d-eebd91559cd3@xen0n.name>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit


On 2024/2/27 下午6:19, WANG Xuerui wrote:
> On 2/27/24 18:12, maobibo wrote:
>>
>>
>> On 2024/2/27 下午5:10, WANG Xuerui wrote:
>>> On 2/27/24 11:14, maobibo wrote:
>>>>
>>>>
>>>> On 2024/2/27 上午4:02, Jiaxun Yang wrote:
>>>>>
>>>>>
>>>>> 在2024年2月26日二月 上午8:04，maobibo写道：
>>>>>> On 2024/2/26 下午2:12, Huacai Chen wrote:
>>>>>>> On Mon, Feb 26, 2024 at 10:04 AM maobibo <maobibo@loongson.cn> 
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 2024/2/24 下午5:13, Huacai Chen wrote:
>>>>>>>>> Hi, Bibo,
>>>>>>>>>
>>>>>>>>> On Thu, Feb 22, 2024 at 11:28 AM Bibo Mao <maobibo@loongson.cn> 
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Instruction cpucfg can be used to get processor features. And 
>>>>>>>>>> there
>>>>>>>>>> is trap exception when it is executed in VM mode, and also it is
>>>>>>>>>> to provide cpu features to VM. On real hardware cpucfg area 0 
>>>>>>>>>> - 20
>>>>>>>>>> is used.  Here one specified area 0x40000000 -- 0x400000ff is 
>>>>>>>>>> used
>>>>>>>>>> for KVM hypervisor to privide PV features, and the area can be 
>>>>>>>>>> extended
>>>>>>>>>> for other hypervisors in future. This area will never be used for
>>>>>>>>>> real HW, it is only used by software.
>>>>>>>>> After reading and thinking, I find that the hypercall method 
>>>>>>>>> which is
>>>>>>>>> used in our productive kernel is better than this cpucfg method.
>>>>>>>>> Because hypercall is more simple and straightforward, plus we 
>>>>>>>>> don't
>>>>>>>>> worry about conflicting with the real hardware.
>>>>>>>> No, I do not think so. cpucfg is simper than hypercall, 
>>>>>>>> hypercall can
>>>>>>>> be in effect when system runs in guest mode. In some scenario 
>>>>>>>> like TCG
>>>>>>>> mode, hypercall is illegal intruction, however cpucfg can work.
>>>>>>> Nearly all architectures use hypercall except x86 for its historical
>>>>>> Only x86 support multiple hypervisors and there is multiple 
>>>>>> hypervisor
>>>>>> in x86 only. It is an advantage, not historical reason.
>>>>>
>>>>> I do believe that all those stuff should not be exposed to guest 
>>>>> user space
>>>>> for security reasons.
>>>> Can you add PLV checking when cpucfg 0x40000000-0x400000FF is 
>>>> emulated? if it is user mode return value is zero and it is kernel 
>>>> mode emulated value will be returned. It can avoid information leaking.
>>>
>>> I've suggested this approach in another reply [1], but I've rechecked 
>>> the manual, and it turns out this behavior is not permitted by the 
>>> current wording. See LoongArch Reference Manual v1.10, Volume 1, 
>>> Section 2.2.10.5 "CPUCFG":
>>>
>>>  > CPUCFG 访问未定义的配置字将读回全 0 值。
>>>  >
>>>  > Reads of undefined CPUCFG configuration words shall return 
>>> all-zeroes.
>>>
>>> This sentence mentions no distinction based on privilege modes, so it 
>>> can only mean the behavior applies universally regardless of 
>>> privilege modes.
>>>
>>> I think if you want to make CPUCFG behavior PLV-dependent, you may 
>>> have to ask the LoongArch spec editors, internally or in public, for 
>>> a new spec revision.
>> No, CPUCFG behavior between CPUCFG0-CPUCFG21 is unchanged, only that 
>> it can be defined by software since CPUCFG 0x400000000 is used by 
>> software.
> 
> The 0x40000000 range is not mentioned in the manuals. I know you've 
> confirmed privately with HW team but this needs to be properly 
> documented for public projects to properly rely on.
> 
>>> (There are already multiple third-party LoongArch implementers as of 
>>> late 2023, so any ISA-level change like this would best be 
>>> coordinated, to minimize surprises.)
>> With document Vol 4-23
>> https://www.intel.com/content/dam/develop/external/us/en/documents/335592-sdm-vol-4.pdf 
>>
>>
>> There is one line "MSR address range between 40000000H - 400000FFH is 
>> marked as a specially reserved range. All existing and
>> future processors will not implement any features using any MSR in 
>> this range."
> 
> Thanks for providing this info, now at least we know why it's this 
> specific range of 0x400000XX that's chosen.
> 
>>
>> It only says that it is reserved, it does not say detailed software 
>> behavior. Software behavior is defined in hypervisor such as:
>> https://github.com/MicrosoftDocs/Virtualization-Documentation/blob/main/tlfs/Requirements%20for%20Implementing%20the%20Microsoft%20Hypervisor%20Interface.pdf 
>>
>> https://kb.vmware.com/s/article/1009458
>>
>> If hypercall method is used, there should be ABI also like aarch64:
>> https://documentation-service.arm.com/static/6013e5faeee5236980d08619
> 
> Yes proper documentation of public API surface is always necessary 
> *before* doing real work. Because right now the hypercall provider is 
> Linux KVM, maybe we can document the existing and planned hypercall 
> usage and ABI in the kernel docs along with code changes.
> 
yes, I will add hypercall in kernel docs.

SMC calling convention is ABI between OS and secure firmware, LoongArch 
has no secure mode, it not necessary to has such doc.The hypercall 
calling convention is relative with hypervisor SW, not relative with CPU 
HW vendor. Each hypervisor maybe has its separate hypercall calling 
convention just like syscall ABIs.

Regards
Bibo Mao