Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Reply-To: Gavin Shan <gshan@redhat.com>
Subject: Re: [PATCH v7 00/22] Support SDEI Virtualization
To:     Shijie Huang <shijie@amperemail.onmicrosoft.com>,
        kvmarm@lists.cs.columbia.edu
Cc:     maz@kernel.org, linux-kernel@vger.kernel.org, eauger@redhat.com,
        oupton@google.com, Jonathan.Cameron@huawei.com,
        pbonzini@redhat.com, vkuznets@redhat.com, will@kernel.org,
        james.morse@arm.com, mark.rutland@arm.com,
        shannon.zhaosl@gmail.com, shan.gavin@gmail.com
References: <20220527080253.1562538-1-gshan@redhat.com>
 <c57e0686-bdaf-2646-43d9-f0ac54e7eb22@amperemail.onmicrosoft.com>
From:   Gavin Shan <gshan@redhat.com>
Message-ID: <4c3e5219-9d77-fa9f-798d-43f261228f59@redhat.com>
Date:   Mon, 30 May 2022 14:57:40 +0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.2.0
MIME-Version: 1.0
In-Reply-To: <c57e0686-bdaf-2646-43d9-f0ac54e7eb22@amperemail.onmicrosoft.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Precedence: bulk

Hi Shijie,

On 5/30/22 2:47 PM, Shijie Huang wrote:
> On 2022/5/27 16:02, Gavin Shan wrote:
>>
>> This series intends to virtualize Software Delegated Exception Interface
>> (SDEI), which is defined by DEN0054C (v1.1). It allows the hypervisor to
>> deliver NMI-alike SDEI event to guest and it's needed by Async PF to
>> deliver page-not-present notification from hypervisor to guest. The code
>> and the required qemu changes can be found from:
>>
>>     https://developer.arm.com/documentation/den0054/c
>>     https://github.com/gwshan/linux    ("kvm/arm64_sdei")
>>     https://github.com/gwshan/qemu     ("kvm/arm64_sdei")
>>
>> The design is quite strightforward by following the specification. The
>> (SDEI) events are classified into the shared and private ones according
>> to their scope. The shared event is system or VM scoped, but the private
>> event is vcpu scoped. This implementation doesn't support the shared
>> event because all the needed events are private. Besides, the critial
>> events aren't supported by the implementation either. It means all events
>> are normal in terms of priority.
>>
>> There are several objects (data structures) introduced to help on the
>> event registration, enablement, disablement, unregistration, reset,
>> delivery and handling.
>>
>>    * kvm_sdei_event_handler
>>      SDEI event handler, which is provided through EVENT_REGISTER
>>      hypercall, is called when the SDEI event is delivered from
>>      host to guest.
>>
>>    * kvm_sdei_event_context
>>      The saved (preempted) context when SDEI event is delivered
>>      for handling.
>>
>>    * kvm_sdei_vcpu
>>      SDEI events and their states.
>>
>> The patches are organized as below:
>>
>>    PATCH[01-02] Preparatory work to extend smccc_get_argx() and refactor
>>                 hypercall routing mechanism
>>    PATCH[03]    Adds SDEI virtualization infrastructure
>>    PATCH[04-16] Supports various SDEI hypercalls and event handling
>>    PATCH[17]    Exposes SDEI capability
>>    PATCH[18-19] Support SDEI migration
>>    PATCH[20]    Adds document about SDEI
>>    PATCH[21-22] SDEI related selftest cases
>>
>> The previous revisions can be found:
>>
>>    v6: https://lore.kernel.org/lkml/20220403153911.12332-4-gshan@redhat.com/T/
>>    v5: https://lore.kernel.org/kvmarm/20220322080710.51727-1-gshan@redhat.com/
>>    v4: https://lore.kernel.org/kvmarm/20210815001352.81927-1-gshan@redhat.com/
>>    v3: https://lore.kernel.org/kvmarm/20210507083124.43347-1-gshan@redhat.com/
>>    v2: https://lore.kernel.org/kvmarm/20210209032733.99996-1-gshan@redhat.com/
>>    v1: https://lore.kernel.org/kvmarm/20200817100531.83045-1-gshan@redhat.com/
>>
>> Testing
>> =======
>> [1] The selftest case included in this series works fine. The default SDEI
>>      event, whose number is zero, can be registered, enabled, raised. The
>>      SDEI event handler can be invoked.
>>
>>      [host]# pwd
>>      /home/gavin/sandbox/linux.main/tools/testing/selftests/kvm
>>      [root@virtlab-arm01 kvm]# ./aarch64/sdei
>>
>>          NR_VCPUS: 2    SDEI Event: 0x00000000
>>
>>      --- VERSION
>>          Version:              1.1 (vendor: 0x4b564d)
>>      --- FEATURES
>>          Shared event slots:   0
>>          Private event slots:  0
>>          Relative mode:        No
>>      --- PRIVATE_RESET
>>      --- SHARED_RESET
>>      --- PE_UNMASK
>>      --- EVENT_GET_INFO
>>          Type:                 Private
>>          Priority:             Normal
>>          Signaled:             Yes
>>      --- EVENT_REGISTER
>>      --- EVENT_ENABLE
>>      --- EVENT_SIGNAL
>>          Handled:              Yes
>>          IRQ:                  No
>>          Status:               Registered-Enabled-Running
>>          PC/PSTATE:            000000000040232c 00000000600003c5
>>          Regs:                 0000000000000000 0000000000000000
>>                                0000000000000000 0000000000000000
>>      --- PE_MASK
>>      --- EVENT_DISABLE
>>      --- EVENT_UNREGISTER
>>
>>          Result: OK
>>
>> [2] There are additional patches in the following repositories to create
>>      procfs entries, allowing to inject SDEI event from host side. The
>>      SDEI client in the guest side registers the SDEI default event, whose
>>      number is zero. Also, the QEMU exports SDEI ACPI table and supports
>>      migration for SDEI.
>>
>>      https://github.com/gwshan/linux    ("kvm/arm64_sdei")
>>      https://github.com/gwshan/qemu     ("kvm/arm64_sdei")
>>
>>      [2.1] Start the guests and migrate the source VM to the destination
>>            VM.
>>
>>      [host]# /home/gavin/sandbox/qemu.main/build/qemu-system-aarch64       \
>>              -accel kvm -machine virt,gic-version=host                     \
>>              -cpu host -smp 6,sockets=2,cores=3,threads=1                  \
>>              -m 1024M,slots=16,maxmem=64G                                  \
>>                 :                                                          \
>>              -kernel /home/gavin/sandbox/linux.guest/arch/arm64/boot/Image \
>>              -initrd /home/gavin/sandbox/images/rootfs.cpio.xz             \
>>              -append earlycon=pl011,mmio,0x9000000                         \
>>                 :
>>
>>      [host]# /home/gavin/sandbox/qemu.main/build/qemu-system-aarch64       \
>>              -accel kvm -machine virt,gic-version=host                     \
>>              -cpu host -smp 6,sockets=2,cores=3,threads=1                  \
>>              -m 1024M,slots=16,maxmem=64G                                  \
>>                 :                                                          \
>>              -kernel /home/gavin/sandbox/linux.guest/arch/arm64/boot/Image \
>>              -initrd /home/gavin/sandbox/images/rootfs.cpio.xz             \
>>              -append earlycon=pl011,mmio,0x9000000                         \
>>              -incoming tcp:0:4444                                          \
>>                 :
>>
>>      [2.2] Check kernel log on the source VM. The SDEI service is enabled
>>            and the default SDEI event (0x0) is enabled.
>>
>>       [guest-src]# dmesg | grep -i sdei
>>       ACPI: SDEI 0x000000005BC80000 000024 \
>>                  (v00 BOCHS  BXPC     00000001 BXPC 00000001)
>>       sdei: SDEIv1.1 (0x4b564d) detected in firmware.
>>       SDEI TEST: Version 1.1, Vendor 0x4b564d
>>       sdei_init: SDEI event (0x0) registered
>>       sdei_init: SDEI event (0x0) enabled
>>
>>
>>       (qemu) migrate -d tcp:localhost:4444
>>
>>      [2.3] Migrate the source VM to the destination VM. Inject SDEI event
>>            to the destination VM. The event is raised and handled.
>>
>>      (qemu) migrate -d tcp:localhost:4444
>>
>>      [host]# echo 0 > /proc/kvm/kvm-5360/vcpu-1
>>
>>      [guest-dst]#
>>      =========== SDEI Event (CPU#1) ===========
>>      Event: 0000000000000000  Parameter: 00000000dabfdabf
>>      PC:    ffff800008cbb554  PSTATE: 00000000604000c5  SP: ffff800009c7bde0
>>      Regs:    00000000000016ee ffff00001ffd2e28 00000000000016ed 0000000000000001
>>               ffff800016c28000 0000000000000000 0000000000000000 0000000000000000
>>               0000000000000000 0000000000000000 0000000000000000 0000000000000000
>>               0000000000000000 0000000000000000 0000000000000000 0000000000000000
>>               0000000000000000 0000000000000000 0000000000000000 ffff800009399008
>>               ffff8000097d9af0 ffff8000097d99f8 ffff8000093a8db8 ffff8000097d9b18
>>               0000000000000000 0000000000000000 ffff000000339d00 0000000000000000
>>               0000000000000000 ffff800009c7bde0 ffff800008cbb5c4
>>      Context: 00000000000016ee ffff00001ffd2e28 00000000000016ed 0000000000000001
>>               ffff800016c28000 03ffffffffffffff 000000024325db59 ffff8000097de190
>>               ffff00000033a790 ffff800008cbb814 0000000000000a30 0000000000000000
> 
> I tested this patch set. It's okay.
> 
> Tested-by: Huang Shijie <shijie@os.amperecomputing.com>
> 

[...]

Appreciate your efforts to test it through. I will have your
tested-by if respin is needed. Thank you for your time on this.

Thanks,
Gavin