Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <48c10cf4-98c7-9203-3edc-ad88fdd1c7bf@linux.ibm.com>
Date:   Mon, 27 Jun 2022 15:34:59 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.3.0
Subject: Re: [PATCH v10 3/3] KVM: s390: resetting the Topology-Change-Report
Content-Language: en-US
To:     Janosch Frank <frankja@linux.ibm.com>, kvm@vger.kernel.org
Cc:     linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org,
        borntraeger@de.ibm.com, cohuck@redhat.com, david@redhat.com,
        thuth@redhat.com, imbrenda@linux.ibm.com, hca@linux.ibm.com,
        gor@linux.ibm.com, wintera@linux.ibm.com, seiden@linux.ibm.com,
        nrb@linux.ibm.com
References: <20220620125437.37122-1-pmorel@linux.ibm.com>
 <20220620125437.37122-4-pmorel@linux.ibm.com>
 <7d50c2df-7cad-dbc6-baa0-ab647f8dde4e@linux.ibm.com>
From:   Pierre Morel <pmorel@linux.ibm.com>
In-Reply-To: <7d50c2df-7cad-dbc6-baa0-ab647f8dde4e@linux.ibm.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk


On 6/24/22 08:50, Janosch Frank wrote:
> On 6/20/22 14:54, Pierre Morel wrote:
>> During a subsystem reset the Topology-Change-Report is cleared.
>> Let's give userland the possibility to clear the MTCR in the case
>> of a subsystem reset.
>>
>> To migrate the MTCR, we give userland the possibility to
>> query the MTCR state.
>>
>> We indicate KVM support for the CPU topology facility with a new
>> KVM capability: KVM_CAP_S390_CPU_TOPOLOGY.
>>
>> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
>> ---
>>   Documentation/virt/kvm/api.rst   | 31 +++++++++++
>>   arch/s390/include/uapi/asm/kvm.h | 10 ++++
>>   arch/s390/kvm/kvm-s390.c         | 96 ++++++++++++++++++++++++++++++++
>>   include/uapi/linux/kvm.h         |  1 +
>>   4 files changed, 138 insertions(+)
>>
>> diff --git a/Documentation/virt/kvm/api.rst 
>> b/Documentation/virt/kvm/api.rst
>> index 11e00a46c610..326f8b7e7671 100644
>> --- a/Documentation/virt/kvm/api.rst
>> +++ b/Documentation/virt/kvm/api.rst
>> @@ -7956,6 +7956,37 @@ should adjust CPUID leaf 0xA to reflect that 
>> the PMU is disabled.
>>   When enabled, KVM will exit to userspace with KVM_EXIT_SYSTEM_EVENT of
>>   type KVM_SYSTEM_EVENT_SUSPEND to process the guest suspend request.
>> +8.37 KVM_CAP_S390_CPU_TOPOLOGY
>> +------------------------------
>> +
>> +:Capability: KVM_CAP_S390_CPU_TOPOLOGY
>> +:Architectures: s390
>> +:Type: vm
>> +
>> +This capability indicates that KVM will provide the S390 CPU Topology
>> +facility which consist of the interpretation of the PTF instruction for
>> +the Function Code 2 along with interception and forwarding of both the
> 
> Making function code capital surprises me when reading.

wanted to highlight FC.
I remove it.

> 
>> +PTF instruction with Function Codes 0 or 1 and the STSI(15,1,x)
>> +instruction to the userland hypervisor.
>> +
>> +The stfle facility 11, CPU Topology facility, should not be provided
> 
> s/provided/indicated
> 
OK

>> +to the guest without this capability.
>> +
>> +When this capability is present, KVM provides a new attribute group
>> +on vm fd, KVM_S390_VM_CPU_TOPOLOGY.
>> +This new attribute allows to get, set or clear the Modified Change
>> +Topology Report (MTCR) bit of the SCA through the kvm_device_attr
>> +structure.
>> +
>> +Getting the MTCR bit is realized by using a kvm_device_attr attr
>> +entry value of KVM_GET_DEVICE_ATTR and with kvm_device_attr addr
>> +entry pointing to the address of a struct kvm_cpu_topology.
>> +The value of the MTCR is return by the bit mtcr of the structure. > +
>> +When using KVM_SET_DEVICE_ATTR the MTCR is set by using the
>> +attr->attr value KVM_S390_VM_CPU_TOPO_MTCR_SET and cleared by
>> +using KVM_S390_VM_CPU_TOPO_MTCR_CLEAR.
> 
> I have the feeling that we can drop the two blocks above and we won't 
> loose information.
> 
>> +/**
>> + * kvm_s390_sca_clear_mtcr
>> + * @kvm: guest KVM description
>> + *
>> + * Is only relevant if the topology facility is present,
>> + * the caller should check KVM facility 11
>> + *
>> + * Updates the Multiprocessor Topology-Change-Report to signal
>> + * the guest with a topology change.
>> + */
>> +static void kvm_s390_sca_clear_mtcr(struct kvm *kvm)
> 
> This is a set operation with the value 0 and that's clearly visible by 
> the copied code. If you make the utility entry a bitfield you can easily 
> set 0/1 via one function without doing the bit manipulation by hand.

OK

> 
> I.e. please only use one set function.
> 
>> +{
>> +    struct bsca_block *sca = kvm->arch.sca; /* SCA version doesn't 
>> matter */
>> +
>> +    ipte_lock(kvm);
>> +    sca->utility &= ~SCA_UTILITY_MTCR;
>> +    ipte_unlock(kvm);
>> +}
>> +
>> +static int kvm_s390_set_topology(struct kvm *kvm, struct 
>> kvm_device_attr *attr)
>> +{
>> +    if (!test_kvm_facility(kvm, 11))
>> +        return -ENXIO;
>> +
>> +    switch (attr->attr) {
>> +    case KVM_S390_VM_CPU_TOPO_MTCR_SET:
>> +        kvm_s390_sca_set_mtcr(kvm);
>> +        break;
>> +    case KVM_S390_VM_CPU_TOPO_MTCR_CLEAR:
>> +        kvm_s390_sca_clear_mtcr(kvm);
>> +        break;
>> +    }
> 
> By having two endpoints here we trade an easy check with having to 
> access process memory to grab the value we want to set.
> 
> I'm still torn about this.
> 
>> +    return 0;
>> +}
>> +
>> +/**
>> + * kvm_s390_sca_get_mtcr
>> + * @kvm: guest KVM description
>> + *
>> + * Is only relevant if the topology facility is present,
>> + * the caller should check KVM facility 11
>> + *
>> + * reports to QEMU the Multiprocessor Topology-Change-Report.
>> + */
>> +static int kvm_s390_sca_get_mtcr(struct kvm *kvm)
>> +{
>> +    struct bsca_block *sca = kvm->arch.sca; /* SCA version doesn't 
>> matter */
> 
> Same comments as with the set_mtcr()

OK

> 
>> +    int val;
>> +
>> +    ipte_lock(kvm);
>> +    val = sca->utility & SCA_UTILITY_MTCR;
>> +    ipte_unlock(kvm);
>> +
>> +    return val;
>> +}
>> +
>> +static int kvm_s390_get_topology(struct kvm *kvm, struct 
>> kvm_device_attr *attr)
>> +{
>> +    struct kvm_cpu_topology topo = {};
>> +
>> +    if (!test_kvm_facility(kvm, 11))
>> +        return -ENXIO;
>> +
>> +    topo.mtcr = kvm_s390_sca_get_mtcr(kvm) ? 1 : 0;
>> +    if (copy_to_user((void __user *)attr->addr, &topo, sizeof(topo)))
>> +        return -EFAULT;
>> +
>> +    return 0;
>> +}
>> +
>>   static int kvm_s390_vm_set_attr(struct kvm *kvm, struct 
>> kvm_device_attr *attr)
>>   {
>>       int ret;
>> @@ -1730,6 +1817,9 @@ static int kvm_s390_vm_set_attr(struct kvm *kvm, 
>> struct kvm_device_attr *attr)
>>       case KVM_S390_VM_MIGRATION:
>>           ret = kvm_s390_vm_set_migration(kvm, attr);
>>           break;
>> +    case KVM_S390_VM_CPU_TOPOLOGY:
>> +        ret = kvm_s390_set_topology(kvm, attr);
>> +        break;
>>       default:
>>           ret = -ENXIO;
>>           break;
>> @@ -1755,6 +1845,9 @@ static int kvm_s390_vm_get_attr(struct kvm *kvm, 
>> struct kvm_device_attr *attr)
>>       case KVM_S390_VM_MIGRATION:
>>           ret = kvm_s390_vm_get_migration(kvm, attr);
>>           break;
>> +    case KVM_S390_VM_CPU_TOPOLOGY:
>> +        ret = kvm_s390_get_topology(kvm, attr);
>> +        break;
>>       default:
>>           ret = -ENXIO;
>>           break;
>> @@ -1828,6 +1921,9 @@ static int kvm_s390_vm_has_attr(struct kvm *kvm, 
>> struct kvm_device_attr *attr)
>>       case KVM_S390_VM_MIGRATION:
>>           ret = 0;
>>           break;
>> +    case KVM_S390_VM_CPU_TOPOLOGY:
>> +        ret = test_kvm_facility(kvm, 11) ? 0 : -ENXIO;
>> +        break;
>>       default:
>>           ret = -ENXIO;
>>           break;
>> diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
>> index 5088bd9f1922..33317d820032 100644
>> --- a/include/uapi/linux/kvm.h
>> +++ b/include/uapi/linux/kvm.h
>> @@ -1157,6 +1157,7 @@ struct kvm_ppc_resize_hpt {
>>   #define KVM_CAP_VM_TSC_CONTROL 214
>>   #define KVM_CAP_SYSTEM_EVENT_DATA 215
>>   #define KVM_CAP_ARM_SYSTEM_SUSPEND 216
>> +#define KVM_CAP_S390_CPU_TOPOLOGY 217
>>   #ifdef KVM_CAP_IRQ_ROUTING
> 

-- 
Pierre Morel
IBM Lab Boeblingen