Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <7d50c2df-7cad-dbc6-baa0-ab647f8dde4e@linux.ibm.com>
Date:   Fri, 24 Jun 2022 08:50:47 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.9.0
Content-Language: en-US
To:     Pierre Morel <pmorel@linux.ibm.com>, kvm@vger.kernel.org
Cc:     linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org,
        borntraeger@de.ibm.com, cohuck@redhat.com, david@redhat.com,
        thuth@redhat.com, imbrenda@linux.ibm.com, hca@linux.ibm.com,
        gor@linux.ibm.com, wintera@linux.ibm.com, seiden@linux.ibm.com,
        nrb@linux.ibm.com
References: <20220620125437.37122-1-pmorel@linux.ibm.com>
 <20220620125437.37122-4-pmorel@linux.ibm.com>
From:   Janosch Frank <frankja@linux.ibm.com>
Subject: Re: [PATCH v10 3/3] KVM: s390: resetting the Topology-Change-Report
In-Reply-To: <20220620125437.37122-4-pmorel@linux.ibm.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 6/20/22 14:54, Pierre Morel wrote:
> During a subsystem reset the Topology-Change-Report is cleared.
> Let's give userland the possibility to clear the MTCR in the case
> of a subsystem reset.
> 
> To migrate the MTCR, we give userland the possibility to
> query the MTCR state.
> 
> We indicate KVM support for the CPU topology facility with a new
> KVM capability: KVM_CAP_S390_CPU_TOPOLOGY.
> 
> Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
> ---
>   Documentation/virt/kvm/api.rst   | 31 +++++++++++
>   arch/s390/include/uapi/asm/kvm.h | 10 ++++
>   arch/s390/kvm/kvm-s390.c         | 96 ++++++++++++++++++++++++++++++++
>   include/uapi/linux/kvm.h         |  1 +
>   4 files changed, 138 insertions(+)
> 
> diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst
> index 11e00a46c610..326f8b7e7671 100644
> --- a/Documentation/virt/kvm/api.rst
> +++ b/Documentation/virt/kvm/api.rst
> @@ -7956,6 +7956,37 @@ should adjust CPUID leaf 0xA to reflect that the PMU is disabled.
>   When enabled, KVM will exit to userspace with KVM_EXIT_SYSTEM_EVENT of
>   type KVM_SYSTEM_EVENT_SUSPEND to process the guest suspend request.
>   
> +8.37 KVM_CAP_S390_CPU_TOPOLOGY
> +------------------------------
> +
> +:Capability: KVM_CAP_S390_CPU_TOPOLOGY
> +:Architectures: s390
> +:Type: vm
> +
> +This capability indicates that KVM will provide the S390 CPU Topology
> +facility which consist of the interpretation of the PTF instruction for
> +the Function Code 2 along with interception and forwarding of both the

Making function code capital surprises me when reading.

> +PTF instruction with Function Codes 0 or 1 and the STSI(15,1,x)
> +instruction to the userland hypervisor.
> +
> +The stfle facility 11, CPU Topology facility, should not be provided

s/provided/indicated

> +to the guest without this capability.
> +
> +When this capability is present, KVM provides a new attribute group
> +on vm fd, KVM_S390_VM_CPU_TOPOLOGY.
> +This new attribute allows to get, set or clear the Modified Change
> +Topology Report (MTCR) bit of the SCA through the kvm_device_attr
> +structure.
> +
> +Getting the MTCR bit is realized by using a kvm_device_attr attr
> +entry value of KVM_GET_DEVICE_ATTR and with kvm_device_attr addr
> +entry pointing to the address of a struct kvm_cpu_topology.
> +The value of the MTCR is return by the bit mtcr of the structure. > +
> +When using KVM_SET_DEVICE_ATTR the MTCR is set by using the
> +attr->attr value KVM_S390_VM_CPU_TOPO_MTCR_SET and cleared by
> +using KVM_S390_VM_CPU_TOPO_MTCR_CLEAR.

I have the feeling that we can drop the two blocks above and we won't 
loose information.

> +/**
> + * kvm_s390_sca_clear_mtcr
> + * @kvm: guest KVM description
> + *
> + * Is only relevant if the topology facility is present,
> + * the caller should check KVM facility 11
> + *
> + * Updates the Multiprocessor Topology-Change-Report to signal
> + * the guest with a topology change.
> + */
> +static void kvm_s390_sca_clear_mtcr(struct kvm *kvm)

This is a set operation with the value 0 and that's clearly visible by 
the copied code. If you make the utility entry a bitfield you can easily 
set 0/1 via one function without doing the bit manipulation by hand.

I.e. please only use one set function.

> +{
> +	struct bsca_block *sca = kvm->arch.sca; /* SCA version doesn't matter */
> +
> +	ipte_lock(kvm);
> +	sca->utility &= ~SCA_UTILITY_MTCR;
> +	ipte_unlock(kvm);
> +}
> +
> +static int kvm_s390_set_topology(struct kvm *kvm, struct kvm_device_attr *attr)
> +{
> +	if (!test_kvm_facility(kvm, 11))
> +		return -ENXIO;
> +
> +	switch (attr->attr) {
> +	case KVM_S390_VM_CPU_TOPO_MTCR_SET:
> +		kvm_s390_sca_set_mtcr(kvm);
> +		break;
> +	case KVM_S390_VM_CPU_TOPO_MTCR_CLEAR:
> +		kvm_s390_sca_clear_mtcr(kvm);
> +		break;
> +	}

By having two endpoints here we trade an easy check with having to 
access process memory to grab the value we want to set.

I'm still torn about this.

> +	return 0;
> +}
> +
> +/**
> + * kvm_s390_sca_get_mtcr
> + * @kvm: guest KVM description
> + *
> + * Is only relevant if the topology facility is present,
> + * the caller should check KVM facility 11
> + *
> + * reports to QEMU the Multiprocessor Topology-Change-Report.
> + */
> +static int kvm_s390_sca_get_mtcr(struct kvm *kvm)
> +{
> +	struct bsca_block *sca = kvm->arch.sca; /* SCA version doesn't matter */

Same comments as with the set_mtcr()

> +	int val;
> +
> +	ipte_lock(kvm);
> +	val = sca->utility & SCA_UTILITY_MTCR;
> +	ipte_unlock(kvm);
> +
> +	return val;
> +}
> +
> +static int kvm_s390_get_topology(struct kvm *kvm, struct kvm_device_attr *attr)
> +{
> +	struct kvm_cpu_topology topo = {};
> +
> +	if (!test_kvm_facility(kvm, 11))
> +		return -ENXIO;
> +
> +	topo.mtcr = kvm_s390_sca_get_mtcr(kvm) ? 1 : 0;
> +	if (copy_to_user((void __user *)attr->addr, &topo, sizeof(topo)))
> +		return -EFAULT;
> +
> +	return 0;
> +}
> +
>   static int kvm_s390_vm_set_attr(struct kvm *kvm, struct kvm_device_attr *attr)
>   {
>   	int ret;
> @@ -1730,6 +1817,9 @@ static int kvm_s390_vm_set_attr(struct kvm *kvm, struct kvm_device_attr *attr)
>   	case KVM_S390_VM_MIGRATION:
>   		ret = kvm_s390_vm_set_migration(kvm, attr);
>   		break;
> +	case KVM_S390_VM_CPU_TOPOLOGY:
> +		ret = kvm_s390_set_topology(kvm, attr);
> +		break;
>   	default:
>   		ret = -ENXIO;
>   		break;
> @@ -1755,6 +1845,9 @@ static int kvm_s390_vm_get_attr(struct kvm *kvm, struct kvm_device_attr *attr)
>   	case KVM_S390_VM_MIGRATION:
>   		ret = kvm_s390_vm_get_migration(kvm, attr);
>   		break;
> +	case KVM_S390_VM_CPU_TOPOLOGY:
> +		ret = kvm_s390_get_topology(kvm, attr);
> +		break;
>   	default:
>   		ret = -ENXIO;
>   		break;
> @@ -1828,6 +1921,9 @@ static int kvm_s390_vm_has_attr(struct kvm *kvm, struct kvm_device_attr *attr)
>   	case KVM_S390_VM_MIGRATION:
>   		ret = 0;
>   		break;
> +	case KVM_S390_VM_CPU_TOPOLOGY:
> +		ret = test_kvm_facility(kvm, 11) ? 0 : -ENXIO;
> +		break;
>   	default:
>   		ret = -ENXIO;
>   		break;
> diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
> index 5088bd9f1922..33317d820032 100644
> --- a/include/uapi/linux/kvm.h
> +++ b/include/uapi/linux/kvm.h
> @@ -1157,6 +1157,7 @@ struct kvm_ppc_resize_hpt {
>   #define KVM_CAP_VM_TSC_CONTROL 214
>   #define KVM_CAP_SYSTEM_EVENT_DATA 215
>   #define KVM_CAP_ARM_SYSTEM_SUSPEND 216
> +#define KVM_CAP_S390_CPU_TOPOLOGY 217
>   
>   #ifdef KVM_CAP_IRQ_ROUTING
>