Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [PATCH 1/2] s390/vfio-ap: r/w lock for PQAP interception handler
 function pointer
To:     Halil Pasic <pasic@linux.ibm.com>,
        Christian Borntraeger <borntraeger@de.ibm.com>
Cc:     linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org,
        cohuck@redhat.com, pasic@linux.vnet.ibm.com, jjherne@linux.ibm.com,
        jgg@nvidia.com, alex.williamson@redhat.com, kwankhede@nvidia.com,
        david@redhat.com
References: <20210719193503.793910-1-akrowiak@linux.ibm.com>
 <20210719193503.793910-2-akrowiak@linux.ibm.com>
 <1a9f15d7-0f4d-00a0-0a8b-f1c08aa52eeb@de.ibm.com>
 <20210819012532.0e9c443c.pasic@linux.ibm.com>
From:   Tony Krowiak <akrowiak@linux.ibm.com>
Message-ID: <8df389f7-44aa-978e-84d8-96c625b0470b@linux.ibm.com>
Date:   Thu, 19 Aug 2021 09:36:34 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <20210819012532.0e9c443c.pasic@linux.ibm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US
Precedence: bulk


On 8/18/21 7:25 PM, Halil Pasic wrote:
> On Wed, 18 Aug 2021 19:03:33 +0200
> Christian Borntraeger <borntraeger@de.ibm.com> wrote:
>
>> On 19.07.21 21:35, Tony Krowiak wrote:
>>> The function pointer to the interception handler for the PQAP instruction
>>> can get changed during the interception process. Let's add a
>>> semaphore to struct kvm_s390_crypto to control read/write access to the
>>> function pointer contained therein.
>>>
>>> The semaphore must be locked for write access by the vfio_ap device driver
>>> when notified that the KVM pointer has been set or cleared. It must be
>>> locked for read access by the interception framework when the PQAP
>>> instruction is intercepted.
>>>
>>> Signed-off-by: Tony Krowiak <akrowiak@linux.ibm.com>
>>> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
>>> ---
>>>    arch/s390/include/asm/kvm_host.h      |  8 +++-----
>>>    arch/s390/kvm/kvm-s390.c              |  1 +
>>>    arch/s390/kvm/priv.c                  | 10 ++++++----
>>>    drivers/s390/crypto/vfio_ap_ops.c     | 23 +++++++++++++++++------
>>>    drivers/s390/crypto/vfio_ap_private.h |  2 +-
>>>    5 files changed, 28 insertions(+), 16 deletions(-)
>>>
>>> diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_host.h
>>> index 9b4473f76e56..f18849d259e6 100644
>>> --- a/arch/s390/include/asm/kvm_host.h
>>> +++ b/arch/s390/include/asm/kvm_host.h
>>> @@ -798,14 +798,12 @@ struct kvm_s390_cpu_model {
>>>    	unsigned short ibc;
>>>    };
>>>    
>>> -struct kvm_s390_module_hook {
>>> -	int (*hook)(struct kvm_vcpu *vcpu);
>>> -	struct module *owner;
>>> -};
>>> +typedef int (*crypto_hook)(struct kvm_vcpu *vcpu);
>>>    
>>>    struct kvm_s390_crypto {
>>>    	struct kvm_s390_crypto_cb *crycb;
>>> -	struct kvm_s390_module_hook *pqap_hook;
>>> +	struct rw_semaphore pqap_hook_rwsem;
>>> +	crypto_hook *pqap_hook;
>>>    	__u32 crycbd;
>>>    	__u8 aes_kw;
>>>    	__u8 dea_kw;
>>> diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
>>> index b655a7d82bf0..a08f242a9f27 100644
>>> --- a/arch/s390/kvm/kvm-s390.c
>>> +++ b/arch/s390/kvm/kvm-s390.c
>>> @@ -2630,6 +2630,7 @@ static void kvm_s390_crypto_init(struct kvm *kvm)
>>>    {
>>>    	kvm->arch.crypto.crycb = &kvm->arch.sie_page2->crycb;
>>>    	kvm_s390_set_crycb_format(kvm);
>>> +	init_rwsem(&kvm->arch.crypto.pqap_hook_rwsem);
>>>    
>>>    	if (!test_kvm_facility(kvm, 76))
>>>    		return;
>>> diff --git a/arch/s390/kvm/priv.c b/arch/s390/kvm/priv.c
>>> index 9928f785c677..6bed9406c1f3 100644
>>> --- a/arch/s390/kvm/priv.c
>>> +++ b/arch/s390/kvm/priv.c
>>> @@ -610,6 +610,7 @@ static int handle_io_inst(struct kvm_vcpu *vcpu)
>>>    static int handle_pqap(struct kvm_vcpu *vcpu)
>>>    {
>>>    	struct ap_queue_status status = {};
>>> +	crypto_hook pqap_hook;
>>>    	unsigned long reg0;
>>>    	int ret;
>>>    	uint8_t fc;
>>> @@ -657,15 +658,16 @@ static int handle_pqap(struct kvm_vcpu *vcpu)
>>>    	 * Verify that the hook callback is registered, lock the owner
>>>    	 * and call the hook.
>>>    	 */
>>> +	down_read(&vcpu->kvm->arch.crypto.pqap_hook_rwsem);
>>>    	if (vcpu->kvm->arch.crypto.pqap_hook) {                     <--- HERE
>>> -		if (!try_module_get(vcpu->kvm->arch.crypto.pqap_hook->owner))
>>> -			return -EOPNOTSUPP;
>>> -		ret = vcpu->kvm->arch.crypto.pqap_hook->hook(vcpu);
>>> -		module_put(vcpu->kvm->arch.crypto.pqap_hook->owner);
>>> +		pqap_hook = *vcpu->kvm->arch.crypto.pqap_hook;
>> Dont we have to check for NULL here? If not can you add a comment why?
> I believe we did the necessary check on the line I just marked with
> "<--- HERE".
>
> I find that "*" operator confusing in this context as it doesn't do
> any good for us. I believe this situation is described in 6.5.3.2.4 of
> the c11 standard. For convenience I will cite from the corresponding
> draft:
> "The unary * operator denotes indirection. If the operand points to a
> function, the result is a function designator; if it points to an
> object, the result is an lvalue designating the object. If the operand
> has type ‘‘pointer to type’’, the result has type ‘‘type’’. If an
> invalid value has been assigned to the pointer, the behavior of the
> unary * operator is undefined."
>
> Frankly I also fail to see the benefit of introducing the local variable
> named "pqap_hook", but back then I decided to not complain about style.

The vcpu->kvm->arch.crypto.pqap_hook is a pointer to a function
pointer. The actual function pointer is stored in matrix_mdev->pqap_hook,
the reason being that the handle_pqap function in vfio_ap_ops.c
retrieves the matrix_mdev via a container_of macro. The dereferencing
of the vcpu->kvm->arch.crypto.pqap_hook into a local variable was
to get the function pointer. There may have been a more stylish
way of doing this, but the functionality is there.

>
> Regards,
> Halil
>
>>
>>> +		ret = pqap_hook(vcpu);
>> [...]