Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <b2dd8ef3-5fdb-5e29-4c75-12b9c3971950@linux.ibm.com>
Date:   Fri, 18 Mar 2022 18:13:55 -0400
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.3.0
Subject: Re: [PATCH v18 12/18] s390/vfio-ap: reset queues after adapter/domain
 unassignment
Content-Language: en-US
From:   Tony Krowiak <akrowiak@linux.ibm.com>
To:     jjherne@linux.ibm.com, linux-s390@vger.kernel.org,
        linux-kernel@vger.kernel.org, kvm@vger.kernel.org
Cc:     freude@linux.ibm.com, borntraeger@de.ibm.com, cohuck@redhat.com,
        mjrosato@linux.ibm.com, pasic@linux.ibm.com,
        alex.williamson@redhat.com, kwankhede@nvidia.com,
        fiuczy@linux.ibm.com
References: <20220215005040.52697-1-akrowiak@linux.ibm.com>
 <20220215005040.52697-13-akrowiak@linux.ibm.com>
 <6083d83b-6867-2525-fdd8-baccde1a599f@linux.ibm.com>
 <e869ab58-432e-e451-9021-71ee65488fb0@linux.ibm.com>
In-Reply-To: <e869ab58-432e-e451-9021-71ee65488fb0@linux.ibm.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk

snip ...

>
>>
>>
>>> + vfio_ap_unlink_mdev_fr_queue(q);
>>> +        hash_del(&q->mdev_qnode);
>>>       }
>>>   }
>> ...
>>> @@ -1273,9 +1320,9 @@ static void vfio_ap_mdev_unset_kvm(struct 
>>> ap_matrix_mdev *matrix_mdev,
>>>           mutex_lock(&kvm->lock);
>>>           mutex_lock(&matrix_dev->mdevs_lock);
>>>   -        kvm_arch_crypto_clear_masks(kvm);
>>> -        vfio_ap_mdev_reset_queues(matrix_mdev);
>>> -        kvm_put_kvm(kvm);
>>> +        kvm_arch_crypto_clear_masks(matrix_mdev->kvm);
>>> + vfio_ap_mdev_reset_queues(&matrix_mdev->qtable);
>>> +        kvm_put_kvm(matrix_mdev->kvm);
>>>           matrix_mdev->kvm = NULL;
>>
>> I understand changing the call to vfio_ap_mdev_reset_queues, but why 
>> are we changing the
>> kvm pointer on the surrounding lines?
>
> In reality, both pointers are one in the same given the two callers pass
> matrix_mdev->kvm into the function. I'm not sure why that is the case,
> it is probably a remnant from the commits that fixed the lockdep splat;
> however, there is no reason other than I've gotten used to retrieving the
> KVM pointer from the ap_matrix_mdev structure. In reality, there is no
> reason to pass a 'struct kvm *kvm' into this function, so I'm going to
> look into that and adjust accordingly.

The 'struct kvm *kvm' parameter was added to the signature of the
vfio_ap_mdev_unset_kvm function with the following commit:

86956e70761b (s390/vfio-ap: replace open coded locks for 
VFIO_GROUP_NOTIFY_SET_KVM notification)

I also noticed the the kernel doc for the vfio_ap_mdev_set_kvm and
vfio_ap_mdev_unset_kvm functions still contained a comment that is no longer
valid by the following commit:

0cc00c8d4050 (s390/vfio-ap: fix circular lockdep when setting/clearing 
crypto masks)

I pushed a patch to our devel branch that removes the invalid comment
from the two functions and removes the 'struct kvm *kvm' parameter
from the vfio_ap_mdev_unset_kvm function. That patch will prereq this 
series.

>
>>
>>
>>> mutex_unlock(&matrix_dev->mdevs_lock);
>>> @@ -1328,14 +1375,17 @@ static int vfio_ap_mdev_reset_queue(struct 
>>> vfio_ap_queue *q, unsigned int retry)
>>>         if (!q)
>>>           return 0;
>>> +    q->reset_rc = 0;