Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Wed, 10 Feb 2021 16:24:29 +0100
From:   Halil Pasic <pasic@linux.ibm.com>
To:     Cornelia Huck <cohuck@redhat.com>
Cc:     Tony Krowiak <akrowiak@linux.ibm.com>, linux-s390@vger.kernel.org,
        linux-kernel@vger.kernel.org, kvm@vger.kernel.org,
        stable@vger.kernel.org, borntraeger@de.ibm.com,
        kwankhede@nvidia.com, pbonzini@redhat.com,
        alex.williamson@redhat.com, pasic@linux.vnet.ibm.com
Subject: Re: [PATCH 1/1] s390/vfio-ap: fix circular lockdep when
 setting/clearing crypto masks
Message-ID: <20210210162429.261fc17c.pasic@linux.ibm.com>
In-Reply-To: <20210210115334.46635966.cohuck@redhat.com>
References: <20210209194830.20271-1-akrowiak@linux.ibm.com>
        <20210209194830.20271-2-akrowiak@linux.ibm.com>
        <20210210115334.46635966.cohuck@redhat.com>
Organization: IBM
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 8bit
Precedence: bulk

On Wed, 10 Feb 2021 11:53:34 +0100
Cornelia Huck <cohuck@redhat.com> wrote:

> On Tue,  9 Feb 2021 14:48:30 -0500
> Tony Krowiak <akrowiak@linux.ibm.com> wrote:
> 
> > This patch fixes a circular locking dependency in the CI introduced by
> > commit f21916ec4826 ("s390/vfio-ap: clean up vfio_ap resources when KVM
> > pointer invalidated"). The lockdep only occurs when starting a Secure
> > Execution guest. Crypto virtualization (vfio_ap) is not yet supported for
> > SE guests; however, in order to avoid CI errors, this fix is being
> > provided.
> > 
> > The circular lockdep was introduced when the masks in the guest's APCB
> > were taken under the matrix_dev->lock. While the lock is definitely
> > needed to protect the setting/unsetting of the KVM pointer, it is not
> > necessarily critical for setting the masks, so this will not be done under
> > protection of the matrix_dev->lock.
> > 
> > Fixes: f21916ec4826 ("s390/vfio-ap: clean up vfio_ap resources when KVM pointer invalidated")
> > Cc: stable@vger.kernel.org
> > Signed-off-by: Tony Krowiak <akrowiak@linux.ibm.com>
> > ---
> >  drivers/s390/crypto/vfio_ap_ops.c | 75 ++++++++++++++++++-------------
> >  1 file changed, 45 insertions(+), 30 deletions(-)
> >   
> 
> >  static void vfio_ap_mdev_unset_kvm(struct ap_matrix_mdev *matrix_mdev)
> >  {
> > -	kvm_arch_crypto_clear_masks(matrix_mdev->kvm);
> > -	matrix_mdev->kvm->arch.crypto.pqap_hook = NULL;
> > -	vfio_ap_mdev_reset_queues(matrix_mdev->mdev);
> > -	kvm_put_kvm(matrix_mdev->kvm);
> > -	matrix_mdev->kvm = NULL;
> > +	if (matrix_mdev->kvm) {  
> 
> If you're doing setting/unsetting under matrix_dev->lock, is it
> possible that matrix_mdev->kvm gets unset between here and the next
> line, as you don't hold the lock?
> 
> Maybe you could
> - grab a reference to kvm while holding the lock
> - call the mask handling functions with that kvm reference
> - lock again, drop the reference, and do the rest of the processing?

I agree, matrix_mdev->kvm can go NULL any time and we are risking
a null pointer dereference here.

Another idea would be to do


static void vfio_ap_mdev_unset_kvm(struct ap_matrix_mdev *matrix_mdev)           
{                                                                               
        struct kvm *kvm;
                                                        
        mutex_lock(&matrix_dev->lock);                                          
        if (matrix_mdev->kvm) {                                                 
                kvm = matrix_mdev->kvm;                                         
                matrix_mdev->kvm = NULL;                                        
                mutex_unlock(&matrix_dev->lock);                                
                kvm_arch_crypto_clear_masks(kvm);                               
                mutex_lock(&matrix_dev->lock);                                  
                matrix_mdev->kvm->arch.crypto.pqap_hook = NULL;                 
                vfio_ap_mdev_reset_queues(matrix_mdev->mdev);                   
                kvm_put_kvm(kvm);                                               
        }                                                                       
        mutex_unlock(&matrix_dev->lock);                                         
}

That way only one unset would actually do the unset and cleanup
and every other invocation would bail out with only checking
matrix_mdev->kvm.

 
> > +		kvm_arch_crypto_clear_masks(matrix_mdev->kvm);
> > +		mutex_lock(&matrix_dev->lock);
> > +		matrix_mdev->kvm->arch.crypto.pqap_hook = NULL;
> > +		vfio_ap_mdev_reset_queues(matrix_mdev->mdev);
> > +		kvm_put_kvm(matrix_mdev->kvm);
> > +		matrix_mdev->kvm = NULL;
> > +		mutex_unlock(&matrix_dev->lock);
> > +	}
> >  }  
>