Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <9837da27-e224-aded-fe3e-4f4db6b1599c@linux.ibm.com>
Date:   Wed, 31 May 2023 10:48:24 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.11.0
Subject: Re: [PATCH 0/3] s390/vfio-ap: fix hang when mdev attached to guest is
 removed
To:     Tony Krowiak <akrowiak@linux.ibm.com>, linux-s390@vger.kernel.org,
        linux-kernel@vger.kernel.org, kvm@vger.kernel.org,
        alex.williamson@redhat.com
Cc:     jjherne@linux.ibm.com, pasic@linux.ibm.com, farman@linux.ibm.com,
        borntraeger@linux.ibm.com, Heiko Carstens <hca@linux.ibm.com>,
        Vasily Gorbik <gor@linux.ibm.com>,
        "agordeev@linux.ibm.com" <agordeev@linux.ibm.com>
References: <20230530223538.279198-1-akrowiak@linux.ibm.com>
Content-Language: en-US
From:   Matthew Rosato <mjrosato@linux.ibm.com>
In-Reply-To: <20230530223538.279198-1-akrowiak@linux.ibm.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
Precedence: bulk

On 5/30/23 6:35 PM, Tony Krowiak wrote:
> When a user attempts to remove a vfio-ap mediated device attached to a
> guest, the operation hangs until the mdev's fd is closed by the guest
> (i.e., the hostdev is detached or the guest is shut down). This patch 
> series provides kernel-side code that allows userspace to set up a 
> communication channel that will allow the vfio_ap device driver to notify 
> userspace when a request to release the mdev is received, so that userspace
> can close the mdev fd and avoid the hang. The patch series provides the 
> following:  
> 
> 1. Introduces code to handle the VFIO_DEVICE_GET_IRQ_INFO and 
>    VFIO_DEVICE_SET_IRQS ioctl calls to set the eventfd_ctx for signaling a
>    device request to userspace. 
> 
> 2. Wires up the VFIO bus driver callback to request a release of the mdev.
>    When invoked, the vfio_ap device driver will use the eventfd_ctx set up
>    in #1 to signal a request to userspace to release the mdev.
> 
> 
> Note:
> ----
> If a user subsequently attempts to restart the guest or re-attach the mdev,
> the operation will fail with a message indicating the domain is already
> active. This is a libvirt problem resolved with the following commit:
> 
> commit ebd004a03dbd ("security: do not remember/recall labels for VFIO 
> MDEVs") 

For the series: 

Reviewed-by: Matthew Rosato <mjrosato@linux.ibm.com>


I also did some testing using the companion qemu series at
https://lore.kernel.org/qemu-devel/20230530225544.280031-1-akrowiak@linux.ibm.com

Before kernel+qemu changes:
1. mdevctl start -u <uuid>, where <uuid> references a vfio-ap mdev
2. start a qemu guest with <uuid> attached
3. mdvectl stop -u <uuid>
4. -mdevctl will now hang indefinitely; the mdev remains in-use by the guest-
Note: detaching the device or powering off the guest will allow the mdevctl command to complete.

After kernel+qemu changes:
1. mdevctl start -u <uuid>, where <uuid> references a vfio-ap mdev
2. start a qemu guest with <uuid> attached
3. mdvectl stop -u <uuid>
4. -device is detached from the guest and stopped-
5. Using a libvirt that includes ebd004a03dbd I also verified that the mdev can be started again and re-attached to the running guest without error.


> 
> Tony Krowiak (3):
>   vfio: ap: realize the VFIO_DEVICE_GET_IRQ_INFO ioctl
>   vfio: ap: realize the VFIO_DEVICE_SET_IRQS ioctl
>   s390/vfio-ap: Wire in the vfio_device_ops request callback
> 
>  drivers/s390/crypto/vfio_ap_ops.c     | 134 +++++++++++++++++++++++++-
>  drivers/s390/crypto/vfio_ap_private.h |   3 +
>  include/uapi/linux/vfio.h             |   9 ++
>  3 files changed, 145 insertions(+), 1 deletion(-)
>