Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Message-ID: <104bb7a4-68e7-adb1-91f4-d6fce09b99e7@linux.ibm.com>
Date:   Tue, 18 Jan 2022 13:37:16 -0500
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.4.0
Subject: Re: [PATCH v2 24/30] vfio-pci/zdev: wire up group notifier
Content-Language: en-US
To:     Pierre Morel <pmorel@linux.ibm.com>, linux-s390@vger.kernel.org
Cc:     alex.williamson@redhat.com, cohuck@redhat.com,
        schnelle@linux.ibm.com, farman@linux.ibm.com,
        borntraeger@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com,
        gerald.schaefer@linux.ibm.com, agordeev@linux.ibm.com,
        frankja@linux.ibm.com, david@redhat.com, imbrenda@linux.ibm.com,
        vneethv@linux.ibm.com, oberpar@linux.ibm.com, freude@linux.ibm.com,
        thuth@redhat.com, pasic@linux.ibm.com, kvm@vger.kernel.org,
        linux-kernel@vger.kernel.org
References: <20220114203145.242984-1-mjrosato@linux.ibm.com>
 <20220114203145.242984-25-mjrosato@linux.ibm.com>
 <0af94334-27ac-7e05-86ea-465857e9dadd@linux.ibm.com>
From:   Matthew Rosato <mjrosato@linux.ibm.com>
In-Reply-To: <0af94334-27ac-7e05-86ea-465857e9dadd@linux.ibm.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk

On 1/18/22 12:34 PM, Pierre Morel wrote:
> 
> 
> On 1/14/22 21:31, Matthew Rosato wrote:
>> KVM zPCI passthrough device logic will need a reference to the associated
>> kvm guest that has access to the device.  Let's register a group notifier
>> for VFIO_GROUP_NOTIFY_SET_KVM to catch this information in order to 
>> create
>> an association between a kvm guest and the host zdev.
>>
>> Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
>> ---
>>   arch/s390/include/asm/kvm_pci.h  |  2 ++
>>   drivers/vfio/pci/vfio_pci_core.c |  2 ++
>>   drivers/vfio/pci/vfio_pci_zdev.c | 46 ++++++++++++++++++++++++++++++++
>>   include/linux/vfio_pci_core.h    | 10 +++++++
>>   4 files changed, 60 insertions(+)
>>
>> diff --git a/arch/s390/include/asm/kvm_pci.h 
>> b/arch/s390/include/asm/kvm_pci.h
>> index fa90729a35cf..97a90b37c87d 100644
>> --- a/arch/s390/include/asm/kvm_pci.h
>> +++ b/arch/s390/include/asm/kvm_pci.h
>> @@ -17,6 +17,7 @@
>>   #include <linux/kvm.h>
>>   #include <linux/pci.h>
>>   #include <linux/mutex.h>
>> +#include <linux/notifier.h>
>>   #include <asm/pci_insn.h>
>>   #include <asm/pci_dma.h>
>> @@ -33,6 +34,7 @@ struct kvm_zdev {
>>       u64 rpcit_count;
>>       struct kvm_zdev_ioat ioat;
>>       struct zpci_fib fib;
>> +    struct notifier_block nb;
>>   };
>>   int kvm_s390_pci_dev_open(struct zpci_dev *zdev);
>> diff --git a/drivers/vfio/pci/vfio_pci_core.c 
>> b/drivers/vfio/pci/vfio_pci_core.c
>> index f948e6cd2993..fc57d4d0abbe 100644
>> --- a/drivers/vfio/pci/vfio_pci_core.c
>> +++ b/drivers/vfio/pci/vfio_pci_core.c
>> @@ -452,6 +452,7 @@ void vfio_pci_core_close_device(struct vfio_device 
>> *core_vdev)
>>       vfio_pci_vf_token_user_add(vdev, -1);
>>       vfio_spapr_pci_eeh_release(vdev->pdev);
>> +    vfio_pci_zdev_release(vdev);
>>       vfio_pci_core_disable(vdev);
>>       mutex_lock(&vdev->igate);
>> @@ -470,6 +471,7 @@ EXPORT_SYMBOL_GPL(vfio_pci_core_close_device);
>>   void vfio_pci_core_finish_enable(struct vfio_pci_core_device *vdev)
>>   {
>>       vfio_pci_probe_mmaps(vdev);
>> +    vfio_pci_zdev_open(vdev);
>>       vfio_spapr_pci_eeh_open(vdev->pdev);
>>       vfio_pci_vf_token_user_add(vdev, 1);
>>   }
>> diff --git a/drivers/vfio/pci/vfio_pci_zdev.c 
>> b/drivers/vfio/pci/vfio_pci_zdev.c
>> index ea4c0d2b0663..5c2bddc57b39 100644
>> --- a/drivers/vfio/pci/vfio_pci_zdev.c
>> +++ b/drivers/vfio/pci/vfio_pci_zdev.c
>> @@ -13,6 +13,7 @@
>>   #include <linux/vfio_zdev.h>
>>   #include <asm/pci_clp.h>
>>   #include <asm/pci_io.h>
>> +#include <asm/kvm_pci.h>
>>   #include <linux/vfio_pci_core.h>
>> @@ -136,3 +137,48 @@ int vfio_pci_info_zdev_add_caps(struct 
>> vfio_pci_core_device *vdev,
>>       return ret;
>>   }
>> +
>> +static int vfio_pci_zdev_group_notifier(struct notifier_block *nb,
>> +                    unsigned long action, void *data)
>> +{
>> +    struct kvm_zdev *kzdev = container_of(nb, struct kvm_zdev, nb);
>> +
>> +    if (action == VFIO_GROUP_NOTIFY_SET_KVM) {
>> +        if (!data || !kzdev->zdev)
>> +            return NOTIFY_DONE;
>> +        kvm_s390_pci_attach_kvm(kzdev->zdev, data);
> 
> Why not just set kzdev->kvm = data ?
> 
> alternatively, define kvm_s390_pci_attach_kvm() as an inline instead of 
> a global function.
> 
> otherwise LGTM

At some point in the past this function did more than just set a 
pointer...  You are correct there's no need for this abstraction now, 
let's just set kzdev->kvm = data directly here and drop the 
kvm_s390_pci_attach_kvm function.