Message-ID: <bb401f7d-7b2d-15ef-51c8-2c63be70f04c@linux.ibm.com>
Date:   Tue, 14 Dec 2021 09:59:11 -0500
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.3.0
Subject: Re: [PATCH 30/32] vfio-pci/zdev: add DTSM to clp group capability
Content-Language: en-US
To:     Pierre Morel <pmorel@linux.ibm.com>, linux-s390@vger.kernel.org
Cc:     alex.williamson@redhat.com, cohuck@redhat.com,
        schnelle@linux.ibm.com, farman@linux.ibm.com,
        borntraeger@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com,
        gerald.schaefer@linux.ibm.com, agordeev@linux.ibm.com,
        frankja@linux.ibm.com, david@redhat.com, imbrenda@linux.ibm.com,
        vneethv@linux.ibm.com, oberpar@linux.ibm.com, freude@linux.ibm.com,
        thuth@redhat.com, pasic@linux.ibm.com, kvm@vger.kernel.org,
        linux-kernel@vger.kernel.org
References: <20211207205743.150299-1-mjrosato@linux.ibm.com>
 <20211207205743.150299-31-mjrosato@linux.ibm.com>
 <b54b2ee9-3d7f-11b7-9aa4-e5dafd01a086@linux.ibm.com>
From:   Matthew Rosato <mjrosato@linux.ibm.com>
In-Reply-To: <b54b2ee9-3d7f-11b7-9aa4-e5dafd01a086@linux.ibm.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk

On 12/14/21 4:58 AM, Pierre Morel wrote:
> 
> 
> On 12/7/21 21:57, Matthew Rosato wrote:
>> The DTSM, or designation type supported mask, indicates what IOAT formats
>> are available to the guest.  For an interpreted device, userspace will 
>> not
>> know what format(s) the IOAT assist supports, so pass it via the
>> capability chain.  Since the value belongs to the Query PCI Function 
>> Group
>> clp, let's extend the existing capability with a new version.
>>
>> Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
>> ---
>>   drivers/vfio/pci/vfio_pci_zdev.c | 9 ++++++---
>>   include/uapi/linux/vfio_zdev.h   | 3 +++
>>   2 files changed, 9 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/vfio/pci/vfio_pci_zdev.c 
>> b/drivers/vfio/pci/vfio_pci_zdev.c
>> index 85be77492a6d..342b59ed36c9 100644
>> --- a/drivers/vfio/pci/vfio_pci_zdev.c
>> +++ b/drivers/vfio/pci/vfio_pci_zdev.c
>> @@ -45,19 +45,22 @@ static int zpci_group_cap(struct zpci_dev *zdev, 
>> struct vfio_info_cap *caps)
>>   {
>>       struct vfio_device_info_cap_zpci_group cap = {
>>           .header.id = VFIO_DEVICE_INFO_CAP_ZPCI_GROUP,
>> -        .header.version = 1,
>> +        .header.version = 2,
>>           .dasm = zdev->dma_mask,
>>           .msi_addr = zdev->msi_addr,
>>           .flags = VFIO_DEVICE_INFO_ZPCI_FLAG_REFRESH,
>>           .mui = zdev->fmb_update,
>>           .noi = zdev->max_msi,
>>           .maxstbl = ZPCI_MAX_WRITE_SIZE,
> 
> This, maxstbl, is not part of the patch but shouldn't we consider it too?
> The maxstbl is fixed for intercepted VFIO because the kernel is handling 
> the STBL instruction in behalf of the guest.
> Here the guest will use STBL directly.
> 
> I think we should report the right maxstbl value.
> 

I think we are OK, I think you missed the line that does this already, 
it was added in patch 27 when we wire up interpretive execution.  So, 
here we are defaulting to reporting ZPCI_MAX_WRITE_SIZE, and then ...

>> -        .version = zdev->version
>> +        .version = zdev->version,
>> +        .dtsm = 0
>>       };
>>       /* Some values are different for interpreted devices */
>> -    if (zdev->kzdev && zdev->kzdev->interp)
>> +    if (zdev->kzdev && zdev->kzdev->interp) {
>>           cap.maxstbl = zdev->maxstbl;

... Here we overwrite this with the hardware value only for interpreted 
devices.  Just like we are also now additionally doing for DTSM with 
this patch.

>> +        cap.dtsm = kvm_s390_pci_get_dtsm(zdev);
>> +    }
>>       return vfio_info_add_capability(caps, &cap.header, sizeof(cap));
>>   }
>> diff --git a/include/uapi/linux/vfio_zdev.h 
>> b/include/uapi/linux/vfio_zdev.h
>> index 1a5229b7bb18..b4c2ba8e71f0 100644
>> --- a/include/uapi/linux/vfio_zdev.h
>> +++ b/include/uapi/linux/vfio_zdev.h
>> @@ -47,6 +47,9 @@ struct vfio_device_info_cap_zpci_group {
>>       __u16 noi;        /* Maximum number of MSIs */
>>       __u16 maxstbl;        /* Maximum Store Block Length */
>>       __u8 version;        /* Supported PCI Version */
>> +    /* End of version 1 */
>> +    __u8 dtsm;        /* Supported IOAT Designations */
>> +    /* End of version 2 */
>>   };
>>   /**
>>
>