Received: by 2002:a05:6a10:8c0a:0:0:0:0 with SMTP id go10csp700210pxb;
        Tue, 2 Feb 2021 16:02:06 -0800 (PST)
X-Google-Smtp-Source: ABdhPJxyJqqNuEeMi4EZJ3Ln5ki0qvfKEgD1xDYmMeYQELvVwdfKemCFDgWblV5hCmkYzyfTjagu
X-Received: by 2002:a17:906:ce49:: with SMTP id se9mr446736ejb.341.1612310526682;
        Tue, 02 Feb 2021 16:02:06 -0800 (PST)
ARC-Seal: i=1; a=rsa-sha256; t=1612310526; cv=none;
        d=google.com; s=arc-20160816;
        b=Yibn9sCN/+vajDwWWQeGwulzKoYQ3DqIX8GkrhN+22V1+NN5vwT+ycbP6cl49Bnoos
         4NuVenREvHY/s7QL8z5eABpkoC4baNBl7KEf//hfrNfHQrncn/hwXww6Rd+Z1h7q5Ot0
         i4A5bXnu6zan4122B5HXO5JeY993G0+1v2vrtPuABzHXIXRIs/lx9n8zxuAGH5kcMCYI
         iYdcBydNsYUJoU0E26Xyt4OH5uVn+aZnU7cPKy7vK3vZZa/WJVw5wgTaCcMxtNDTdF6v
         kXzPR56cJU6P+Md7EFgidnKbU5CltuSynPVledTJ5Jmbnp17y5vZiuOFjwE2cKd/nXG+
         vzLQ==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816;
        h=list-id:precedence:dkim-signature:content-language
         :content-transfer-encoding:in-reply-to:mime-version:user-agent:date
         :message-id:from:references:cc:to:subject;
        bh=T9u2m8ib6L6wjnHIIPMQb8zM6/4WXaLdn05UdqoTT0c=;
        b=JotMK64jUj7QdefbET/Yj9TAlqFxlkqIrb9NTqdLig3LjK95MECWa95DA4wzhI+VVN
         g5SBCAhmuvU3aaoU9rvi/C+j1p7ErSwFFO/e0G4gvOdjhusqCpA9ObppoXTm1WFXAyRV
         im46VhfuDyAVebohFQz5fnl9tn12NvKFh6uRMp+GvsaJbjAoVkG8ol7eLPP0KX2yZmiT
         TOWvtYG4jXwrX/y5ja6IQ5OfWyog41VR9r5m6XMT6OcNdyW+1OSnUaZjE5CYwUedrT3a
         skV9sJuB089GS8tlUCrZJFFm+OpNkfQxq41UVswwv5gRflo8f6pxw5qze5fLs9jkvRYF
         ywVA==
ARC-Authentication-Results: i=1; mx.google.com;
       dkim=pass header.i=@nvidia.com header.s=n1 header.b=N7eZ8ReE;
       spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org;
       dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nvidia.com
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18])
        by mx.google.com with ESMTP id dm10si272855ejc.64.2021.02.02.16.01.39;
        Tue, 02 Feb 2021 16:02:06 -0800 (PST)
Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Authentication-Results: mx.google.com;
       dkim=pass header.i=@nvidia.com header.s=n1 header.b=N7eZ8ReE;
       spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org;
       dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nvidia.com
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S237843AbhBBRoo (ORCPT <rfc822;luelistan@gmail.com> + 99 others);
        Tue, 2 Feb 2021 12:44:44 -0500
Received: from hqnvemgate26.nvidia.com ([216.228.121.65]:18566 "EHLO
        hqnvemgate26.nvidia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S237751AbhBBRmG (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Tue, 2 Feb 2021 12:42:06 -0500
Received: from hqmail.nvidia.com (Not Verified[216.228.121.13]) by hqnvemgate26.nvidia.com (using TLS: TLSv1.2, AES256-SHA)
        id <B60198ec50000>; Tue, 02 Feb 2021 09:41:25 -0800
Received: from [172.27.0.48] (172.20.145.6) by HQMAIL107.nvidia.com
 (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Tue, 2 Feb
 2021 17:41:19 +0000
Subject: Re: [PATCH 8/9] vfio/pci: use x86 naming instead of igd
To:     Cornelia Huck <cohuck@redhat.com>,
        Alex Williamson <alex.williamson@redhat.com>
CC:     Matthew Rosato <mjrosato@linux.ibm.com>, <jgg@nvidia.com>,
        <kvm@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
        <liranl@nvidia.com>, <oren@nvidia.com>, <tzahio@nvidia.com>,
        <leonro@nvidia.com>, <yarong@nvidia.com>, <aviadye@nvidia.com>,
        <shahafs@nvidia.com>, <artemp@nvidia.com>, <kwankhede@nvidia.com>,
        <ACurrid@nvidia.com>, <gmataev@nvidia.com>, <cjia@nvidia.com>,
        <yishaih@nvidia.com>, <aik@ozlabs.ru>
References: <20210201162828.5938-1-mgurtovoy@nvidia.com>
 <20210201162828.5938-9-mgurtovoy@nvidia.com>
 <20210201181454.22112b57.cohuck@redhat.com>
 <599c6452-8ba6-a00a-65e7-0167f21eac35@linux.ibm.com>
 <20210201114230.37c18abd@omen.home.shazbot.org>
 <20210202170659.1c62a9e8.cohuck@redhat.com>
From:   Max Gurtovoy <mgurtovoy@nvidia.com>
Message-ID: <a413334c-3319-c6a3-3d8a-0bb68a10b9c1@nvidia.com>
Date:   Tue, 2 Feb 2021 19:41:16 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.7.0
MIME-Version: 1.0
In-Reply-To: <20210202170659.1c62a9e8.cohuck@redhat.com>
Content-Type: text/plain; charset="utf-8"; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US
X-Originating-IP: [172.20.145.6]
X-ClientProxiedBy: HQMAIL107.nvidia.com (172.20.187.13) To
 HQMAIL107.nvidia.com (172.20.187.13)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1;
        t=1612287685; bh=T9u2m8ib6L6wjnHIIPMQb8zM6/4WXaLdn05UdqoTT0c=;
        h=Subject:To:CC:References:From:Message-ID:Date:User-Agent:
         MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding:
         Content-Language:X-Originating-IP:X-ClientProxiedBy;
        b=N7eZ8ReEx0/Bqg8xyxrqzQMO8fa7xE7tK+5n1zO5nKxn2aYadp7UDLxOdfBN4PSSe
         jaiYRM/E0kIc4jRblHs/SaiNHjJ1jxm+uqMe8sH0sKUkui7HPuNc4b/T3WygdpT0Fs
         bVxhgrB9M3V1aKw6FkHNIUq6bEnRf38pCsmYhM6J1CqsSu+IOT5Ki9g0aSjidqqIF1
         t62ePGc3K4aYGh/H69wI4+ypPBwQZkih1aNx4aqUOc8u6J9hHW9WPxw4Wt23P3FVzQ
         6VXZAy3ZTiVS85rAJgkB/woIpeIX+MwVm+d8aTCVdkEjAodn6ogQZ1h2oeXsrTkRff
         xfJCT2DFJ5I+g==
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 2/2/2021 6:06 PM, Cornelia Huck wrote:
> On Mon, 1 Feb 2021 11:42:30 -0700
> Alex Williamson <alex.williamson@redhat.com> wrote:
>
>> On Mon, 1 Feb 2021 12:49:12 -0500
>> Matthew Rosato <mjrosato@linux.ibm.com> wrote:
>>
>>> On 2/1/21 12:14 PM, Cornelia Huck wrote:
>>>> On Mon, 1 Feb 2021 16:28:27 +0000
>>>> Max Gurtovoy <mgurtovoy@nvidia.com> wrote:
>>>>      
>>>>> This patch doesn't change any logic but only align to the concept of
>>>>> vfio_pci_core extensions. Extensions that are related to a platform
>>>>> and not to a specific vendor of PCI devices should be part of the core
>>>>> driver. Extensions that are specific for PCI device vendor should go
>>>>> to a dedicated vendor vfio-pci driver.
>>>> My understanding is that igd means support for Intel graphics, i.e. a
>>>> strict subset of x86. If there are other future extensions that e.g.
>>>> only make sense for some devices found only on AMD systems, I don't
>>>> think they should all be included under the same x86 umbrella.
>>>>
>>>> Similar reasoning for nvlink, that only seems to cover support for some
>>>> GPUs under Power, and is not a general platform-specific extension IIUC.
>>>>
>>>> We can arguably do the zdev -> s390 rename (as zpci appears only on
>>>> s390, and all PCI devices will be zpci on that platform), although I'm
>>>> not sure about the benefit.
>>> As far as I can tell, there isn't any benefit for s390 it's just
>>> "re-branding" to match the platform name rather than the zdev moniker,
>>> which admittedly perhaps makes it more clear to someone outside of s390
>>> that any PCI device on s390 is a zdev/zpci type, and thus will use this
>>> extension to vfio_pci(_core).  This would still be true even if we added
>>> something later that builds atop it (e.g. a platform-specific device
>>> like ism-vfio-pci).  Or for that matter, mlx5 via vfio-pci on s390x uses
>>> these zdev extensions today and would need to continue using them in a
>>> world where mlx5-vfio-pci.ko exists.
>>>
>>> I guess all that to say: if such a rename matches the 'grand scheme' of
>>> this design where we treat arch-level extensions to vfio_pci(_core) as
>>> "vfio_pci_(arch)" then I'm not particularly opposed to the rename.  But
>>> by itself it's not very exciting :)
>> This all seems like the wrong direction to me.  The goal here is to
>> modularize vfio-pci into a core library and derived vendor modules that
>> make use of that core library.  If existing device specific extensions
>> within vfio-pci cannot be turned into vendor modules through this
>> support and are instead redefined as platform specific features of the
>> new core library, that feels like we're already admitting failure of
>> this core library to support known devices, let alone future devices.
>>
>> IGD is a specific set of devices.  They happen to rely on some platform
>> specific support, whose availability should be determined via the
>> vendor module probe callback.  Packing that support into an "x86"
>> component as part of the core feels not only short sighted, but also
>> avoids addressing the issues around how userspace determines an optimal
>> module to use for a device.
> Hm, it seems that not all current extensions to the vfio-pci code are
> created equal.
>
> IIUC, we have igd and nvlink, which are sets of devices that only show
> up on x86 or ppc, respectively, and may rely on some special features
> of those architectures/platforms. The important point is that you have
> a device identifier that you can match a driver against.

maybe you can supply the ids ?

Alexey K, I saw you've been working on the NVLINK2 for P9. can you 
supply the exact ids for that should be bounded to this driver ?

I'll add it to V3.

>
> On the other side, we have the zdev support, which both requires s390
> and applies to any pci device on s390. If we added special handling for
> ISM on s390, ISM would be in a category similar to igd/nvlink.
>
> Now, if somebody plugs a mlx5 device into an s390, we would want both
> the zdev support and the specialized mlx5 driver. Maybe zdev should be
> some kind of library that can be linked into normal vfio-pci, into
> vfio-pci-mlx5, and a hypothetical vfio-pci-ism? You always want zdev on
> s390 (if configured into the kernel).
>