Received: by 2002:a05:6a10:8c0a:0:0:0:0 with SMTP id go10csp726800pxb;
        Tue, 2 Feb 2021 16:51:17 -0800 (PST)
X-Google-Smtp-Source: ABdhPJzio2vyxqvwkJ+6XhjhnN+qDiqkcCO8hYI9oGlVhUIgdNTfCOTFoZmtelYfZgKaIpoKuW0z
X-Received: by 2002:a17:907:1701:: with SMTP id le1mr671583ejc.68.1612313476871;
        Tue, 02 Feb 2021 16:51:16 -0800 (PST)
ARC-Seal: i=1; a=rsa-sha256; t=1612313476; cv=none;
        d=google.com; s=arc-20160816;
        b=ADfJgoN6JuZUwnkTBoEJWJyTnPxclm5maZaxbVOVx66Be2ioEbvvUGcpvQo2gKbX4n
         WccX3UibRpW/Hxr2kTbzsVS+6NGtlYBGlUV5YL7VCvlYWmgP3tqeWNGvWF/cabUPL0Ud
         HEgBESTNLv+lWsYr+H6X3xH7/nutk5izTnYk5vclRUOaob/px4cMM9YRCcp/mQ/6tJZw
         Wa/rdMg2rvXYywL8ijDnCSTL1eiApj3otrPlZKIV3E2fGDo3ecm+9x4mltt6mdmTxmeo
         hYBXmoQmMnuebZCuFrzw3G8iF3PRpDZeAp79u7Qdntx8m8HRyuW5+5wzDBHrOOfp8t2p
         p6dA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816;
        h=list-id:precedence:dkim-signature:content-language
         :content-transfer-encoding:in-reply-to:mime-version:user-agent:date
         :message-id:from:references:cc:to:subject;
        bh=VzeO8fn5oKUBcIe3AbDvCWI0BQgaZOFMZoUoYnQg2tg=;
        b=cPdhiSBuJ9NrcI//9FjyB37Jje/wNIbk22R3SPwqouh60WGfwIQ+ICbrZ0a3jjo9kL
         Uw7sav3G3/F3mUC8BvSpv6hHWD2iMeGYeeD2iSBxG7CcrXF1HwC6wL4lVamRc2H//749
         mY2+IxKrx1H1KxQaFyWCjjEUXMKdzw0Jw43BeMKJK5AgAR50/3g8WWS8wHospP0z5ABR
         Ik0A5XRPssbTDV5109iAXTEdRHokUMN5G8v+hHmope2+VVP8DJGP5XP+Ry6Uq7TEobes
         jY5tp+p+klNh0Y1nUsFVqf1wvRZ7dYy6fomH3Pi3KBeU5ia+29fliOHQQaQQGOgFDnnY
         oasQ==
ARC-Authentication-Results: i=1; mx.google.com;
       dkim=pass header.i=@nvidia.com header.s=n1 header.b=Q+CtvyvM;
       spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org;
       dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nvidia.com
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18])
        by mx.google.com with ESMTP id o15si296732ejx.665.2021.02.02.16.50.52;
        Tue, 02 Feb 2021 16:51:16 -0800 (PST)
Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Authentication-Results: mx.google.com;
       dkim=pass header.i=@nvidia.com header.s=n1 header.b=Q+CtvyvM;
       spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org;
       dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nvidia.com
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S233723AbhBBVA3 (ORCPT <rfc822;lizhong0730@gmail.com>
        + 99 others); Tue, 2 Feb 2021 16:00:29 -0500
Received: from hqnvemgate24.nvidia.com ([216.228.121.143]:7520 "EHLO
        hqnvemgate24.nvidia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S233274AbhBBVAU (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Tue, 2 Feb 2021 16:00:20 -0500
Received: from hqmail.nvidia.com (Not Verified[216.228.121.13]) by hqnvemgate24.nvidia.com (using TLS: TLSv1.2, AES256-SHA)
        id <B6019bd3c0000>; Tue, 02 Feb 2021 12:59:40 -0800
Received: from [172.27.0.48] (172.20.145.6) by HQMAIL107.nvidia.com
 (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Tue, 2 Feb
 2021 20:59:30 +0000
Subject: Re: [PATCH 8/9] vfio/pci: use x86 naming instead of igd
To:     Jason Gunthorpe <jgg@nvidia.com>,
        Alex Williamson <alex.williamson@redhat.com>
CC:     Cornelia Huck <cohuck@redhat.com>,
        Matthew Rosato <mjrosato@linux.ibm.com>, <kvm@vger.kernel.org>,
        <linux-kernel@vger.kernel.org>, <liranl@nvidia.com>,
        <oren@nvidia.com>, <tzahio@nvidia.com>, <leonro@nvidia.com>,
        <yarong@nvidia.com>, <aviadye@nvidia.com>, <shahafs@nvidia.com>,
        <artemp@nvidia.com>, <kwankhede@nvidia.com>, <ACurrid@nvidia.com>,
        <gmataev@nvidia.com>, <cjia@nvidia.com>, <yishaih@nvidia.com>,
        <aik@ozlabs.ru>
References: <20210201162828.5938-1-mgurtovoy@nvidia.com>
 <20210201162828.5938-9-mgurtovoy@nvidia.com>
 <20210201181454.22112b57.cohuck@redhat.com>
 <599c6452-8ba6-a00a-65e7-0167f21eac35@linux.ibm.com>
 <20210201114230.37c18abd@omen.home.shazbot.org>
 <20210202170659.1c62a9e8.cohuck@redhat.com>
 <a413334c-3319-c6a3-3d8a-0bb68a10b9c1@nvidia.com>
 <20210202105455.5a358980@omen.home.shazbot.org>
 <20210202185017.GZ4247@nvidia.com>
 <20210202123723.6cc018b8@omen.home.shazbot.org>
 <20210202204432.GC4247@nvidia.com>
From:   Max Gurtovoy <mgurtovoy@nvidia.com>
Message-ID: <5e9ee84e-d950-c8d9-ac70-df042f7d8b47@nvidia.com>
Date:   Tue, 2 Feb 2021 22:59:27 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.7.0
MIME-Version: 1.0
In-Reply-To: <20210202204432.GC4247@nvidia.com>
Content-Type: text/plain; charset="utf-8"; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US
X-Originating-IP: [172.20.145.6]
X-ClientProxiedBy: HQMAIL101.nvidia.com (172.20.187.10) To
 HQMAIL107.nvidia.com (172.20.187.13)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1;
        t=1612299580; bh=VzeO8fn5oKUBcIe3AbDvCWI0BQgaZOFMZoUoYnQg2tg=;
        h=Subject:To:CC:References:From:Message-ID:Date:User-Agent:
         MIME-Version:In-Reply-To:Content-Type:Content-Transfer-Encoding:
         Content-Language:X-Originating-IP:X-ClientProxiedBy;
        b=Q+CtvyvMHflZh/Clwh4OHJNijKdiI76UyAEj58pPtOU6Pvuastj2UFSYud50DDPLP
         9GSAAKJ3kdJjy5flRmGiHJtvOeg1rPdyNy25OkqRYl2Bf/VWtK1M01W9CRBUsmt90S
         MXhYztrU7Lbwq8fw9HkQm5cWQNPuaoPdyS5eIGCXDr8T0wQBu+Jjqo2ZVrcp/x0+Gw
         hlhzsLSbeMLho2eVtyx33vlnfFSOgsJETYVo1tWgwnJlFrSN1YNiCi7Ksdd1vzrAZa
         eu/MiwQCyrd3/ArsC6JAHEvh6Oguldg+kS6ST9mLs1xXqj4zroTNyi8J7HDYSnD4cO
         x/STV1yKoFfWA==
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 2/2/2021 10:44 PM, Jason Gunthorpe wrote:
> On Tue, Feb 02, 2021 at 12:37:23PM -0700, Alex Williamson wrote:
>
>> For the most part, this explicit bind interface is redundant to
>> driver_override, which already avoids the duplicate ID issue.
> No, the point here is to have the ID tables in the PCI drivers because
> they fundamentally only work with their supported IDs. The normal
> driver core ID tables are a replacement for all the hardwired if's in
> vfio_pci.
>
> driver_override completely disables all the ID checking, it seems only
> useful for vfio_pci which works with everything. It should not be used
> with something like nvlink_vfio_pci.ko that needs ID checking.

This mechanism of driver_override seems weird to me. In case of hotplug 
and both capable drivers (native device driver and vfio-pci) are loaded, 
both will compete on the device.

I think the proposed flags is very powerful and it does fix the original 
concern Alex had ("if we start adding ids for vfio drivers then we 
create conflicts with the native host driver") and it's very deterministic.

In this way we'll bind explicitly to a driver.

And the way we'll choose a vfio-pci driver is by device_id + vendor_id + 
subsystem_device + subsystem_vendor.

There shouldn't be 2 vfio-pci drivers that support a device with same 
above 4 ids.

if you don't find a suitable vendor-vfio-pci.ko, you'll try binding 
vfio-pci.ko.

Each driver will publish its supported ids in sysfs to help the user to 
decide.

>
> Yes, this DRIVER_EXPLICIT_BIND_ONLY idea somewhat replaces
> driver_override because we could set the PCI any match on vfio_pci and
> manage the driver binding explicitly instead.
>
>> A driver id table doesn't really help for binding the device,
>> ultimately even if a device is in the id table it might fail to
>> probe due to the missing platform support that each of these igd and
>> nvlink drivers expose,
> What happens depends on what makes sense for the driver, some missing
> optional support could continue without it, or it could fail.
>
> IGD and nvlink can trivially go onwards and work if they don't find
> the platform support.
>
> Or they might want to fail, I think the mlx5 and probably nvlink
> drivers should fail as they are intended to be coupled with userspace
> that expects to use their extended features.
>
> In those cases failing is a feature because it prevents the whole
> system from going into an unexpected state.
>
> Jason