Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   "Tian, Kevin" <kevin.tian@intel.com>
To:     Alex Williamson <alex.williamson@redhat.com>
CC:     Jason Gunthorpe <jgg@ziepe.ca>,
        Zhenyu Wang <zhenyuw@linux.intel.com>,
        "Wang, Zhi A" <zhi.a.wang@intel.com>,
        Jani Nikula <jani.nikula@linux.intel.com>,
        Joonas Lahtinen <joonas.lahtinen@linux.intel.com>,
        "Vivi, Rodrigo" <rodrigo.vivi@intel.com>,
        Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>,
        David Airlie <airlied@linux.ie>,
        Daniel Vetter <daniel@ffwll.ch>,
        Eric Farman <farman@linux.ibm.com>,
        Matthew Rosato <mjrosato@linux.ibm.com>,
        Halil Pasic <pasic@linux.ibm.com>,
        Vineeth Vijayan <vneethv@linux.ibm.com>,
        "Peter Oberparleiter" <oberpar@linux.ibm.com>,
        Heiko Carstens <hca@linux.ibm.com>,
        Vasily Gorbik <gor@linux.ibm.com>,
        Alexander Gordeev <agordeev@linux.ibm.com>,
        Christian Borntraeger <borntraeger@linux.ibm.com>,
        Sven Schnelle <svens@linux.ibm.com>,
        Tony Krowiak <akrowiak@linux.ibm.com>,
        Jason Herne <jjherne@linux.ibm.com>,
        Harald Freudenberger <freude@linux.ibm.com>,
        Diana Craciun <diana.craciun@oss.nxp.com>,
        "Cornelia Huck" <cohuck@redhat.com>,
        Longfang Liu <liulongfang@huawei.com>,
        "Shameer Kolothum" <shameerali.kolothum.thodi@huawei.com>,
        Yishai Hadas <yishaih@nvidia.com>,
        Eric Auger <eric.auger@redhat.com>,
        Kirti Wankhede <kwankhede@nvidia.com>,
        Leon Romanovsky <leon@kernel.org>,
        Abhishek Sahu <abhsahu@nvidia.com>,
        "intel-gvt-dev@lists.freedesktop.org" 
        <intel-gvt-dev@lists.freedesktop.org>,
        "intel-gfx@lists.freedesktop.org" <intel-gfx@lists.freedesktop.org>,
        "dri-devel@lists.freedesktop.org" <dri-devel@lists.freedesktop.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-s390@vger.kernel.org" <linux-s390@vger.kernel.org>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "Liu, Yi L" <yi.l.liu@intel.com>
Subject: RE: [PATCH 15/15] vfio: Add struct device to vfio_device
Thread-Topic: [PATCH 15/15] vfio: Add struct device to vfio_device
Thread-Index: AQHYufqqXjqbWwGvv0ujITvL457lEa3ICPIAgAAacoCAAGT9wIAAvh4AgAB8a9A=
Date:   Thu, 1 Sep 2022 00:46:51 +0000
Message-ID: <BN9PR11MB527652B0557897BDA08B5FE38C7B9@BN9PR11MB5276.namprd11.prod.outlook.com>
References: <20220827171037.30297-1-kevin.tian@intel.com>
        <20220827171037.30297-16-kevin.tian@intel.com>
        <20220830161838.4aa47045.alex.williamson@redhat.com>
        <Yw6i7btDKcUDPADP@ziepe.ca>
        <BN9PR11MB5276BF3B8D65B66DB292CAE58C789@BN9PR11MB5276.namprd11.prod.outlook.com>
 <20220831111512.4924e152.alex.williamson@redhat.com>
In-Reply-To: <20220831111512.4924e152.alex.williamson@redhat.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?2tAN5jaXHGZRETiTGyZFXLF8adjGTYrHSMCzt464YZbC90hB1kMzasLdaYZI?=
 =?us-ascii?Q?2/LTfbbGqJMInhcf+HoqUELem3vP70VAgYwrOIYAnrFE4iKIqDIInqrMZs81?=
 =?us-ascii?Q?8Hlq3XEzTwONvAkdMP17Zf94ZqhETnpXp/+6I0BaWkUtHRskwE/GHAuvOM91?=
 =?us-ascii?Q?lDZrTW0leykKhX6OHM32e5ojLFxO0TbaFU1kkrJSjrQ6ROn1sSfCW050qXjy?=
 =?us-ascii?Q?bF6+Ap8HW3THwbIQZhdXJdcYUUM52z1u7I1yWklLzuyTnyPNKhnDhHFIeSbu?=
 =?us-ascii?Q?JQrwGChUyNJb3+GyjbWR4QglJTcK4zMeOlRT7BtPZT5r412x57sbGkqeEkpl?=
 =?us-ascii?Q?7nrEJyN6fQxFm7J9YwAobMOJ9/Ode1Ss5pi0+RqgNxAJRIAExnRk5ybNXNhw?=
 =?us-ascii?Q?MQrIf6EnwCDggrKgX0UvB8bkCuqPfVg4oGPQRPVPYCcfA+q3oAlpI2d7G4i7?=
 =?us-ascii?Q?89qhoidE9zydVfcvVJjnD0ni/5iIuUuyY8VmLT28AFqDWKE+f4FRN1ysq0HR?=
 =?us-ascii?Q?ONFk+H62bwl1+xpjjSjFQPcGIoKpw9/GSQXQ/DklKW2AcX2sPGpvyEGU7mGV?=
 =?us-ascii?Q?sQvHexv71fJv0kltkYEcnH1CzpQ2Ynv5ak5lX5WKEmuZWK6l0fodpXDU1ccm?=
 =?us-ascii?Q?+P5lvuef6g7omG3HTQPsdDtarrczjS6I/0mPyZbQopLPzbcRQctrcc20+uJz?=
 =?us-ascii?Q?XYORfY6QKHCDqgwXK/zfxzuNYlXgTpXhHC4FOKkNKHVm4f5FmGW7CwU62pTV?=
 =?us-ascii?Q?8tcnpTb9BdBx0OEVuEwTbBSx6RX9aIgyaxMMQbBTD4aGgRbfL5CCPZnBWEJb?=
 =?us-ascii?Q?rKlaS8J03b4g3O5LOIJutKmS4lPakM9AbHqPO7fTW+NULzKl8Plfs7BBVOgY?=
 =?us-ascii?Q?nKuretqKN2p0XOYESC1/jah+cydk0cVo24emZ7MuejEnBUb8AYqbVxqq7kza?=
 =?us-ascii?Q?8bgTYhEMkCgq+wuwQQcx4BH13aoWu7Kwdjba5Op6MHMmpgmio5HlfmTp3OSY?=
 =?us-ascii?Q?jIwdP6ZGLBrHYrY5NEEz6rGcIM4RZ39+wOHj4V4AFISL8e6YQlvS54q20qtl?=
 =?us-ascii?Q?GkMEMa7x7xPEhK2oJ4+UA33oT4Im0zwvGpJawjQcR3Et8Yy8lYkgoqIet/TW?=
 =?us-ascii?Q?ute7CnrJ3TChp+ksVh4BXXZce/wqBsdOPRk36Y06hOgT5WxAPGvEMpfd6FfA?=
 =?us-ascii?Q?hKRs1ilZZux50a1q1hd+uIvTdtP2PcYnuu1qjcnE5jWxJxNWUN05opkUCc7T?=
 =?us-ascii?Q?ZHYTT/jm6wlIKzKdCH0ufEgXcS+Dwp2SnxeYCzf4gEQIjKmoojBCtE/1apNz?=
 =?us-ascii?Q?uSA+9RGsR608fcPsJ4P652EHA/y2J+sBzom0xi2xaSFL/5U+Zw1oXoWnTxRi?=
 =?us-ascii?Q?Vo1C8nYFaZ43hPRrWDgNN/qXg5ESmgs9xA3VhFOHdOW5qc3UV2shg4HBMZnB?=
 =?us-ascii?Q?iZA19zoK9aCcHYNHZNniRGBN174awSjID+V5cxnfqN5NdFAzAnUl53M5BkUz?=
 =?us-ascii?Q?SiXw8Xiu9s/+6drrowBJ1+hIQUoCEHBH+JjSNNG3WJh9JI/+EN2WNYxQLEFU?=
 =?us-ascii?Q?HCvcqrD6BLiw40J0uGtwgVd54qnorZtAluS8NsIM?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5276.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bc937e53-14c1-430c-d8e8-08da8bb375cf
X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Sep 2022 00:46:51.7840
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: qfhWhfJXCFsBQkcaA2EHAuV4bt0W4PA4uywKc7CLrCMdMpp+QPQNgduU15mRdn/Akd20Lhp1ZlEyTLBcIURLuQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR11MB5234
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI,
        SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

> From: Alex Williamson <alex.williamson@redhat.com>
> Sent: Thursday, September 1, 2022 1:15 AM
>=20
> On Wed, 31 Aug 2022 06:10:51 +0000
> "Tian, Kevin" <kevin.tian@intel.com> wrote:
>=20
> > > From: Jason Gunthorpe <jgg@ziepe.ca>
> > > Sent: Wednesday, August 31, 2022 7:53 AM
> > >
> > > On Tue, Aug 30, 2022 at 04:18:38PM -0600, Alex Williamson wrote:
> > > > On Sun, 28 Aug 2022 01:10:37 +0800
> > > > Kevin Tian <kevin.tian@intel.com> wrote:
> > > >
> > > > > From: Yi Liu <yi.l.liu@intel.com>
> > > > >
> > > > > and replace kref. With it a 'vfio-dev/vfioX' node is created unde=
r the
> > > > > sysfs path of the parent, indicating the device is bound to a vfi=
o
> > > > > driver, e.g.:
> > > > >
> > > > > /sys/devices/pci0000\:6f/0000\:6f\:01.0/vfio-dev/vfio0
> > > > >
> > > > > It is also a preparatory step toward adding cdev for supporting f=
uture
> > > > > device-oriented uAPI.
> > > >
> > > > Shall we start Documentation/ABI/testing/vfio-dev now?  Thanks.
> > >
> > > I always thought that was something to use when adding new custom
> > > sysfs attributes?
> > >
> > > Here we are just creating a standard struct device with its standard
> > > sysfs?
> > >
> >
> > There is nothing special for vfio-dev/vfioX. But from pci device p.o.v
> > this does introduce a custom node in the directory, which is probably
> > what Alex referred to?
>=20
> Yup, but not just for pci, we're adding a node into the device
> directory for any device bound to vfio.
>=20
> > Anyway if required following can be introduced:
> >
> > diff --git a/Documentation/ABI/testing/sysfs-devices-vfio-dev
> b/Documentation/ABI/testing/sysfs-devices-vfio-dev
> > new file mode 100644
> > index 000000000000..dfe8baaf1ccb
> > --- /dev/null
> > +++ b/Documentation/ABI/testing/sysfs-devices-vfio-dev
> > @@ -0,0 +1,8 @@
> > +What:		 /sys/.../<device>/vfio-dev/vfioX/
> > +Date:		 September 2022
> > +Contact:	 Yi Liu <yi.l.liu@intel.com>
> > +Description:
> > +		 This directory is created when the device is bound to a
> > +		 vfio driver. The layout under this directory matches what
> > +		 exists for a standard 'struct device'. 'X' is a random
> > +		 number marking this device in vfio.
>=20
> It's not really random, it's a unique index.  Seems like a good
> starting point.
>=20
> >
> > At the start I thought it might make more sense to add it into an
> > existing vfio ABI file. But looks it doesn't exist.
> >
> > Curious why nobody asked for ABI doc for /dev/vfio/vfio, /sys/class/vfi=
o,
> etc...
>=20
> Oversight, there should probably be a sysfs-class-vfio file.  Thanks,
>=20

I can help add one.

btw I plan to respin v2 tomorrow. Regarding to this ABI thing there are
three options:

1) Just add sysfs-devices-vfio-dev in this series. Later merge to
   sysfs-class-vfio once the latter is introduced in a separate patch.

2) Do sysfs-class-vfio in this series, including both existing vfio ABIs an=
d
   the new vfio-dev.

3) No ABI file in this series. Handle it in a separate patch with
   sysfs-class-vfio.

Which one do  you prefer to?

Thanks
Kevin