Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   "Tian, Kevin" <kevin.tian@intel.com>
To:     Zhenyu Wang <zhenyuw@linux.intel.com>,
        "Wang, Zhi A" <zhi.a.wang@intel.com>,
        Jani Nikula <jani.nikula@linux.intel.com>,
        "Joonas Lahtinen" <joonas.lahtinen@linux.intel.com>,
        "Vivi, Rodrigo" <rodrigo.vivi@intel.com>,
        Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>,
        David Airlie <airlied@linux.ie>,
        Daniel Vetter <daniel@ffwll.ch>,
        Eric Farman <farman@linux.ibm.com>,
        Matthew Rosato <mjrosato@linux.ibm.com>,
        Halil Pasic <pasic@linux.ibm.com>,
        Vineeth Vijayan <vneethv@linux.ibm.com>,
        "Peter Oberparleiter" <oberpar@linux.ibm.com>,
        Heiko Carstens <hca@linux.ibm.com>,
        Vasily Gorbik <gor@linux.ibm.com>,
        Alexander Gordeev <agordeev@linux.ibm.com>,
        Christian Borntraeger <borntraeger@linux.ibm.com>,
        Sven Schnelle <svens@linux.ibm.com>,
        Tony Krowiak <akrowiak@linux.ibm.com>,
        Jason Herne <jjherne@linux.ibm.com>,
        Harald Freudenberger <freude@linux.ibm.com>,
        Diana Craciun <diana.craciun@oss.nxp.com>,
        "Alex Williamson" <alex.williamson@redhat.com>,
        Cornelia Huck <cohuck@redhat.com>,
        Longfang Liu <liulongfang@huawei.com>,
        Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>,
        Jason Gunthorpe <jgg@ziepe.ca>,
        Yishai Hadas <yishaih@nvidia.com>,
        Eric Auger <eric.auger@redhat.com>,
        "Kirti Wankhede" <kwankhede@nvidia.com>,
        Leon Romanovsky <leon@kernel.org>,
        "Abhishek Sahu" <abhsahu@nvidia.com>,
        "intel-gvt-dev@lists.freedesktop.org" 
        <intel-gvt-dev@lists.freedesktop.org>,
        "intel-gfx@lists.freedesktop.org" <intel-gfx@lists.freedesktop.org>,
        "dri-devel@lists.freedesktop.org" <dri-devel@lists.freedesktop.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-s390@vger.kernel.org" <linux-s390@vger.kernel.org>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>
CC:     "Liu, Yi L" <yi.l.liu@intel.com>
Subject: RE: [PATCH 00/15] Tidy up vfio_device life cycle
Thread-Topic: [PATCH 00/15] Tidy up vfio_device life cycle
Thread-Index: AQHYufpaEERp1jw5Hk6i2eOqrykz5a3DVopg
Date:   Sat, 27 Aug 2022 22:41:35 +0000
Message-ID: <BN9PR11MB5276B696C6446952D3E670BE8C749@BN9PR11MB5276.namprd11.prod.outlook.com>
References: <20220827171037.30297-1-kevin.tian@intel.com>
In-Reply-To: <20220827171037.30297-1-kevin.tian@intel.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?AR7/0cs7uVav4cz89zzqC4H8Nnn8zzEaH3Jckockg1LXqSAi/fCpgsC7erxq?=
 =?us-ascii?Q?2wC7PzXAkRsZ7AO2iN0LQdzAgKBucpwnJvDB3H5BCzqRFRG+wiO+DZrkdpy3?=
 =?us-ascii?Q?VztKW1S7oSIUkzI2NS6FCKEHsaO7h2ZhBemJau19+xEKtC2oAOjm3JSo1nIL?=
 =?us-ascii?Q?0ucuxYJCsAbESzirObM8NUeBn345xSgBRXm01yo7jq+fP0KjzRXmvyLuoZlp?=
 =?us-ascii?Q?wiG3ebQwTdCMpbMBpt2ySj1nry+INvEbrCdSR3UUpoWW43MyIeYHFWSOMZqk?=
 =?us-ascii?Q?y+UWVEuRsK8g6Aa24fGNkArHjDfNj0KousiYGaSAfKUHwlq0D5Ye3lvNroMF?=
 =?us-ascii?Q?O7hUnwq1pkdADajOEJDDqQ9c6GcpJXdDs2H8uHdVIMdYssEREnWYFy0q5XXL?=
 =?us-ascii?Q?IzjIH0cVBsZ/7L2OUiTZmnSurs1UPztsCUbY91Ic8xijvNI9v7ZfNwduxURW?=
 =?us-ascii?Q?UEDdAjsKt8WYXzK/NjtT0WXH/9BAfqSl5cBKQ8ak3oQABmGjrqBd2xEkDqXD?=
 =?us-ascii?Q?3u2HSUjE67P/t8NeCuuakeOTCdbEsaK3ypLZ+yPUlC3F1NSwrDEugjf01oya?=
 =?us-ascii?Q?T4sovIl3Z29s0KK9C/ngEB0gUqYJ6e6OjrDRlIe+pdIWiWuYwJ63mhXlQVox?=
 =?us-ascii?Q?bRI/27G3l723THtXxk5S/X+NaYcYXeO7OMw9/0DZXgv+BfYWitMxb97mxyUv?=
 =?us-ascii?Q?SMX2NUXQs3y9kw+910hLLNwcetVdFZ99HtCKloapz12tVvjwmyoPPxlERMF1?=
 =?us-ascii?Q?DasA666i+9c7XIrLVg9I1UK+xTeYqqLmpXiL1ckomHFS3cWIEdQkSYyw/kV8?=
 =?us-ascii?Q?+8fM0MXndKGkEnYav5DDZzG3N6b/8sH/NXDh/StidEklx4fJJBcz3C9a/cwA?=
 =?us-ascii?Q?EpCXZcwcvpFPaciyoHHY6CA4qGvHJ/EtBxA8RcJ1fkXAWmrTYltQ6WDT9Ul1?=
 =?us-ascii?Q?9pwq5Aes9/rblW+b3ZNJF4vRtUVlk018tZBEHJj7JwSf4i6xO0iVLiZ2hDnm?=
 =?us-ascii?Q?56TIinOhGulADp1qe9EpTabqACBKrbZZr70cm/sEyjDA1g0XSN4lkVIRcHQi?=
 =?us-ascii?Q?W4BRDN+Q9gD/6u9zhlYv/g2rrgpu59mRhMZuxiT/aGYQze/BpIqrF3NzZic+?=
 =?us-ascii?Q?U47BzA+kzbWwnbG2yU9KScNEQRye12AiwtBmugsIrrx59GAgQ58aLYTNVLXN?=
 =?us-ascii?Q?jB3DjU++5Frj351ECYlSdcmyyB7YMmr0bFZpZnuvu9vB/uYev+RPwX9OqurB?=
 =?us-ascii?Q?3wjjF4xShxlPFA/zXndvQ8+cg3mK0nElHbTgI9ski0je1vo/b/aJNFzb37GD?=
 =?us-ascii?Q?7TXiOVoL9l4J+tAPWq1hQfXOvcZLdzSW4Vs0kipE4zB1wma7J9QOFQu/qPZH?=
 =?us-ascii?Q?/I/hjRbXiesMr+m/QYqk6LYb6kHd/sLHQ4wuA72P5SYJfu0nQ88Le12HV0y/?=
 =?us-ascii?Q?BuY83xZMaGK2msww973G3qhjTLbFdtMsXeWO7Ht7O7+7Wp3ZI1sG1Gy5lLEq?=
 =?us-ascii?Q?ZdaGoTErRmNJjbAJNowB/zCgH3yNPN3Rk3RSUFOZOnTMT3NE4fbLaisJCNvO?=
 =?us-ascii?Q?o4kn3L7Mxs2WaInMEHV6aRZHhkqII0Gs4KHN8M6s?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5276.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d4971ad3-b789-409a-7130-08da887d4bee
X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Aug 2022 22:41:35.1915
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: dYSFL2M5cmaNNagGotu/68+VRMD2PkRuKTdyyHujXL/OdLoSY61lT2mUvkmAC/OmMTFQOGUJ5n5cdO3cwGYBog==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR11MB2566
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI,
        SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

lkp reported some warnings below. I'll got them fixed in the next
version. Not sure why they are not captured in my builds though
I did turn on treating warnings as error.

drivers/vfio/fsl-mc/vfio_fsl_mc.c:570:6: warning: no previous prototype for=
 'vfio_fsl_mc_release_dev' [-Wmissing-prototypes]
drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.c:1216:5: warning: no previous=
 prototype for 'hisi_acc_vfio_pci_migrn_init_dev' [-Wmissing-prototypes]
drivers/vfio/platform/vfio_amba.c:110:37: warning: unused variable 'vfio_pl=
atform_ops' [-Wunused-const-variable]
drivers/vfio/platform/vfio_platform_common.c:608:37: warning: unused variab=
le 'vfio_platform_ops' [-Wunused-const-variable]


> From: Tian, Kevin <kevin.tian@intel.com>
> Sent: Sunday, August 28, 2022 1:10 AM
>=20
> The idea is to let vfio core manage the vfio_device life cycle instead
> of duplicating the logic cross drivers. Besides cleaner code in driver
> side this also allows adding struct device to vfio_device as the first
> step toward adding cdev uAPI in the future. Another benefit is that
> user can now look at sysfs to decide whether a device is bound to
> vfio [1], e.g.:
>=20
> 	/sys/devices/pci0000\:6f/0000\:6f\:01.0/vfio-dev/vfio0
>=20
> Though most drivers can fit the new model naturally:
>=20
>  - vfio_alloc_device() to allocate and initialize vfio_device
>  - vfio_put_device() to release vfio_device
>  - dev_ops->init() for driver private initialization
>  - dev_ops->release() for driver private cleanup
>=20
> vfio-ccw is the only exception due to a life cycle mess that its private
> structure mixes both parent and mdev info hence must be alloc/free'ed
> outside of the life cycle of vfio device.
>=20
> Per prior discussions this won't be fixed in short term by IBM folks [2].
>=20
> Instead of waiting this series introduces a few tricks to move forward:
>=20
>  - vfio_init_device() to initialize a pre-allocated device structure;
>=20
>  - require *EVERY* driver to implement @release and free vfio_device
>    inside. Then vfio-ccw can use a completion mechanism to delay the
>    free to css driver;
>=20
> The second trick is not a real burden to other drivers because they
> all require a @release for private cleanup anyay. Later once the ccw
> mess is fixed a simple cleanup can be done by moving free from @release
> to vfio core.
>=20
> Thanks
> Kevin
>=20
> [1] https://listman.redhat.com/archives/libvir-list/2022-August/233482.ht=
ml
> [2]
> https://lore.kernel.org/all/0ee29bd6583f17f0ee4ec0769fa50e8ea6703623.ca
> mel@linux.ibm.com/
>=20
> Kevin Tian (6):
>   vfio: Add helpers for unifying vfio_device life cycle
>   drm/i915/gvt: Use the new device life cycle helpers
>   vfio/platform: Use the new device life cycle helpers
>   vfio/amba: Use the new device life cycle helpers
>   vfio/ccw: Use the new device life cycle helpers
>   vfio: Rename vfio_device_put() and vfio_device_try_get()
>=20
> Yi Liu (9):
>   vfio/pci: Use the new device life cycle helpers
>   vfio/mlx5: Use the new device life cycle helpers
>   vfio/hisi_acc: Use the new device life cycle helpers
>   vfio/mdpy: Use the new device life cycle helpers
>   vfio/mtty: Use the new device life cycle helpers
>   vfio/mbochs: Use the new device life cycle helpers
>   vfio/ap: Use the new device life cycle helpers
>   vfio/fsl-mc: Use the new device life cycle helpers
>   vfio: Add struct device to vfio_device
>=20
>  drivers/gpu/drm/i915/gvt/gvt.h                |   5 +-
>  drivers/gpu/drm/i915/gvt/kvmgt.c              |  52 ++++--
>  drivers/gpu/drm/i915/gvt/vgpu.c               |  31 ++--
>  drivers/s390/cio/vfio_ccw_ops.c               |  52 +++++-
>  drivers/s390/cio/vfio_ccw_private.h           |   3 +
>  drivers/s390/crypto/vfio_ap_ops.c             |  50 +++---
>  drivers/vfio/fsl-mc/vfio_fsl_mc.c             |  87 +++++----
>  .../vfio/pci/hisilicon/hisi_acc_vfio_pci.c    |  80 ++++-----
>  drivers/vfio/pci/mlx5/main.c                  |  49 ++++--
>  drivers/vfio/pci/vfio_pci.c                   |  20 +--
>  drivers/vfio/pci/vfio_pci_core.c              |  23 ++-
>  drivers/vfio/platform/vfio_amba.c             |  72 ++++++--
>  drivers/vfio/platform/vfio_platform.c         |  66 +++++--
>  drivers/vfio/platform/vfio_platform_common.c  |  61 +++----
>  drivers/vfio/platform/vfio_platform_private.h |  18 +-
>  drivers/vfio/vfio_main.c                      | 165 +++++++++++++++---
>  include/linux/vfio.h                          |  29 ++-
>  include/linux/vfio_pci_core.h                 |   6 +-
>  samples/vfio-mdev/mbochs.c                    |  73 +++++---
>  samples/vfio-mdev/mdpy.c                      |  81 +++++----
>  samples/vfio-mdev/mtty.c                      |  67 ++++---
>  21 files changed, 724 insertions(+), 366 deletions(-)
>=20
>=20
> base-commit: 1c23f9e627a7b412978b4e852793c5e3c3efc555
> --
> 2.21.3