Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 7 Sep 2022 09:11:34 -0300
From:   Jason Gunthorpe <jgg@ziepe.ca>
To:     Christoph Hellwig <hch@infradead.org>
Cc:     "Tian, Kevin" <kevin.tian@intel.com>,
        Matthew Rosato <mjrosato@linux.ibm.com>,
        David Airlie <airlied@linux.ie>,
        Joonas Lahtinen <joonas.lahtinen@linux.intel.com>,
        "dri-devel@lists.freedesktop.org" <dri-devel@lists.freedesktop.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Kirti Wankhede <kwankhede@nvidia.com>,
        Vineeth Vijayan <vneethv@linux.ibm.com>,
        Diana Craciun <diana.craciun@oss.nxp.com>,
        Alexander Gordeev <agordeev@linux.ibm.com>,
        Longfang Liu <liulongfang@huawei.com>,
        "linux-s390@vger.kernel.org" <linux-s390@vger.kernel.org>,
        "Liu, Yi L" <yi.l.liu@intel.com>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        Leon Romanovsky <leon@kernel.org>,
        Halil Pasic <pasic@linux.ibm.com>,
        Christian Borntraeger <borntraeger@linux.ibm.com>,
        "intel-gfx@lists.freedesktop.org" <intel-gfx@lists.freedesktop.org>,
        "Wang, Zhi A" <zhi.a.wang@intel.com>,
        Tony Krowiak <akrowiak@linux.ibm.com>,
        Eric Farman <farman@linux.ibm.com>,
        Vasily Gorbik <gor@linux.ibm.com>,
        Heiko Carstens <hca@linux.ibm.com>,
        Jani Nikula <jani.nikula@linux.intel.com>,
        Eric Auger <eric.auger@redhat.com>,
        Alex Williamson <alex.williamson@redhat.com>,
        Harald Freudenberger <freude@linux.ibm.com>,
        Zhenyu Wang <zhenyuw@linux.intel.com>,
        "Vivi, Rodrigo" <rodrigo.vivi@intel.com>,
        "intel-gvt-dev@lists.freedesktop.org" 
        <intel-gvt-dev@lists.freedesktop.org>,
        Jason Herne <jjherne@linux.ibm.com>,
        Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>,
        Yishai Hadas <yishaih@nvidia.com>,
        Cornelia Huck <cohuck@redhat.com>,
        Peter Oberparleiter <oberpar@linux.ibm.com>,
        Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>,
        Sven Schnelle <svens@linux.ibm.com>,
        Daniel Vetter <daniel@ffwll.ch>,
        Abhishek Sahu <abhsahu@nvidia.com>
Subject: Re: [PATCH v2 01/15] vfio: Add helpers for unifying vfio_device life
 cycle
Message-ID: <YxiKdh6EDBZnTAGH@ziepe.ca>
References: <20220901143747.32858-1-kevin.tian@intel.com>
 <20220901143747.32858-2-kevin.tian@intel.com>
 <YxcV05AVN4kqdPX6@infradead.org>
 <BN9PR11MB5276EE6209C1E3D4662368DC8C419@BN9PR11MB5276.namprd11.prod.outlook.com>
 <YxiGpryRNrxvEoiY@infradead.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <YxiGpryRNrxvEoiY@infradead.org>
Precedence: bulk

On Wed, Sep 07, 2022 at 04:55:18AM -0700, Christoph Hellwig wrote:
> On Wed, Sep 07, 2022 at 12:43:30AM +0000, Tian, Kevin wrote:
> > > From: Christoph Hellwig
> > > Sent: Tuesday, September 6, 2022 5:42 PM
> > > 
> > > What is the point?  This adds indirect calls, and actually creates
> > > more boilerplate code in the drivers.  i.g. when using this code there
> > > is more, and harder to read code.
> > 
> > The point is to align with struct device life cycle when it's introduced
> > to vfio_device. The object is released via put_device() then what would
> > be the alternative if the driver doesn't provide a @release callback?
> > 
> > and with @release then naturally @init is also expected.
> 
> No, with a release no @init is expected.  The init method is one
> of the major obsfucations here, only topped by the weird
> vfio_alloc_device macro.  Yes, that saves about 4 lines of code
> in every driver, but places a burden on the struct layout and
> very much obsfucated things.  Without vfio_alloc_device and
> the init method I think much of this would make a lot more sense.
> 
> See the patch below that goes on top of this series to show how
> undoing these two would look on mbochs.  It it a slight reduction
> lines of code, but more readable and much less churn compared
> to the status before this series.

I've seen alot of error handling bugs caused by open-coding patterns
like this. People get confused about what the lifecycle is and botch
the error unwinds, almost 100% of the time :\ They call kfree when
they should call put_device, they call put_device before initing
enough stuff that the release callback doesn't crash, double free
stuff by calling put_device at the wrong point, and so on.

The advantage of init/release is the strict pairing and the core code
helping get the error unwind right, by not calling release until init
succeeds.

The advantage of the vfio_alloc_device() is not saving 4 lines, it is
giving the drivers a simple/sane error handling strategy. Goto unwind
inside init, release undoes everything init does and the probe path
only calls put_device(). It is simple and logical to implement and
hard to make subtle bugs.

Specifically it eliminates the open coded transition of kfree to
put_device that seems so difficult for people to get right.

netdev has done a version of this, so has rdma, and it works well.

Jason