Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Thu, 3 Oct 2019 15:36:48 -0700
From:   Jakub Kicinski <jakub.kicinski@netronome.com>
To:     Bjorn Helgaas <helgaas@kernel.org>
Cc:     CREGUT Pierre IMT/OLN <pierre.cregut@orange.com>,
        linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org,
        Donald Dutile <ddutile@redhat.com>,
        Alexander Duyck <alexander.h.duyck@intel.com>
Subject: Re: [PATCH] PCI/IOV: update num_VFs earlier
Message-ID: <20191003153648.786ad0bf@cakuba.hsd1.ca.comcast.net>
In-Reply-To: <20191003221007.GA209602@google.com>
References: <49b0ad6d-7b6f-adbd-c4a3-5f9328a7ad9d@orange.com>
        <20191003221007.GA209602@google.com>
Organization: Netronome Systems, Ltd.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Thu, 3 Oct 2019 17:10:07 -0500, Bjorn Helgaas wrote:
> On Thu, Oct 03, 2019 at 11:04:45AM +0200, CREGUT Pierre IMT/OLN wrote:
> > Le 02/10/2019 =C3=A0 01:45, Bjorn Helgaas a =C3=A9crit=C2=A0: =20
> > > On Fri, Apr 26, 2019 at 10:11:54AM +0200, CREGUT Pierre IMT/OLN wrote=
: =20
> > > > I also initially thought that kobject_uevent generated the netlink =
event
> > > > but this is not the case. This is generated by the specific driver =
in use.
> > > > For the Intel i40e driver, this is the call to i40e_do_reset_safe in
> > > > i40e_pci_sriov_configure that sends the event.
> > > > It is followed by i40e_pci_sriov_enable that calls i40e_alloc_vfs t=
hat
> > > > finally calls the generic pci_enable_sriov function. =20
> > > I don't know anything about netlink.  The script from the bugzilla
> > > (https://bugzilla.kernel.org/show_bug.cgi?id=3D202991) looks like it
> > > runs
> > >=20
> > >    ip monitor dev enp9s0f2
> > >=20
> > > What are the actual netlink events you see?  Are they related to a
> > > device being removed? =20
> >=20
> > We have netlink events both when num_vfs goes from 0 to N and from N to=
 0.
> > Indeed you have to go to 0 before going to M with M !=3D N. =20
>=20
> Right.

FWIW I think this netlink event is an artefact of i40e implementation,
and is not something the networking stack generates. Hopefully Alex can
correct me if I'm wrong, but I don't think most drivers will generate
such an event.

> commit 0940fc95da45
> Author: Pierre Cr=C3=A9gut <pierre.cregut@orange.com>
> Date:   Wed Sep 11 09:27:36 2019 +0200
>=20
>     PCI/IOV: Serialize sysfs sriov_numvfs reads vs writes
>    =20
>     When sriov_numvfs is being updated, drivers may notify about new devi=
ces
>     before they are reflected in sriov->num_VFs, so concurrent sysfs reads
>     previously returned stale values.
>    =20
>     Serialize the sysfs read vs the write so the read returns the correct
>     num_VFs value.
>    =20
>     Link: https://bugzilla.kernel.org/show_bug.cgi?id=3D202991
>     Link: https://lore.kernel.org/r/20190911072736.32091-1-pierre.cregut@=
orange.com
>     Signed-off-by: Pierre Cr=C3=A9gut <pierre.cregut@orange.com>
>     Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>
>=20
> diff --git a/drivers/pci/iov.c b/drivers/pci/iov.c
> index b3f972e8cfed..e77562aabbae 100644
> --- a/drivers/pci/iov.c
> +++ b/drivers/pci/iov.c
> @@ -254,8 +254,14 @@ static ssize_t sriov_numvfs_show(struct device *dev,
>  				 char *buf)
>  {
>  	struct pci_dev *pdev =3D to_pci_dev(dev);
> +	u16 num_vfs;
> +
> +	/* Serialize vs sriov_numvfs_store() so readers see valid num_VFs */
> +	device_lock(&pdev->dev);
> +	num_vfs =3D pdev->sriov->num_VFs;
> +	device_lock(&pdev->dev);
> =20
> -	return sprintf(buf, "%u\n", pdev->sriov->num_VFs);
> +	return sprintf(buf, "%u\n", num_vfs);
>  }
> =20
>  /*

The change makes sense to me!