Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
From:   NeilBrown <neilb@suse.com>
To:     James Simmons <jsimmons@infradead.org>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        devel@driverdev.osuosl.org,
        Andreas Dilger <andreas.dilger@intel.com>,
        Oleg Drokin <oleg.drokin@intel.com>
Date:   Tue, 15 May 2018 13:40:27 +1000
Cc:     Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        Lustre Development List <lustre-devel@lists.lustre.org>,
        Lai Siyao <lai.siyao@intel.com>,
        James Simmons <jsimmons@infradead.org>
Subject: Re: [PATCH v2] staging: lustre: obdclass: change object lookup to no wait mode
In-Reply-To: <1526350548-4402-1-git-send-email-jsimmons@infradead.org>
References: <1526350548-4402-1-git-send-email-jsimmons@infradead.org>
Message-ID: <87bmdh7fis.fsf@notabene.neil.brown.name>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
        micalg=pgp-sha256; protocol="application/pgp-signature"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

--=-=-=
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Mon, May 14 2018, James Simmons wrote:

> From: Lai Siyao <lai.siyao@intel.com>
>
> Currently we set LU_OBJECT_HEARD_BANSHEE on object when we want
> to remove object from cache, but this may lead to deadlock, because
> when other process lookup such object, it needs to wait for this
> object until release (done at last refcount put), while that process
> maybe already hold an LDLM lock.
>
> Now that current code can handle dying object correctly, we can just
> return such object in lookup, thus the above deadlock can be avoided.
>
> Signed-off-by: Lai Siyao <lai.siyao@intel.com>
> Intel-bug-id: https://jira.hpdd.intel.com/browse/LU-9049
> Reviewed-on: https://review.whamcloud.com/26965
> Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
> Tested-by: Cliff White <cliff.white@intel.com>
> Reviewed-by: Fan Yong <fan.yong@intel.com>
> Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
> Signed-off-by: James Simmons <jsimmons@infradead.org>

Reviewed-by: NeilBrown <neilb@suse.com>

Thanks :-)

NeilBrown

> ---
> Changelog:
>
> v1) Initial patch that didn't apply to staging-testing branch
> v2) Rebased after Neil's patches landed. Remove unlikely() test
>     as requested by Dan Carpenter
>
>  drivers/staging/lustre/lustre/obdclass/lu_object.c | 39 +++++-----------=
------
>  1 file changed, 9 insertions(+), 30 deletions(-)
>
> diff --git a/drivers/staging/lustre/lustre/obdclass/lu_object.c b/drivers=
/staging/lustre/lustre/obdclass/lu_object.c
> index f14e350..e0abd4f 100644
> --- a/drivers/staging/lustre/lustre/obdclass/lu_object.c
> +++ b/drivers/staging/lustre/lustre/obdclass/lu_object.c
> @@ -593,15 +593,10 @@ static struct lu_object *htable_lookup(struct lu_si=
te *s,
>  				       const struct lu_fid *f,
>  				       __u64 *version)
>  {
> -	struct cfs_hash		*hs =3D s->ls_obj_hash;
>  	struct lu_site_bkt_data *bkt;
>  	struct lu_object_header *h;
>  	struct hlist_node	*hnode;
> -	__u64 ver;
> -	wait_queue_entry_t waiter;
> -
> -retry:
> -	ver =3D cfs_hash_bd_version_get(bd);
> +	u64 ver =3D cfs_hash_bd_version_get(bd);
>=20=20
>  	if (*version =3D=3D ver)
>  		return ERR_PTR(-ENOENT);
> @@ -618,31 +613,13 @@ static struct lu_object *htable_lookup(struct lu_si=
te *s,
>  	}
>=20=20
>  	h =3D container_of(hnode, struct lu_object_header, loh_hash);
> -	if (likely(!lu_object_is_dying(h))) {
> -		cfs_hash_get(s->ls_obj_hash, hnode);
> -		lprocfs_counter_incr(s->ls_stats, LU_SS_CACHE_HIT);
> -		if (!list_empty(&h->loh_lru)) {
> -			list_del_init(&h->loh_lru);
> -			percpu_counter_dec(&s->ls_lru_len_counter);
> -		}
> -		return lu_object_top(h);
> +	cfs_hash_get(s->ls_obj_hash, hnode);
> +	lprocfs_counter_incr(s->ls_stats, LU_SS_CACHE_HIT);
> +	if (!list_empty(&h->loh_lru)) {
> +		list_del_init(&h->loh_lru);
> +		percpu_counter_dec(&s->ls_lru_len_counter);
>  	}
> -
> -	/*
> -	 * Lookup found an object being destroyed this object cannot be
> -	 * returned (to assure that references to dying objects are eventually
> -	 * drained), and moreover, lookup has to wait until object is freed.
> -	 */
> -
> -	init_waitqueue_entry(&waiter, current);
> -	add_wait_queue(&bkt->lsb_marche_funebre, &waiter);
> -	set_current_state(TASK_UNINTERRUPTIBLE);
> -	lprocfs_counter_incr(s->ls_stats, LU_SS_CACHE_DEATH_RACE);
> -	cfs_hash_bd_unlock(hs, bd, 1);
> -	schedule();
> -	remove_wait_queue(&bkt->lsb_marche_funebre, &waiter);
> -	cfs_hash_bd_lock(hs, bd, 1);
> -	goto retry;
> +	return lu_object_top(h);
>  }
>=20=20
>  /**
> @@ -683,6 +660,8 @@ static void lu_object_limit(const struct lu_env *env,=
 struct lu_device *dev)
>  }
>=20=20
>  /**
> + * Core logic of lu_object_find*() functions.
> + *
>   * Much like lu_object_find(), but top level device of object is specifi=
cally
>   * \a dev rather than top level device of the site. This interface allows
>   * objects of different "stacking" to be created within the same site.
> --=20
> 1.8.3.1

--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEG8Yp69OQ2HB7X0l6Oeye3VZigbkFAlr6VqwACgkQOeye3VZi
gbnLHQ//XwpQhuYbubF114KewUnGrMBgj+tX384BqfXbxS/3ddyj41xQtuDOy7Bd
CllvqRhTcJZKRWYCy9vunBNVNJJ/dzXpG0zqeKRf3GK7VrJbMreX4Q+AvOqOfiVz
jGpKLJ5prrAKr2FJyMZZrXOrjyI2GQ36kOVjwa01O+UoEdyIXFpRURsFj2I85nrd
unTbYL2rEvN0Tx8gqIaQd43APOEFfPHWBg1atifgPXrSN2AoEe30Qq2FwJ7DvanS
bM7GFT0Ch63oQ2b+Cmuhj5i6+Evx5O6IOcW2RzJ3orFIS+P6E/Cm+JsIOmRsWUw4
eQKvjR8sYq+DpaOuGb5kaWKTfI+m/W8Jk4hHDwDxfTYcQUxZOxST3HWmGodVKNBf
YdTkl/cv0pedAM4QZRanH6PAhpp6V5su4fyQANYM1lAtpZfHb45/ehfnNIw8r8t5
nmmtZ6m3Cd+01rh2PmwdiF8dyt6T/98kcCoKX40yY4G/gfs1NmWebUotfAkRwOYZ
yCkFieh/OG+M9pGD1f7kfd8VXfoUOlPstAoCtPDKoepw4c5igze06fZa5/Uyilo2
DykPGZ5tsXHxhr5cyLvb7JMZDCBSOgdici5uO+loRyZgGEdSwg83lx/t37z8dOxz
vItkxPZC7kNITUvA0ZyeMRQiOfNtfWXn7x8D7wJAWGRDuXpH5a0=
=9IzA
-----END PGP SIGNATURE-----
--=-=-=--