Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Content-Type: text/plain;
        charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\))
Subject: Re: Question about the barrier() in hlist_nulls_for_each_entry_rcu()
From:   Alan Huang <mmpgouride@gmail.com>
In-Reply-To: <CANn89iLqU=huOuCt2kXmrXf68TUU-N90aQnMykkYcZ+Arx9-aA@mail.gmail.com>
Date:   Fri, 21 Jul 2023 22:31:13 +0800
Cc:     linux-kernel@vger.kernel.org, netdev@vger.kernel.org,
        rcu@vger.kernel.org, "Paul E. McKenney" <paulmck@kernel.org>,
        roman.gushchin@linux.dev
Content-Transfer-Encoding: quoted-printable
Message-Id: <EDE1C0AE-B479-49F9-995D-DA9CC1A6EA57@gmail.com>
References: <04C1E631-725C-47AD-9914-25D5CE04DFF4@gmail.com>
 <CANn89iKJWw7zUP-E_d=Yhaz=Qw0R3Ae7ULaGgrtsi1yf2pfpGg@mail.gmail.com>
 <E9CF24C7-3080-4720-B540-BAF03068336B@gmail.com>
 <CANn89iLqU=huOuCt2kXmrXf68TUU-N90aQnMykkYcZ+Arx9-aA@mail.gmail.com>
To:     Eric Dumazet <edumazet@google.com>
Precedence: bulk


> 2023=E5=B9=B47=E6=9C=8821=E6=97=A5 05:11=EF=BC=8CEric Dumazet =
<edumazet@google.com> =E5=86=99=E9=81=93=EF=BC=9A
>=20
> On Thu, Jul 20, 2023 at 10:00=E2=80=AFPM Alan Huang =
<mmpgouride@gmail.com> wrote:
>>=20
>>=20
>>> 2023=E5=B9=B47=E6=9C=8821=E6=97=A5 03:22=EF=BC=8CEric Dumazet =
<edumazet@google.com> =E5=86=99=E9=81=93=EF=BC=9A
>>>=20
>>> On Thu, Jul 20, 2023 at 8:54=E2=80=AFPM Alan Huang =
<mmpgouride@gmail.com> wrote:
>>>>=20
>>>> Hi,
>>>>=20
>>>> I noticed a commit c87a124a5d5e(=E2=80=9Cnet: force a reload of =
first item in hlist_nulls_for_each_entry_rcu=E2=80=9D)
>>>> and a related discussion [1].
>>>>=20
>>>> After reading the whole discussion, it seems like that ptr->field =
was cached by gcc even with the deprecated
>>>> ACCESS_ONCE(), so my question is:
>>>>=20
>>>>       Is that a compiler bug? If so, has this bug been fixed today, =
ten years later?
>>>>=20
>>>>       What about READ_ONCE(ptr->field)?
>>>=20
>>> Make sure sparse is happy.
>>=20
>> It caused a problem without barrier(), and the deprecated =
ACCESS_ONCE() didn=E2=80=99t help:
>>=20
>>        https://lore.kernel.org/all/519D19DA.50400@yandex-team.ru/
>>=20
>> So, my real question is: With READ_ONCE(ptr->field), are there still =
some unusual cases where gcc
>> decides not to reload ptr->field?
>=20
> I can not really answer without seeing an actual patch...

The content of the potential patch:

diff --git a/include/linux/rculist_nulls.h =
b/include/linux/rculist_nulls.h
index 89186c499dd4..bcd39670f359 100644
--- a/include/linux/rculist_nulls.h
+++ b/include/linux/rculist_nulls.h
@@ -158,15 +158,9 @@ static inline void hlist_nulls_add_fake(struct =
hlist_nulls_node *n)
  * @pos:       the &struct hlist_nulls_node to use as a loop cursor.
  * @head:      the head of the list.
  * @member:    the name of the hlist_nulls_node within the struct.
- *
- * The barrier() is needed to make sure compiler doesn't cache first =
element [1],
- * as this loop can be restarted [2]
- * [1] Documentation/memory-barriers.txt around line 1533
- * [2] Documentation/RCU/rculist_nulls.rst around line 146
  */
 #define hlist_nulls_for_each_entry_rcu(tpos, pos, head, member)         =
               \
-       for (({barrier();}),                                             =
       \
-            pos =3D rcu_dereference_raw(hlist_nulls_first_rcu(head));   =
         \
+       for (pos =3D rcu_dereference_raw(hlist_nulls_first_rcu(head));   =
         \
                (!is_a_nulls(pos)) &&                                    =
       \
                ({ tpos =3D hlist_nulls_entry(pos, typeof(*tpos), =
member); 1; }); \
                pos =3D rcu_dereference_raw(hlist_nulls_next_rcu(pos)))
@@ -180,8 +174,7 @@ static inline void hlist_nulls_add_fake(struct =
hlist_nulls_node *n)
  * @member:    the name of the hlist_nulls_node within the struct.
  */
 #define hlist_nulls_for_each_entry_safe(tpos, pos, head, member)        =
       \
-       for (({barrier();}),                                             =
       \
-            pos =3D rcu_dereference_raw(hlist_nulls_first_rcu(head));   =
         \
+       for (pos =3D rcu_dereference_raw(hlist_nulls_first_rcu(head));   =
         \
                (!is_a_nulls(pos)) &&                                    =
       \
                ({ tpos =3D hlist_nulls_entry(pos, typeof(*tpos), =
member);        \
                   pos =3D =
rcu_dereference_raw(hlist_nulls_next_rcu(pos)); 1; });)


>=20
> Why are you asking ? Are you tracking compiler bug fixes ?

The barrier() here makes me confused.
=20
If we really need that, do we need:

	READ_ONCE(head->first);
	barrier();
	READ_ONCE(head->first);

?

>=20
>>=20
>>>=20
>>> Do you have a patch for review ?
>>=20
>> Possibly next month. :)
>>=20
>>>=20
>>>=20
>>>>=20
>>>>=20
>>>> [1] =
https://lore.kernel.org/all/1369699930.3301.494.camel@edumazet-glaptop/
>>>>=20
>>>> Thanks,
>>>> Alan