Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) client-ip=2620:137:e000::1:18;
MIME-Version: 1.0
References: <20220601084149.13097-1-zhoufeng.zf@bytedance.com>
 <20220601084149.13097-2-zhoufeng.zf@bytedance.com> <CAADnVQJcbDXtQsYNn=j0NzKx3SFSPE1YTwbmtkxkpzmFt-zh9Q@mail.gmail.com>
 <21ec90e3-2e89-09c1-fd22-de76e6794d68@bytedance.com>
In-Reply-To: <21ec90e3-2e89-09c1-fd22-de76e6794d68@bytedance.com>
From:   Alexei Starovoitov <alexei.starovoitov@gmail.com>
Date:   Wed, 1 Jun 2022 13:35:19 +0200
Message-ID: <CAADnVQKdU-3uBE9tKifChUunmr=c=32M4GwP8qG1-S=Atf7fvw@mail.gmail.com>
Subject: Re: [External] Re: [PATCH v4 1/2] bpf: avoid grabbing spin_locks of
 all cpus when no free elems
To:     Feng Zhou <zhoufeng.zf@bytedance.com>
Cc:     Alexei Starovoitov <ast@kernel.org>,
        Daniel Borkmann <daniel@iogearbox.net>,
        Andrii Nakryiko <andrii@kernel.org>,
        Martin KaFai Lau <kafai@fb.com>,
        Song Liu <songliubraving@fb.com>, Yonghong Song <yhs@fb.com>,
        John Fastabend <john.fastabend@gmail.com>,
        KP Singh <kpsingh@kernel.org>,
        Network Development <netdev@vger.kernel.org>,
        bpf <bpf@vger.kernel.org>, LKML <linux-kernel@vger.kernel.org>,
        Xiongchun Duan <duanxiongchun@bytedance.com>,
        Muchun Song <songmuchun@bytedance.com>,
        Dongdong Wang <wangdongdong.6@bytedance.com>,
        Cong Wang <cong.wang@bytedance.com>,
        Chengming Zhou <zhouchengming@bytedance.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Wed, Jun 1, 2022 at 1:11 PM Feng Zhou <zhoufeng.zf@bytedance.com> wrote:
>
> =E5=9C=A8 2022/6/1 =E4=B8=8B=E5=8D=885:50, Alexei Starovoitov =E5=86=99=
=E9=81=93:
> > On Wed, Jun 1, 2022 at 10:42 AM Feng zhou <zhoufeng.zf@bytedance.com> w=
rote:
> >>   static inline void ___pcpu_freelist_push(struct pcpu_freelist_head *=
head,
> >> @@ -130,14 +134,19 @@ static struct pcpu_freelist_node *___pcpu_freeli=
st_pop(struct pcpu_freelist *s)
> >>          orig_cpu =3D cpu =3D raw_smp_processor_id();
> >>          while (1) {
> >>                  head =3D per_cpu_ptr(s->freelist, cpu);
> >> +               if (READ_ONCE(head->is_empty))
> >> +                       goto next_cpu;
> >>                  raw_spin_lock(&head->lock);
> >>                  node =3D head->first;
> >>                  if (node) {
> > extra bool is unnecessary.
> > just READ_ONCE(head->first)
>
> As for why to add is_empty instead of directly judging head->first, my
> understanding is this, head->first is frequently modified during updating
> map, which will lead to invalid other cpus's cache, and is_empty is after
> freelist having no free elems will be changed, the performance will be
> better.

maybe. pls benchmark it.
imo wasting a bool for the corner case is not a good trade off.