Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) client-ip=2620:137:e000::1:18;
MIME-Version: 1.0
References: <20220224000531.1265030-1-haoluo@google.com> <a7f26f93-c5f8-2abc-e186-5d179706ae8e@soleen.com>
 <CA+khW7hwT0PSiToAJcdX1Te9QwhWL671sMX+92VS+V6Zsp+0Vg@mail.gmail.com>
In-Reply-To: <CA+khW7hwT0PSiToAJcdX1Te9QwhWL671sMX+92VS+V6Zsp+0Vg@mail.gmail.com>
From:   Andrii Nakryiko <andrii.nakryiko@gmail.com>
Date:   Fri, 11 Mar 2022 11:11:13 -0800
Message-ID: <CAEf4BzYa1o5FYWGY3tt6O_7LK5h+NmiFzM9QhnKA5n3w-3=p7Q@mail.gmail.com>
Subject: Re: [PATCH bpf-next v2] bpf: Cache the last valid build_id.
To:     Hao Luo <haoluo@google.com>
Cc:     Pasha Tatashin <pasha.tatashin@soleen.com>,
        Alexei Starovoitov <ast@kernel.org>,
        Daniel Borkmann <daniel@iogearbox.net>,
        Andrii Nakryiko <andrii@kernel.org>,
        Song Liu <songliubraving@fb.com>,
        Namhyung Kim <namhyung@kernel.org>,
        Blake Jones <blakejones@google.com>, bpf <bpf@vger.kernel.org>,
        open list <linux-kernel@vger.kernel.org>,
        Greg Thelen <gthelen@google.com>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Thu, Mar 10, 2022 at 5:16 PM Hao Luo <haoluo@google.com> wrote:
>
> On Fri, Feb 25, 2022 at 12:43 PM Pasha Tatashin
> <pasha.tatashin@soleen.com> wrote:
> >
> > On 2/23/22 19:05, Hao Luo wrote:
> > > For binaries that are statically linked, consecutive stack frames are
> > > likely to be in the same VMA and therefore have the same build id.
> > > As an optimization for this case, we can cache the previous frame's
> > > VMA, if the new frame has the same VMA as the previous one, reuse the
> > > previous one's build id. We are holding the MM locks as reader across
> > > the entire loop, so we don't need to worry about VMA going away.
> > >
> > > Tested through "stacktrace_build_id" and "stacktrace_build_id_nmi" in
> > > test_progs.
> > >
> > > Suggested-by: Greg Thelen <gthelen@google.com>
> > > Signed-off-by: Hao Luo <haoluo@google.com>
> >
> > Reviewed-by: Pasha Tatashin <pasha.tatashin@soleen.com>
> >
>
> An update with performance numbers. Thanks to Blake Jones for
> collecting the stats:
>
> In a production workload, with BPF probes sampling stack trace, we see
> the following changes:
>
>  - stack_map_get_build_id_offset() is taking 70% of the time of
> __bpf_get_stackid(); it was 80% before.

Great, thanks for following up with updated numbers!

>
>  - find_get_page() and find_vma() together are taking 75% of the time
> of stack_map_get_build_id_offset(); it was 83% before.
>
> Note the call chain is
>
> __bpf_get_stackid()
>   -> stack_map_get_build_id_offset()
>     -> find_get_page()
>     -> find_vma()
>
> > Thanks,
> > Pasha