Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20220728020250.1699-1-iecedge@gmail.com> <20221107144931.GA20793@willie-the-truck>
 <Y2klCLj7F7fKsza+@FVFF77S0Q05N> <20221107153506.GA21157@willie-the-truck>
 <Y2ktAisfFAr0aU2V@FVFF77S0Q05N> <20221108141009.d39d1d02f07e8993c1f37349@kernel.org>
In-Reply-To: <20221108141009.d39d1d02f07e8993c1f37349@kernel.org>
From:   Jianlin Lv <iecedge@gmail.com>
Date:   Tue, 8 Nov 2022 13:53:13 +0800
Message-ID: <CAFA-uR_B2iRaNj7Da2tZOu7JHUhrm=wZASia7nOfP-xJipzfiQ@mail.gmail.com>
Subject: Re: [PATCH] arm64/kprobes: Add support for KPROBES_ON_FTRACE
To:     Masami Hiramatsu <mhiramat@kernel.org>
Cc:     Mark Rutland <mark.rutland@arm.com>, Will Deacon <will@kernel.org>,
        corbet@lwn.net, catalin.marinas@arm.com, rostedt@goodmis.org,
        mingo@redhat.com, naveen.n.rao@linux.ibm.com,
        anil.s.keshavamurthy@intel.com, davem@davemloft.net, arnd@arndb.de,
        zhengzengkai@huawei.com, jianlv@ebay.com,
        linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org,
        linux-arm-kernel@lists.infradead.org
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Tue, Nov 8, 2022 at 1:10 PM Masami Hiramatsu <mhiramat@kernel.org> wrote:
>
> On Mon, 7 Nov 2022 16:06:26 +0000
> Mark Rutland <mark.rutland@arm.com> wrote:
>
> > On Mon, Nov 07, 2022 at 03:35:07PM +0000, Will Deacon wrote:
> > > On Mon, Nov 07, 2022 at 03:32:24PM +0000, Mark Rutland wrote:
> > > > On Mon, Nov 07, 2022 at 02:49:31PM +0000, Will Deacon wrote:
> > > > > [+Mark R]
> > > > >
> > > > > On Thu, Jul 28, 2022 at 02:02:50AM +0000, Jianlin Lv wrote:
> > > > > > This is the arm64 version of ftrace-based kprobes to avoid the overhead
> > > > > > with regular kprobes, by using the ftrace infrastructure.
> > > > > >
> > > > > > Signed-off-by: Jianlin Lv <iecedge@gmail.com>
> > > > > > ---
> > > > > >  .../debug/kprobes-on-ftrace/arch-support.txt  |  2 +-
> > > > > >  arch/arm64/Kconfig                            |  1 +
> > > > > >  arch/arm64/kernel/probes/Makefile             |  1 +
> > > > > >  arch/arm64/kernel/probes/kprobes-ftrace.c     | 81 +++++++++++++++++++
> > > > > >  include/linux/kprobes.h                       |  2 +
> > > > > >  kernel/kprobes.c                              |  4 +-
> > > > > >  6 files changed, 88 insertions(+), 3 deletions(-)
> > > > > >  create mode 100644 arch/arm64/kernel/probes/kprobes-ftrace.c
> > > > >
> > > > > Sorry for the slow reply on this, but I think this deserved to be split
> > > > > into two patches: the first one reworking the core check_ftrace_location()
> > > > > logic to work properly with branch-and-link style architectures, and the
> > > > > second one adding support for arm64.
> > > >
> > > > I'd prefer we don't do this at all; there a bunch of issues with kprobes *not*
> > > > taking an exception, since we get a dodgy not-quite-real pt_regs, and to clean
> > > > up the existing issues the plan is:
> > > >
> > > > 1) Move ftrace over to ftrace_regs
> > > > 2) Implement fprobes using ftrace_regs
> > > > 3) Remove kretprobes
>
> Yes, that is what we agreed at the tracing summit.

Sorry to miss TS2022.
Are there documents that can be shared to document the conclusions
reached at the tracing summit?
This will be helpful to understand what changes may be coming to tracing.

>
> When we finished to move on the fprobe and fprobe-events, kretprobes
> (and kprobe-on-ftrace if CONFIG_HAVE_DYNAMIC_FTRACE_WITH_REGS is not
> supported) are not needed from the user viewpoint.
> So we can mark the kretprobe API obsolete.
>
> > > >
> > > > ... and regular kprobes will need to take an exception (via BRK) to get a real
> > > > pt_regs, so that can't be optimized to use ftrace.
> > >
> > > OKey doke. Does that mean that other architectures will follow the same
> > > approach of taking an exception,
> >
> > I think once everyone has FPROBE, KPROBES_ON_FTRACE becomes redundant, and
> > could be removed (leaving kprobes to always follow a take-an-exception flow on
> > all architectures).
>
> Anyway I will give some window to transit to the fprobe with ftrace_regs.
>
> >
> > > or do they somehow work by magic?
> >
> > Some architectures don't need to take an exception to be able to create a full
> > pt_regs (e.g. x86's flags are accessible in a way arm64's PSTATE isn't), but
> > that needs to be generated / restored differently to exception entry/return,
> > and so even where it's possible it can be painful to maintain (and slower than
> > using ftrace_regs), so I suspect KPROBES_ON_FTRACE would be removed.
>
> I think KPROBES_ON_FTRACE support depends on CONFIG_HAVE_DYNAMIC_FTRACE_WITH_REGS.
> When the all architecture removed it, I remove it. But it means that if function
> tracer is enabled, we can not put any kprobes on the entry of functions on x86
> because there is no space to put a software breakpoint at the function entry
> on x86.
>
> For the ftrace (tracefs) user, I will minimize the effect of this change, but
> the tools developers (including eBPF developer) this change may be critical.
>
> Thank you,
>
>
> >
> > So different constaints more than magic.
> >
> > Thanks,
> > Mark.
>
>
> --
> Masami Hiramatsu (Google) <mhiramat@kernel.org>