Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Thu, 12 Jan 2023 23:36:29 +0900
From:   Masami Hiramatsu (Google) <mhiramat@kernel.org>
To:     Tiezhu Yang <yangtiezhu@loongson.cn>
Cc:     "Naveen N. Rao" <naveen.n.rao@linux.ibm.com>,
        Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com>,
        "David S. Miller" <davem@davemloft.net>,
        linux-trace-kernel@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: kernel hangs when kprobe memcpy
Message-Id: <20230112233629.fafdbbe07dddf364f8078df6@kernel.org>
In-Reply-To: <19666c03-4bf6-7aac-3f1d-cd31ab7de2d5@loongson.cn>
References: <d179086d-78d8-d0e3-e113-9072cffa55f4@loongson.cn>
        <19666c03-4bf6-7aac-3f1d-cd31ab7de2d5@loongson.cn>
Mime-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Precedence: bulk

Hi Tiezhu,

On Thu, 12 Jan 2023 21:32:51 +0800
Tiezhu Yang <yangtiezhu@loongson.cn> wrote:

> 
> 
> On 01/11/2023 07:38 PM, Tiezhu Yang wrote:
> > Hi all,
> >
> > (1) I have the following test environment, kernel hangs when kprobe memcpy:
> >
> > system: x86_64 fedora 36
> > kernel version: Linux 5.7 (compile and update)
> > test case: modprobe kprobe_example symbol="memcpy"
> > (CONFIG_SAMPLE_KPROBES=m)
> >
> > In order to fix build errors, it needs to unset CONFIG_NFP and do the
> > following changes:
> > commit 52a9dab6d892 ("libsubcmd: Fix use-after-free for realloc(..., 0)")
> > commit de979c83574a ("x86/entry: Build thunk_$(BITS) only if
> > CONFIG_PREEMPTION=y")
> >
> > (2) Using the latest upstream mainline kernel, no hang problem due to the
> > commit e3a9e681adb7 ("x86/entry: Fixup bad_iret vs noinstr") to prohibit
> > probing memcpy which is put into the .noinstr.text section.
> >
> >   # modprobe kprobe_example symbol="memcpy"
> >   modprobe: ERROR: could not insert 'kprobe_example': Invalid argument
> >
> > In my opinion, according to the commit message, the above commit is not
> > intended to fix the memcpy hang problem, the problem was fixed by accident.
> >
> > (3) If make handler_pre() and handler_post() as empty functions in the 5.7
> > kernel code, the above hang problem does not exist.


> >
> > diff --git a/samples/kprobes/kprobe_example.c
> > b/samples/kprobes/kprobe_example.c
> > index fd346f58ddba..c194171d8a46 100644
> > --- a/samples/kprobes/kprobe_example.c
> > +++ b/samples/kprobes/kprobe_example.c
> > @@ -28,8 +28,6 @@ static struct kprobe kp = {
> >  static int __kprobes handler_pre(struct kprobe *p, struct pt_regs *regs)
> >  {
> >  #ifdef CONFIG_X86
> > -    pr_info("<%s> p->addr = 0x%p, ip = %lx, flags = 0x%lx\n",
> > -        p->symbol_name, p->addr, regs->ip, regs->flags);
> >  #endif
> >  #ifdef CONFIG_PPC
> >      pr_info("<%s> p->addr = 0x%p, nip = 0x%lx, msr = 0x%lx\n",
> > @@ -65,8 +63,6 @@ static void __kprobes handler_post(struct kprobe *p,
> > struct pt_regs *regs,
> >                  unsigned long flags)
> >  {
> >  #ifdef CONFIG_X86
> > -    pr_info("<%s> p->addr = 0x%p, flags = 0x%lx\n",
> > -        p->symbol_name, p->addr, regs->flags);
> >  #endif
> >  #ifdef CONFIG_PPC
> >      pr_info("<%s> p->addr = 0x%p, msr = 0x%lx\n",
> >
> > I want to know what is the real reason of the hang problem when kprobe
> > memcpy,
> > I guess it may be kprobe recursion, what do you think? Thank you.
> >
> > By the way, kprobe memset has no problem whether or not handler_pre() and
> > handler_post() are empty functions.
> >
> > Thanks,
> > Tiezhu
> 
> I find out the following call trace:
> 
> handler_pre()
>    pr_info()
>      printk()
>        _printk()
>          vprintk()
>            vprintk_store()
>              memcpy()
> 
> I think it may cause recursive exceptions, so we should
> mark memcpy as non-kprobe-able, right?

Yes, and the .noinstr.text (noinstr function attribute) is including
non-kprobe-able (nokprobe function attribute). I a function is nokprobe
and notrace, it should be noinstr. "NOKPROBE_SYMBOL" is used for the
symbol which is called in the kprobe processing path (e.g. x86 int3
handler etc.).

BTW, that the bug you reported is interesting. Even if another kprobe
is called inside kprobe pre/post handler, it must be skipped.
If you can share your kconfig, I can try to reproduce it.

Thank you,

-- 
Masami Hiramatsu (Google) <mhiramat@kernel.org>