Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Subject: Re: [PATCH 6/6] LoongArch: Add generic ex-handler unwind in prologue
 unwinder
To:     Qing Zhang <zhangqing@loongson.cn>,
        Huacai Chen <chenhuacai@kernel.org>,
        WANG Xuerui <kernel@xen0n.name>
Cc:     loongarch@lists.linux.dev, linux-kernel@vger.kernel.org,
        Steven Rostedt <rostedt@goodmis.org>,
        Masami Hiramatsu <mhiramat@kernel.org>,
        Mark Rutland <mark.rutland@arm.com>
References: <20221215040141.18610-1-hejinyang@loongson.cn>
 <20221215040141.18610-7-hejinyang@loongson.cn>
 <2aa963e1-e202-73b8-725f-3ca207d622a7@loongson.cn>
From:   Jinyang He <hejinyang@loongson.cn>
Message-ID: <2bba3021-b4d8-f23e-c924-3e951ce3f768@loongson.cn>
Date:   Fri, 16 Dec 2022 09:44:09 +0800
User-Agent: Mozilla/5.0 (X11; Linux loongarch64; rv:68.0) Gecko/20100101
 Thunderbird/68.7.0
MIME-Version: 1.0
In-Reply-To: <2aa963e1-e202-73b8-725f-3ca207d622a7@loongson.cn>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US
Precedence: bulk

On 2022-12-15 20:04, Qing Zhang wrote:

> Hi, Jinyang
>
> On 2022/12/15 下午12:01, Jinyang He wrote:
>> When exception is triggered, code flow go handle_\exception in some
>> cases. One of stackframe in this case as follows,
>>
>> high -> +-------+
>>          | REGS  |  <- a pt_regs
>>          |       |
>>          |       |  <- ex trigger
>>          | REGS  |  <- ex pt_regs   <-+
>>          |       |                    |
>>          |       |                    |
>> low  -> +-------+           ->unwind-+
>>
>> When unwinder unwind to handler_\exception it cannot go on prologue
>> analysis. It is asynchronous code flow, we should get the next frame
>> PC from regs->csr_era but not from regs->regs[1]. And we copy the
>> handler codes to eentry in the early time and copy the handler codes
>> to NUMA-relative memory named pcpu_handlers if NUMA is enabled. Thus,
>> unwinder cannot unwind normally. Therefore, try to give some hint in
>> handler_\exception and fixup it in unwind_next_frame.
>>
>> Reported-by: Qing Zhang <zhangqing@loongson.cn>
>> Signed-off-by: Jinyang He <hejinyang@loongson.cn>
>> ---
>>   arch/loongarch/include/asm/unwind.h     |   2 +-
>>   arch/loongarch/kernel/genex.S           |   3 +
>>   arch/loongarch/kernel/unwind_prologue.c | 100 +++++++++++++++++++++---
>>   arch/loongarch/mm/tlb.c                 |   2 +-
>>   4 files changed, 92 insertions(+), 15 deletions(-)
>>
> The others look good to me, but there is still a small problem:
> When I tested hw_breakpoint.ko with prologue unwinder,
> sometimes output address [<9000000100302724>] 0x9000000100302724, eg: 
> CPU: 3 PID: 0
> But some processes are normal, eg: CPU: 0 PID: 0
>
> [27.655549] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 6.1.0-rc8 #9
> [27.655552] Hardware name: Loongson Loongson-3A5000-7A1000-1w-A2101/
> Loongson-LS3A5000-7A1000-1w-A2101,  BIOS 
> vUDK2018-LoongArch-V2.0.pre-beta8 06/15/2022
>
> [27.655604]...
> [27.655606] Call Trace:
> [27.655607] [<9000000000222f88>] show_stack+0x60/0x184
> [27.655613] [<90000000010e9b8c>] dump_stack_lvl+0x60/0x88
> [27.655618] [] sample_hbp_handler+0x30/0x4c [data_breakpoint]
> [27.655626] [<900000000037c8a0>] __perf_event_overflow+0x84/0x26c
> [27.655629] [<900000000038980c>] perf_bp_event+0xc0/0xc8
> [27.655633] [<900000000022e3bc>] watchpoint_handler+0x54/0x88
> [27.655637] [<90000000010ea2f8>] do_watch+0x30/0x48
> [27.655640] [<9000000100302724>] 0x9000000100302724      // Not natural
> [27.655642] [<9000000000ab4cbc>] add_interrupt_randomness+0x60/0xbc
> [27.655646] [<90000000002a0fa0>] handle_irq_event_percpu+0x28/0x70
> [27.655650] [<90000000002a6f9c>] handle_percpu_irq+0x54/0x88
> [27.655652] [<90000000002a025c>] generic_handle_domain_irq+0x28/0x40
> [27.655655] [<9000000000995458>] handle_cpu_irq+0x68/0xa4
> [27.655658] [<90000000010ea8dc>] handle_loongarch_irq+0x34/0x4c
> [27.655661] [<90000000010ea974>] do_vint+0x80/0xb4
> [27.655664] [<90000000002216a0>] __arch_cpu_idle+0x20/0x24
> [27.655667] [<90000000010f8178>] default_idle_call+0x30/0x58
> [27.655670] [<90000000002825cc>] do_idle+0xb4/0x118
> [27.655674] [<900000000028281c>] cpu_startup_entry+0x20/0x24
> [27.655677] [<900000000022b198>] start_secondary+0x9c/0xa4
> [27.655679] [<90000000010eb124>] smpboot_entry+0x60/0x64
>
>
> [27.658940] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.1.0-rc8 #9
> ...
> [28.229978] Call Trace:
> [28.229979] [<9000000000222f88>] show_stack+0x60/0x184
> [28.237503] [<90000000010e9b8c>] dump_stack_lvl+0x60/0x88
> [28.242866] [] sample_hbp_handler+0x30/0x4c [data_breakpoint]
> [28.250132] [<900000000037c8a0>] __perf_event_overflow+0x84/0x26c
> [28.256186] [<900000000038980c>] perf_bp_event+0xc0/0xc8
> [28.261462] [<900000000022e3bc>] watchpoint_handler+0x54/0x88
> [28.267170] [<90000000010ea2f8>] do_watch+0x30/0x48
> [28.272013] [<90000000017d2724>] exception_handlers+0x2724/0x1000  //...

There is not in kernel text section but in kernel bss section. Because
the boot cpu set csr.eentry to eentry and set others cpus set csr.eentry
to pcpu_handlers[cpu]. All of these eentry are not in orginal position.
So we cannot find its real symbol. But I still give a chance to go on 
and record
PC value when unwind_state_fixup return true in unwind_by_prologue().


Thanks,

Jinyang


> [28.278155] [<9000000000ab4cbc>] add_interrupt_randomness+0x60/0xbc
> [28.284381] [<90000000002a0fa0>] handle_irq_event_percpu+0x28/0x70
> [28.290520] [<90000000002a6f9c>] handle_percpu_irq+0x54/0x88
> [28.296140] [<90000000002a025c>] generic_handle_domain_irq+0x28/0x40
> [28.302452] [<9000000000995458>] handle_cpu_irq+0x68/0xa4
> [28.307813] [<90000000010ea8dc>] handle_loongarch_irq+0x34/0x4c
> [28.313693] [<90000000010ea974>] do_vint+0x80/0xb4
> [28.318450] [<90000000002216a0>] __arch_cpu_idle+0x20/0x24
> [28.323897] [<90000000010f8178>] default_idle_call+0x30/0x58
> [28.329518] [<90000000002825cc>] do_idle+0xb4/0x118
> [28.334361] [<900000000028281c>] cpu_startup_entry+0x20/0x24
> [28.339982] [<90000000010ec0dc>] kernel_init+0x0/0x110
> [28.345085] [<90000000011106f8>] arch_post_acpi_subsys_init+0x0/0x4
>
> Maybe sometimes assembly kallsyms is not recognized, let me think...
>
> Thanks,
> -Qing
>