2021-12-29 08:55:24

by yaowenbin

[permalink] [raw]
Subject: [PATCH] perf top: add concurrent access protection of the SLsmg screen management

When the following command is executed several times, a coredump file is generated.
$ timeout -k 9 5 perf top -e task-clock
*******
*******
*******
0.01% [kernel] [k] __do_softirq
0.01% libpthread-2.28.so [.] __pthread_mutex_lock
0.01% [kernel] [k] __ll_sc_atomic64_sub_return
double free or corruption (!prev) perf top --sort comm,dso
timeout: the monitored command dumped core

When we terminate "perf top" using sending signal method, SLsmg_reset_smg
function is called. SLsmg_reset_smg resets the SLsmg screen management routines
by freeing all memory allocated while it was active. However SLsmg_reinit_smg
function maybe be called by another thread. SLsmg_reinit_smg function will
free the same memory accessed by SLsmg_reset_smg functon, thus it results
in double free. SLsmg_reinit_smg function is called already protected by
ui__lock, so we fix the problem by adding pthread_mutex_trylock of ui__lock
when calling SLsmg_reset_smg function.

Signed-off-by: yaowenbin <[email protected]>
Signed-off-by: hewenliang <[email protected]>
Signed-off-by: Wenyu Liu <[email protected]>
---
tools/perf/ui/tui/setup.c | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/tools/perf/ui/tui/setup.c b/tools/perf/ui/tui/setup.c
index d4ac41679..1fdf92062 100644
--- a/tools/perf/ui/tui/setup.c
+++ b/tools/perf/ui/tui/setup.c
@@ -170,9 +170,11 @@ void ui__exit(bool wait_for_ok)
"Press any key...", 0);

SLtt_set_cursor_visibility(1);
- SLsmg_refresh();
- SLsmg_reset_smg();
+ if (!pthread_mutex_trylock(&ui__lock)) {
+ SLsmg_refresh();
+ SLsmg_reset_smg();
+ pthread_mutex_unlock(&ui__lock);
+ }
SLang_reset_tty();
-
perf_error__unregister(&perf_tui_eops);
}
--
2.27.0


2021-12-31 14:24:26

by Jiri Olsa

[permalink] [raw]
Subject: Re: [PATCH] perf top: add concurrent access protection of the SLsmg screen management

On Wed, Dec 29, 2021 at 04:55:19PM +0800, yaowenbin wrote:
> When the following command is executed several times, a coredump file is generated.
> $ timeout -k 9 5 perf top -e task-clock
> *******
> *******
> *******
> 0.01% [kernel] [k] __do_softirq
> 0.01% libpthread-2.28.so [.] __pthread_mutex_lock
> 0.01% [kernel] [k] __ll_sc_atomic64_sub_return
> double free or corruption (!prev) perf top --sort comm,dso
> timeout: the monitored command dumped core
>
> When we terminate "perf top" using sending signal method, SLsmg_reset_smg
> function is called. SLsmg_reset_smg resets the SLsmg screen management routines
> by freeing all memory allocated while it was active. However SLsmg_reinit_smg
> function maybe be called by another thread. SLsmg_reinit_smg function will
> free the same memory accessed by SLsmg_reset_smg functon, thus it results
> in double free. SLsmg_reinit_smg function is called already protected by
> ui__lock, so we fix the problem by adding pthread_mutex_trylock of ui__lock
> when calling SLsmg_reset_smg function.
>
> Signed-off-by: yaowenbin <[email protected]>
> Signed-off-by: hewenliang <[email protected]>
> Signed-off-by: Wenyu Liu <[email protected]>
> ---
> tools/perf/ui/tui/setup.c | 8 +++++---
> 1 file changed, 5 insertions(+), 3 deletions(-)
>
> diff --git a/tools/perf/ui/tui/setup.c b/tools/perf/ui/tui/setup.c
> index d4ac41679..1fdf92062 100644
> --- a/tools/perf/ui/tui/setup.c
> +++ b/tools/perf/ui/tui/setup.c
> @@ -170,9 +170,11 @@ void ui__exit(bool wait_for_ok)
> "Press any key...", 0);
>
> SLtt_set_cursor_visibility(1);
> - SLsmg_refresh();
> - SLsmg_reset_smg();
> + if (!pthread_mutex_trylock(&ui__lock)) {

trylock because it can be called from signal, right?
could you plase add some comment about that

thanks,
jirka

> + SLsmg_refresh();
> + SLsmg_reset_smg();
> + pthread_mutex_unlock(&ui__lock);
> + }
> SLang_reset_tty();
> -
> perf_error__unregister(&perf_tui_eops);
> }
> --
> 2.27.0
>


2022-01-02 14:48:00

by Arnaldo Carvalho de Melo

[permalink] [raw]
Subject: Re: [PATCH] perf top: add concurrent access protection of the SLsmg screen management

Em Wed, Dec 29, 2021 at 04:55:19PM +0800, yaowenbin escreveu:
> When the following command is executed several times, a coredump file is generated.
> $ timeout -k 9 5 perf top -e task-clock
> *******
> *******
> *******
> 0.01% [kernel] [k] __do_softirq
> 0.01% libpthread-2.28.so [.] __pthread_mutex_lock
> 0.01% [kernel] [k] __ll_sc_atomic64_sub_return
> double free or corruption (!prev) perf top --sort comm,dso
> timeout: the monitored command dumped core

Thanks, tested, reproduced the problem without the patch, not managed to
reproduce in 50 calls to the reproducer after applying the patch.

- Arnaldo

> When we terminate "perf top" using sending signal method, SLsmg_reset_smg
> function is called. SLsmg_reset_smg resets the SLsmg screen management routines
> by freeing all memory allocated while it was active. However SLsmg_reinit_smg
> function maybe be called by another thread. SLsmg_reinit_smg function will
> free the same memory accessed by SLsmg_reset_smg functon, thus it results
> in double free. SLsmg_reinit_smg function is called already protected by
> ui__lock, so we fix the problem by adding pthread_mutex_trylock of ui__lock
> when calling SLsmg_reset_smg function.
>
> Signed-off-by: yaowenbin <[email protected]>
> Signed-off-by: hewenliang <[email protected]>
> Signed-off-by: Wenyu Liu <[email protected]>
> ---
> tools/perf/ui/tui/setup.c | 8 +++++---
> 1 file changed, 5 insertions(+), 3 deletions(-)
>
> diff --git a/tools/perf/ui/tui/setup.c b/tools/perf/ui/tui/setup.c
> index d4ac41679..1fdf92062 100644
> --- a/tools/perf/ui/tui/setup.c
> +++ b/tools/perf/ui/tui/setup.c
> @@ -170,9 +170,11 @@ void ui__exit(bool wait_for_ok)
> "Press any key...", 0);
>
> SLtt_set_cursor_visibility(1);
> - SLsmg_refresh();
> - SLsmg_reset_smg();
> + if (!pthread_mutex_trylock(&ui__lock)) {
> + SLsmg_refresh();
> + SLsmg_reset_smg();
> + pthread_mutex_unlock(&ui__lock);
> + }
> SLang_reset_tty();
> -
> perf_error__unregister(&perf_tui_eops);
> }
> --
> 2.27.0

--

- Arnaldo

2022-01-04 07:21:53

by yaowenbin

[permalink] [raw]
Subject: Re: [PATCH] perf top: add concurrent access protection of the SLsmg screen management

In ui__refresh_dimensions function, ui__lock will also be acquired, so we use trylock
in the signal handler ui__exit to avoid deadlock.
Thanks.

在 2021/12/31 22:10, Jiri Olsa 写道:
> On Wed, Dec 29, 2021 at 04:55:19PM +0800, yaowenbin wrote:
>> When the following command is executed several times, a coredump file is generated.
>> $ timeout -k 9 5 perf top -e task-clock
>> *******
>> *******
>> *******
>> 0.01% [kernel] [k] __do_softirq
>> 0.01% libpthread-2.28.so [.] __pthread_mutex_lock
>> 0.01% [kernel] [k] __ll_sc_atomic64_sub_return
>> double free or corruption (!prev) perf top --sort comm,dso
>> timeout: the monitored command dumped core
>>
>> When we terminate "perf top" using sending signal method, SLsmg_reset_smg
>> function is called. SLsmg_reset_smg resets the SLsmg screen management routines
>> by freeing all memory allocated while it was active. However SLsmg_reinit_smg
>> function maybe be called by another thread. SLsmg_reinit_smg function will
>> free the same memory accessed by SLsmg_reset_smg functon, thus it results
>> in double free. SLsmg_reinit_smg function is called already protected by
>> ui__lock, so we fix the problem by adding pthread_mutex_trylock of ui__lock
>> when calling SLsmg_reset_smg function.
>>
>> Signed-off-by: yaowenbin <[email protected]>
>> Signed-off-by: hewenliang <[email protected]>
>> Signed-off-by: Wenyu Liu <[email protected]>
>> ---
>> tools/perf/ui/tui/setup.c | 8 +++++---
>> 1 file changed, 5 insertions(+), 3 deletions(-)
>>
>> diff --git a/tools/perf/ui/tui/setup.c b/tools/perf/ui/tui/setup.c
>> index d4ac41679..1fdf92062 100644
>> --- a/tools/perf/ui/tui/setup.c
>> +++ b/tools/perf/ui/tui/setup.c
>> @@ -170,9 +170,11 @@ void ui__exit(bool wait_for_ok)
>> "Press any key...", 0);
>>
>> SLtt_set_cursor_visibility(1);
>> - SLsmg_refresh();
>> - SLsmg_reset_smg();
>> + if (!pthread_mutex_trylock(&ui__lock)) {
>
> trylock because it can be called from signal, right?
> could you plase add some comment about that
>
> thanks,
> jirka
>
>> + SLsmg_refresh();
>> + SLsmg_reset_smg();
>> + pthread_mutex_unlock(&ui__lock);
>> + }
>> SLang_reset_tty();
>> -
>> perf_error__unregister(&perf_tui_eops);
>> }
>> --
>> 2.27.0
>>
>
> .