2022-06-25 22:55:37

by Yuwen Chen

[permalink] [raw]
Subject: [PATCH] erofs: Wake up all waiters after z_erofs_lzma_head ready.

When the user mounts the erofs second times, the decompression thread
may hung. The problem happens due to a sequence of steps like the
following:

1) Task A called z_erofs_load_lzma_config which obtain all of the node
from the z_erofs_lzma_head.

2) At this time, task B called the z_erofs_lzma_decompress and wanted to
get a node. But the z_erofs_lzma_head was empty, the Task B had to
sleep.

3) Task A release nodes and push nodes into the z_erofs_lzma_head. But
task B was still sleeping.

One example report when the hung happens:
task:kworker/u3:1 state:D stack:14384 pid: 86 ppid: 2 flags:0x00004000
Workqueue: erofs_unzipd z_erofs_decompressqueue_work
Call Trace:
<TASK>
__schedule+0x281/0x760
schedule+0x49/0xb0
z_erofs_lzma_decompress+0x4bc/0x580
? cpu_core_flags+0x10/0x10
z_erofs_decompress_pcluster+0x49b/0xba0
? __update_load_avg_se+0x2b0/0x330
? __update_load_avg_se+0x2b0/0x330
? update_load_avg+0x5f/0x690
? update_load_avg+0x5f/0x690
? set_next_entity+0xbd/0x110
? _raw_spin_unlock+0xd/0x20
z_erofs_decompress_queue.isra.0+0x2e/0x50
z_erofs_decompressqueue_work+0x30/0x60
process_one_work+0x1d3/0x3a0
worker_thread+0x45/0x3a0
? process_one_work+0x3a0/0x3a0
kthread+0xe2/0x110
? kthread_complete_and_exit+0x20/0x20
ret_from_fork+0x22/0x30
</TASK>

Signed-off-by: Yuwen Chen <[email protected]>
---
fs/erofs/decompressor_lzma.c | 1 +
1 file changed, 1 insertion(+)

diff --git a/fs/erofs/decompressor_lzma.c b/fs/erofs/decompressor_lzma.c
index 05a3063cf2bc..5e59b3f523eb 100644
--- a/fs/erofs/decompressor_lzma.c
+++ b/fs/erofs/decompressor_lzma.c
@@ -143,6 +143,7 @@ int z_erofs_load_lzma_config(struct super_block *sb,
DBG_BUGON(z_erofs_lzma_head);
z_erofs_lzma_head = head;
spin_unlock(&z_erofs_lzma_lock);
+ wake_up_all(&z_erofs_lzma_wq);

z_erofs_lzma_max_dictsize = dict_size;
mutex_unlock(&lzma_resize_mutex);
--
2.25.1


2022-06-27 03:34:52

by Gao Xiang

[permalink] [raw]
Subject: Re: [PATCH] erofs: Wake up all waiters after z_erofs_lzma_head ready.

On Sat, Jun 25, 2022 at 10:50:00PM +0800, Yuwen Chen wrote:
> When the user mounts the erofs second times, the decompression thread
> may hung. The problem happens due to a sequence of steps like the
> following:
>
> 1) Task A called z_erofs_load_lzma_config which obtain all of the node
> from the z_erofs_lzma_head.
>
> 2) At this time, task B called the z_erofs_lzma_decompress and wanted to
> get a node. But the z_erofs_lzma_head was empty, the Task B had to
> sleep.
>
> 3) Task A release nodes and push nodes into the z_erofs_lzma_head. But
> task B was still sleeping.
>
> One example report when the hung happens:
> task:kworker/u3:1 state:D stack:14384 pid: 86 ppid: 2 flags:0x00004000
> Workqueue: erofs_unzipd z_erofs_decompressqueue_work
> Call Trace:
> <TASK>
> __schedule+0x281/0x760
> schedule+0x49/0xb0
> z_erofs_lzma_decompress+0x4bc/0x580
> ? cpu_core_flags+0x10/0x10
> z_erofs_decompress_pcluster+0x49b/0xba0
> ? __update_load_avg_se+0x2b0/0x330
> ? __update_load_avg_se+0x2b0/0x330
> ? update_load_avg+0x5f/0x690
> ? update_load_avg+0x5f/0x690
> ? set_next_entity+0xbd/0x110
> ? _raw_spin_unlock+0xd/0x20
> z_erofs_decompress_queue.isra.0+0x2e/0x50
> z_erofs_decompressqueue_work+0x30/0x60
> process_one_work+0x1d3/0x3a0
> worker_thread+0x45/0x3a0
> ? process_one_work+0x3a0/0x3a0
> kthread+0xe2/0x110
> ? kthread_complete_and_exit+0x20/0x20
> ret_from_fork+0x22/0x30
> </TASK>
>
> Signed-off-by: Yuwen Chen <[email protected]>

Reviewed-by: Gao Xiang <[email protected]>

Thanks,
Gao Xiang

2022-06-27 06:02:44

by Yue Hu

[permalink] [raw]
Subject: Re: [PATCH] erofs: Wake up all waiters after z_erofs_lzma_head ready.

On Sat, 25 Jun 2022 22:50:00 +0800
Yuwen Chen <[email protected]> wrote:

> When the user mounts the erofs second times, the decompression thread
> may hung. The problem happens due to a sequence of steps like the
> following:
>
> 1) Task A called z_erofs_load_lzma_config which obtain all of the node
> from the z_erofs_lzma_head.
>
> 2) At this time, task B called the z_erofs_lzma_decompress and wanted to
> get a node. But the z_erofs_lzma_head was empty, the Task B had to
> sleep.
>
> 3) Task A release nodes and push nodes into the z_erofs_lzma_head. But
> task B was still sleeping.
>
> One example report when the hung happens:
> task:kworker/u3:1 state:D stack:14384 pid: 86 ppid: 2 flags:0x00004000
> Workqueue: erofs_unzipd z_erofs_decompressqueue_work
> Call Trace:
> <TASK>
> __schedule+0x281/0x760
> schedule+0x49/0xb0
> z_erofs_lzma_decompress+0x4bc/0x580
> ? cpu_core_flags+0x10/0x10
> z_erofs_decompress_pcluster+0x49b/0xba0
> ? __update_load_avg_se+0x2b0/0x330
> ? __update_load_avg_se+0x2b0/0x330
> ? update_load_avg+0x5f/0x690
> ? update_load_avg+0x5f/0x690
> ? set_next_entity+0xbd/0x110
> ? _raw_spin_unlock+0xd/0x20
> z_erofs_decompress_queue.isra.0+0x2e/0x50
> z_erofs_decompressqueue_work+0x30/0x60
> process_one_work+0x1d3/0x3a0
> worker_thread+0x45/0x3a0
> ? process_one_work+0x3a0/0x3a0
> kthread+0xe2/0x110
> ? kthread_complete_and_exit+0x20/0x20
> ret_from_fork+0x22/0x30
> </TASK>
>
> Signed-off-by: Yuwen Chen <[email protected]>
> ---
> fs/erofs/decompressor_lzma.c | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/fs/erofs/decompressor_lzma.c b/fs/erofs/decompressor_lzma.c
> index 05a3063cf2bc..5e59b3f523eb 100644
> --- a/fs/erofs/decompressor_lzma.c
> +++ b/fs/erofs/decompressor_lzma.c
> @@ -143,6 +143,7 @@ int z_erofs_load_lzma_config(struct super_block *sb,
> DBG_BUGON(z_erofs_lzma_head);
> z_erofs_lzma_head = head;
> spin_unlock(&z_erofs_lzma_lock);
> + wake_up_all(&z_erofs_lzma_wq);
>
> z_erofs_lzma_max_dictsize = dict_size;
> mutex_unlock(&lzma_resize_mutex);

Please do not end the summary line(title) with a period.

2022-06-27 06:27:30

by Gao Xiang

[permalink] [raw]
Subject: Re: [PATCH] erofs: Wake up all waiters after z_erofs_lzma_head ready.

On Mon, Jun 27, 2022 at 01:57:54PM +0800, Yue Hu wrote:
> On Sat, 25 Jun 2022 22:50:00 +0800
> Yuwen Chen <[email protected]> wrote:
>

...

> >
> > Signed-off-by: Yuwen Chen <[email protected]>
> > ---
> > fs/erofs/decompressor_lzma.c | 1 +
> > 1 file changed, 1 insertion(+)
> >
> > diff --git a/fs/erofs/decompressor_lzma.c b/fs/erofs/decompressor_lzma.c
> > index 05a3063cf2bc..5e59b3f523eb 100644
> > --- a/fs/erofs/decompressor_lzma.c
> > +++ b/fs/erofs/decompressor_lzma.c
> > @@ -143,6 +143,7 @@ int z_erofs_load_lzma_config(struct super_block *sb,
> > DBG_BUGON(z_erofs_lzma_head);
> > z_erofs_lzma_head = head;
> > spin_unlock(&z_erofs_lzma_lock);
> > + wake_up_all(&z_erofs_lzma_wq);
> >
> > z_erofs_lzma_max_dictsize = dict_size;
> > mutex_unlock(&lzma_resize_mutex);
>
> Please do not end the summary line(title) with a period.

Okay, that is a good point, also it'd be better to de-capitalize the 'W'
in "Wake" since they are common practices for kernel patches.

If it's convenient to you to send another version, please go with my
r-v-b tag.

Thanks,
Gao Xiang

2022-06-27 06:43:24

by Yuwen Chen

[permalink] [raw]
Subject: [PATCH] erofs: wake up all waiters after z_erofs_lzma_head ready

When the user mounts the erofs second times, the decompression thread
may hung. The problem happens due to a sequence of steps like the
following:

1) Task A called z_erofs_load_lzma_config which obtain all of the node
from the z_erofs_lzma_head.

2) At this time, task B called the z_erofs_lzma_decompress and wanted to
get a node. But the z_erofs_lzma_head was empty, the Task B had to
sleep.

3) Task A release nodes and push nodes into the z_erofs_lzma_head. But
task B was still sleeping.

One example report when the hung happens:
task:kworker/u3:1 state:D stack:14384 pid: 86 ppid: 2 flags:0x00004000
Workqueue: erofs_unzipd z_erofs_decompressqueue_work
Call Trace:
<TASK>
__schedule+0x281/0x760
schedule+0x49/0xb0
z_erofs_lzma_decompress+0x4bc/0x580
? cpu_core_flags+0x10/0x10
z_erofs_decompress_pcluster+0x49b/0xba0
? __update_load_avg_se+0x2b0/0x330
? __update_load_avg_se+0x2b0/0x330
? update_load_avg+0x5f/0x690
? update_load_avg+0x5f/0x690
? set_next_entity+0xbd/0x110
? _raw_spin_unlock+0xd/0x20
z_erofs_decompress_queue.isra.0+0x2e/0x50
z_erofs_decompressqueue_work+0x30/0x60
process_one_work+0x1d3/0x3a0
worker_thread+0x45/0x3a0
? process_one_work+0x3a0/0x3a0
kthread+0xe2/0x110
? kthread_complete_and_exit+0x20/0x20
ret_from_fork+0x22/0x30
</TASK>

Signed-off-by: Yuwen Chen <[email protected]>
Reviewed-by: Gao Xiang <[email protected]>
---
fs/erofs/decompressor_lzma.c | 1 +
1 file changed, 1 insertion(+)

diff --git a/fs/erofs/decompressor_lzma.c b/fs/erofs/decompressor_lzma.c
index 05a3063cf2bc..5e59b3f523eb 100644
--- a/fs/erofs/decompressor_lzma.c
+++ b/fs/erofs/decompressor_lzma.c
@@ -143,6 +143,7 @@ int z_erofs_load_lzma_config(struct super_block *sb,
DBG_BUGON(z_erofs_lzma_head);
z_erofs_lzma_head = head;
spin_unlock(&z_erofs_lzma_lock);
+ wake_up_all(&z_erofs_lzma_wq);

z_erofs_lzma_max_dictsize = dict_size;
mutex_unlock(&lzma_resize_mutex);
--
2.25.1

2022-06-27 07:33:40

by Yuwen Chen

[permalink] [raw]
Subject: Re: [PATCH] erofs: wake up all waiters after z_erofs_lzma_head ready

The new patch has been upload. The modification points are as follower:

1. delete the period.

2. change 'W' to lowercase.

3. add a Reviewed-by tag.

On 6/27/22 06:40, Yuwen Chen wrote:
> When the user mounts the erofs second times, the decompression thread
> may hung. The problem happens due to a sequence of steps like the
> following:
>
> 1) Task A called z_erofs_load_lzma_config which obtain all of the node
> from the z_erofs_lzma_head.
>
> 2) At this time, task B called the z_erofs_lzma_decompress and wanted to
> get a node. But the z_erofs_lzma_head was empty, the Task B had to
> sleep.
>
> 3) Task A release nodes and push nodes into the z_erofs_lzma_head. But
> task B was still sleeping.
>
> One example report when the hung happens:
> task:kworker/u3:1 state:D stack:14384 pid: 86 ppid: 2 flags:0x00004000
> Workqueue: erofs_unzipd z_erofs_decompressqueue_work
> Call Trace:
> <TASK>
> __schedule+0x281/0x760
> schedule+0x49/0xb0
> z_erofs_lzma_decompress+0x4bc/0x580
> ? cpu_core_flags+0x10/0x10
> z_erofs_decompress_pcluster+0x49b/0xba0
> ? __update_load_avg_se+0x2b0/0x330
> ? __update_load_avg_se+0x2b0/0x330
> ? update_load_avg+0x5f/0x690
> ? update_load_avg+0x5f/0x690
> ? set_next_entity+0xbd/0x110
> ? _raw_spin_unlock+0xd/0x20
> z_erofs_decompress_queue.isra.0+0x2e/0x50
> z_erofs_decompressqueue_work+0x30/0x60
> process_one_work+0x1d3/0x3a0
> worker_thread+0x45/0x3a0
> ? process_one_work+0x3a0/0x3a0
> kthread+0xe2/0x110
> ? kthread_complete_and_exit+0x20/0x20
> ret_from_fork+0x22/0x30
> </TASK>
>
> Signed-off-by: Yuwen Chen <[email protected]>
> Reviewed-by: Gao Xiang <[email protected]>
> ---
> fs/erofs/decompressor_lzma.c | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/fs/erofs/decompressor_lzma.c b/fs/erofs/decompressor_lzma.c
> index 05a3063cf2bc..5e59b3f523eb 100644
> --- a/fs/erofs/decompressor_lzma.c
> +++ b/fs/erofs/decompressor_lzma.c
> @@ -143,6 +143,7 @@ int z_erofs_load_lzma_config(struct super_block *sb,
> DBG_BUGON(z_erofs_lzma_head);
> z_erofs_lzma_head = head;
> spin_unlock(&z_erofs_lzma_lock);
> + wake_up_all(&z_erofs_lzma_wq);
>
> z_erofs_lzma_max_dictsize = dict_size;
> mutex_unlock(&lzma_resize_mutex);