Subject: File system hung on kernel 4.14

Lưu ý: Đề nghị không tải email hoặc file đính kèm trực tiếp lên các nền tảng phân tích mã độc bên thứ ba như VirusTotal, Anyrun... nhằm tránh nguy cơ làm lộ thông tin trong email.
Notice: Please do not upload this email or attachments directly to third-party threat analysis platform such as VirusTotal, Anyrun... to avoid the risk of leaking email content.



Dear Linux kernel ext4, Dear Ted,

We are developing project on Android 9 which used kernel 4.14.133
version.
And while normal using device, we have a issue device freezing and
reboot after few minute.
I checked the log and saw kernel hung on filesystem as below:

10-02 20:34:17.586 2461 2461 W Kernel : [15159.873157] Call trace:
10-02 20:34:17.586 2461 2461 W Kernel : [15159.873177]
[<000000005e1d80e5>] __switch_to+0x94/0xd8
10-02 20:34:17.586 2461 2461 W Kernel : [15159.873186]
[<00000000a85c7184>] __schedule+0x274/0x940
10-02 20:34:17.586 2461 2461 W Kernel : [15159.873190]
[<00000000c9c2c8f0>] schedule+0x40/0xa8
10-02 20:34:17.586 2461 2461 W Kernel : [15159.873197]
[<00000000a97520e1>] io_schedule+0x20/0x40
10-02 20:34:17.586 2461 2461 W Kernel : [15159.873206]
[<000000001d0a54c7>] wait_on_page_bit+0x144/0x218
10-02 20:34:17.586 2461 2461 W Kernel : [15159.873210]
[<00000000a2541efa>] __filemap_fdatawait_range+0xd4/0x150
10-02 20:34:17.586 2461 2461 W Kernel : [15159.873213]
[<0000000035aea854>] filemap_fdatawait_keep_errors+0x28/0x58
10-02 20:34:17.586 2461 2461 W Kernel : [15159.873223]
[<0000000084ab312f>] jbd2_journal_commit_transaction+0x6bc/0x1838
10-02 20:34:17.586 2461 2461 W Kernel : [15159.873228]
[<000000007e52cd07>] kjournald2+0xd8/0x268
10-02 20:34:17.586 2461 2461 W Kernel : [15159.873234]
[<00000000f46ec2f2>] kthread+0x138/0x140
10-02 20:34:17.586 2461 2461 W Kernel : [15159.873238]
[<000000009e914ea4>] ret_from_fork+0x10/0x1c
10-02 20:34:17.586 2461 2461 I Kernel : [15159.873274]
HwBinder:2156_3 D 0 4116 1 0x00000000
10-02 20:34:17.586 2461 2461 W Kernel : [15159.873279] Call trace:
10-02 20:34:17.586 2461 2461 W Kernel : [15159.873282]
[<000000005e1d80e5>] __switch_to+0x94/0xd8
10-02 20:34:17.586 2461 2461 W Kernel : [15159.873286]
[<00000000a85c7184>] __schedule+0x274/0x940
10-02 20:34:17.586 2461 2461 W Kernel : [15159.873288]
[<00000000c9c2c8f0>] schedule+0x40/0xa8
10-02 20:34:17.587 2461 2461 W Kernel : [15159.873292]
[<00000000228469bd>] schedule_timeout+0x204/0x430
10-02 20:34:17.587 2461 2461 W Kernel : [15159.873295]
[<000000006c4e1588>] wait_for_common+0xbc/0x178
10-02 20:34:17.587 2461 2461 W Kernel : [15159.873298]
[<0000000085ed017c>] wait_for_completion+0x28/0x38
10-02 20:34:17.587 2461 2461 W Kernel : [15159.873303]
[<00000000ce130c93>] flush_work+0x134/0x220
10-02 20:34:17.587 2461 2461 W Kernel : [15159.873309]
[<00000000cd75762d>] drm_mode_rmfb+0x130/0x1a8
10-02 20:34:17.587 2461 2461 W Kernel : [15159.873317]
[<000000002c2f9c43>] drm_ioctl_kernel+0x70/0xd0
10-02 20:34:17.587 2461 2461 W Kernel : [15159.873320]
[<000000005902551e>] drm_ioctl+0x1e4/0x450
10-02 20:34:17.587 2461 2461 W Kernel : [15159.873327]
[<00000000d4238d57>] do_vfs_ioctl+0xc4/0xb40
10-02 20:34:17.587 2461 2461 W Kernel : [15159.873330]
[<0000000003b80d06>] SyS_ioctl+0x8c/0xa8
10-02 20:34:17.587 2461 2461 W Kernel : [15159.873334] Exception
stack(0xffff00000fc13ec0 to 0xffff00000fc14000)
10-02 20:34:17.587 2461 2461 W Kernel : [15159.873340] 3ec0:
0000000000000003 00000000c00464af 0000fe8cb41ee684 0000000000000001
10-02 20:34:17.587 2461 2461 W Kernel : [15159.873344] 3ee0:
000000000000002c 0000fe8cb4c630c0 000000090000001e 0000000007800000
10-02 20:34:17.587 2461 2461 W Kernel : [15159.873348] 3f00:
000000000000001d 0000fe8cb41ee5f8 0000fe8cb41ee5f8 0000fe8cb41ee640
10-02 20:34:17.587 2461 2461 W Kernel : [15159.873352] 3f20:
000000000000000c 0000000000000001 0000000000000000 fffffffffffffff0
10-02 20:34:17.587 2461 2461 W Kernel : [15159.873356] 3f40:
0000fe8cb5c6fd90 0000fe8cb51e90e0 0000fe8cb4c630c0 0000fe8cb41ef588
10-02 20:34:17.587 2461 2461 W Kernel : [15159.873360] 3f60:
00000000c00464af 0000000000000003 0000fe8cb4446960 0000fe8cb41ee89c
10-02 20:34:17.587 2461 2461 W Kernel : [15159.873364] 3f80:
0000c04ee2bd66c5 0000c04ee2bd74f3 00000000000000ff 00000000ffffffff
10-02 20:34:17.587 2461 2461 W Kernel : [15159.873368] 3fa0:
0000fe8cb4c2b540 0000fe8cb41ee630 0000fe8cb51e916c 0000fe8cb41ee540
10-02 20:34:17.587 2461 2461 W Kernel : [15159.873372] 3fc0:
0000fe8cb522f758 00000000a0000000 0000000000000003 000000000000001d
10-02 20:34:17.587 2461 2461 W Kernel : [15159.873376] 3fe0:
0000000000000000 0000000000000000 0000000000000000 0000000000000000
10-02 20:34:17.587 2461 2461 W Kernel : [15159.873381]
[<000000008094bac4>] el0_svc_naked+0x34/0x38
10-02 20:34:17.588 2461 2461 I Kernel : [15159.873452]
PackageManager D 0 2657 2121 0x00000009
10-02 20:34:17.588 2461 2461 W Kernel : [15159.873457] Call trace:
10-02 20:34:17.588 2461 2461 W Kernel : [15159.873461]
[<000000005e1d80e5>] __switch_to+0x94/0xd8
10-02 20:34:17.588 2461 2461 W Kernel : [15159.873464]
[<00000000a85c7184>] __schedule+0x274/0x940
10-02 20:34:17.588 2461 2461 W Kernel : [15159.873467]
[<00000000c9c2c8f0>] schedule+0x40/0xa8
10-02 20:34:17.588 2461 2461 W Kernel : [15159.873472]
[<00000000d3740507>] jbd2_log_wait_commit+0xc0/0x158
10-02 20:34:17.588 2461 2461 W Kernel : [15159.873477]
[<0000000036159c5c>] jbd2_complete_transaction+0x90/0xc0
10-02 20:34:17.588 2461 2461 W Kernel : [15159.873483]
[<0000000019ee473a>] ext4_sync_file+0x440/0x488
10-02 20:34:17.588 2461 2461 W Kernel : [15159.873490]
[<00000000665d0f69>] vfs_fsync_range+0x64/0xc0
10-02 20:34:17.588 2461 2461 W Kernel : [15159.873493]
[<0000000035c0b348>] do_fsync+0x48/0x88
10-02 20:34:17.588 2461 2461 W Kernel : [15159.873496]
[<000000006f18433e>] SyS_fsync+0x24/0x38
10-02 20:34:17.588 2461 2461 W Kernel : [15159.873499] Exception
stack(0xffff00000b01bec0 to 0xffff00000b01c000)
10-02 20:34:17.588 2461 2461 W Kernel : [15159.873503] bec0:
00000000000000fb 0000e6a41540bb64 000000006f4c7d34 0000000000000000
10-02 20:34:17.588 2461 2461 W Kernel : [15159.873507] bee0:
00000000137c08e4 000000000000216e 0000216e0000216e 000000000000216e
10-02 20:34:17.588 2461 2461 W Kernel : [15159.873511] bf00:
0000000000000052 d4d524bf9be44442 0000000000430000 0000e6a42f7e0688
10-02 20:34:17.588 2461 2461 W Kernel : [15159.873515] bf20:
0000000077ef52e8 0000000000000000 000000006f71f9d8 0000000000003f0d
10-02 20:34:17.588 2461 2461 W Kernel : [15159.873519] bf40:
0000e6a429b23e30 0000e6a4b1eb2c00 0000000077ef52e8 00000000000000fb
10-02 20:34:17.588 2461 2461 W Kernel : [15159.873523] bf60:
00000000ffffffff 00000000137c08d0 0000000000e60ce4 0000000000e60ce4
10-02 20:34:17.588 2461 2461 W Kernel : [15159.873527] bf80:
00000000137c01e8 00000000137c03a8 00000000137c0850 00000000137c08e0
10-02 20:34:17.588 2461 2461 W Kernel : [15159.873531] bfa0:
00000000137c2908 0000e6a41540bb20 0000e6a429b076b4 0000e6a41540bb10
10-02 20:34:17.588 2461 2461 W Kernel : [15159.873535] bfc0:
0000e6a4b1eb2c08 0000000060000000 00000000000000fb 0000000000000052
10-02 20:34:17.588 2461 2461 W Kernel : [15159.873539] bfe0:
0000000000000000 0000000000000000 0000000000000000 0000000000000000
10-02 20:34:17.589 2461 2461 W Kernel : [15159.873542]
[<000000008094bac4>] el0_svc_naked+0x34/0x38
10-02 20:34:17.589 2461 2461 I Kernel : [15159.873869] Thread-52
D 0 4475 2121 0x00000008
10-02 20:34:17.589 2461 2461 W Kernel : [15159.873874] Call trace:
10-02 20:34:17.589 2461 2461 W Kernel : [15159.873878]
[<000000005e1d80e5>] __switch_to+0x94/0xd8
10-02 20:34:17.589 2461 2461 W Kernel : [15159.873882]
[<00000000a85c7184>] __schedule+0x274/0x940
10-02 20:34:17.589 2461 2461 W Kernel : [15159.873885]
[<00000000c9c2c8f0>] schedule+0x40/0xa8
10-02 20:34:17.589 2461 2461 W Kernel : [15159.873889]
[<00000000d3740507>] jbd2_log_wait_commit+0xc0/0x158
10-02 20:34:17.589 2461 2461 W Kernel : [15159.873892]
[<0000000036159c5c>] jbd2_complete_transaction+0x90/0xc0
10-02 20:34:17.589 2461 2461 W Kernel : [15159.873896]
[<0000000019ee473a>] ext4_sync_file+0x440/0x488
10-02 20:34:17.589 2461 2461 W Kernel : [15159.873901]
[<00000000665d0f69>] vfs_fsync_range+0x64/0xc0
10-02 20:34:17.589 2461 2461 W Kernel : [15159.873903]
[<0000000035c0b348>] do_fsync+0x48/0x88
10-02 20:34:17.589 2461 2461 W Kernel : [15159.873906]
[<000000006f18433e>] SyS_fsync+0x24/0x38
10-02 20:34:17.589 2461 2461 W Kernel : [15159.873909] Exception
stack(0xffff00002228bec0 to 0xffff00002228c000)
10-02 20:34:17.589 2461 2461 W Kernel : [15159.873913] bec0:
000000000000001e 0000e6a404184824 000000006f4c7d34 0000000077f8f00c
10-02 20:34:17.589 2461 2461 W Kernel : [15159.873918] bee0:
0000000000000002 00000000000000a9 0000e6a4b5237000 0000000000729bee
10-02 20:34:17.589 2461 2461 W Kernel : [15159.873922] bf00:
0000000000000052 d4d524bf9be44442 0000000000430000 0000e6a42f7e0688
10-02 20:34:17.589 2461 2461 W Kernel : [15159.873926] bf20:
0000000000000018 00000003e8000000 002e5f5680000000 0000474a00000000
10-02 20:34:17.589 2461 2461 W Kernel : [15159.873930] bf40:
0000e6a429b23e30 0000e6a4b1eb2c00 0000000077f7e15e 000000000000001e
10-02 20:34:17.589 2461 2461 W Kernel : [15159.873934] bf60:
00000000ffffffff 0000000013049b60 0000000013207c88 0000000013049ae0
10-02 20:34:17.589 2461 2461 W Kernel : [15159.873938] bf80:
00000000000001b4 00000000000001b0 0000000013207d30 0000018af09743e6
10-02 20:34:17.590 2461 2461 W Kernel : [15159.873942] bfa0:
0000e6a42f70ac23 0000e6a4041847e0 0000e6a429b076b4 0000e6a4041847d0
10-02 20:34:17.590 2461 2461 W Kernel : [15159.873946] bfc0:
0000e6a4b1eb2c08 0000000060000000 000000000000001e 0000000000000052
10-02 20:34:17.590 2461 2461 W Kernel : [15159.873950] bfe0:
0000000000000000 0000000000000000 0000000000000000 0000000000000000
10-02 20:34:17.590 2461 2461 W Kernel : [15159.873952]
[<000000008094bac4>] el0_svc_naked+0x34/0x38
10-02 20:34:17.590 2461 2461 I Kernel : [15159.874135] kworker/u12:3
D 0 9578 2 0x00000020
10-02 20:34:17.590 2461 2461 I Kernel : [15159.874149] Workqueue:
writeback wb_workfn (flush-202:3)
10-02 20:34:17.590 2461 2461 W Kernel : [15159.874154] Call trace:
10-02 20:34:17.590 2461 2461 W Kernel : [15159.874157]
[<000000005e1d80e5>] __switch_to+0x94/0xd8
10-02 20:34:17.590 2461 2461 W Kernel : [15159.874161]
[<00000000a85c7184>] __schedule+0x274/0x940
10-02 20:34:17.590 2461 2461 W Kernel : [15159.874163]
[<00000000c9c2c8f0>] schedule+0x40/0xa8
10-02 20:34:17.590 2461 2461 W Kernel : [15159.874167]
[<00000000a97520e1>] io_schedule+0x20/0x40
10-02 20:34:17.590 2461 2461 W Kernel : [15159.874175]
[<000000008d5c36d0>] blk_mq_get_tag+0x194/0x340
10-02 20:34:17.590 2461 2461 W Kernel : [15159.874178]
[<00000000f97393cb>] blk_mq_get_request+0x164/0x3b0
10-02 20:34:17.590 2461 2461 W Kernel : [15159.874181]
[<00000000e69c8e58>] blk_mq_make_request+0xc8/0x6f8
10-02 20:34:17.590 2461 2461 W Kernel : [15159.874187]
[<00000000e88e7453>] generic_make_request+0xf4/0x288
10-02 20:34:17.590 2461 2461 W Kernel : [15159.874190]
[<0000000016cf2add>] submit_bio+0x5c/0x1d0
10-02 20:34:17.590 2461 2461 W Kernel : [15159.874196]
[<0000000097be4708>] ext4_io_submit+0x54/0x68
10-02 20:34:17.590 2461 2461 W Kernel : [15159.874199]
[<00000000e9c80f00>] ext4_bio_write_page+0x1b0/0x528
10-02 20:34:17.590 2461 2461 W Kernel : [15159.874202]
[<000000009bf94bad>] mpage_submit_page+0x60/0x90
10-02 20:34:17.590 2461 2461 W Kernel : [15159.874205]
[<00000000da1a9585>] mpage_map_and_submit_buffers+0x138/0x238
10-02 20:34:17.590 2461 2461 W Kernel : [15159.874208]
[<00000000c84c3e14>] ext4_writepages+0x8dc/0xe08
10-02 20:34:17.590 2461 2461 W Kernel : [15159.874214]
[<000000007a345b10>] do_writepages+0x5c/0x108
10-02 20:34:17.590 2461 2461 W Kernel : [15159.874217]
[<00000000ef1a522d>] __writeback_single_inode+0x48/0x4f8
10-02 20:34:17.590 2461 2461 W Kernel : [15159.874220]
[<00000000aeac1ba5>] writeback_sb_inodes+0x1c0/0x470
10-02 20:34:17.590 2461 2461 W Kernel : [15159.874223]
[<00000000f7f818b4>] __writeback_inodes_wb+0x78/0xc8
10-02 20:34:17.591 2461 2461 W Kernel : [15159.874225]
[<00000000ae30bb56>] wb_writeback+0x24c/0x3d8
10-02 20:34:17.591 2461 2461 W Kernel : [15159.874228]
[<00000000b489a91f>] wb_workfn+0x1c4/0x490
10-02 20:34:17.591 2461 2461 W Kernel : [15159.874233]
[<000000004df896a9>] process_one_work+0x1d8/0x498
10-02 20:34:17.591 2461 2461 W Kernel : [15159.874236]
[<00000000d7fa6dd7>] worker_thread+0x4c/0x478
10-02 20:34:17.591 2461 2461 W Kernel : [15159.874240]
[<00000000f46ec2f2>] kthread+0x138/0x140
10-02 20:34:17.591 2461 2461 W Kernel : [15159.874243]
[<000000009e914ea4>] ret_from_fork+0x10/0x1c

In the linux file system, is there the commits to fixed the issue. 
Could you please help to check it.

Thank you so much
------------------------------------------------------------------------------

Email này cùng các tệp tin đính kèm là các thông tin bảo mật của VINGROUP và có mục đích chỉ gửi cho những người nhận được nêu tại email. Nếu Quý vị không phải là người nhận dự kiến của email này cùng các tập tin kèm theo, vui lòng không thực hiện bất cứ hành động nào trên cơ sở email và các tập tin này. Việc chia sẻ hoặc tiết lộ bất cứ nội dung nào của email này hoặc bất kỳ tập tin đính kèm nào là không được phép, trừ khi có sự đồng ý trước bằng văn bản của VINGROUP. Trường hợp Quý vị nhận được email này do có sự nhầm lẫn hoặc lỗi hệ thống, vui lòng thông báo ngay cho VINGROUP bằng việc phản hồi lại email này, đồng thời xóa email này cùng các tập tin đính kèm khỏi hệ thống của Quý vị. VINGROUP không chịu bất kỳ trách nhiệm nào về bất cứ sự truyền tải không chính xác, không đầy đủ hay không kịp thời của bất kỳ thông tin nào trong email. Mặc dù VINGROUP đã thực hiện các biện pháp cần thiết để đảm bảo email này không có chứa virus, VINGROUP không chịu trách nhiệm đối với bất cứ thiệt hại hay tổn thất nào phát sinh từ việc nhận hoặc sử dụng email hoặc bất kỳ tệp tin đính kèm nào.

This email and any file transmitted with it are confidential information of VINGROUP and intended solely for the use of the individual or entity to whom they are addressed. If you are not the intended recipient, please do not take any action in reliance on this email. Without prior written consent of VINGROUP, any disclosure or distribution of this email or any of its attachments is strictly prohibited. If you have received this email in error, kindly notify VINGROUP immediately by replying to this email and then delete this email along with all attachments from your system. VINGROUP shall not take any responsibility for any improper, incomplete or delayed transmission of any information contained in this email. Although VINGROUP has taken reasonable precautions to ensure no viruses are present in this email, VINGROUP shall not accept responsibility for any loss or damage arising from the receipt or use of this email or any attachment.

------------------------------------------------------------------------------


2023-10-31 03:06:50

by Theodore Ts'o

[permalink] [raw]
Subject: Re: File system hung on kernel 4.14

On Mon, Oct 30, 2023 at 07:37:09AM +0000, Đỗ Văn Ngọc (VF-KHTVPMDDT-TTTKLKDDT) wrote:
> Dear Linux kernel ext4, Dear Ted,
>
> We are developing project on Android 9 which used kernel 4.14.133
> version.
> And while normal using device, we have a issue device freezing and
> reboot after few minute.
> I checked the log and saw kernel hung on filesystem as below:

I've already replied to your first e-mail sent directly to me. It
appears to be a hardware problem, or possibly a device driver issue.
But it's not a file system issue. Basically, ext4 submitted an I/O,
and then calls io_schedule(), and I/O never completes....

- Ted

[15159.874154] Call trace:
[15159.874157] [<000000005e1d80e5>] __switch_to+0x94/0xd8
[15159.874161] [<00000000a85c7184>] __schedule+0x274/0x940
[15159.874163] [<00000000c9c2c8f0>] schedule+0x40/0xa8
[15159.874167] [<00000000a97520e1>] io_schedule+0x20/0x40
[15159.874175] [<000000008d5c36d0>] blk_mq_get_tag+0x194/0x340
[15159.874178] [<00000000f97393cb>] blk_mq_get_request+0x164/0x3b0
[15159.874181] [<00000000e69c8e58>] blk_mq_make_request+0xc8/0x6f8
[15159.874187] [<00000000e88e7453>] generic_make_request+0xf4/0x288
[15159.874190] [<0000000016cf2add>] submit_bio+0x5c/0x1d0
[15159.874196] [<0000000097be4708>] ext4_io_submit+0x54/0x68
[15159.874199] [<00000000e9c80f00>] ext4_bio_write_page+0x1b0/0x528
[15159.874202] [<000000009bf94bad>] mpage_submit_page+0x60/0x90
[15159.874205] [<00000000da1a9585>] mpage_map_and_submit_buffers+0x138/0x238
[15159.874208] [<00000000c84c3e14>] ext4_writepages+0x8dc/0xe08
[15159.874214] [<000000007a345b10>] do_writepages+0x5c/0x108
[15159.874217] [<00000000ef1a522d>] __writeback_single_inode+0x48/0x4f8
[15159.874220] [<00000000aeac1ba5>] writeback_sb_inodes+0x1c0/0x470
[15159.874223] [<00000000f7f818b4>] __writeback_inodes_wb+0x78/0xc8
[15159.874225] [<00000000ae30bb56>] wb_writeback+0x24c/0x3d8
[15159.874228] [<00000000b489a91f>] wb_workfn+0x1c4/0x490
[15159.874233] [<000000004df896a9>] process_one_work+0x1d8/0x498
[15159.874236] [<00000000d7fa6dd7>] worker_thread+0x4c/0x478
[15159.874240] [<00000000f46ec2f2>] kthread+0x138/0x140
[15159.874243] [<000000009e914ea4>] ret_from_fork+0x10/0x1c