Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S264896AbTFLQyV (ORCPT ); Thu, 12 Jun 2003 12:54:21 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S264898AbTFLQyV (ORCPT ); Thu, 12 Jun 2003 12:54:21 -0400 Received: from over.ny.us.ibm.com ([32.97.182.111]:43438 "EHLO over.ny.us.ibm.com") by vger.kernel.org with ESMTP id S264896AbTFLQxm (ORCPT ); Thu, 12 Jun 2003 12:53:42 -0400 Subject: Re: 2.5.70-mm6 From: Mingming Cao To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, pbadari@us.ibm.com In-Reply-To: <20030610201242.7fde819b.akpm@digeo.com> References: <20030607151440.6982d8c6.akpm@digeo.com> <3EE690AC.70500@us.ibm.com> <20030610201242.7fde819b.akpm@digeo.com> Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Mailer: Ximian Evolution 1.0.8 (1.0.8-10) Date: 12 Jun 2003 09:37:42 -0700 Message-Id: <1055435864.1466.9.camel@w-ming2.beaverton.ibm.com> Mime-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 26444 Lines: 676 > Mingming Cao wrote: > > > > I run 50 fsx tests on ext3 filesystem on 2.5.70-mm6 kernel. Serveral fsx > > tests hang with the status D, after the tests run for a while. No oops, > > no error messages. I found same problem on mm5, but 2.5.70 is fine. Sorry, the tests in 2.5.70 also failed, same problem. On Tue, 2003-06-10 at 20:12, Andrew Morton wrote > If you could, please retest with "elevator=deadline"? > Thanks for your feedback. This time I got more fsx tests hang(about 25). Before normally I saw 5 or 10 tests fail. Here is the stack info. kjournald D 00000001 849 1 900 847 (L-TLB) Call Trace: [] io_schedule+0x26/0x30 [] __wait_on_buffer_wq+0xf5/0x100 [] autoremove_wake_function+0x0/0x50 [] autoremove_wake_function+0x0/0x50 [] sync_dirty_buffer+0x6c/0xd0 [] __wait_on_buffer+0x17/0x20 [] journal_commit_transaction+0xbf7/0x122b [] autoremove_wake_function+0x0/0x50 [] autoremove_wake_function+0x0/0x50 [] schedule+0x207/0x4f0 [] kjournald+0x236/0x260 [] autoremove_wake_function+0x0/0x50 [] autoremove_wake_function+0x0/0x50 [] ret_from_fork+0x6/0x14 [] commit_timeout+0x0/0x10 [] kjournald+0x0/0x260 [] kernel_thread_helper+0x5/0x18 fsx D 00000001 900 1 901 849 (NOTLB) Call Trace: [] do_get_write_access+0x522/0x680 [] default_wake_function+0x0/0x30 [] ext3_get_inode_loc+0x103/0x1a0 [] journal_get_write_access+0x39/0x60 [] ext3_reserve_inode_write+0x86/0xe0 [] ext3_mark_inode_dirty+0x2b/0x60 [] ext3_dirty_inode+0x8c/0x90 [] __mark_inode_dirty+0xf6/0x100 [] inode_update_time+0x80/0xb0 [] generic_file_aio_write_nolock+0x215/0xa50 [] ext3_do_update_inode+0x1c8/0x3d0 [] __bread+0x27/0x50 [] ext3_get_inode_loc+0x103/0x1a0 [] generic_file_aio_write+0x9e/0x120 [] ext3_file_write+0x44/0xe0 [] do_sync_write+0xb6/0xf0 [] __mark_inode_dirty+0xf6/0x100 [] cp_new_stat64+0xfe/0x110 [] autoremove_wake_function+0x0/0x50 [] scheduler_tick+0x16e/0x3b0 [] vfs_write+0xbe/0x130 [] generic_file_llseek+0x0/0xf0 [] sys_write+0x42/0x70 [] syscall_call+0x7/0xb fsx D C3812E48 901 1 902 900 (NOTLB) Call Trace: [] apic_timer_interrupt+0x1a/0x20 [] block_sync_page+0x0/0x10 [] io_schedule+0x26/0x30 [] wait_on_page_bit_wq+0xcc/0x100 [] autoremove_wake_function+0x0/0x50 [] autoremove_wake_function+0x0/0x50 [] truncate_inode_pages+0x2e5/0x2f0 [] vmtruncate+0x6b/0x100 [] inode_setattr+0x134/0x150 [] ext3_setattr+0x7a/0x1a0 [] cp_new_stat64+0xfe/0x110 [] notify_change+0x160/0x19d [] do_truncate+0x6a/0x90 [] sys_fstat64+0x37/0x40 [] sys_ftruncate+0x118/0x1b0 [] generic_file_llseek+0x0/0xf0 [] sys_lseek+0x69/0xb0 [] syscall_call+0x7/0xb fsx D 00000001 902 1 903 901 (NOTLB) Call Trace: [] block_sync_page+0x0/0x10 [] io_schedule+0x26/0x30 [] wait_on_page_bit_wq+0xcc/0x100 [] autoremove_wake_function+0x0/0x50 [] autoremove_wake_function+0x0/0x50 [] truncate_inode_pages+0x2e5/0x2f0 [] vmtruncate+0x6b/0x100 [] inode_setattr+0x134/0x150 [] ext3_setattr+0x7a/0x1a0 [] cp_new_stat64+0xfe/0x110 [] notify_change+0x160/0x19d [] do_truncate+0x6a/0x90 [] sys_fstat64+0x37/0x40 [] sys_ftruncate+0x118/0x1b0 [] generic_file_llseek+0x0/0xf0 [] sys_lseek+0x69/0xb0 [] syscall_call+0x7/0xb fsx D 00000001 903 1 904 902 (NOTLB) Call Trace: [] io_schedule+0x26/0x30 [] __wait_on_buffer_wq+0xf5/0x100 [] autoremove_wake_function+0x0/0x50 [] submit_bh+0x90/0x1e0 [] autoremove_wake_function+0x0/0x50 [] submit_bio+0x54/0xa0 [] __bread_slow_wq+0xad/0xf0 [] __bread+0x46/0x50 [] read_block_bitmap+0x58/0xa0 [] ext3_new_block+0x1a2/0x590 [] __find_get_block+0x73/0x100 [] ext3_alloc_block+0x37/0x40 [] ext3_alloc_branch+0x4a/0x2c0 [] ext3_get_branch+0x72/0x100 [] ext3_get_block_handle+0x18c/0x360 [] alloc_buffer_head+0x41/0x50 [] ext3_get_block+0x64/0xb0 [] __block_prepare_write+0x227/0x4b0 [] block_prepare_write+0x34/0x50 [] ext3_get_block+0x0/0xb0 [] ext3_prepare_write+0x71/0x130 [] ext3_get_block+0x0/0xb0 [] generic_file_aio_write_nolock+0x38d/0xa50 [] ext3_do_update_inode+0x1c8/0x3d0 [] update_atime+0x92/0xf0 [] __generic_file_aio_read+0x1c4/0x210 [] generic_file_aio_write+0x9e/0x120 [] ext3_file_write+0x44/0xe0 [] do_sync_write+0xb6/0xf0 [] __mark_inode_dirty+0xf6/0x100 [] cp_new_stat64+0xfe/0x110 [] autoremove_wake_function+0x0/0x50 [] sys_fstat64+0x37/0x40 [] vfs_write+0xbe/0x130 [] generic_file_llseek+0x0/0xf0 [] sys_write+0x42/0x70 [] syscall_call+0x7/0xb fsx D 00000001 904 1 925 903 (NOTLB) Call Trace: [] apic_timer_interrupt+0x1a/0x20 [] io_schedule+0x26/0x30 [] wait_on_page_bit_wq+0xcc/0x100 [] autoremove_wake_function+0x0/0x50 [] find_get_pages+0x36/0x60 [] autoremove_wake_function+0x0/0x50 [] pagevec_lookup+0x2e/0x40 [] truncate_inode_pages+0x1cc/0x2f0 [] vmtruncate+0x6b/0x100 [] inode_setattr+0x134/0x150 [] ext3_setattr+0x7a/0x1a0 [] cp_new_stat64+0xfe/0x110 [] notify_change+0x160/0x19d [] do_truncate+0x6a/0x90 [] sys_fstat64+0x37/0x40 [] sys_ftruncate+0x118/0x1b0 [] generic_file_llseek+0x0/0xf0 [] sys_lseek+0x69/0xb0 [] syscall_call+0x7/0xb fsx D EF963DD4 925 1 926 904 (NOTLB) Call Trace: [] generic_unplug_device+0x67/0x70 [] io_schedule+0x26/0x30 [] wait_on_page_bit_wq+0xcc/0x100 [] autoremove_wake_function+0x0/0x50 [] autoremove_wake_function+0x0/0x50 [] truncate_inode_pages+0x2e5/0x2f0 [] vmtruncate+0x6b/0x100 [] inode_setattr+0x134/0x150 [] ext3_setattr+0x7a/0x1a0 [] cp_new_stat64+0xfe/0x110 [] notify_change+0x160/0x19d [] do_truncate+0x6a/0x90 [] sys_fstat64+0x37/0x40 [] sys_ftruncate+0x118/0x1b0 [] generic_file_llseek+0x0/0xf0 [] sys_lseek+0x69/0xb0 [] syscall_call+0x7/0xb fsx D C3810004 926 1 927 925 (NOTLB) Call Trace: [] apic_timer_interrupt+0x1a/0x20 [] block_sync_page+0x0/0x10 [] io_schedule+0x26/0x30 [] wait_on_page_bit_wq+0xcc/0x100 [] autoremove_wake_function+0x0/0x50 [] autoremove_wake_function+0x0/0x50 [] truncate_inode_pages+0x2e5/0x2f0 [] vmtruncate+0x6b/0x100 [] inode_setattr+0x134/0x150 [] ext3_setattr+0x7a/0x1a0 [] cp_new_stat64+0xfe/0x110 [] notify_change+0x160/0x19d [] do_truncate+0x6a/0x90 [] sys_fstat64+0x37/0x40 [] sys_ftruncate+0x118/0x1b0 [] generic_file_llseek+0x0/0xf0 [] sys_lseek+0x69/0xb0 [] syscall_call+0x7/0xb fsx D EF95FDA8 927 1 928 926 (NOTLB) Call Trace: [] apic_timer_interrupt+0x1a/0x20 [] io_schedule+0x26/0x30 [] wait_on_page_bit_wq+0xcc/0x100 [] autoremove_wake_function+0x0/0x50 [] find_get_pages+0x36/0x60 [] autoremove_wake_function+0x0/0x50 [] pagevec_lookup+0x2e/0x40 [] truncate_inode_pages+0x1cc/0x2f0 [] vmtruncate+0x6b/0x100 [] inode_setattr+0x134/0x150 [] ext3_setattr+0x7a/0x1a0 [] cp_new_stat64+0xfe/0x110 [] notify_change+0x160/0x19d [] do_truncate+0x6a/0x90 [] sys_fstat64+0x37/0x40 [] sys_ftruncate+0x118/0x1b0 [] generic_file_llseek+0x0/0xf0 [] sys_lseek+0x69/0xb0 [] syscall_call+0x7/0xb fsx D 00000001 928 1 929 927 (NOTLB) Call Trace: [] io_schedule+0x26/0x30 [] __wait_on_buffer_wq+0xf5/0x100 [] autoremove_wake_function+0x0/0x50 [] submit_bh+0x90/0x1e0 [] autoremove_wake_function+0x0/0x50 [] submit_bio+0x54/0xa0 [] __bread_slow_wq+0xad/0xf0 [] __bread+0x46/0x50 [] read_block_bitmap+0x58/0xa0 [] ext3_new_block+0x1a2/0x590 [] __find_get_block+0x73/0x100 [] ext3_alloc_block+0x37/0x40 [] ext3_alloc_branch+0x4a/0x2c0 [] ext3_get_branch+0x72/0x100 [] ext3_get_block_handle+0x18c/0x360 [] alloc_buffer_head+0x41/0x50 [] ext3_get_block+0x64/0xb0 [] __block_prepare_write+0x227/0x4b0 [] block_prepare_write+0x34/0x50 [] ext3_get_block+0x0/0xb0 [] ext3_prepare_write+0x71/0x130 [] ext3_get_block+0x0/0xb0 [] generic_file_aio_write_nolock+0x38d/0xa50 [] __wake_up_common+0x3a/0x60 [] __bread+0x27/0x50 [] ext3_get_inode_loc+0x103/0x1a0 [] generic_file_aio_write+0x9e/0x120 [] ext3_file_write+0x44/0xe0 [] do_sync_write+0xb6/0xf0 [] __mark_inode_dirty+0xf6/0x100 [] cp_new_stat64+0xfe/0x110 [] autoremove_wake_function+0x0/0x50 [] sys_fstat64+0x37/0x40 [] vfs_write+0xbe/0x130 [] generic_file_llseek+0x0/0xf0 [] sys_write+0x42/0x70 [] syscall_call+0x7/0xb fsx D 00000001 929 1 935 928 (NOTLB) Call Trace: [] do_get_write_access+0x522/0x680 [] default_wake_function+0x0/0x30 [] __find_get_block+0x73/0x100 [] journal_get_write_access+0x39/0x60 [] ext3_free_data+0x39/0x150 [] ext3_free_branches+0xed/0x280 [] ext3_truncate+0x47e/0x4e0 [] invalidate_mmap_range+0x7c/0x100 [] ext3_truncate+0x0/0x4e0 [] vmtruncate+0x8c/0x100 [] inode_setattr+0x134/0x150 [] ext3_setattr+0x7a/0x1a0 [] cp_new_stat64+0xfe/0x110 [] notify_change+0x160/0x19d [] do_truncate+0x6a/0x90 [] sys_fstat64+0x37/0x40 [] sys_ftruncate+0x118/0x1b0 [] syscall_call+0x7/0xb fsx D 00000001 935 1 936 929 (NOTLB) Call Trace: [] generic_unplug_device+0x67/0x70 [] io_schedule+0x26/0x30 [] wait_on_page_bit_wq+0xcc/0x100 [] autoremove_wake_function+0x0/0x50 [] find_get_pages+0x36/0x60 [] autoremove_wake_function+0x0/0x50 [] pagevec_lookup+0x2e/0x40 [] truncate_inode_pages+0x1cc/0x2f0 [] vmtruncate+0x6b/0x100 [] inode_setattr+0x134/0x150 [] ext3_setattr+0x7a/0x1a0 [] cp_new_stat64+0xfe/0x110 [] notify_change+0x160/0x19d [] do_truncate+0x6a/0x90 [] sys_fstat64+0x37/0x40 [] sys_ftruncate+0x118/0x1b0 [] generic_file_llseek+0x0/0xf0 [] sys_lseek+0x69/0xb0 [] syscall_call+0x7/0xb fsx D EF8FBDD4 936 1 937 935 (NOTLB) Call Trace: [] apic_timer_interrupt+0x1a/0x20 [] ll_front_merge_fn+0x4b/0x120 [] io_schedule+0x26/0x30 [] wait_on_page_bit_wq+0xcc/0x100 [] autoremove_wake_function+0x0/0x50 [] autoremove_wake_function+0x0/0x50 [] truncate_inode_pages+0x2e5/0x2f0 [] vmtruncate+0x6b/0x100 [] inode_setattr+0x134/0x150 [] ext3_setattr+0x7a/0x1a0 [] cp_new_stat64+0xfe/0x110 [] notify_change+0x160/0x19d [] do_truncate+0x6a/0x90 [] sys_fstat64+0x37/0x40 [] sys_ftruncate+0x118/0x1b0 [] generic_file_llseek+0x0/0xf0 [] sys_lseek+0x69/0xb0 [] syscall_call+0x7/0xb fsx D 00000001 937 1 938 936 (NOTLB) Call Trace: [] io_schedule+0x26/0x30 [] wait_on_page_bit_wq+0xcc/0x100 [] autoremove_wake_function+0x0/0x50 [] find_get_pages+0x36/0x60 [] autoremove_wake_function+0x0/0x50 [] pagevec_lookup+0x2e/0x40 [] truncate_inode_pages+0x1cc/0x2f0 [] vmtruncate+0x6b/0x100 [] inode_setattr+0x134/0x150 [] ext3_setattr+0x7a/0x1a0 [] cp_new_stat64+0xfe/0x110 [] notify_change+0x160/0x19d [] do_truncate+0x6a/0x90 [] sys_fstat64+0x37/0x40 [] sys_ftruncate+0x118/0x1b0 [] generic_file_llseek+0x0/0xf0 [] sys_lseek+0x69/0xb0 [] syscall_call+0x7/0xb fsx D 00000001 938 1 939 937 (NOTLB) Call Trace: [] do_get_write_access+0x522/0x680 [] default_wake_function+0x0/0x30 [] journal_get_write_access+0x39/0x60 [] ext3_free_data+0x39/0x150 [] ext3_truncate+0x28b/0x4e0 [] invalidate_mmap_range+0x7c/0x100 [] ext3_truncate+0x0/0x4e0 [] vmtruncate+0x8c/0x100 [] inode_setattr+0x134/0x150 [] ext3_setattr+0x7a/0x1a0 [] cp_new_stat64+0xfe/0x110 [] notify_change+0x160/0x19d [] do_truncate+0x6a/0x90 [] sys_fstat64+0x37/0x40 [] sys_ftruncate+0x118/0x1b0 [] generic_file_llseek+0x0/0xf0 [] sys_lseek+0x69/0xb0 [] syscall_call+0x7/0xb fsx D 00000001 939 1 940 938 (NOTLB) Call Trace: [] io_schedule+0x26/0x30 [] __wait_on_buffer_wq+0xf5/0x100 [] autoremove_wake_function+0x0/0x50 [] submit_bh+0x90/0x1e0 [] autoremove_wake_function+0x0/0x50 [] submit_bio+0x54/0xa0 [] __bread_slow_wq+0xad/0xf0 [] __bread+0x46/0x50 [] read_block_bitmap+0x58/0xa0 [] ext3_new_block+0x1a2/0x590 [] __wake_up_common+0x3a/0x60 [] ext3_alloc_block+0x37/0x40 [] ext3_alloc_branch+0x4a/0x2c0 [] bh_lru_install+0xb9/0xf0 [] ext3_get_block_handle+0x18c/0x360 [] alloc_buffer_head+0x41/0x50 [] ext3_get_block+0x64/0xb0 [] __block_prepare_write+0x227/0x4b0 [] block_prepare_write+0x34/0x50 [] ext3_get_block+0x0/0xb0 [] ext3_prepare_write+0x71/0x130 [] ext3_get_block+0x0/0xb0 [] generic_file_aio_write_nolock+0x38d/0xa50 [] ext3_do_update_inode+0x1c8/0x3d0 [] default_wake_function+0x2a/0x30 [] autoremove_wake_function+0x27/0x50 [] __wake_up_common+0x3a/0x60 [] generic_file_aio_write+0x9e/0x120 [] ext3_file_write+0x44/0xe0 [] do_sync_write+0xb6/0xf0 [] scsi_put_command+0x6e/0x90 [] scsi_end_request+0xae/0xc0 [] autoremove_wake_function+0x0/0x50 [] __wake_up_common+0x3a/0x60 [] qla2x00_intr_handler+0x203/0x220 [] vfs_write+0xbe/0x130 [] generic_file_llseek+0x0/0xf0 [] sys_write+0x42/0x70 [] syscall_call+0x7/0xb fsx D 00000001 940 1 941 939 (NOTLB) Call Trace: [] generic_unplug_device+0x67/0x70 [] io_schedule+0x26/0x30 [] wait_on_page_bit_wq+0xcc/0x100 [] autoremove_wake_function+0x0/0x50 [] find_get_pages+0x36/0x60 [] autoremove_wake_function+0x0/0x50 [] pagevec_lookup+0x2e/0x40 [] truncate_inode_pages+0x1cc/0x2f0 [] vmtruncate+0x6b/0x100 [] inode_setattr+0x134/0x150 [] ext3_setattr+0x7a/0x1a0 [] cp_new_stat64+0xfe/0x110 [] notify_change+0x160/0x19d [] do_truncate+0x6a/0x90 [] sys_fstat64+0x37/0x40 [] sys_ftruncate+0x118/0x1b0 [] generic_file_llseek+0x0/0xf0 [] sys_lseek+0x69/0xb0 [] syscall_call+0x7/0xb fsx D C3815338 941 1 942 940 (NOTLB) Call Trace: [] apic_timer_interrupt+0x1a/0x20 [] block_sync_page+0x0/0x10 [] io_schedule+0x26/0x30 [] wait_on_page_bit_wq+0xcc/0x100 [] autoremove_wake_function+0x0/0x50 [] find_get_pages+0x36/0x60 [] autoremove_wake_function+0x0/0x50 [] pagevec_lookup+0x2e/0x40 [] truncate_inode_pages+0x1cc/0x2f0 [] vmtruncate+0x6b/0x100 [] inode_setattr+0x134/0x150 [] ext3_setattr+0x7a/0x1a0 [] cp_new_stat64+0xfe/0x110 [] notify_change+0x160/0x19d [] do_truncate+0x6a/0x90 [] sys_fstat64+0x37/0x40 [] sys_ftruncate+0x118/0x1b0 [] generic_file_llseek+0x0/0xf0 [] sys_lseek+0x69/0xb0 [] syscall_call+0x7/0xb fsx D 00000001 942 1 943 941 (NOTLB) Call Trace: [] apic_timer_interrupt+0x1a/0x20 [