Greetings,
Yesterday morning and this morning, 4.4-master has exploded as below.
(have crash dumps, but returning from vacation, also have backlog)
[ 1576.519298] ------------[ cut here ]------------
[ 1576.519306] WARNING: CPU: 6 PID: 287 at block/blk-merge.c:435 blk_rq_map_sg+0x43a/0x440()
[ 1576.519307] Modules linked in: tun bnep bluetooth rfkill nfsd auth_rpcgss oid_registry nfs_acl lockd grace sunrpc fuse af_packet bridge stp llc iscsi_ibft iscsi_boot_sysfs ip6t_REJECT nf_reject_ipv6 xt_tcpudp nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_raw ipt_REJECT nf_reject_ipv4 iptable_raw iptable_filter ip6table_mangle nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_ipv4 nf_defrag_ipv4 ip_tables xt_conntrack nf_conntrack ip6table_filter ip6_tables x_tables snd_hda_codec_realtek snd_hda_codec_generic snd_hda_codec_hdmi snd_hda_intel snd_hda_codec x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm nls_iso8859_1 nls_cp437 snd_hwdep vfat fat snd_hda_core tpm_infineon tpm_tis irqbypass crct10dif_pclmul crc32_pclmul snd_pcm tpm crc32c_intel snd_timer snd aesni_intel sr_mod r8169
[ 1576.519331] cdrom iTCO_wdt iTCO_vendor_support lpc_ich thermal battery aes_x86_64 lrw mei_me mii mei gf128mul glue_helper ablk_helper intel_smartconnect mfd_core serio_raw i2c_i801 joydev soundcore fan cryptd dm_mod efivarfs ums_realtek uas usb_storage nouveau mxm_wmi wmi i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm video button sg autofs4
[ 1576.519347] CPU: 6 PID: 287 Comm: kworker/6:1H Not tainted 4.4.0-master #483
[ 1576.519348] Hardware name: MEDION MS-7848/MS-7848, BIOS M7848W08.20C 09/23/2013
[ 1576.519350] Workqueue: kblockd cfq_kick_queue
[ 1576.519351] ffffffff81837694 ffff8800dc9ebb50 ffffffff81299276 0000000000000000
[ 1576.519353] ffff8800dc9ebb88 ffffffff810552c6 0000000000000000 0000000000000000
[ 1576.519354] 00000000000000e0 ffff8800db4a4a80 000000000000000f ffff8800dc9ebb98
[ 1576.519355] Call Trace:
[ 1576.519359] [<ffffffff81299276>] dump_stack+0x44/0x5e
[ 1576.519362] [<ffffffff810552c6>] warn_slowpath_common+0x86/0xc0
[ 1576.519364] [<ffffffff810553ba>] warn_slowpath_null+0x1a/0x20
[ 1576.519365] [<ffffffff8127c64a>] blk_rq_map_sg+0x43a/0x440
[ 1576.519368] [<ffffffff813d2e24>] scsi_init_sgtable+0x44/0x70
[ 1576.519369] [<ffffffff813d2e9a>] scsi_init_io+0x4a/0x1c0
[ 1576.519372] [<ffffffff813ded98>] sd_setup_read_write_cmnd+0x38/0x9d0
[ 1576.519375] [<ffffffff813cb007>] ? scsi_host_alloc_command+0x47/0xc0
[ 1576.519376] [<ffffffff813e2927>] sd_init_command+0x27/0xa0
[ 1576.519378] [<ffffffff813d3100>] scsi_setup_cmnd+0xf0/0x150
[ 1576.519379] [<ffffffff813d32b1>] scsi_prep_fn+0xd1/0x170
[ 1576.519381] [<ffffffff81277a7f>] blk_peek_request+0x14f/0x260
[ 1576.519383] [<ffffffff813d47af>] scsi_request_fn+0x3f/0x600
[ 1576.519384] [<ffffffff81273c73>] __blk_run_queue+0x33/0x40
[ 1576.519385] [<ffffffff81292e4f>] cfq_kick_queue+0x2f/0x40
[ 1576.519388] [<ffffffff8106ac33>] process_one_work+0x143/0x3c0
[ 1576.519389] [<ffffffff8106afca>] worker_thread+0x11a/0x450
[ 1576.519392] [<ffffffff815b837d>] ? __schedule+0x31d/0x830
[ 1576.519393] [<ffffffff8106aeb0>] ? process_one_work+0x3c0/0x3c0
[ 1576.519395] [<ffffffff8106fe29>] kthread+0xc9/0xe0
[ 1576.519396] [<ffffffff8106fd60>] ? kthread_park+0x60/0x60
[ 1576.519398] [<ffffffff815bc46f>] ret_from_fork+0x3f/0x70
[ 1576.519399] [<ffffffff8106fd60>] ? kthread_park+0x60/0x60
[ 1576.519400] ---[ end trace 0e3b884028bc69c0 ]---
[ 1576.519407] ------------[ cut here ]------------
[ 1576.519408] kernel BUG at drivers/scsi/scsi_lib.c:1096!
[ 1576.519409] invalid opcode: 0000 [#1] SMP
[ 1576.519411] Dumping ftrace buffer:
[ 1576.519413] (ftrace buffer empty)
[ 1576.519414] Modules linked in: tun bnep bluetooth rfkill nfsd auth_rpcgss oid_registry nfs_acl lockd grace sunrpc fuse af_packet bridge stp llc iscsi_ibft iscsi_boot_sysfs ip6t_REJECT nf_reject_ipv6 xt_tcpudp nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_raw ipt_REJECT nf_reject_ipv4 iptable_raw iptable_filter ip6table_mangle nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_ipv4 nf_defrag_ipv4 ip_tables xt_conntrack nf_conntrack ip6table_filter ip6_tables x_tables snd_hda_codec_realtek snd_hda_codec_generic snd_hda_codec_hdmi snd_hda_intel snd_hda_codec x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm nls_iso8859_1 nls_cp437 snd_hwdep vfat fat snd_hda_core tpm_infineon tpm_tis irqbypass crct10dif_pclmul crc32_pclmul snd_pcm tpm crc32c_intel snd_timer snd aesni_intel sr_mod r8169
[ 1576.519430] cdrom iTCO_wdt iTCO_vendor_support lpc_ich thermal battery aes_x86_64 lrw mei_me mii mei gf128mul glue_helper ablk_helper intel_smartconnect mfd_core serio_raw i2c_i801 joydev soundcore fan cryptd dm_mod efivarfs ums_realtek uas usb_storage nouveau mxm_wmi wmi i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm video button sg autofs4
[ 1576.519440] CPU: 6 PID: 287 Comm: kworker/6:1H Tainted: G W 4.4.0-master #483
[ 1576.519440] Hardware name: MEDION MS-7848/MS-7848, BIOS M7848W08.20C 09/23/2013
[ 1576.519441] Workqueue: kblockd cfq_kick_queue
[ 1576.519442] task: ffff8800dcde9b40 ti: ffff8800dc9e8000 task.ti: ffff8800dc9e8000
[ 1576.519443] RIP: 0010:[<ffffffff813d2e4a>] [<ffffffff813d2e4a>] scsi_init_sgtable+0x6a/0x70
[ 1576.519445] RSP: 0018:ffff8800dc9ebc20 EFLAGS: 00010002
[ 1576.519446] RAX: 0000000000000005 RBX: ffff880409a4dae0 RCX: 0000000000000006
[ 1576.519447] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000009
[ 1576.519448] RBP: ffff8800dc9ebc38 R08: 000000000000000a R09: 0000000000000000
[ 1576.519448] R10: 0000000000000000 R11: 00000000000003ba R12: ffff8804088bfec0
[ 1576.519449] R13: 0000000000000000 R14: ffff88040c1c9000 R15: ffff880409a4dae0
[ 1576.519450] FS: 0000000000000000(0000) GS:ffff88041ed80000(0000) knlGS:0000000000000000
[ 1576.519451] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 1576.519452] CR2: 00007fa1663ee000 CR3: 0000000001a0b000 CR4: 00000000001406e0
[ 1576.519452] Stack:
[ 1576.519453] ffff8804088bfe00 ffff8800dc8d8700 ffff88040c1c9000 ffff8800dc9ebc78
[ 1576.519454] ffffffff813d2e9a ffff88040c804e18 ffff880409a4dae0 ffff8800dc8d8758
[ 1576.519456] ffff880409fc0000 ffff88040c1c9000 000000000003f000 ffff8800dc9ebce8
[ 1576.519457] Call Trace:
[ 1576.519459] [<ffffffff813d2e9a>] scsi_init_io+0x4a/0x1c0
[ 1576.519460] [<ffffffff813ded98>] sd_setup_read_write_cmnd+0x38/0x9d0
[ 1576.519462] [<ffffffff813cb007>] ? scsi_host_alloc_command+0x47/0xc0
[ 1576.519463] [<ffffffff813e2927>] sd_init_command+0x27/0xa0
[ 1576.519465] [<ffffffff813d3100>] scsi_setup_cmnd+0xf0/0x150
[ 1576.519466] [<ffffffff813d32b1>] scsi_prep_fn+0xd1/0x170
[ 1576.519468] [<ffffffff81277a7f>] blk_peek_request+0x14f/0x260
[ 1576.519469] [<ffffffff813d47af>] scsi_request_fn+0x3f/0x600
[ 1576.519471] [<ffffffff81273c73>] __blk_run_queue+0x33/0x40
[ 1576.519472] [<ffffffff81292e4f>] cfq_kick_queue+0x2f/0x40
[ 1576.519473] [<ffffffff8106ac33>] process_one_work+0x143/0x3c0
[ 1576.519475] [<ffffffff8106afca>] worker_thread+0x11a/0x450
[ 1576.519476] [<ffffffff815b837d>] ? __schedule+0x31d/0x830
[ 1576.519478] [<ffffffff8106aeb0>] ? process_one_work+0x3c0/0x3c0
[ 1576.519479] [<ffffffff8106fe29>] kthread+0xc9/0xe0
[ 1576.519480] [<ffffffff8106fd60>] ? kthread_park+0x60/0x60
[ 1576.519482] [<ffffffff815bc46f>] ret_from_fork+0x3f/0x70
[ 1576.519483] [<ffffffff8106fd60>] ? kthread_park+0x60/0x60
[ 1576.519484] Code: e8 ec 93 ea ff 41 3b 44 24 08 77 1f 41 89 44 24 08 8b 43 5c 41 89 44 24 10 5b 44 89 e8 41 5c 41 5d 5d c3 41 bd 02 00 00 00 eb ee <0f> 0b 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 41 57 41 56 41 55
[ 1576.519497] RIP [<ffffffff813d2e4a>] scsi_init_sgtable+0x6a/0x70
[ 1576.519499] RSP <ffff8800dc9ebc20>
[ 6875.958897] kvm: zapping shadow pages for mmio generation wraparound
[ 6883.121830] kvm [10716]: vcpu0 unhandled rdmsr: 0x570
[ 6883.121888] kvm [10716]: vcpu1 unhandled rdmsr: 0x570
[ 6883.121911] kvm [10716]: vcpu2 unhandled rdmsr: 0x570
[ 6883.121925] kvm [10716]: vcpu3 unhandled rdmsr: 0x570
[ 6884.452090] kvm [10716]: vcpu2 unhandled rdmsr: 0x606
[ 6884.452107] kvm [10716]: vcpu2 unhandled rdmsr: 0x34
[ 6887.917565] ------------[ cut here ]------------
[ 6887.917573] WARNING: CPU: 2 PID: 279 at block/blk-merge.c:435 blk_rq_map_sg+0x43a/0x440()
[ 6887.917574] Modules linked in: vhost_net vhost tun bnep bluetooth rfkill nfsd auth_rpcgss oid_registry nfs_acl lockd grace sunrpc fuse af_packet bridge stp llc iscsi_ibft iscsi_boot_sysfs ip6t_REJECT nf_reject_ipv6 xt_tcpudp nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_raw ipt_REJECT nf_reject_ipv4 iptable_raw iptable_filter ip6table_mangle nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_ipv4 nf_defrag_ipv4 ip_tables xt_conntrack nf_conntrack ip6table_filter ip6_tables x_tables x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass nls_iso8859_1 crct10dif_pclmul crc32_pclmul nls_cp437 snd_hda_codec_hdmi vfat fat crc32c_intel snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hwdep snd_hda_core aesni_intel snd_pcm snd_timer tpm_infineon aes_x86_64
[ 6887.917597] snd lrw tpm_tis iTCO_wdt gf128mul tpm glue_helper ablk_helper mei_me r8169 iTCO_vendor_support sr_mod cdrom thermal battery lpc_ich cryptd mei mii serio_raw i2c_i801 joydev mfd_core intel_smartconnect soundcore fan dm_mod efivarfs ums_realtek uas usb_storage nouveau mxm_wmi wmi i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm video button sg autofs4
[ 6887.917615] CPU: 2 PID: 279 Comm: kworker/2:1H Not tainted 4.4.0-master #483
[ 6887.917616] Hardware name: MEDION MS-7848/MS-7848, BIOS M7848W08.20C 09/23/2013
[ 6887.917618] Workqueue: kblockd cfq_kick_queue
[ 6887.917619] ffffffff81837694 ffff8804093f3b50 ffffffff81299276 0000000000000000
[ 6887.917620] ffff8804093f3b88 ffffffff810552c6 0000000000000000 0000000000000000
[ 6887.917621] 00000000000001f0 ffff880355a57a80 0000000000000020 ffff8804093f3b98
[ 6887.917623] Call Trace:
[ 6887.917626] [<ffffffff81299276>] dump_stack+0x44/0x5e
[ 6887.917629] [<ffffffff810552c6>] warn_slowpath_common+0x86/0xc0
[ 6887.917631] [<ffffffff810553ba>] warn_slowpath_null+0x1a/0x20
[ 6887.917633] [<ffffffff8127c64a>] blk_rq_map_sg+0x43a/0x440
[ 6887.917635] [<ffffffff813d2e24>] scsi_init_sgtable+0x44/0x70
[ 6887.917637] [<ffffffff813d2e9a>] scsi_init_io+0x4a/0x1c0
[ 6887.917638] [<ffffffff81291d63>] ? cfq_rb_erase+0x23/0x40
[ 6887.917641] [<ffffffff813ded98>] sd_setup_read_write_cmnd+0x38/0x9d0
[ 6887.917643] [<ffffffff813cb007>] ? scsi_host_alloc_command+0x47/0xc0
[ 6887.917645] [<ffffffff813e2927>] sd_init_command+0x27/0xa0
[ 6887.917646] [<ffffffff813d3100>] scsi_setup_cmnd+0xf0/0x150
[ 6887.917647] [<ffffffff813d32b1>] scsi_prep_fn+0xd1/0x170
[ 6887.917649] [<ffffffff81277a7f>] blk_peek_request+0x14f/0x260
[ 6887.917650] [<ffffffff813d47af>] scsi_request_fn+0x3f/0x600
[ 6887.917652] [<ffffffff81273c73>] __blk_run_queue+0x33/0x40
[ 6887.917653] [<ffffffff81292e4f>] cfq_kick_queue+0x2f/0x40
[ 6887.917655] [<ffffffff8106ac33>] process_one_work+0x143/0x3c0
[ 6887.917657] [<ffffffff8106afca>] worker_thread+0x11a/0x450
[ 6887.917660] [<ffffffff815b837d>] ? __schedule+0x31d/0x830
[ 6887.917661] [<ffffffff8106aeb0>] ? process_one_work+0x3c0/0x3c0
[ 6887.917663] [<ffffffff8106fe29>] kthread+0xc9/0xe0
[ 6887.917664] [<ffffffff8106fd60>] ? kthread_park+0x60/0x60
[ 6887.917665] [<ffffffff815bc46f>] ret_from_fork+0x3f/0x70
[ 6887.917666] [<ffffffff8106fd60>] ? kthread_park+0x60/0x60
[ 6887.917667] ---[ end trace 6944837ca9dd37ff ]---
[ 6887.917678] ------------[ cut here ]------------
[ 6887.917678] kernel BUG at drivers/scsi/scsi_lib.c:1096!
[ 6887.917679] invalid opcode: 0000 [#1] SMP
[ 6887.917681] Dumping ftrace buffer:
[ 6887.917684] (ftrace buffer empty)
[ 6887.917685] Modules linked in: vhost_net vhost tun bnep bluetooth rfkill nfsd auth_rpcgss oid_registry nfs_acl lockd grace sunrpc fuse af_packet bridge stp llc iscsi_ibft iscsi_boot_sysfs ip6t_REJECT nf_reject_ipv6 xt_tcpudp nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_raw ipt_REJECT nf_reject_ipv4 iptable_raw iptable_filter ip6table_mangle nf_conntrack_netbios_ns nf_conntrack_broadcast nf_conntrack_ipv4 nf_defrag_ipv4 ip_tables xt_conntrack nf_conntrack ip6table_filter ip6_tables x_tables x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm irqbypass nls_iso8859_1 crct10dif_pclmul crc32_pclmul nls_cp437 snd_hda_codec_hdmi vfat fat crc32c_intel snd_hda_codec_realtek snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hwdep snd_hda_core aesni_intel snd_pcm snd_timer tpm_infineon aes_x86_64
[ 6887.917703] snd lrw tpm_tis iTCO_wdt gf128mul tpm glue_helper ablk_helper mei_me r8169 iTCO_vendor_support sr_mod cdrom thermal battery lpc_ich cryptd mei mii serio_raw i2c_i801 joydev mfd_core intel_smartconnect soundcore fan dm_mod efivarfs ums_realtek uas usb_storage nouveau mxm_wmi wmi i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm drm video button sg autofs4
[ 6887.917717] CPU: 2 PID: 279 Comm: kworker/2:1H Tainted: G W 4.4.0-master #483
[ 6887.917717] Hardware name: MEDION MS-7848/MS-7848, BIOS M7848W08.20C 09/23/2013
[ 6887.917719] Workqueue: kblockd cfq_kick_queue
[ 6887.917720] task: ffff880409aa9b40 ti: ffff8804093f0000 task.ti: ffff8804093f0000
[ 6887.917721] RIP: 0010:[<ffffffff813d2e4a>] [<ffffffff813d2e4a>] scsi_init_sgtable+0x6a/0x70
[ 6887.917723] RSP: 0000:ffff8804093f3c20 EFLAGS: 00010002
[ 6887.917724] RAX: 0000000000000005 RBX: ffff880409f76560 RCX: 0000000000000006
[ 6887.917725] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000009
[ 6887.917725] RBP: ffff8804093f3c38 R08: 000000000000000a R09: 0000000000000000
[ 6887.917726] R10: 0000000000000000 R11: 00000000000003c6 R12: ffff8800dc90a3c0
[ 6887.917726] R13: 0000000000000000 R14: ffff880409f78000 R15: ffff880409f76560
[ 6887.917727] FS: 0000000000000000(0000) GS:ffff88041ec80000(0000) knlGS:0000000000000000
[ 6887.917728] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 6887.917729] CR2: 00002ada68ac2008 CR3: 0000000352e81000 CR4: 00000000001426e0
[ 6887.917729] Stack:
[ 6887.917730] ffff8800dc90a300 ffff880409f80000 ffff880409f78000 ffff8804093f3c78
[ 6887.917731] ffffffff813d2e9a ffffffff81291d63 ffff880409f76560 ffff880409f80000
[ 6887.917732] ffff880409f41000 ffff880409f78000 0000000000038000 ffff8804093f3ce8
[ 6887.917734] Call Trace:
[ 6887.917736] [<ffffffff813d2e9a>] scsi_init_io+0x4a/0x1c0
[ 6887.917738] [<ffffffff81291d63>] ? cfq_rb_erase+0x23/0x40
[ 6887.917740] [<ffffffff813ded98>] sd_setup_read_write_cmnd+0x38/0x9d0
[ 6887.917742] [<ffffffff813cb007>] ? scsi_host_alloc_command+0x47/0xc0
[ 6887.917744] [<ffffffff813e2927>] sd_init_command+0x27/0xa0
[ 6887.917745] [<ffffffff813d3100>] scsi_setup_cmnd+0xf0/0x150
[ 6887.917746] [<ffffffff813d32b1>] scsi_prep_fn+0xd1/0x170
[ 6887.917748] [<ffffffff81277a7f>] blk_peek_request+0x14f/0x260
[ 6887.917749] [<ffffffff813d47af>] scsi_request_fn+0x3f/0x600
[ 6887.917751] [<ffffffff81273c73>] __blk_run_queue+0x33/0x40
[ 6887.917752] [<ffffffff81292e4f>] cfq_kick_queue+0x2f/0x40
[ 6887.917754] [<ffffffff8106ac33>] process_one_work+0x143/0x3c0
[ 6887.917755] [<ffffffff8106afca>] worker_thread+0x11a/0x450
[ 6887.917757] [<ffffffff815b837d>] ? __schedule+0x31d/0x830
[ 6887.917759] [<ffffffff8106aeb0>] ? process_one_work+0x3c0/0x3c0
[ 6887.917760] [<ffffffff8106fe29>] kthread+0xc9/0xe0
[ 6887.917762] [<ffffffff8106fd60>] ? kthread_park+0x60/0x60
[ 6887.917764] [<ffffffff815bc46f>] ret_from_fork+0x3f/0x70
[ 6887.917765] [<ffffffff8106fd60>] ? kthread_park+0x60/0x60
[ 6887.917765] Code: e8 ec 93 ea ff 41 3b 44 24 08 77 1f 41 89 44 24 08 8b 43 5c 41 89 44 24 10 5b 44 89 e8 41 5c 41 5d 5d c3 41 bd 02 00 00 00 eb ee <0f> 0b 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 41 57 41 56 41 55
[ 6887.917778] RIP [<ffffffff813d2e4a>] scsi_init_sgtable+0x6a/0x70
[ 6887.917779] RSP <ffff8804093f3c20>
On 12/02/2015 12:29 AM, Mike Galbraith wrote:
> Greetings,
>
> Yesterday morning and this morning, 4.4-master has exploded as below.
>
> (have crash dumps, but returning from vacation, also have backlog)
>
> [ 1576.519298] ------------[ cut here ]------------
> [ 1576.519306] WARNING: CPU: 6 PID: 287 at block/blk-merge.c:435 blk_rq_map_sg+0x43a/0x440()
Should be fixed in for-linus in my tree, going upstream today.
--
Jens Axboe