Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934544AbcCPEmi (ORCPT ); Wed, 16 Mar 2016 00:42:38 -0400 Received: from mail-lf0-f49.google.com ([209.85.215.49]:36856 "EHLO mail-lf0-f49.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752119AbcCPEmd (ORCPT ); Wed, 16 Mar 2016 00:42:33 -0400 From: Evgenii Lepikhin To: linux-kernel@vger.kernel.org Subject: Re: kernel OOPS in MM(?) Organization: 1Gb.ru References: User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.0.50 (gnu/linux) X-Url: http://www.1gb.ru Cancel-Lock: sha1:JZIzYq7abmCB54yFFPuStCuEd9I= Date: Wed, 16 Mar 2016 07:42:29 +0300 In-Reply-To: (Evgenii Lepikhin's message of "Thu, 10 Mar 2016 12:31:54 UTC") Message-ID: <87vb4n6nka.fsf@eugene-vaio.home> MIME-Version: 1.0 Content-Type: text/plain Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 7530 Lines: 94 Hello, On 2016-03-10 12:31, Evgenii Lepikhin wrote: > We need help to understand the source of the problem and may be to create a bugreport. Here is crash report: > > Mar 10 04:03:51 l28 kernel: [2075560.434445] BUG: unable to handle kernel paging request at 0000000040008021 > Mar 10 04:03:51 l28 kernel: [2075560.434669] IP: [] __kmalloc+0x69/0x100 > Mar 10 04:03:51 l28 kernel: [2075560.434800] PGD b7e462067 PUD 0 > Mar 10 04:03:51 l28 kernel: [2075560.434913] Oops: 0000 [#1] SMP > Mar 10 04:03:51 l28 kernel: [2075560.435044] Modules linked in: > tcm_loop iscsi_target_mod target_core_pscsi target_core_file > target_core_iblock target_core_mod ipt_NETFLOW(O) configfs iscsi_tcp > libis > csi_tcp libiscsi scsi_transport_iscsi fuse [last unloaded: ipfw_mod] > Mar 10 04:03:51 l28 kernel: [2075560.435539] CPU: 4 PID: 27141 Comm: rm Tainted: G O 3.12.51-jl-2015-12-25 #1 > Mar 10 04:03:51 l28 kernel: [2075560.435734] Hardware name: Intel Corporation S2600IP ........../S2600IP, BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013 > Mar 10 04:03:51 l28 kernel: [2075560.435939] task: ffff880e622ccba0 ti: ffff880eeb008000 task.ti: ffff880eeb008000 > Mar 10 04:03:51 l28 kernel: [2075560.436131] RIP: 0010:[] [] __kmalloc+0x69/0x100 > Mar 10 04:03:51 l28 kernel: [2075560.436333] RSP: 0018:ffff880eeb009b38 EFLAGS: 00010282 > Mar 10 04:03:51 l28 kernel: [2075560.436439] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 00000000a8a73dc2 > Mar 10 04:03:51 l28 kernel: [2075560.436632] RDX: 00000000a8a73dc1 RSI: 0000000000000000 RDI: 0000000000013500 > Mar 10 04:03:51 l28 kernel: [2075560.438248] RBP: ffff880eeb009b58 R08: ffff88103fc13500 R09: ffffffff811a0267 > Mar 10 04:03:51 l28 kernel: [2075560.438446] R10: ffff880eeb009d84 R11: 0000000000000000 R12: ffff88081f803a00 > Mar 10 04:03:51 l28 kernel: [2075560.438656] R13: 0000000040008021 R14: 0000000000000250 R15: ffff880250e833b0 > Mar 10 04:03:51 l28 kernel: [2075560.438851] FS: 00007fe2316dd700(0000) GS:ffff88103fc00000(0000) knlGS:0000000000000000 > Mar 10 04:03:51 l28 kernel: [2075560.439045] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > Mar 10 04:03:51 l28 kernel: [2075560.439152] CR2: 0000000040008021 CR3: 0000000a20736000 CR4: 00000000000407e0 > Mar 10 04:03:51 l28 kernel: [2075560.439343] Stack: > Mar 10 04:03:51 l28 kernel: [2075560.439439] 0000000000000000 0000000000000250 0000000000000060 0000000000000000 > Mar 10 04:03:51 l28 kernel: [2075560.439663] ffff880eeb009b88 ffffffff811a0267 ffff881015fb7fe0 0000000000000060 > Mar 10 04:03:51 l28 kernel: [2075560.439898] ffff880250e83490 0000000000000000 ffff880eeb009ba8 ffffffff811a02f8 > Mar 10 04:03:51 l28 kernel: [2075560.440153] Call Trace: > Mar 10 04:03:51 l28 kernel: [2075560.440257] [] kmem_alloc+0x67/0xe0 > Mar 10 04:03:51 l28 kernel: [2075560.440365] [] kmem_zalloc+0x18/0x40 > Mar 10 04:03:51 l28 kernel: [2075560.440473] [] xfs_log_commit_cil+0x373/0x4c0 > Mar 10 04:03:51 l28 kernel: [2075560.440585] [] ? xfs_bmap_search_multi_extents+0xe0/0x110 > Mar 10 04:03:51 l28 kernel: [2075560.440783] [] xfs_trans_commit+0x6c/0x250 > Mar 10 04:03:51 l28 kernel: [2075560.440899] [] xfs_bmap_finish+0xb7/0x1a0 Another issue on the same server, same instruction pointer: Mar 16 04:53:54 l28 kernel: [521052.387878] BUG: unable to handle kernel paging request at 0000000040008021 Mar 16 04:53:54 l28 kernel: [521052.388022] IP: [] __kmalloc+0x69/0x100 Mar 16 04:53:54 l28 kernel: [521052.388171] PGD 0 Mar 16 04:53:54 l28 kernel: [521052.388289] Oops: 0000 [#1] SMP Mar 16 04:53:54 l28 kernel: [521052.388410] Modules linked in: tcm_loop iscsi_target_mod target_core_pscsi target_core_file target_core_iblock target_core_mod ipt_NETFLOW(O) configfs iscsi_tcp libis csi_tcp libiscsi scsi_transport_iscsi fuse Mar 16 04:53:54 l28 kernel: [521052.388913] CPU: 6 PID: 5947 Comm: iscsi_trx Tainted: G O 3.12.51-jl-2015-12-25 #1 Mar 16 04:53:54 l28 kernel: [521052.389125] Hardware name: Intel Corporation S2600IP ........../S2600IP, BIOS SE5C600.86B.01.08.0003.022620131521 02/26/2013 Mar 16 04:53:54 l28 kernel: [521052.389351] task: ffff88081a3a6720 ti: ffff8808162de000 task.ti: ffff8808162de000 Mar 16 04:53:54 l28 kernel: [521052.389566] RIP: 0010:[] [] __kmalloc+0x69/0x100 Mar 16 04:53:54 l28 kernel: [521052.389782] RSP: 0018:ffff8808162dfd18 EFLAGS: 00010286 Mar 16 04:53:54 l28 kernel: [521052.389899] RAX: 0000000000000000 RBX: ffff880819a51800 RCX: 0000000003b305d3 Mar 16 04:53:54 l28 kernel: [521052.390112] RDX: 0000000003b305d2 RSI: 0000000000000000 RDI: 0000000000013500 Mar 16 04:53:54 l28 kernel: [521052.390309] RBP: ffff8808162dfd38 R08: ffff88103fd13500 R09: ffffffffa00e7072 Mar 16 04:53:54 l28 kernel: [521052.390503] R10: 0000000000100000 R11: 0000000000000030 R12: ffff88081f803a00 Mar 16 04:53:54 l28 kernel: [521052.390694] R13: 0000000040008021 R14: 00000000000080d0 R15: ffff8808162dfdd0 Mar 16 04:53:54 l28 kernel: [521052.390888] FS: 0000000000000000(0000) GS:ffff88103fd00000(0000) knlGS:0000000000000000 Mar 16 04:53:54 l28 kernel: [521052.391082] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Mar 16 04:53:54 l28 kernel: [521052.391190] CR2: 0000000040008021 CR3: 000000000180b000 CR4: 00000000000407e0 Mar 16 04:53:54 l28 kernel: [521052.391382] Stack: Mar 16 04:53:54 l28 kernel: [521052.391477] ffff880819a51800 ffff8808162dfe50 ffff880816319cc0 0000000000000006 Mar 16 04:53:54 l28 kernel: [521052.393132] ffff8808162dfeb8 ffffffffa00e7072 ffff8808162dfd78 ffff88103fd10880 Mar 16 04:53:54 l28 kernel: [521052.393345] 0000000000000000 ffff880800000000 0000000000001000 ffff880816318124 Mar 16 04:53:54 l28 kernel: [521052.393585] Call Trace: Mar 16 04:53:54 l28 kernel: [521052.393703] [] iscsi_target_rx_thread+0x592/0xd80 [iscsi_target_mod] Mar 16 04:53:54 l28 kernel: [521052.393909] [] ? __schedule+0x2e0/0x8e0 Mar 16 04:53:54 l28 kernel: [521052.394052] [] ? iscsi_target_tx_thread+0x280/0x280 [iscsi_target_mod] Mar 16 04:53:54 l28 kernel: [521052.394274] [] kthread+0xbb/0xc0 Mar 16 04:53:54 l28 kernel: [521052.394392] [] ? __kthread_parkme+0x80/0x80 Mar 16 04:53:54 l28 kernel: [521052.394508] [] ret_from_fork+0x58/0x90 Mar 16 04:53:54 l28 kernel: [521052.394619] [] ? __kthread_parkme+0x80/0x80 Mar 16 04:53:54 l28 kernel: [521052.394740] Code: 65 4c 03 04 25 48 bc 00 00 49 8b 50 08 4d 8b 28 49 8b 40 10 4d 85 ed 74 6b 48 85 c0 74 66 49 63 44 24 20 48 8d 4a 01 49 8b 3c 24 <49> 8b 5c 05 00 4c 89 e8 65 48 0f c7 0f 0f 94 c0 84 c0 74 bd 49 Mar 16 04:53:54 l28 kernel: [521052.395596] RIP [] __kmalloc+0x69/0x100 Mar 16 04:53:54 l28 kernel: [521052.395721] RSP Mar 16 04:53:54 l28 kernel: [521052.395841] CR2: 0000000040008021 Mar 16 04:53:54 l28 kernel: [521052.396883] ---[ end trace 20dde4e477f78d24 ]--- > > Kernel 3.12.51. Gdb listing: > > (gdb) list *(__kmalloc+0x69) > 0xffffffff810ee519 is in __kmalloc (mm/slub.c:260). > [...] > 258 static inline void *get_freepointer(struct kmem_cache *s, void *object) > 259 { > 260 return *(void **)(object + s->offset); > 261 } > > What whould be the next step? Thank you. -- UNIX/Ocaml engineer at 1Gb.ru. Telegram: johnlepikhin