Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S933303AbeAHVYZ (ORCPT <rfc822;ralf@linux-mips.org> + 1 other);
        Mon, 8 Jan 2018 16:24:25 -0500
Received: from aserp2120.oracle.com ([141.146.126.78]:41692 "EHLO
        aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
        with ESMTP id S933242AbeAHVYX (ORCPT
        <rfc822;linux-kernel@vger.kernel.org>);
        Mon, 8 Jan 2018 16:24:23 -0500
X-Google-Smtp-Source: ACJfBouCP8DZim/TgHNNWhbZaV9bEtM1OasKTj+MNa59K0jBJ2LVT8OwqDm76rNt3zQQ7Z/B1LIfBOvesGxMX+UOcm4=
MIME-Version: 1.0
In-Reply-To: <CAOAebxu_gGE5chc-Mn=zBo_M3ojExvooNfvFRWOT=FygoffjLQ@mail.gmail.com>
References: <20180105175229.GA29834@kroah.com> <CALCETrW3hfkeUAPzFhGdE56vn23Q462h2reMaT=3ftbNGeSb3Q@mail.gmail.com>
 <CAOAebxvbXrVV7-x6LdEsAEhjR=qrxNBxxBrWzk7RaBECcWqsiw@mail.gmail.com>
 <CAOAebxv47m34R1qteFaQB1-x4jsUVJPPvPLVe4jxRGZeFUY2qA@mail.gmail.com>
 <CAOAebxuqK4TOktLrHNLdW-S3oSxVO2og84js7N5u_QDC7mE2Ow@mail.gmail.com>
 <20180105204557.GA8839@kroah.com> <a93f30d1-e00e-419b-532d-203be16e2299@oracle.com>
 <20180107104540.GB14783@kroah.com> <CAOAebxsBVTyS2gSXh4c-g1b8Lvjv9MBG3Wjr_AC+LJpMLHRqEA@mail.gmail.com>
 <CAOAebxsP1LLV-Eg_h8brbxfsn_tpRDR46yrVZeokrFt3ArRuMA@mail.gmail.com>
 <20180108074645.GA24062@kroah.com> <CAOAebxu_gGE5chc-Mn=zBo_M3ojExvooNfvFRWOT=FygoffjLQ@mail.gmail.com>
From: Pavel Tatashin <pasha.tatashin@oracle.com>
Date: Mon, 8 Jan 2018 16:24:18 -0500
X-Gmail-Original-Message-ID: <CAOAebxuecMGOVe4zfNgk+t3mXVnDaWVA-GLdb+HiX3t9LNnxLg@mail.gmail.com>
Message-ID: <CAOAebxuecMGOVe4zfNgk+t3mXVnDaWVA-GLdb+HiX3t9LNnxLg@mail.gmail.com>
Subject: Re: [PATCH 4.4 00/37] 4.4.110-stable review
To: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Andy Lutomirski <luto@kernel.org>, Hugh Dickins <hughd@google.com>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        Thomas Voegtle <tv@lio96.de>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        Guenter Roeck <linux@roeck-us.net>,
        Shuah Khan <shuahkh@osg.samsung.com>, patches@kernelci.org,
        Ben Hutchings <ben.hutchings@codethink.co.uk>,
        lkft-triage@lists.linaro.org, stable <stable@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8768 signatures=668652
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=1 malwarescore=0
 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999
 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.0.1-1711220000 definitions=main-1801080303
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
Return-Path: <linux-kernel-owner@vger.kernel.org>

Here is one more:

[    6.284763] EFI Variables Facility v0.08 2004-May-17
[    6.555990] ------------[ cut here ]------------
[    6.561145] kernel BUG at
/scratch/ptatashi/linux-stable/mm/slub.c:3627!
[    6.568625] invalid opcode: 0000 [#1] SMP
[    6.573219] Modules linked in:
[    6.576639] CPU: 1 PID: 364 Comm: kworker/1:1 Not tainted
4.4.110_pt_stable #3
[    6.584692] Hardware name: Oracle Corporation ORACLE SERVER
X6-2/ASM,MOTHERBOARD,1U, BIOS 38050100 08/30/2016
[    6.595766] Workqueue: events clocksource_watchdog_work
[    6.601611] task: ffff881fecd82b00 ti: ffff881fecda4000 task.ti:
ffff881fecda4000
[    6.609963] RIP: 0010:[<ffffffff811e704a>]  [<ffffffff811e704a>]
kfree+0x14a/0x150
[    6.618419] RSP: 0000:ffff881fecda7d40  EFLAGS: 00010246
[    6.624348] RAX: ffffffff8106c280 RBX: ffff883ff114bfc0 RCX:
00000000ffffffd8
[    6.632314] RDX: 000077ff80000000 RSI: 0000000000000246 RDI:
ffff883ff114bfc0
[    6.640280] RBP: ffff881fecda7d58 R08: 0000000000000000 R09:
ffff881fff917300
[    6.648244] R10: 0000000000000000 R11: ffffea00ffc452c0 R12:
ffff883fec2f4080
[    6.656208] R13: ffffffff810a5bee R14: 00000000ffffffff R15:
0000000000000000
[    6.664175] FS:  0000000000000000(0000) GS:ffff881fff840000(0000)
knlGS:0000000000000000
[    6.673208] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    6.679623] CR2: 0000000000000000 CR3: 0000000001aa2000 CR4:
0000000000360670
[    6.687587] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[    6.695553] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7:
0000000000000400
[    6.703516] Stack:
[    6.705759]  ffff883ff114bfc0 ffff883fec2f4080 ffffffff819a26e8
ffff881fecda7e00
[    6.714061]  ffffffff810a5bee ffff881f00000020 ffff881fecda7e10
ffff881fecda7da8
[    6.722363]  ffffffff00000000 ffff881f00000000 ffff881fecda7d90
ffff881fecda7d90
[    6.730666] Call Trace:
[    6.733400]  [<ffffffff810a5bee>]
kthread_create_on_node+0x14e/0x1a0
[    6.740495]  [<ffffffff810f9dd5>]
clocksource_watchdog_work+0x25/0x40
[    6.747679]  [<ffffffff8109ef6f>] process_one_work+0x14f/0x400
[    6.754181]  [<ffffffff8109fbc4>] worker_thread+0x114/0x480
[    6.760402]  [<ffffffff8109fab0>] ? rescuer_thread+0x310/0x310
[    6.766913]  [<ffffffff810a56b5>] kthread+0xe5/0x100
[    6.772456]  [<ffffffff810a55d0>] ? kthread_park+0x60/0x60
[    6.778580]  [<ffffffff8170fa0f>] ret_from_fork+0x3f/0x70
[    6.784608]  [<ffffffff810a55d0>] ? kthread_park+0x60/0x60
[    6.790721] Code: 8b 03 31 f6 f6 c4 40 74 04 41 8b 73 6c 4c 89 df
e8 1c a8 fa ff e9 73 ff ff ff 4c 8d 58 ff e9 20 ff ff ff 49 8b 43 20
a8 01 75 d4 <0f> 0b 0f 1f 40 00 0f 1f 44 00 00 55 48 89 e5 41 57 41 56
41 55
[    6.812429] RIP  [<ffffffff811e704a>] kfree+0x14a/0x150
[    6.818273]  RSP <ffff881fecda7d40>
[    6.822177] ---[ end trace 4ce44d21c6d68eed ]---

On Mon, Jan 8, 2018 at 3:38 PM, Pavel Tatashin
<pasha.tatashin@oracle.com> wrote:
> Hi Greg,
>
>
>
> On Mon, Jan 8, 2018 at 2:46 AM, Greg Kroah-Hartman
> <gregkh@linuxfoundation.org> wrote:
>> On Sun, Jan 07, 2018 at 10:06:59AM -0500, Pavel Tatashin wrote:
>>> Hi Greg,
>>>
>>> I reverted suse12 back to:
>>> 13dae54cb229d078635f159dd8afe16ae683980b
>>> x86/kaiser: Move feature detection up (bsc#1068032).
>>>
>>> And, still do not see the problem. So, whatever fixes the issue comes
>>> before kaiser.
>>
>> Ok, thanks for the hint.
>>
>> As I can't duplicate this here at all, any specifics as to what
>> hardware/procesor type this is?
>>
>
> BIOS:
> Version 2.17.1249. Copyright (C) 2016 American Megatrends, Inc.
> BIOS Date: 08/30/2016 10:35:36 Ver: 38050100
>
> ca-ostest442:linux-stable$ lscpu
> Architecture:          x86_64
> CPU op-mode(s):        32-bit, 64-bit
> Byte Order:            Little Endian
> CPU(s):                40
> On-line CPU(s) list:   0-39
> Thread(s) per core:    2
> Core(s) per socket:    10
> Socket(s):             2
> NUMA node(s):          2
> Vendor ID:             GenuineIntel
> CPU family:            6
> Model:                 79
> Model name:            Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz
> Stepping:              1
> CPU MHz:               1738.601
> BogoMIPS:              4396.18
> Virtualization:        VT-x
> L1d cache:             32K
> L1i cache:             32K
> L2 cache:              256K
> L3 cache:              25600K
> NUMA node0 CPU(s):     0-9,20-29
> NUMA node1 CPU(s):     10-19,30-39
>
> Note, if I boot with nr_cpus=1, hang never happens, with nr_cpus=4
> happens but seldomly, and with all 40 CPUs happens on almost every
> reboot.
>
> As Hugh Dickins suggested, I am going to show panic outputs, as I get
> them. Here is one more panic (note output is not complete because
> machine reboots):
>
> [    6.276456] EFI Variables Facility v0.08 2004-May-17
> [    6.384665] BUG: unable to handle kernel paging request at
> ffff901fff5a6000
> [    6.392461] IP: [<ffffffff8106bb08>] vmalloc_fault+0x1f8/0x340
> [    6.398987] PGD 0
> [    6.401242] Oops: 0000 [#1] SMP
> [    6.404866] Modules linked in:
> [    6.408287] CPU: 10 PID: 0 Comm: swapper/10 Not tainted
> 4.4.110_pt_stable #2
> [    6.416156] Hardware name: Oracle Corporation ORACLE SERVER
> X6-2/ASM,MOTHERBOARD,1U, BIOS 3
> 8050100 08/30/2016
> [    6.427226] task: ffff883ff1e28000 ti: ffff883ff1e24000 task.ti:
> ffff883ff1e24000
> [    6.435580] RIP: 0010:[<ffffffff8106bb08>]  [<ffffffff8106bb08>]
> vmalloc_fault+0x1f8/0x340
> [    6.444819] RSP: 0000:ffff883ff1e27cc0  EFLAGS: 00010086
> [    6.450749] RAX: ffff881fff5a6058 RBX: 00003ffffffff000 RCX:
> 0000081fff5a6000
> [    6.458714] RDX: ffff880000000000 RSI: ffff901fff5a6000 RDI:
> 0000000000000000
> [    6.466681] RBP: ffff883ff1e27cf0 R08: 0000000000000018 R09:
> 000000000002d2de
> [    6.474647] R10: 0000000000032ef3 R11: 0000000000002e04 R12:
> ffffc900000000f0
> [    6.482615] R13: ffff880000000000 R14: ffff901fff5a6000 R15:
> ffff881fff5a6000
> [    6.490574] FS:  0000000000000000(0000) GS:ffff88407e600000(0000)
> knlGS:0000000000000000
> [    6.499607] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [    6.506022] CR2: ffff901fff5a6000 CR3: 0000000001aa2000 CR4:
> 0000000000360670
> [    6.513989] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
> 0000000000000000
> [    6.521956] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7:
> 0000000000000400
> [    6.529923] Stack:
> [    6.532169]  ffff881fff5a6000[    6.532405] ------------[ cut here
> ]------------
> [    6.532414] WARNING: CPU: 22 PID: 162