Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Tue, 17 Apr 2018 14:12:07 +0200
From:   Jan Kara <jack@suse.cz>
To:     Pavlos Parissis <pavlos.parissis@gmail.com>
Cc:     Jan Kara <jack@suse.cz>, Guillaume Morin <guillaume@morinfr.org>,
        stable@vger.kernel.org, decui@microsoft.com, jack@suse.com,
        linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org,
        mszeredi@redhat.com
Subject: Re: kernel panics with 4.14.X versions
Message-ID: <20180417121207.cs7eijrndovbplgz@quack2.suse.cz>
References: <20180416132550.d25jtdntdvpy55l3@bender.morinfr.org>
 <20180416144041.t2mt7ugzwqr56ka3@quack2.suse.cz>
 <9b11cfba-4bdc-8a3e-cd33-2f7e8d513bdf@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <9b11cfba-4bdc-8a3e-cd33-2f7e8d513bdf@gmail.com>
User-Agent: NeoMutt/20170421 (1.8.2)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Tue 17-04-18 01:31:24, Pavlos Parissis wrote:
> On 16/04/2018 04:40 μμ, Jan Kara wrote:

<snip>

> > How easily can you hit this?
> 
> Very easily, I only need to wait 1-2 days for a crash to occur.

I wouldn't call that very easily but opinions may differ :). Anyway it's
good (at least for debugging) that it's reproducible.

> > Are you able to run debug kernels
> 
> Well, I was under the impression I do as I have:
>   grep -E 'DEBUG_KERNEL|DEBUG_INFO' /boot/config-4.14.32-1.el7.x86_64
>   CONFIG_DEBUG_INFO=y
>   # CONFIG_DEBUG_INFO_REDUCED is not set
>   # CONFIG_DEBUG_INFO_SPLIT is not set
>   # CONFIG_DEBUG_INFO_DWARF4 is not set
>   CONFIG_DEBUG_KERNEL=y
> 
> Do you think that my kernel doesn't produce a proper crash dump?
> I have a production cluster where I can run any kernel we need, so if I need
> to compile again with different settings I can certainly do that.

OK, good. So please try running 4.16 as you mention below to verify whether
this is just a -stable regression or also a problem in the current upstream
kernel. Based on your results with 4.16 I'll prepare a debug patch for you to
apply on top of 4.14.32 so that we can debug this further.

> > / inspect
> > crash dumps when the issue occurs?
> 
> I can't do that as the server isn't responsive and I can only power cycle it.

Well, kernel crash dumps work in that situation as well - when the kernel
panics, it will kexec into a new kernel and dump memory of the old kernel
to disk. It can then be investigated with the 'crash' utility. But
obviously you don't have this set up and don't have experience with this so
let's go via a standard 'debug patch' route.

> > Also testing with the latest mainline
> > kernel (4.16) would be welcome whether this isn't just an issue with the
> > backport of fsnotify fixes from Miklos.
> 
> I can try the kernel-ml-4.16.2 from elrepo (we use CentOS 7).

Yes, that would be good.

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR