Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Fri, 21 Jun 2019 17:30:52 +0200 (CEST)
From:   Thomas Gleixner <tglx@linutronix.de>
To:     Linus Torvalds <torvalds@linux-foundation.org>
cc:     Chris Wilson <chris@chris-wilson.co.uk>,
        Linux List Kernel Mailing <linux-kernel@vger.kernel.org>,
        Steven Rostedt <rostedt@goodmis.org>,
        Josh Poimboeuf <jpoimboe@redhat.com>,
        Joerg Roedel <joro@8bytes.org>
Subject: Re: NMI hardlock stacktrace deadlock [was Re: Linux 5.2-rc5]
In-Reply-To: <CAHk-=wjoeZ9_aiu+642ur=iGhGjfBQhRPURxX9Py+-B6coctXw@mail.gmail.com>
Message-ID: <alpine.DEB.2.21.1906211725200.5503@nanos.tec.linutronix.de>
References: <CAHk-=whEQPvLpDbx+WR4Q4jf2FxXjf_zTX3uLy_6ZzHtgTV4LA@mail.gmail.com> <156094799629.21217.4574572565333265288@skylake-alporthouse-com> <CAHk-=wjhJNKVfHgwd0QX_bq769sxfP4jvfy0dd-WtFMfdivMwg@mail.gmail.com> <156097197830.664.13418742301997062555@skylake-alporthouse-com>
 <CAHk-=wjoeZ9_aiu+642ur=iGhGjfBQhRPURxX9Py+-B6coctXw@mail.gmail.com>
User-Agent: Alpine 2.21 (DEB 202 2017-01-01)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Wed, 19 Jun 2019, Linus Torvalds wrote:

> On Wed, Jun 19, 2019 at 12:19 PM Chris Wilson <chris@chris-wilson.co.uk> wrote:
> >
> > > Do you have the oops itself at all?
> >
> > An example at
> > https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6310/fi-kbl-x1275/dmesg0.log
> > https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6310/fi-kbl-x1275/boot0.log

The second once contains a lockdep splat which warns about a potential
deadlock in the iommu code.

> > The bug causing the oops is clearly a driver problem. The rc5 fallout
> > just seems to be because of some shrinker changes affecting some object
> > reaping that were unfortunately still active. What perturbed the CI
> > team was the machine failed to panic & reboot.
> 
> Hmm. It's hard to guess at the cause of that. The oopses themselves
> don't look like they are happening in any particularly bad context, so
> all the normal reboot-on-oops etc stuff _should_ work.
> 
> So it would help a lot if you could bisect the bad problem at least a
> bit, if it is at all reproducible. Because with no other clues, it's
> hard to even guess at what might be up.
> 
> The fact that you say "NMI watchdog firing as we dumped the ftrace"
> means that maybe it might be some ftrace / stacktrace issue where the
> dumping itself leads to some endless loop, but who knows.
> 
> For example, one thing that has happened during this development cycle
> is the stacktrace common infrastructure changes (arch_stack_walk() and
> friends). I'm, not seeing why that would cause your issues, but I'm
> adding a few random people for ftrace / stacktrace changes.

/me whistels innocently.

Chris, do you have the actual NMI lockup detector splats somewhere?

Thanks,

	tglx