MIME-Version: 1.0
In-Reply-To: <CALCETrWTh6YccfBgiTNE5C0FWBegH9LCqUTaj88s81LgUn1MsQ@mail.gmail.com>
References: <alpine.DEB.2.20.1709080826110.1888@nanos> <20170908080536.ninspvplibd37fj2@pd.tnic>
 <20170908091614.nmdxjnukxowlsjja@pd.tnic> <20170908094815.GA278@x4>
 <20170908103513.npjmb2kcjt2zljb2@gmail.com> <20170908103906.GB278@x4>
 <20170908113039.GA285@x4> <CALCETrXkWrQjSqpTQw_vJwN_hRp6eWXEOcibM+57NmC+HWxGNg@mail.gmail.com>
 <20170908171633.GA279@x4> <CALCETrX6mFHWa1oZxS7DQUtnbqyrRRg53bvgGFv19ka8Sy_KcA@mail.gmail.com>
 <20170908215656.qw66lgfsfgpoqrdm@pd.tnic> <CALCETrUYGv_8GmWpRanYOuJfRfBijCMwMfCtUunKcZ3B0=8M8g@mail.gmail.com>
 <CA+55aFx91B5_eBvuWm2=SGmn_NOTRmooR=cKoP6Nj-DLeM2PMA@mail.gmail.com> <CALCETrWTh6YccfBgiTNE5C0FWBegH9LCqUTaj88s81LgUn1MsQ@mail.gmail.com>
From: Linus Torvalds <torvalds@linux-foundation.org>
Date: Fri, 8 Sep 2017 18:05:29 -0700
Message-ID: <CA+55aFzf1C+WXRtfkdwAbb0kniYJfkfm=rghouM3u8x7-ZJGMg@mail.gmail.com>
Subject: Re: Current mainline git (24e700e291d52bd2) hangs when building e.g. perf
To: Andy Lutomirski <luto@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>,
        Markus Trippelsdorf <markus@trippelsdorf.de>,
        Ingo Molnar <mingo@kernel.org>, Thomas Gleixner <tglx@linutronix.de>,
        Peter Zijlstra <peterz@infradead.org>,
        LKML <linux-kernel@vger.kernel.org>, Ingo Molnar <mingo@redhat.com>,
        Tom Lendacky <thomas.lendacky@amd.com>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 1838
Lines: 41

On Fri, Sep 8, 2017 at 5:00 PM, Andy Lutomirski <luto@kernel.org> wrote:
>
> I'm not convinced.  The SDM says (Vol 3, 11.3, under WC):
>
> If the WC buffer is partially filled, the writes may be delayed until
> the next occurrence of a serializing event; such as, an SFENCE or
> MFENCE instruction, CPUID execution, a read or write to uncached
> memory, an interrupt occurrence, or a LOCK instruction execution.
>
> Thanks, Intel, for definiing "serializing event" differently here than
> anywhere else in the whole manual.

Yeah, it's really badly defined. Ok, maybe a locked instruction does
actually wait for it.. It should be invisible to anything, regardless.

> 1. The kernel wants to reclaim a page of normal memory, so it unmaps
> it and flushes.  Another CPU has an entry for that page in its WC
> buffer.  I don't think we care whether the flush causes the WC write
> to really hit RAM because it's unobservable -- we just need to make
> sure it is ordered, as seen by software, before the flush operation
> completes.  From the quote above, I think we're okay here.

Agreed.

> 2. The kernel is unmapping some IO memory (e.g. a GPU command buffer).
> It wants a guarantee that, when flush_tlb_mm_range returns, all CPUs
> are really done writing to it.  Here I'm less convinced.  The SDM
> quote certainly suggests to me that we have a promise that the WC
> write has *started* before flush_tlb_mm_range returns, but I'm not
> sure I believe that it's guaranteed to have retired.

If others have writable TLB entries, what keeps them from just
continuing to write for a long time afterwards?

> I'd prefer to leave it as is except on the buggy AMD CPUs, though,
> since the current code is nice and fast.

So is there a patch to detect the 383 erratum and serialize for those?
I may have missed that part.

              Linus