Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <CAFULd4b=5-=WfF9OPCX+H9VDnsgbN7OBFj-XP=MZ0QqF5WpvQA@mail.gmail.com>
 <alpine.DEB.2.21.1907151020320.1669@nanos.tec.linutronix.de>
In-Reply-To: <alpine.DEB.2.21.1907151020320.1669@nanos.tec.linutronix.de>
From:   Uros Bizjak <ubizjak@gmail.com>
Date:   Mon, 15 Jul 2019 14:04:15 +0200
Message-ID: <CAFULd4aHwv9+dkTtNLtNGHu=wmt62cZDkjr-vbCVzYftJbOpEg@mail.gmail.com>
Subject: Re: [RFC PATCH, x86]: Disable CPA cache flush for selfsnoop targets
To:     Thomas Gleixner <tglx@linutronix.de>
Cc:     LKML <linux-kernel@vger.kernel.org>, X86 ML <x86@kernel.org>,
        Andrew Lutomirski <luto@kernel.org>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Mon, Jul 15, 2019 at 10:24 AM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> Uros,
>
> On Thu, 11 Jul 2019, Uros Bizjak wrote:
> > Recent patch [1] disabled a self-snoop feature on a list of processor
> > models with a known errata, so we are confident that the feature
> > should work on remaining models also for other purposes than to speed
> > up MTRR programming.
> >
> > I would like to resurrect an old patch [2] that avoids calling clflush
> > and wbinvd
> > to invalidate caches when CPU supports selfsnoop.
>
> Please do not attach patches, send them inline and please add a proper
> changelog. Just saying 'Disable CPA cache flush for selfsnoop targets' in
> the subject line then nada gives absolutely zero information.

Thanks for your remarks and instructions!

I'll send a new revision of the patch with expanded ChangeLog later today,
saying something along the lines of:

"CPUs which have self-snooping capability can handle conflicting
memory type across CPUs by snooping its own cache. Commit #fd329f276ecaa
("x86/mtrr: Skip cache flushes on CPUs with cache self-snooping")
avoids cache flushes when MTRR registers are programmed. The Page
Attribute Table (PAT) is a companion feature to the MTRRs, and according
to section 11.12.4 of the Intel 64 and IA 32 Architectures Software
Developer's Manual, if the CPU supports cache self-snooping, it is not
necessary to flush caches when remapping a page that was previously
mapped as a different memory type.

Note that commit #1e03bff360010
("x86/cpu/intel: Clear cache self-snoop capability in CPUs with known errata")
cleared cache self-snoop capability for CPUs where conflicting memory types
lead to unpredictable behavior, machine check errors, or hangs."

> > The patch was ported to latest Fedora kernel (5.1.16) and tested with
> > CONFIG_CPA_DEBUG on INTEL_FAM6_IVYBRIDGE_X. The relevant ports of
> > dmesg show:
> >
> > ...
> > < hundreds of CPA protect messages, resulting from set_memory_rw CPA
> > undo test in mm/init_64.c >
> > CPA  protect  Rodata RO: 0xffffffffbd1fe000 - 0xffffffffbd1fefff PFN
> > 1461fe req 8000000000000063 prevent 0000000000000002
> > CPA  protect  Rodata RO: 0xffff889c461fe000 - 0xffff889c461fefff PFN
> > 1461fe req 8000000000000063 prevent 0000000000000002
> > Testing CPA: again
> > Freeing unused kernel image memory: 2016K
> > Freeing unused kernel image memory: 4K
> > x86/mm: Checked W+X mappings: passed, no W+X pages found.
> > rodata_test: all tests were successful
> > x86/mm: Checking user space page tables
> > x86/mm: Checked W+X mappings: passed, no W+X pages found.
> >
> > and from CPA selftest:
> >
> > CPA self-test:
> >  4k 36352 large 4021 gb 0 x 81[ffff889b00098000-ffff889bdf7ff000] miss 133120
> >  4k 180224 large 3740 gb 0 x 81[ffff889b00098000-ffff889bdf7ff000] miss 133120
> >  4k 180224 large 3740 gb 0 x 81[ffff889b00098000-ffff889bdf7ff000] miss 133120
> > ok.
>
> These outputs are pretty useless simply because the selftest only verifies
> the inner workings of CPA itself, but has nothing to do with the
> correctness vs. cache flushing.

Please note that CONFIG_CPA_DEBUG also spawns a pageattr-test kthread
which remaps a memory page every 30 seconds. I was confident enough to
run the patched kernel (with CONFIG_CPA_DEBUG) on my main workstation
(Ivybridge-X, Fedora 30), already for a week without a single problem.

Uros.