Received-SPF: pass (google.com: domain of linux-kernel+bounces-99761-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Feedback-ID: ieff94742:Fastmail
User-Agent: Cyrus-JMAP/3.11.0-alpha0-251-g8332da0bf6-fm-20240305.001-g8332da0b
Precedence: bulk
MIME-Version: 1.0
Message-Id: <0645946c-f4f5-43b1-a9a0-03ed139036b3@app.fastmail.com>
In-Reply-To: <D4D747C5-2B0E-44B0-A550-BA05BE0AF2FA@zytor.com>
References: <20240311164638.2015063-1-pasha.tatashin@soleen.com>
 <20240311164638.2015063-12-pasha.tatashin@soleen.com>
 <3e180c07-53db-4acb-a75c-1a33447d81af@app.fastmail.com>
 <bedfa55b-b1d0-4e59-8c94-dbc5f8485a7f@intel.com>
 <ef0419dd-9d7b-4b77-b63a-5f11aaefb570@app.fastmail.com>
 <08EFDEDB-7BBB-4D9C-B7E5-D7370EC609BE@gmail.com>
 <dd67a2b9-35ef-4cf8-b303-9e6b8692b390@intel.com>
 <D4D747C5-2B0E-44B0-A550-BA05BE0AF2FA@zytor.com>
Date: Mon, 11 Mar 2024 19:16:38 -0700
From: "Andy Lutomirski" <luto@kernel.org>
To: "H. Peter Anvin" <hpa@zytor.com>, "Dave Hansen" <dave.hansen@intel.com>,
 "Nadav Amit" <nadav.amit@gmail.com>
Cc: "Pasha Tatashin" <pasha.tatashin@soleen.com>,
 "Linux Kernel Mailing List" <linux-kernel@vger.kernel.org>,
 linux-mm <linux-mm@kvack.org>,
 "Andrew Morton" <akpm@linux-foundation.org>,
 "the arch/x86 maintainers" <x86@kernel.org>,
 "Borislav Petkov" <bp@alien8.de>,
 "Christian Brauner" <brauner@kernel.org>, bristot@redhat.com,
 "Ben Segall" <bsegall@google.com>,
 "Dave Hansen" <dave.hansen@linux.intel.com>, dianders@chromium.org,
 dietmar.eggemann@arm.com, eric.devolder@oracle.com, hca@linux.ibm.com,
 "hch@infradead.org" <hch@infradead.org>,
 "Jacob Pan" <jacob.jun.pan@linux.intel.com>,
 "Jason Gunthorpe" <jgg@ziepe.ca>, jpoimboe@kernel.org,
 "Joerg Roedel" <jroedel@suse.de>, juri.lelli@redhat.com,
 "Kent Overstreet" <kent.overstreet@linux.dev>, kinseyho@google.com,
 "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>,
 lstoakes@gmail.com, mgorman@suse.de, mic@digikod.net,
 michael.christie@oracle.com, "Ingo Molnar" <mingo@redhat.com>,
 mjguzik@gmail.com, "Michael S. Tsirkin" <mst@redhat.com>,
 "Nicholas Piggin" <npiggin@gmail.com>,
 "Peter Zijlstra (Intel)" <peterz@infradead.org>,
 "Petr Mladek" <pmladek@suse.com>,
 "Rick P Edgecombe" <rick.p.edgecombe@intel.com>,
 "Steven Rostedt" <rostedt@goodmis.org>,
 "Suren Baghdasaryan" <surenb@google.com>,
 "Thomas Gleixner" <tglx@linutronix.de>,
 "Uladzislau Rezki" <urezki@gmail.com>, vincent.guittot@linaro.org,
 vschneid@redhat.com
Subject: Re: [RFC 11/14] x86: add support for Dynamic Kernel Stacks
Content-Type: text/plain;charset=utf-8
Content-Transfer-Encoding: quoted-printable

On Mon, Mar 11, 2024, at 6:25 PM, H. Peter Anvin wrote:
> On March 11, 2024 5:53:33 PM PDT, Dave Hansen <dave.hansen@intel.com> =
wrote:
>>On 3/11/24 16:56, Nadav Amit wrote:
>>> So you can look on the dirty-bit, which is not being set
>>> speculatively and save yourself one problem.
>>Define "set speculatively". :)
>>
>>> If software on one logical processor writes to a page while software
>>> on another logical processor concurrently clears the R/W flag in the
>>> paging-structure entry that maps the page, execution on some
>>> processors may result in the entry=E2=80=99s dirty flag being set (d=
ue to the
>>> write on the first logical processor) and the entry=E2=80=99s R/W fl=
ag being
>>> clear (due to the update to the entry on the second logical
>>> processor).
>>
>>In other words, you'll see both a fault *AND* the dirty bit.  The write
>>never retired and the dirty bit is set.
>>
>>Does that count as being set speculatively?
>>
>>That's just the behavior that the SDM explicitly admits to.
>
> Indeed; both the A and D bits are by design permissive; that is, the=20
> hardware can set them at any time.
>
> The only guarantees are:
>
> 1. The hardware will not set the A bit on a not present late, nor the =
D=20
> bit on a read only page.

Wait a sec.  What about setting the D bit on a not-present page?

I always assumed that the actual intended purpose of the D bit was for t=
hings like file mapping.  Imagine an alternate universe in which Linux u=
sed hardware dirty tracking instead of relying on do_wp_page, etc.

mmap(..., MAP_SHARED): PTE is created, read-write, clean

user program may or may not write to the page.

Now either munmap is called or the kernel needs to reclaim memory.  So t=
he kernel checks if the page is dirty and, if so, writes it back, and th=
en unmaps it.

Now some silly people invented SMP, so this needs an atomic operation:  =
xchg the PTE to all-zeros, see if the dirty bit is set, and, if itt's se=
t, write back the page.  Otherwise discard it.

Does this really not work on Intel CPU?