Received-SPF: pass (google.com: domain of linux-kernel+bounces-180224-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161;
Precedence: bulk
MIME-Version: 1.0
References: <FA5F6719-8824-4B04-803E-82990E65E627@akamai.com>
In-Reply-To: <FA5F6719-8824-4B04-803E-82990E65E627@akamai.com>
From: Ard Biesheuvel <ardb@kernel.org>
Date: Wed, 15 May 2024 19:42:49 +0200
Message-ID: <CAMj1kXE2ZvaKout=nSfv08Hn5yvf8SRGhQeTikZcUeQOmyDgnw@mail.gmail.com>
Subject: Re: Regression in 6.1.81: Missing memory in pmem device
To: "Chaney, Ben" <bchaney@akamai.com>, Kees Cook <keescook@chromium.org>
Cc: "gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>, 
	"linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>, 
	"stable@vger.kernel.org" <stable@vger.kernel.org>, "bp@alien8.de" <bp@alien8.de>, 
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>, 
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, "mingo@redhat.com" <mingo@redhat.com>, 
	"tglx@linutronix.de" <tglx@linutronix.de>, "Tottenham, Max" <mtottenh@akamai.com>, 
	"Hunt, Joshua" <johunt@akamai.com>, "Galaxy, Michael" <mgalaxy@akamai.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

(cc Kees)

On Wed, 15 May 2024 at 19:32, Chaney, Ben <bchaney@akamai.com> wrote:
>
> Hello,
>                 I encountered an issue when upgrading to 6.1.89 from 6.1.=
77. This upgrade caused a breakage in emulated persistent memory. Significa=
nt amounts of memory are missing from a pmem device:
>
> fdisk -l /dev/pmem*
> Disk /dev/pmem0: 355.9 GiB, 382117871616 bytes, 746323968 sectors
> Units: sectors of 1 * 512 =3D 512 bytes
> Sector size (logical/physical): 512 bytes / 4096 bytes
> I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>
> Disk /dev/pmem1: 25.38 GiB, 27246198784 bytes, 53215232 sectors
> Units: sectors of 1 * 512 =3D 512 bytes
> Sector size (logical/physical): 512 bytes / 4096 bytes
> I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>
>         The memmap parameter that created these pmem devices is =E2=80=9C=
memmap=3D364416M!28672M,367488M!419840M=E2=80=9D, which should cause a much=
 larger amount of memory to be allocated to /dev/pmem1. The amount of missi=
ng memory and the device it is missing from is randomized on each reboot. T=
here is some amount of memory missing in almost all cases, but not 100% of =
the time. Notably, the memory that is missing from these devices is not rec=
laimed by the system for general use. This system in question has 768GB of =
memory split evenly across two NUMA nodes.
>
>         When the error occurs, there are also the following error message=
s showing up in dmesg:
>
> [    5.318317] nd_pmem namespace1.0: [mem 0x5c2042c000-0x5ff7ffffff flags=
 0x200] misaligned, unable to map
> [    5.335073] nd_pmem: probe of namespace1.0 failed with error -95
>
>         Bisection implicates 2dfaeac3f38e4e550d215204eedd97a061fdc118 as =
the patch that first caused the issue. I believe the cause of the issue is =
that the EFI stub is randomizing the location of the decompressed kernel wi=
thout accounting for the memory map, and it is clobbering some of the memor=
y that has been reserved for pmem.
>

Does using 'nokaslr' on the kernel command line work around this?

I think in this particular case, we could just disable physical KASLR
(but retain virtual KASLR) if memmap=3D appears on the kernel command
line, on the basis that emulated persistent memory is somewhat of a
niche use case, and physical KASLR is not as important as virtual
KASLR (which shouldn't be implicated in this).