Received-SPF: pass (google.com: domain of linux-kernel+bounces-171905-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Precedence: bulk
MIME-Version: 1.0
References: <20231214223423.1133074-1-yang@os.amperecomputing.com>
 <1e8f5ac7-54ce-433a-ae53-81522b2320e1@arm.com> <Zav3UK7ESNxCMjyP@casper.infradead.org>
 <b75cb59a-734f-43d5-b565-fc9bb8c5ed05@arm.com> <CAHbLzkpT6padaDo8GimCcQReSGybQn_ntzj+wsZbTXe3urtK-g@mail.gmail.com>
 <bad7ec4a-1507-4ec4-996a-ea29d07d47a0@arm.com> <CAHbLzkrtcsU=pW13AyAMvF72A03fUV5iFcM0HwQoEemeajtqxg@mail.gmail.com>
 <b84e2799-2b6c-4670-b017-3a04ec18c0f2@arm.com> <dea802da-2e5e-4c91-b817-43afdde68958@huawei.com>
 <1dc9a561-55f7-4d65-8b86-8a40fa0e84f9@arm.com> <6016c0e9-b567-4205-8368-1f1c76184a28@huawei.com>
 <2c14d9ad-c5a3-4f29-a6eb-633cdf3a5e9e@redhat.com> <da24d8bb-3723-48fa-86f4-8b24457d3556@huawei.com>
 <2b403705-a03c-4cfe-8d95-b38dd83fca52@arm.com>
In-Reply-To: <2b403705-a03c-4cfe-8d95-b38dd83fca52@arm.com>
From: Yang Shi <shy828301@gmail.com>
Date: Tue, 7 May 2024 10:17:32 -0700
Message-ID: <CAHbLzkq9BQFfpjxG_ozrgzWOO3XJmtre-mgY03McY6guVn7U9g@mail.gmail.com>
Subject: Re: [RESEND PATCH] mm: align larger anonymous mappings on THP boundaries
To: Ryan Roberts <ryan.roberts@arm.com>
Cc: Kefeng Wang <wangkefeng.wang@huawei.com>, David Hildenbrand <david@redhat.com>, 
	Matthew Wilcox <willy@infradead.org>, Yang Shi <yang@os.amperecomputing.com>, riel@surriel.com, 
	cl@linux.com, akpm@linux-foundation.org, linux-kernel@vger.kernel.org, 
	linux-mm@kvack.org, Ze Zuo <zuoze1@huawei.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, May 7, 2024 at 8:53=E2=80=AFAM Ryan Roberts <ryan.roberts@arm.com> =
wrote:
>
> On 07/05/2024 14:53, Kefeng Wang wrote:
> >
> >
> > On 2024/5/7 19:13, David Hildenbrand wrote:
> >>
> >>> https://github.com/intel/lmbench/blob/master/src/lat_mem_rd.c#L95
> >>>
> >>>> suggest. If you want to try something semi-randomly; it might be use=
ful to rule
> >>>> out the arm64 contpte feature. I don't see how that would be interac=
ting
> >>>> here if
> >>>> mTHP is disabled (is it?). But its new for 6.9 and arm64 only. Disab=
le with
> >>>> ARM64_CONTPTE (needs EXPERT) at compile time.
> >>> I don't enabled mTHP, so it should be not related about ARM64_CONTPTE=
,
> >>> but will have a try.
> >
> > After ARM64_CONTPTE disabled, memory read latency is similar with ARM64=
_CONTPTE
> > enabled(default 6.9-rc7), still larger than align anon reverted.
>
> OK thanks for trying.
>
> Looking at the source for lmbench, its malloc'ing (512M + 8K) up front an=
d using
> that for all sizes. That will presumably be considered "large" by malloc =
and
> will be allocated using mmap. So with the patch, it will be 2M aligned. W=
ithout
> it, it probably won't. I'm still struggling to understand why not alignin=
g it in
> virtual space would make it more performant though...

Yeah, I'm confused too.

I just ran the same command on 6.6.13 (w/o the thp alignment patch and
mTHP stuff) and 6.9-rc4 (w/ the thp alignment patch and all mTHP
stuff) on my arm64 machine, but I didn't see such a pattern.

The result has a little bit fluctuation, for example, 6.6.13 has
better result with 4M/6M/8M, but 6.9-rc4 has better result for
12M/16M/32M/48M/64M, and the difference may be quite noticeable. But
anyway I didn't see such a regression pattern.

The benchmark is supposed to measure cache and memory latency, its
result strongly relies on the cache and memory subsystem, for example,
hw prefetcher, etc.

>
> Is it possible to provide the smaps output for at least that 512M+8K bloc=
k for
> both cases? It might give a bit of a clue.
>
> Do you have traditional (PMD-sized) THP enabled? If its enabled and unali=
gned
> then the front of the buffer wouldn't be mapped with THP, but if it is al=
igned,
> it will. That could affect it.
>
> >
> >>
> >> cont-pte can get active if we're just lucky when allocating pages in t=
he right
> >> order, correct Ryan?
> >>
>