From:   William Kucharski <william.kucharski@oracle.com>
To:     Hillf Danton <hdanton@sina.com>
CC:     David Hildenbrand <david@redhat.com>,
        Zach O'Keefe <zokeefe@google.com>,
        Mike Kravetz <mike.kravetz@oracle.com>,
        Peter Xu <peterx@redhat.com>, Rik van Riel <riel@surriel.com>,
        Mike Rapoport <rppt@kernel.org>, Linux-MM <linux-mm@kvack.org>,
        LKML <linux-kernel@vger.kernel.org>
Subject: Re: THP backed thread stacks
Thread-Topic: THP backed thread stacks
Thread-Index: AQHZUId6EUNxRdiQ4UKEGoIYcgiIL67zHhkAgAAI0ICAABpxgIAAo6sAgAGi3ACAANHVgIAAPmGA
Date:   Sun, 12 Mar 2023 04:39:05 +0000
Message-ID: <3A5D4B5D-FF0A-4AB5-8E86-24893FE82A9A@oracle.com>
References: <20230306235730.GA31451@monkey> <ZAaCISgq4A/GnkCk@x1n>
 <20230307004049.GC4956@monkey> <20230308190206.GA4005@monkey>
 <CAAa6QmSdTo77dP2q2nU==C_2PdyF611+PVF32uPOTUQbp1kc9Q@mail.gmail.com>
 <20230309233340.GC3700@monkey>
 <CAAa6QmSiuFF6Oe0-j+eD0ma2tZAbhZuwENDYSZQSBrh1oeaLdA@mail.gmail.com>
 <9F855331-33B2-4366-9375-988B0D3DAC98@oracle.com>
 <9f44de08-b484-baa7-80c8-0a02a7abb717@redhat.com>
 <20230312005549.2609-1-hdanton@sina.com>
In-Reply-To: <20230312005549.2609-1-hdanton@sina.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?XcHwdfF7ML8p/TLkTuK1GwSEBYwFf54WP6TGIE4DLtzjq8xojXkOoaGtgiGX?=
 =?us-ascii?Q?4b+SRdBU/kOYqthEKcIc+lori1G7GFFVXy6SU6g4ob9iKl+tfGjQbiAegDOa?=
 =?us-ascii?Q?ASv08k30p/tFvEBAKfLkXWKclFndsXRVzcTRM0LwPRw45cx51g9+1d+iaK8B?=
 =?us-ascii?Q?Pz+4kMbdgoOqrTuuiNmiJKbl9eh0YPNjTQhhTjEHBGlzDAIv+LENbYjglWZt?=
 =?us-ascii?Q?VePUzlsWonGmpzDt3iUlwAn106Hru2SG66l4ua9gNYmSHrAm30HleNz05MWp?=
 =?us-ascii?Q?XgN1FPlY7mepFBzaSfe+rxqvvpnr62kDBEWp1+5EF4HCpko4ZWy+CkvMFsja?=
 =?us-ascii?Q?rKR4FS7HuadyQaPqBO8ikeVzmiu91cHqWghzMCH8TLTEyWvK+8GD0g/H6ydd?=
 =?us-ascii?Q?pE9Pz7V8pWZjsqXHHeg3D1qQdPmVig1mwXrNNnre1ZqaLrNU6wTas8p2clcc?=
 =?us-ascii?Q?iRot117uo5dVLt7wFwP0zUiQgTaR6eVWsVJf76y7eVZFimyW5XyEW6UOTMJ1?=
 =?us-ascii?Q?mx5pFegCFmwBevEk+cehz/iqsJLtocg95WyBbS/q02iDzVmx8D60oMYbDjSG?=
 =?us-ascii?Q?VMZrryduBKSfuFmKl4d6uHKTQkOv9bSw7zfqn8lrDthPWcnJgvxbLADdqJ4A?=
 =?us-ascii?Q?G6cxyBrR9uxWs4EeP34EDwVjO3Za2nwUzHLDnWEY2yZf5cQV/2VTuQtvba/i?=
 =?us-ascii?Q?77r2IzSlxbx/+zS01NHSpLwqcnDm9ZMr/tblNYayRYUMwQ/wZjLUN6I765rl?=
 =?us-ascii?Q?qSTv+QG1yZZwmMIUst4//MBLpWeYlgdxk8dElS61paiurThj0Yho3IykBFFu?=
 =?us-ascii?Q?X+jvn/gcV8RrlNLEknSr6COp4559AlyvPjhU6/iHkkI1/fiD/hjFrR1k8usy?=
 =?us-ascii?Q?EGfFPW2R9Itv8msKO+iDt+25e4mY/Y6gy/UjbnfuCMGlarzwA9PXQcIE98zd?=
 =?us-ascii?Q?XvvQFRbJy+iIwXBEWU6E5+pkFHAHzOaZqte+Ir9KSe1x1UcaPU+OufVJ/DvX?=
 =?us-ascii?Q?5s9LXPMxoMRNSXv1HmIIeqvYQ12UDAoGlzEJwnmKIbgdbuX9HWHW12bEIAdk?=
 =?us-ascii?Q?febojGno8M3qm49X6wR2sYj6BgJOuaTsUFZ163bxdI3EZAtA07k+CmoQlmZ9?=
 =?us-ascii?Q?BVwb8N+YOYh4Y6qulDvovILTn8SCYhAttqcJ2VPysFPqWPGTJnT3ktxWRHmx?=
 =?us-ascii?Q?X3vGwFdysi6OuK2AEXDXVl1SHUYrQWaVZRIhVf0SHzWF1hUuZFlkSoJeZgHF?=
 =?us-ascii?Q?ggeVdpBhrwCZ1EoDq9CPAtBY7F69TToadqdPjA7evWr3g3/rg5PWLaEW4QNa?=
 =?us-ascii?Q?YeFxDKCYVSWGS7ZPjIWi5A8lVeCU4zkdDeiPfy6KKexJyPNEn0uJUm+F4Qbp?=
 =?us-ascii?Q?SrsKjhviJS+E1ytHfkQCYj6lAAkDKajf1OZ00aqFIhv134RewapodHvps7jx?=
 =?us-ascii?Q?LcXRf2ujc0kGbPaNY9+1dEkOdNJPMeb+aFiYEPGIBSWZDeT9t7q4Mou6+xOI?=
 =?us-ascii?Q?cPLA7Pk+Jyh6HlmExd9lY5ip4NJEitJzZJPegHpVXZ/KI8rtZEe8Y8g6vnEe?=
 =?us-ascii?Q?yoIuXGHUX/e/yZB7zIal+JsTLsvZFb+szzYk/kt9T1fOtR2Llzo/WbGhI+it?=
 =?us-ascii?Q?d9vF+421gG92ZoKKYJ+hIZcM1ctWV42B+eAnZVQjZyaptiWlFu0ctVkWsWxt?=
 =?us-ascii?Q?3DvFE4Vj7LlsX4gnI6z84xCvJ+k=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-ID: <A7904E2C6C8F464E90C1DD5FB611906F@namprd10.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: =?us-ascii?Q?2nwesG+4rjfSO8uP91Jxg92LU865rXvJxB6ihQbcBUMOrUyPtIaCC2kKQrEK?=
 =?us-ascii?Q?QnKYqTPgjvAPzxbFlHI5fPBJzuBjrHOzbz4DDN6vCoKqxIuw3foO4fpRoM/M?=
 =?us-ascii?Q?fh3pg7uygvgSo0OOJUq/8dzuYzXroQ2xIJnTMp5onrkJQzy8ObxvYgi1U+R2?=
 =?us-ascii?Q?lBFRhYuQB4HFQozhwvOHvRKVeT7Ju1Zw/LzpOZIzkuO8SIZTqYCxU0qTJ8xY?=
 =?us-ascii?Q?4PjlNPFiTXX3GDFuYmyz8h6ESsNXUUDNq5NK6KnktHuPbgaE2jw9L2MqV2FN?=
 =?us-ascii?Q?OPpYwmB7h66oD9wGfUQzma2H2PyHNP+xA/qmSekIKK4m+INR5SOfG0We0aqJ?=
 =?us-ascii?Q?K22Ob+SlUmlqtgVb04kABmo+ez4XF20s81YL20Zqtwok53Ml0lJzFV/3JO17?=
 =?us-ascii?Q?6YbZS/w8UkdZzbPVumRMZMaH9cI7ZLgd2AhNaiMlQbr4taktklP96uScJnan?=
 =?us-ascii?Q?mcGUBwtY0WpvtgLMFbfM5jyve2i/Ci0SESReqw45C2g9hfYDuLTpMrjNTF/K?=
 =?us-ascii?Q?H3Sf0v+RyljtW4Sa/dooHWkWhJPaKoegEN5Ooq0q7RXW+1TBUcqsWPbgIvl/?=
 =?us-ascii?Q?eNIGrcLu6xJYdmDYmgXEG9EcyNNcZTeOghu49eKEBePt1svdQDDhFinClZKy?=
 =?us-ascii?Q?oMWS8uSO14bwp6FeMIbURGR4pq9Wnsss4H3vgUbUBvKmjbSSZWF7IwpgQX8S?=
 =?us-ascii?Q?2EMUiGsHOC8fexOWH584CX8du3l7q694z4f/f0xYTjRy3F7Ew7Xj4vp838rh?=
 =?us-ascii?Q?Ix8u5Do+boLlYrJwBPkLU/NHZSfKegT7azTqrNnWcxXD4XERlC8J1BZYDk9a?=
 =?us-ascii?Q?fJ8yt3gYzhPRApo79yIUY1ivJYd/OTll6c2eEG5wR7j+kmUwJF2xqUOHvLU0?=
 =?us-ascii?Q?ZrA8ntVy3LjacC/4Dkg7lsLi20stmCg5QqxXW5NXlTyGuEB2rWHcKVWCthP6?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SN4PR10MB5559.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 536609bd-6815-4d33-288b-08db22b3b644
X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Mar 2023 04:39:05.4841
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: BoKlQp8T1bGjeDVKzgUD/aFkV8WmupgwN9rBrZ70OgEv10CgLxjYIikeX07ePbmloSBbTe5pZQEOqLy2w1ErcpLOE19IAyY/eY6kxfp01Qc=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR10MB7481
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22
 definitions=2023-03-11_04,2023-03-10_01,2023-02-09_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 phishscore=0
 suspectscore=0 spamscore=0 malwarescore=0 adultscore=0 mlxscore=0
 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2212070000 definitions=main-2303120038
X-Proofpoint-GUID: HZWl6gFklb0XjnoSF5sUVci82CYPyRrm
X-Proofpoint-ORIG-GUID: HZWl6gFklb0XjnoSF5sUVci82CYPyRrm
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


> On Mar 11, 2023, at 5:55 PM, Hillf Danton <hdanton@sina.com> wrote:
>=20
> On 11 Mar 2023 12:24:58 +0000 William Kucharski <william.kucharski@oracle=
.com>
>>> On Mar 10, 2023, at 04:25, David Hildenbrand <david@redhat.com> wrote:
>>> On 10.03.23 02:40, William Kucharski wrote:
>>>>> On Mar 9, 2023, at 17:05, Zach O'Keefe <zokeefe@google.com> wrote:
>>>>> =3D20
>>>>>> I think the hugepage alignment in their environment was somewhat luc=
k.
>>>>>> One suggestion made was to change stack size to avoid alignment and
>>>>>> hugepage usage.  That 'works' but seems kind of hackish.
>>>>> =3D20
>>>>> That was my first thought, if the alignment was purely due to luck,
>>>>> and not somebody manually specifying it. Agreed it's kind of hackish
>>>>> if anyone can get bit by this by sheer luck.
>>>> I don't agree it's "hackish" at all, but I go more into that below.
>>>>> =3D20
>>>>>> Also, David H pointed out the somewhat recent commit to align suffic=
ie=3D
>> ntly
>>>>>> large mappings to THP boundaries.  This is going to make all stacks =
hu=3D
>> ge
>>>>>> page aligned.
>>>>> =3D20
>>>>> I think that change was reverted by Linus in commit 0ba09b173387
>>>>> ("Revert "mm: align larger anonymous mappings on THP boundaries""),
>>>>> until it's perf regressions were better understood -- and I haven't
>>>>> seen a revamp of it.
>>>> It's too bad it was reverted, though I understand the concerns regardi=
ng=3D
>> it.
>>>> From my point of view, if an address is properly aligned and a caller =
is
>>>> asking for 2M+ to be mapped, it's going to be advantageous from a pure=
ly
>>>> system-focused point of view to do that mapping with a THP.=3D20
>>> =3D20
>>> Just noting that, if user space requests multiple smaller mappings, and=
 t=3D
>> he kernel decides to all place them in the same PMD, all VMAs might get =
mer=3D
>> ged and you end up with a properly aligned VMA where khugepaged would ha=
ppi=3D
>> ly place a THP.
>>> =3D20
>>> That case is, of course, different to the "user space asks for 2M+" map=
pi=3D
>> ng case, but from khugepaged perspective they might look alike -- and it=
 mi=3D
>> ght be unclear if a THP is valuable or not (IOW maybe that THP could be =
bet=3D
>> ter used somewhere else).
>>=20
>> That's a really, really good point.
>>=20
>> My general philosophy on the subject (if the address is aligned and the =
cal=3D
>> ler is asking for a THP-sized allocation, why not map it with a THP if y=
ou =3D
>> can) kind of falls apart when it's the system noticing it can coalesce a=
 bu=3D
>> nch of smaller allocations into one THP via khugepaged.
>>=20
>> Arguably it's the difference between the caller knowing it's asking for =
som=3D
>> ething THP-sized on its behalf and the system deciding to remap a bunch =
of =3D
>> disparate mappings using a THP because _it_ can.
>>=20
>> If we were to say allow a caller's request for a THP-sized allocation/ma=
ppi=3D
>> ng take priority over those from khugepaged, it would not only be a majo=
r v=3D
>> ector for abuse, it would also lead to completely indeterminate behavior=
 ("=3D
>> When I start my browser after a reboot I get a bunch of THPs, but after =
the=3D
>> system's been up for a few weeks, I don't, how come?")
>=20
> Given transparent_hugepage_flags, how would it be abused?  And indetermin=
ed?

I was speaking in terms of heuristics, if we allowed callers making THP-map=
pable requests to have priority over khugepaged requests by default, it wou=
ld be easy for callers to abuse that to request most THP-mappable memory, l=
eaving little with which khugepaged could coalesce smaller pages.

This is much the way hugetlbfs is sometimes used now where if callers can't=
 get the allocations they require, users reboot the machine and make sure t=
heir applications requiring such allocations run first.

My apologies if I am missing an existing mechanism preventing this, it's be=
en a bit since I walked through that code.

   -- Bill