From:   William Kucharski <william.kucharski@oracle.com>
To:     David Hildenbrand <david@redhat.com>
CC:     Zach O'Keefe <zokeefe@google.com>,
        Mike Kravetz <mike.kravetz@oracle.com>,
        Peter Xu <peterx@redhat.com>, Rik van Riel <riel@surriel.com>,
        Mike Rapoport <rppt@kernel.org>, Linux-MM <linux-mm@kvack.org>,
        LKML <linux-kernel@vger.kernel.org>
Subject: Re: THP backed thread stacks
Thread-Topic: THP backed thread stacks
Thread-Index: AQHZUId6EUNxRdiQ4UKEGoIYcgiIL67zHhkAgAAI0ICAABpxgIAAo6sAgAGi3AA=
Date:   Sat, 11 Mar 2023 12:24:58 +0000
Message-ID: <F5DD5318-F762-4FE8-AF1D-7D9AC6A73D39@oracle.com>
References: <20230306235730.GA31451@monkey> <ZAaCISgq4A/GnkCk@x1n>
 <20230307004049.GC4956@monkey> <20230308190206.GA4005@monkey>
 <CAAa6QmSdTo77dP2q2nU==C_2PdyF611+PVF32uPOTUQbp1kc9Q@mail.gmail.com>
 <20230309233340.GC3700@monkey>
 <CAAa6QmSiuFF6Oe0-j+eD0ma2tZAbhZuwENDYSZQSBrh1oeaLdA@mail.gmail.com>
 <9F855331-33B2-4366-9375-988B0D3DAC98@oracle.com>
 <9f44de08-b484-baa7-80c8-0a02a7abb717@redhat.com>
In-Reply-To: <9f44de08-b484-baa7-80c8-0a02a7abb717@redhat.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?wFrL1/p7my8Cr3jnqGJFtlaT3H75HYFRjCBzsgFP8ZimefRfaBolq8DBlv9H?=
 =?us-ascii?Q?i+iDea5cPo/Dpi3ygNt2Zn2IwTNqZn67JjXChEG+MeHO3OF5uii+mR5DyB32?=
 =?us-ascii?Q?2+JhSvQCcVi9hO18/S8zjHu8IQf7BsmzAybSN65HQM8u279o2b6RoHQoCmrc?=
 =?us-ascii?Q?ZmergcN1nms3Bat0FTOFSZc9KM9V4Qo5HMJ59sE7irBNI3SE+a8Y5ZJ/UUwk?=
 =?us-ascii?Q?eQGq8ZNdogujtwLzzy9+KH+ZOlHsm0M5k6sjzOEhAAcuNcKI/g/VSEJgAuE0?=
 =?us-ascii?Q?YZQy3rfpHr9xpWI1BIm1ot/tGQkj78kKQTV7gxazIJu7m0GlNe4BHkr7SBaQ?=
 =?us-ascii?Q?26M+XPlFmFz3ax2WaLNfaf7L2LAY+AwWGIoQo/bpGN/RDU3m95Gnyy8HobF8?=
 =?us-ascii?Q?rz3NuWewxZnM/N/NnyNr1Etv1NcWqUopumqElDHfwla9HNjWs0tZcFVuJAJd?=
 =?us-ascii?Q?tQohx8HKI0A9unsW60fqy5HK8CigMAVDALjbUW3CzHuCeFqKQ3MckOHXQgzs?=
 =?us-ascii?Q?OMfbt0H/0b/I2XmfFnAHhZrtXlSVjm3zP92cbK/c+FSRjNWVabnWiBev5hQL?=
 =?us-ascii?Q?u1KJELMGQiSVbvuDdZWiHu1RprbmaqIhYYOrFAElPGj+fqalTlqsQJlmJ6kb?=
 =?us-ascii?Q?fRcjguvbXko7y+FfyJsJli4a5ljBMISBPMdF5/sH3vI617DqMlto1CvKc+Z7?=
 =?us-ascii?Q?qc5XunXDQ6V06Ehpi92sGDfwA0Rcy9h1d+iXOI5NU5MsbpE+EeTHmQDFxPFB?=
 =?us-ascii?Q?61GDraSx9rg/VpsK4msBfbV7DK83+IO7SbCbjIJ/VQYW8DIJMK1qycBmKuZH?=
 =?us-ascii?Q?6ouDyzYlB+gl6rBuxmt4W28S/64V/kfcUADOUWnPB+TV6PxvF1779c7ZzjIq?=
 =?us-ascii?Q?6YzWe5kmThkUN7+2qaqBkmtUQYbheRRqSNEr9tWqEBtUnSi7BEd94M+TT4n/?=
 =?us-ascii?Q?lPIVbVA3woFSRnimTNLPTkmEoBVPV9Hh643PKQQ3mGmbpqO+6xb3ITd1Ieva?=
 =?us-ascii?Q?YKCd67Wzge/U7m6OfymlQXGnJn4DsKB2kRXCu0+mt55iMe7nRxn9OEnGbfgw?=
 =?us-ascii?Q?j0wUT+P5pIRxp8NJTDZwC6Lm7PU1lKOoYpX3TszX/vuJBUvPpCnS1dtjYl3w?=
 =?us-ascii?Q?brtot0VvzjidwoypDFQ20zgsJbj1Fvkjkz0Z5/hvSji5d8dL1hVeB6kYjFwc?=
 =?us-ascii?Q?/3CanHTvMADrZHsb5bG/eZa+Gx/M3VERi4Vuje3HSxocvzSUvqSLsr7WnLpO?=
 =?us-ascii?Q?7Y8FdWrasaGW7napHLUpOrrUBvTtjG17ZU1tjtq8nTreMIVFKUHhXAz/ySwp?=
 =?us-ascii?Q?g9x0E8mSdSPRu9wVSXFy1AonDqZCnj6PNCdOdLi2qnL3abWDFUj829rWTpFh?=
 =?us-ascii?Q?Z4exiHEg5YrCioG+Z5PJyGM3sgbeYRXrcgCH3Kyu6fJeD/W2gGIG40G2mFy0?=
 =?us-ascii?Q?eZhPOX1G0yZtI1CzvS/uaRgN1vUDaheDMxjQBDb8HGWhRaf/pTvahfdQYPrz?=
 =?us-ascii?Q?xW7YkL0nJ2qjj8+3ha/o2UO2xS8VffMHlLOaurNre4EKjCSUf1woIMkG2rcA?=
 =?us-ascii?Q?1dPhC/Sos33IuRF6DJhOAEJJm5iguoYsrhZ/b6TVQqxH1UBKDlxKN2PP8SM5?=
 =?us-ascii?Q?jgnO5mHUC4G2G4l5pKqYE7hFZcMEyujI0PTVPjS4p1VzGWzJTBfrinABYqfB?=
 =?us-ascii?Q?sv/crwtHImw2KnY8bLVun7bg8JE=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-ID: <8704A9509AA6A04194864E119D0973F8@namprd10.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: O4Fg6cBbikoi7kIX0JZVp+NU8r3oDqpCspTS0TvhjNLgExYcYFRKf0b6cKq0JyQGgM0aSH4KVbYnyhio6kqkgXHsi52S3HZYm1gY6QkISOzn1b6hKb2/WvC6OtXH5HYs5lr8yl4QxiQBABGQp72SuVQ7lVYC/z2qmIpk/k10IcE6S1quEhezS1Y7YVTqk9bDIqr54DjSyDKcBZTIC2YhWwJq8bNkVvs4ELck3It5ivV2LwDwPQe8Ub4CyyBWAd07GA16xzp0ka/lzRQO7ScOxs4M/ls4RvxkS8wXFxrXqoZbSUY0ewIPX8lPC0PRxk+gPb5OZX0WjvmLAq2E6J4uaB+7+U0ZWKcs7I/3dctERB6HeUalGGynggU299lBfbtCKsCTIUftWPy1F2TZKLiVbk9Lemg0n0XtWPeRSu5HWfzBLsCrIYFHzCwMzixF2Uur8x499/vTNFGjoVeIzlReC1cTeNSUoKxAx63dIPJ3OtOopwPHVwnyAXGj3lRM4jr49dEuTDEJYxG9/kUStgn3OmM8mh4tUdX0NWa6X/FM8etr4zMYM4AI/KSSQUJzL9y6pGPsHykuQP3wojT+aITAC6xWefA0EU+9kuOBHCj3XHMPyguVj8Th9EV/9teeG37ed90OR4hFuhHNOc9OaPSj52NK1isUbfwKrH9GCHXuShPXJGE07i/XCveYkG7e8Zsbf/ksrv+qmxyLQ7nEKj76+3VM1oH03O5p6sWRo3eX8/7TaHhlJy2g2FgmI09ZAAvAcQispT2/oRBZU7wu6fCFpCx2nJAeAMMRm83HPWZicmL1pcVmxwFZSyEqdqW/I4qE0tkHa3FWjrKddAQoShvqDE9JvKkpZVGBajxW8J3cX5O0fq082n8OLlk+AkNiG8xK8AckgFsAqxitm3HG265zvHVwTe2cZ9USH07r/I4ina0=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SN4PR10MB5559.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e3336b85-5f3e-40d8-4463-08db222ba133
X-MS-Exchange-CrossTenant-originalarrivaltime: 11 Mar 2023 12:24:58.5705
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: aauQ1n61hWpmg1lOhFQumtBELlPjnA0RcPkeDq1X2THXrADLbCkoTg7uFLwj0sqSU/nqDeyTibJyNyuV/V7X4x3VoLywHPXKj520sY68vb0=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR10MB4218
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22
 definitions=2023-03-11_04,2023-03-10_01,2023-02-09_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxscore=0 bulkscore=0
 adultscore=0 mlxlogscore=999 malwarescore=0 spamscore=0 suspectscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000
 definitions=main-2303110110
X-Proofpoint-ORIG-GUID: 7N69igobjJ2RsTgBz4z0IWIkXiz2OmG0
X-Proofpoint-GUID: 7N69igobjJ2RsTgBz4z0IWIkXiz2OmG0
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


> On Mar 10, 2023, at 04:25, David Hildenbrand <david@redhat.com> wrote:
>=20
> On 10.03.23 02:40, William Kucharski wrote:
>>> On Mar 9, 2023, at 17:05, Zach O'Keefe <zokeefe@google.com> wrote:
>>>=20
>>>> I think the hugepage alignment in their environment was somewhat luck.
>>>> One suggestion made was to change stack size to avoid alignment and
>>>> hugepage usage.  That 'works' but seems kind of hackish.
>>>=20
>>> That was my first thought, if the alignment was purely due to luck,
>>> and not somebody manually specifying it. Agreed it's kind of hackish
>>> if anyone can get bit by this by sheer luck.
>> I don't agree it's "hackish" at all, but I go more into that below.
>>>=20
>>>> Also, David H pointed out the somewhat recent commit to align sufficie=
ntly
>>>> large mappings to THP boundaries.  This is going to make all stacks hu=
ge
>>>> page aligned.
>>>=20
>>> I think that change was reverted by Linus in commit 0ba09b173387
>>> ("Revert "mm: align larger anonymous mappings on THP boundaries""),
>>> until it's perf regressions were better understood -- and I haven't
>>> seen a revamp of it.
>> It's too bad it was reverted, though I understand the concerns regarding=
 it.
>> From my point of view, if an address is properly aligned and a caller is
>> asking for 2M+ to be mapped, it's going to be advantageous from a purely
>> system-focused point of view to do that mapping with a THP.=20
>=20
> Just noting that, if user space requests multiple smaller mappings, and t=
he kernel decides to all place them in the same PMD, all VMAs might get mer=
ged and you end up with a properly aligned VMA where khugepaged would happi=
ly place a THP.
>=20
> That case is, of course, different to the "user space asks for 2M+" mappi=
ng case, but from khugepaged perspective they might look alike -- and it mi=
ght be unclear if a THP is valuable or not (IOW maybe that THP could be bet=
ter used somewhere else).

That's a really, really good point.

My general philosophy on the subject (if the address is aligned and the cal=
ler is asking for a THP-sized allocation, why not map it with a THP if you =
can) kind of falls apart when it's the system noticing it can coalesce a bu=
nch of smaller allocations into one THP via khugepaged.

Arguably it's the difference between the caller knowing it's asking for som=
ething THP-sized on its behalf and the system deciding to remap a bunch of =
disparate mappings using a THP because _it_ can.

If we were to say allow a caller's request for a THP-sized allocation/mappi=
ng take priority over those from khugepaged, it would not only be a major v=
ector for abuse, it would also lead to completely indeterminate behavior ("=
When I start my browser after a reboot I get a bunch of THPs, but after the=
 system's been up for a few weeks, I don't, how come?")

I don't have a good answer here.

    -- Bill=