Date:   Tue, 21 Feb 2023 13:46:06 -0800
From:   Mike Kravetz <mike.kravetz@oracle.com>
To:     James Houghton <jthoughton@google.com>
Cc:     Muchun Song <songmuchun@bytedance.com>,
        Peter Xu <peterx@redhat.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        David Hildenbrand <david@redhat.com>,
        David Rientjes <rientjes@google.com>,
        Axel Rasmussen <axelrasmussen@google.com>,
        Mina Almasry <almasrymina@google.com>,
        Zach O'Keefe <zokeefe@google.com>,
        Manish Mishra <manish.mishra@nutanix.com>,
        Naoya Horiguchi <naoya.horiguchi@nec.com>,
        "Dr . David Alan Gilbert" <dgilbert@redhat.com>,
        "Matthew Wilcox (Oracle)" <willy@infradead.org>,
        Vlastimil Babka <vbabka@suse.cz>,
        Baolin Wang <baolin.wang@linux.alibaba.com>,
        Miaohe Lin <linmiaohe@huawei.com>,
        Yang Shi <shy828301@gmail.com>,
        Frank van der Linden <fvdl@google.com>,
        Jiaqi Yan <jiaqiyan@google.com>, linux-mm@kvack.org,
        linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2 00/46] hugetlb: introduce HugeTLB high-granularity
 mapping
Message-ID: <Y/U7ngTyPlg2XCRj@monkey>
References: <20230218002819.1486479-1-jthoughton@google.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20230218002819.1486479-1-jthoughton@google.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Y8PSLRDKC2cYyqJkALshK5ZEEO1kU5aAdXpZuIUl4Nume8aHFRiDx3n/AcsA?=
 =?us-ascii?Q?qmlbhAytr1+8rd8yyngTF2G/0AEmy5NRJeois6DodTGTC+mz5ysSb5xThlP+?=
 =?us-ascii?Q?bXGo9741sQrQMB3RGPb7i0pJnM+IW1E578xy7ckLiejLGThy2Tku/X1Rtm7a?=
 =?us-ascii?Q?3EWOR7/fCCDUI23YPEt1/+2Q7OEAXKt4+tILJ1ftzwLSSqlDh2Wn/4Fp+WWk?=
 =?us-ascii?Q?DlwHKWHPSt3CmcDALjUvAnceDS+gpidIrGgWt4YGI66D/9YZNWLARKGswazL?=
 =?us-ascii?Q?mMV4E8Yukav4MuB5IixRR7C4xRvgdWCyeYTVVO4E7V0LkmqWEUsCj17HNf/c?=
 =?us-ascii?Q?Ep/tx9JnhxZwMYvqTybHw6WVtz3kJH3GymOFBD/4WLGxGDT70AnLjLFqwZ8J?=
 =?us-ascii?Q?1ZeZMwcQu8jW1DGPoCoIcDWEmdwQO9uPC4yAeEwTrcBhQ5MAFWmFMEHaZxKd?=
 =?us-ascii?Q?cApj46y6XwH/SCvK2A4m44d0QUwrviXiSUuF6ltPtd5FS7vCmESBXdvgkVv4?=
 =?us-ascii?Q?cTMoBFm6y9HDQLvvyr4wQITlcsDtM7cOO85X8hln3f9szRbrgR2nvjvpMSQy?=
 =?us-ascii?Q?FPsnHxs+aIrBpKf17mUgP/15ikb/pTpix3ipICVnV3Jc7gqB4uXiB0TUzJHm?=
 =?us-ascii?Q?oPN5MrnkrALN59/WO7ReLfLbndkYtUJFGRcoVuwPhak/sdKnyVq1Cp77mgWJ?=
 =?us-ascii?Q?XdRp6jAIOUIK3C5LsTVRR1+giPbfHzrdJmiy9OuSgu8gvsyVubmKVtmqf+u6?=
 =?us-ascii?Q?k+KQNDHBPTsOeOaIh72eAR5twye7T/7+1siPAdfMOGQw1A7cc8AilhAX2EHw?=
 =?us-ascii?Q?nkYxTgV+Dxwu1KXmY0EXOaXcwhmIwVDV6UQHZ839yZ1YEkT8njFz7unj34yN?=
 =?us-ascii?Q?803WMa77+3hXqwDSZENdIIjUU3Lds6656sXfENFShohU8EVBnTcmnocKrqn2?=
 =?us-ascii?Q?nytuXok5fkNRnsCBs6Lic7vjmyCaUQ8xNO9E49ey5D1LmS7SPjTkJyiGLv6A?=
 =?us-ascii?Q?gNh7L+rBsJW8FDsDp6QdJoJSsUjVy6i+QNA1KNAUahl8M57jmLZzOWmYtlEF?=
 =?us-ascii?Q?KJzgSJsSCxtX41R7V5J22xSVVQcfLVcBIqYLkf2CdjZ/tD94HiL41pGykL2L?=
 =?us-ascii?Q?e/J2MWZClrEKKpE9+ZHXT3SGnVM3n0Gi4SqkKRP/zg+a81IgaT+F7jvc+uON?=
 =?us-ascii?Q?+KmcB0WJ1rJxf253j1bXhYgZnvYqUrZlzSbBAkC0xG/dKw98vduHAhq8WlVL?=
 =?us-ascii?Q?RCic81Evi3EKeZW8PHeSN2na0djBESJuzo6/5WF3ohvk/AXCIYg5ZGiK24i/?=
 =?us-ascii?Q?Zsx4lM9Vo3hwq2hjX7CvtPzebMi5Owl6uFR4HlvYWfQZpLFtzF6IneOLlPCV?=
 =?us-ascii?Q?myDovnYVF2ixqge3DtqR8knlpa+peZnbhsevAoZcKckBMu3WZA5McMp5M4H4?=
 =?us-ascii?Q?7G3Ce3W72OGpBINt2gscQAV+plHAzehj8YLJS+BKsTHFYar0TXxChq5QW8eJ?=
 =?us-ascii?Q?M9d/CibZ0HHans4TQmKWmflMgRBiE0C7lnMPwAs3SIEIw6hL5jMCbRP7rOMh?=
 =?us-ascii?Q?ojFFO46mcoJ0DmrmuYPSXJKVR/b1exfrbqZ+vxnJkI/kB24IUJctOLi8VBsa?=
 =?us-ascii?Q?CA=3D=3D?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: =?us-ascii?Q?19uhoK7+/1nFWKgbm43fPX7DC1vzjM2zFsk8vMtpl6WigUcYnLC8Qg3HwaDT?=
 =?us-ascii?Q?NTYvBokE6SAmsWiiAXYEybg+cs8Vg68XzWO0DjFtnVx1gFhwiVQiaZjDMMHt?=
 =?us-ascii?Q?9DYh7v/GebqLgPIr5L7t0XSDPzUb75hSoZBlICLZAhcDEOU2wLEdGS603DD/?=
 =?us-ascii?Q?GUmyettMcJWdR0jiMWghXFzWeiru5brj+9SFmXfMU6T35slxwy0d41mSitrg?=
 =?us-ascii?Q?SYMRKH5sNpU0YRiertB7r4kXDJ2sLu/bbTfLYGOKhNOseFff/4uHdhaHJ+be?=
 =?us-ascii?Q?hlEbuVNwI7KB3uPwKIZGwJU3rQPuhmLgodSRXzLCt4WxgiMYog24r5ibQVpe?=
 =?us-ascii?Q?C1xnXHOIEd3yWGXeZH0G/iPxBPMtAdj8bbknugtKWN1sdPNLtkcmbhC+PCT4?=
 =?us-ascii?Q?Z3LINHOwsU41g87L99Pav6Bki5dCn2JCdB1asquw8xKf3eZV9sSan77fKV0E?=
 =?us-ascii?Q?TJOdnaYj2TXZXHYwaqft55yoeWyiYficGktJ7qOrmoVsxeRxl3Z7oyYamhQm?=
 =?us-ascii?Q?TvcpYkO271a7EMCo6GbGMadFtunRTS03onjAanqAQ3fdY2ZiwxQDJG0kxJNb?=
 =?us-ascii?Q?KTepPgqqRiNPWw1TuiamCWXMhSkgH2DTki01TQqC65g4TnHv4U4L+Y+BAdPO?=
 =?us-ascii?Q?TWE/7sbFvvklP38Q5NnmOTKfG7rFn1CxqwqOkFzC0By4v16HP72BbJa0sgIJ?=
 =?us-ascii?Q?0x6fTVaJ5E8QNMVGQ6MeuaBRLpOpkOW11AjvztSVugUsqAtx7lYNgQCisa4F?=
 =?us-ascii?Q?qwj1ApfwClnVP66Qjf6vNOEGob0BNi4XbVdPuMU4EaTd5bN19olEkcEzffCV?=
 =?us-ascii?Q?LnTXxNDdsge0Xp10AkuS4JHUT4oeLrPrmNAZ0NKkj3dKQm6U3jrV8RQZPXHu?=
 =?us-ascii?Q?JmmLs0vtDZH4bMvuYJwuy9UYzc7bydLGV1BzYEOQ5sEYAXVUUVGd1MA4iXEK?=
 =?us-ascii?Q?wHIKtUskreBlF+QoaF8i98UknW60sdJY67PT0e7TxLnW4SImJsrD2WKxYXe7?=
 =?us-ascii?Q?5/RRFbt5b2ZSH1mQRE/iCyUsW3PatOK54mi7jZoOSS+ayfsVel9Zx5ZxbFgu?=
 =?us-ascii?Q?qrwGNXzUspiDYzITVfNlkhafSrkSB82y2JObM9f61QG9SZ24P3VfLg2+Pl0X?=
 =?us-ascii?Q?Z3sBfJjMSA0qJTvoF0q3Cy0LiTzbwG9t7A=3D=3D?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9ad105fb-d8bd-43dd-cfce-08db14550af9
X-MS-Exchange-CrossTenant-AuthSource: BY5PR10MB4196.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Feb 2023 21:46:09.4007
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: LzmQI/8FiUcSWLBl0UlOvq8p/x9DKye0vVggiSxrDx0CYIZXMTbCok3BkO4Xhc23Uax4L8q7JbrPeyjizxYylQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB4797
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.219,Aquarius:18.0.930,Hydra:6.0.562,FMLib:17.11.170.22
 definitions=2023-02-21_12,2023-02-20_02,2023-02-09_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 bulkscore=0 phishscore=0
 suspectscore=0 malwarescore=0 mlxscore=0 mlxlogscore=999 adultscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000
 definitions=main-2302210188
X-Proofpoint-GUID: EjsFHQ9H5MgEzA5eAL9bLMRDGtY9pYe6
X-Proofpoint-ORIG-GUID: EjsFHQ9H5MgEzA5eAL9bLMRDGtY9pYe6
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 02/18/23 00:27, James Houghton wrote:
> This series introduces the concept of HugeTLB high-granularity mapping
> (HGM). This series teaches HugeTLB how to map HugeTLB pages at
> high-granularity, similar to how THPs can be PTE-mapped.
> 
> Support for HGM in this series is for MAP_SHARED VMAs on x86_64 only. Other
> architectures and (some) support for MAP_PRIVATE will come later.
> 
> This series is based on latest mm-unstable (ccd6a73daba9).
> 
> Notable changes with this series
> ================================
> 
>  - hugetlb_add_file_rmap / hugetlb_remove_rmap are added to handle
>    mapcounting for non-anon hugetlb.
>  - The mapcounting scheme uses subpages' mapcounts for high-granularity
>    mappings, but it does not use subpages_mapcount(). This scheme
>    prevents the HugeTLB VMEMMAP optimization from being used, so it
>    will be improved in a later series.
>  - page_add_file_rmap and page_remove_rmap are updated so they can be
>    used by hugetlb_add_file_rmap / hugetlb_remove_rmap.
>  - MADV_SPLIT has been added to enable the userspace API changes that
>    HGM allows for: high-granularity UFFDIO_CONTINUE (and maybe other
>    changes in the future). MADV_SPLIT does NOT force all the mappings to
>    be PAGE_SIZE.
>  - MADV_COLLAPSE is expanded to include HugeTLB mappings.
> 
> Old versions:
> v1: https://lore.kernel.org/linux-mm/20230105101844.1893104-1-jthoughton@google.com/
> RFC v2: https://lore.kernel.org/linux-mm/20221021163703.3218176-1-jthoughton@google.com/
> RFC v1: https://lore.kernel.org/linux-mm/20220624173656.2033256-1-jthoughton@google.com/
> 
> Changelog:
> v1 -> v2 (thanks Peter for all your suggestions!):
> - Changed mapcount to be more THP-like, and make HGM incompatible with
>   HVO.
> - HGM is now disabled by default to leave HVO enabled by default.

I understand the reasoning behind the move to THP-like mapcounting, and the
incompatibility with HVO.  However, I just got to patch 5 and realized either
HGM or HVO will need to be chosen at kernel build time.  That may not be an
issue for cloud providers or others building their own kernels for internal
use.  However, distro kernels will need to pick one option or the other.
Right now, my Fedora desktop has HVO enabled so it would likely not have
HGM enabled.  That is not a big deal for a desktop.

Just curious, do we have distro kernel users that want to use HGM?

I see it mentioned that this incompatibility will be addressed in a future
series. This certainly will be required before HGM can be expanded for
use cases such as memory errors and page poisoning.

Just curious of other thoughts?  Does the first version of HGM need to be
compatible with HVO?
-- 
Mike Kravetz