Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 12 Dec 2022 16:13:52 -0800
From:   Mike Kravetz <mike.kravetz@oracle.com>
To:     James Houghton <jthoughton@google.com>
Cc:     Muchun Song <songmuchun@bytedance.com>,
        Peter Xu <peterx@redhat.com>,
        David Hildenbrand <david@redhat.com>,
        David Rientjes <rientjes@google.com>,
        Axel Rasmussen <axelrasmussen@google.com>,
        Mina Almasry <almasrymina@google.com>,
        Zach O'Keefe <zokeefe@google.com>,
        Manish Mishra <manish.mishra@nutanix.com>,
        Naoya Horiguchi <naoya.horiguchi@nec.com>,
        "Dr . David Alan Gilbert" <dgilbert@redhat.com>,
        "Matthew Wilcox (Oracle)" <willy@infradead.org>,
        Vlastimil Babka <vbabka@suse.cz>,
        Baolin Wang <baolin.wang@linux.alibaba.com>,
        Miaohe Lin <linmiaohe@huawei.com>,
        Yang Shi <shy828301@gmail.com>,
        Andrew Morton <akpm@linux-foundation.org>, linux-mm@kvack.org,
        linux-kernel@vger.kernel.org
Subject: Re: [RFC PATCH v2 08/47] hugetlb: add HGM enablement functions
Message-ID: <Y5fDwH6XiM808oUM@monkey>
References: <20221021163703.3218176-1-jthoughton@google.com>
 <20221021163703.3218176-9-jthoughton@google.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20221021163703.3218176-9-jthoughton@google.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Dkm2fRu2SlBj8oWbLn+Mdld5kPVdu1lNG90+j7jRCUe0E+lx36P78YaYFj6I?=
 =?us-ascii?Q?6lSc17BJ5Qh5F92ik+4WDPVHhimyeXQUWh7kZMaG1GR47Jm+qDIy9tJB0RQD?=
 =?us-ascii?Q?+oUQBn8roMtrYj08xx27CwK6or73cGESpexSwtPvWA5JsS4bOs5vRtNlisVB?=
 =?us-ascii?Q?Pr9n28RsL5Lp6w3UPYT20+hQDrf/BM+d0RioHJIkSGKZvKEBI123r2EYQVpX?=
 =?us-ascii?Q?ZPFZo6jwLY9lIro5TPPGUZiekwBZulMgMaacwlk8F6q6wdsC+KHNcWB7LZcN?=
 =?us-ascii?Q?gjQAfPt0oYMUOCIOw/G6G3QwoBWKgoaHfuxWBKHHPCOC5Nllzdl0vT3S+Lpm?=
 =?us-ascii?Q?kVZA3TD9d2DTqs5x0YPec2gsITnuPch4Ux7u6laFRkLhvDThQeuI4qf4xuQD?=
 =?us-ascii?Q?BvcqBE48UqUUXIlDFs4pfUfzPtVw6OCMkf67McgIYhk2kuV4H7mvzoP+kOz2?=
 =?us-ascii?Q?NARqmZMsuyAFvEhEiJY0j6W+8wVmSR2PGe/VTLw2+qNSVworZ/mXRsxH8yQ5?=
 =?us-ascii?Q?hVJ3L3EPYaKZjBUTNfQfLZjJicsSV48A5IDnbi+R4/z7mzaZdUGjwnTAa8GW?=
 =?us-ascii?Q?o09Nga1DSZmftM495/9AZb99lbjog53Awars/6OJU+csWobF7XdqZ8wkQQdc?=
 =?us-ascii?Q?4V0i/j7x13Hf9NmrYQueQcK78nXkyiS6Jl2mXiNQcmMg4sqNwWhqTA3xhOxS?=
 =?us-ascii?Q?uyjIgrdK1ZV6ksl9/bDBxuPpqgXUsgJDqCpe3a0hSrqQp5KFTmUlJsmb6thk?=
 =?us-ascii?Q?pTu6o73jWX87ovglu0yNB1FUpldzyg35vm7tkOh9ht1fqlbYlUPzLR0ItXPe?=
 =?us-ascii?Q?AaUonf0z7VUEw3KSLAzg4KlH613+RsugtjoRestO0NTozqy97ARvvgFuKBgZ?=
 =?us-ascii?Q?xwHqNAcBOLdbkoe3v82qAsqXodds+pKzO7+4QC0mAYHE/H4/kFCfmvPyIiTR?=
 =?us-ascii?Q?xr4EYmjWRez5eYts4hyw0qcBcuBOCe7dsOU8w62ibXbX4N/jBTMFqCjInmMx?=
 =?us-ascii?Q?GWeGjT2VTqwMXB+wuygqlEU/khPoOno84QAG3GFgCO3F7rJfv4hcZowyGF0F?=
 =?us-ascii?Q?/P4mBBjNocJddF3fYA6rJbBACEqf2PlCitpqOiuiPawW3u2f7Lkk5Gisk624?=
 =?us-ascii?Q?WfhZ1mLWg8HGj2bZRuXw9SCDF2IOX7G8/u2NoU48NRkGtHXRfgTwDpQ+zOEc?=
 =?us-ascii?Q?V+ptdsj5dTUL6c3qA7vaROFWkXVmjYG4tzm733j9tQMNcd0vrimtKPi7nPvG?=
 =?us-ascii?Q?+tl7lZJfiHF8s0Wwdk9YxJ0/4IJrIgrwh36dk7Km2Yj8gTre+PxO/75NWY5F?=
 =?us-ascii?Q?fLOj0dStN5eDLzt3Q0dXX6UmJUeTh5H8pmZzvSlGQY2kc672AQvvu9U+ECLY?=
 =?us-ascii?Q?RP9FTAIitywDKTRB6AAjdV1MJVemEKP8KbiJX/nC7SGldcI2GQwyagaW/ReW?=
 =?us-ascii?Q?XlGQ326+m8TgjiXABPbzqqht7SdyVk9bnQbRyNKuisa/Fo6GsojBtB7BdRwj?=
 =?us-ascii?Q?WdK6dM9EPmIW9nMMIJE9xhW472h3zh67+YT1/DN93VoLCj9Y3pgZzfvpnvee?=
 =?us-ascii?Q?furGw4tFLqmF0cyJWHcBX6yZm6Zc0LulsenGydqWNCHm/4K8mlOFbUPLsk07?=
 =?us-ascii?Q?Ag=3D=3D?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a6fd009b-aae9-4fc4-0742-08dadc9eec5b
X-MS-Exchange-CrossTenant-AuthSource: BY5PR10MB4196.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Dec 2022 00:13:55.9209
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: MFvCjA3Ao8wvm/qLBEuQOG72OAN0FovB2rl6rjK8IRkKyLLV8sbaAzR9CgSsxyKsh0UpIvpfPFs8nRlME/4gew==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR10MB4598
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1
 definitions=2022-12-12_02,2022-12-12_02,2022-06-22_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 phishscore=0 bulkscore=0
 malwarescore=0 mlxlogscore=913 mlxscore=0 adultscore=0 suspectscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000
 definitions=main-2212130000
X-Proofpoint-GUID: 7gZNslZeZA7fYgxRoNriLzXnocjoaFRv
X-Proofpoint-ORIG-GUID: 7gZNslZeZA7fYgxRoNriLzXnocjoaFRv
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 10/21/22 16:36, James Houghton wrote:
> Currently it is possible for all shared VMAs to use HGM, but it must be
> enabled first. This is because with HGM, we lose PMD sharing, and page
> table walks require additional synchronization (we need to take the VMA
> lock).

Not sure yet, but I expect Peter's series will help with locking for
hugetlb specific page table walks.

> 
> Signed-off-by: James Houghton <jthoughton@google.com>
> ---
>  include/linux/hugetlb.h | 22 +++++++++++++
>  mm/hugetlb.c            | 69 +++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 91 insertions(+)
> 
> diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
> index 534958499ac4..6e0c36b08a0c 100644
> --- a/include/linux/hugetlb.h
> +++ b/include/linux/hugetlb.h
> @@ -123,6 +123,9 @@ struct hugetlb_vma_lock {
>  
>  struct hugetlb_shared_vma_data {
>  	struct hugetlb_vma_lock vma_lock;
> +#ifdef CONFIG_HUGETLB_HIGH_GRANULARITY_MAPPING
> +	bool hgm_enabled;
> +#endif
>  };
>  
>  extern struct resv_map *resv_map_alloc(void);
> @@ -1179,6 +1182,25 @@ static inline void hugetlb_unregister_node(struct node *node)
>  }
>  #endif	/* CONFIG_HUGETLB_PAGE */
>  
> +#ifdef CONFIG_HUGETLB_HIGH_GRANULARITY_MAPPING
> +bool hugetlb_hgm_enabled(struct vm_area_struct *vma);
> +bool hugetlb_hgm_eligible(struct vm_area_struct *vma);
> +int enable_hugetlb_hgm(struct vm_area_struct *vma);
> +#else
> +static inline bool hugetlb_hgm_enabled(struct vm_area_struct *vma)
> +{
> +	return false;
> +}
> +static inline bool hugetlb_hgm_eligible(struct vm_area_struct *vma)
> +{
> +	return false;
> +}
> +static inline int enable_hugetlb_hgm(struct vm_area_struct *vma)
> +{
> +	return -EINVAL;
> +}
> +#endif
> +
>  static inline spinlock_t *huge_pte_lock(struct hstate *h,
>  					struct mm_struct *mm, pte_t *pte)
>  {
> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> index 5ae8bc8c928e..a18143add956 100644
> --- a/mm/hugetlb.c
> +++ b/mm/hugetlb.c
> @@ -6840,6 +6840,10 @@ static bool pmd_sharing_possible(struct vm_area_struct *vma)
>  #ifdef CONFIG_USERFAULTFD
>  	if (uffd_disable_huge_pmd_share(vma))
>  		return false;
> +#endif
> +#ifdef CONFIG_HUGETLB_HIGH_GRANULARITY_MAPPING
> +	if (hugetlb_hgm_enabled(vma))
> +		return false;
>  #endif
>  	/*
>  	 * Only shared VMAs can share PMDs.
> @@ -7033,6 +7037,9 @@ static int hugetlb_vma_data_alloc(struct vm_area_struct *vma)
>  	kref_init(&data->vma_lock.refs);
>  	init_rwsem(&data->vma_lock.rw_sema);
>  	data->vma_lock.vma = vma;
> +#ifdef CONFIG_HUGETLB_HIGH_GRANULARITY_MAPPING
> +	data->hgm_enabled = false;
> +#endif
>  	vma->vm_private_data = data;
>  	return 0;
>  }
> @@ -7290,6 +7297,68 @@ __weak unsigned long hugetlb_mask_last_page(struct hstate *h)
>  
>  #endif /* CONFIG_ARCH_WANT_GENERAL_HUGETLB */
>  
> +#ifdef CONFIG_HUGETLB_HIGH_GRANULARITY_MAPPING
> +bool hugetlb_hgm_eligible(struct vm_area_struct *vma)
> +{
> +	/*
> +	 * All shared VMAs may have HGM.
> +	 *
> +	 * HGM requires using the VMA lock, which only exists for shared VMAs.
> +	 * To make HGM work for private VMAs, we would need to use another
> +	 * scheme to prevent collapsing/splitting from invalidating other
> +	 * threads' page table walks.
> +	 */
> +	return vma && (vma->vm_flags & VM_MAYSHARE);

I am not yet 100% convinced you can/will take care of all possible code
paths where hugetlb_vma_data allocation may fail.  If not, then you
should be checking vm_private_data here as well.

> +}
> +bool hugetlb_hgm_enabled(struct vm_area_struct *vma)
> +{
> +	struct hugetlb_shared_vma_data *data = vma->vm_private_data;
> +
> +	if (!vma || !(vma->vm_flags & VM_MAYSHARE))
> +		return false;
> +
> +	return data && data->hgm_enabled;
> +}
> +
> +/*
> + * Enable high-granularity mapping (HGM) for this VMA. Once enabled, HGM
> + * cannot be turned off.
> + *
> + * PMDs cannot be shared in HGM VMAs.
> + */
> +int enable_hugetlb_hgm(struct vm_area_struct *vma)
> +{
> +	int ret;
> +	struct hugetlb_shared_vma_data *data;
> +
> +	if (!hugetlb_hgm_eligible(vma))
> +		return -EINVAL;
> +
> +	if (hugetlb_hgm_enabled(vma))
> +		return 0;
> +
> +	/*
> +	 * We must hold the mmap lock for writing so that callers can rely on
> +	 * hugetlb_hgm_enabled returning a consistent result while holding
> +	 * the mmap lock for reading.
> +	 */
> +	mmap_assert_write_locked(vma->vm_mm);
> +
> +	/* HugeTLB HGM requires the VMA lock to synchronize collapsing. */
> +	ret = hugetlb_vma_data_alloc(vma);
> +	if (ret)
> +		return ret;
> +
> +	data = vma->vm_private_data;
> +	BUG_ON(!data);

Would rather have hugetlb_hgm_eligible check for vm_private_data as
suggested above instead of the BUG here.

-- 
Mike Kravetz

> +	data->hgm_enabled = true;
> +
> +	/* We don't support PMD sharing with HGM. */
> +	hugetlb_unshare_all_pmds(vma);
> +	return 0;
> +}
> +#endif /* CONFIG_HUGETLB_HIGH_GRANULARITY_MAPPING */
> +
>  /*
>   * These functions are overwritable if your architecture needs its own
>   * behavior.
> -- 
> 2.38.0.135.g90850a2211-goog
>