Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Tue, 13 Dec 2022 11:32:29 -0800
From:   Mike Kravetz <mike.kravetz@oracle.com>
To:     James Houghton <jthoughton@google.com>
Cc:     Muchun Song <songmuchun@bytedance.com>,
        Peter Xu <peterx@redhat.com>,
        David Hildenbrand <david@redhat.com>,
        David Rientjes <rientjes@google.com>,
        Axel Rasmussen <axelrasmussen@google.com>,
        Mina Almasry <almasrymina@google.com>,
        Zach O'Keefe <zokeefe@google.com>,
        Manish Mishra <manish.mishra@nutanix.com>,
        Naoya Horiguchi <naoya.horiguchi@nec.com>,
        "Dr . David Alan Gilbert" <dgilbert@redhat.com>,
        "Matthew Wilcox (Oracle)" <willy@infradead.org>,
        Vlastimil Babka <vbabka@suse.cz>,
        Baolin Wang <baolin.wang@linux.alibaba.com>,
        Miaohe Lin <linmiaohe@huawei.com>,
        Yang Shi <shy828301@gmail.com>,
        Andrew Morton <akpm@linux-foundation.org>, linux-mm@kvack.org,
        linux-kernel@vger.kernel.org
Subject: Re: [RFC PATCH v2 11/47] hugetlb: add hugetlb_pmd_alloc and
 hugetlb_pte_alloc
Message-ID: <Y5jTTYwzFHgmm5l6@monkey>
References: <20221021163703.3218176-1-jthoughton@google.com>
 <20221021163703.3218176-12-jthoughton@google.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20221021163703.3218176-12-jthoughton@google.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?m/D4Sx4uvqZ6Td/8naZPbqTb3lO0fJg1fN7Q0vTut7/j16KobPAcR9ifa3j4?=
 =?us-ascii?Q?nyVubfI9/UD+2/HmxaTJBPjIiwihg3RXLBKrBCJq92zdPA3l+2B7rIuoLBJw?=
 =?us-ascii?Q?KfhEJuTGYnj1Ln6+jcs96/IuYvo1g1Ro9dcw6b8Kls/Zlvo+Ky43016nbC85?=
 =?us-ascii?Q?nwTMBKMPmlSmcX/XH2ZrKyVcoPgtrk6hg5qZFa0HjDbAR3WotlDbO9wXrJYZ?=
 =?us-ascii?Q?ttlU057Qq/86xRYlis2c8/lqDpfjpRdwRdOhBHFpyjyZX3ihAP3BDuMryJec?=
 =?us-ascii?Q?vjhqMTI0VU75qUloQf2GTz/WkwNDDZFSPZe3MDj1z5Smn7+XHRh9pSY9aRWz?=
 =?us-ascii?Q?wThLN4TztPi+HYBH0G7mb+7v/m8tW1FQEFWTieAhPWq4AHP3xA1sI2ul6cVA?=
 =?us-ascii?Q?D6zP/yyd2IZ2uWUYMim+2oc+rY391H65rdyWU7nWV51g545vAekRgte9+DDv?=
 =?us-ascii?Q?4vNOeQCd+TklsDmSWrbfQwLMrfqoNu/BzWw3IqqeFeudcU9qcLhxiTwQEv/t?=
 =?us-ascii?Q?bX362AlMNhYKaQI3sO45b86F6sIqeFGuauYV2MNuR5qobatWaF2aoGZMTerG?=
 =?us-ascii?Q?D4AINV2KXKN22bn3P6EUAnXOc1U1GR++cU6d+MTw7cMukZkKe5QioEBX0M2w?=
 =?us-ascii?Q?dW9RtwFqE9GdBcQKptHmcb3nFTD2smtKZBf4uSMpq8wpChqtsntOjLqGB423?=
 =?us-ascii?Q?i6JXLYlU3FYQ7YnG02/JXVR+LgD38+f7/981DMA13rJ1wxSfykpwhFWWbnFR?=
 =?us-ascii?Q?D1Sf/WTUaOZhSatAQRVqFw8/Ytkt1KaRP5XPFfidHzq0cruRGJp5fPp7R0/Q?=
 =?us-ascii?Q?NqOEk2rHKdHFRENRZ8mlLvvxAKwgTQC5MGpzbFV0rBIaxCrXW0V494QSyQAe?=
 =?us-ascii?Q?+dn9MTtkiZyVxy7kp7nwN+1W80LD5X28YzXGkPE8zM2Dxnva8PqFArfSspdk?=
 =?us-ascii?Q?bXqTT/+9txEXOBP9SeSI02nY52laVhWLe7hBVTJEYagkMgvkVI3uBoUDPjL5?=
 =?us-ascii?Q?qKu5K65eki1813R4nXegGlW9Qy6+3Zy2az0/3iT8SX0BPTf1j3xVfFE957Bd?=
 =?us-ascii?Q?4DgGtrOziUy/ZprBaFZ1fIwVqXJOdmX1hkhrjFMjndzXdEG4Tyn04f4d+qN/?=
 =?us-ascii?Q?foKG4l+q0lcrWD/vVIz+YwvxPEllsF66tRgEkewEuW+qTMaeflF5IFUDAlUo?=
 =?us-ascii?Q?n1DxQMDMrOBXKwHnELfvFqkzgfWf/burBCMXbSLX4QfLSyQ85dcwe/9pK39R?=
 =?us-ascii?Q?mG/vXPICfPs3Y/SoO2/RGc974vsQ9aNZRHemntQR/ykKmjaqurgVK6clV/QC?=
 =?us-ascii?Q?/QqrjMXWztBjVjFkdpxwoZ5iU1pUeUnGqJq+uGix2AEXsT6zmZk6nemTAY0T?=
 =?us-ascii?Q?Nu8OvdgZPT0+9aRLQv1NESNgfrJ6Xe4zZtO1TUOc+pZUnQD88Mv6XMwLVOVY?=
 =?us-ascii?Q?+Bu+AeiEWssB/7yNSIW/xNhVVIuzRQKepxsxwIoU3rSmIqmGAKC1mTo8fCVg?=
 =?us-ascii?Q?SHawI9oDjGWh2Gk1u5djipip4iCgXZ2dBcR0jFKW4c0BbpFBAe+hbRjp2zZF?=
 =?us-ascii?Q?dANiYXAUvQujZtKOqT3XrC/zSYWGyIjjjMYIajR3d0XapKUrVNh/QImea+eJ?=
 =?us-ascii?Q?sQ=3D=3D?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 662b23e8-9920-49ab-cac5-08dadd40c776
X-MS-Exchange-CrossTenant-AuthSource: BY5PR10MB4196.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Dec 2022 19:32:32.2317
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Rw54ow3m2wdIHHJDihUMQyl6oUGgn1ThrHj/QIIzi8o1iV+M3KCLt1P7eKF3ypGuOpWCeVJ11+zLyiXgZ7dscQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR10MB7080
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1
 definitions=2022-12-13_03,2022-12-13_01,2022-06-22_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 suspectscore=0 bulkscore=0
 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 malwarescore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000
 definitions=main-2212130170
X-Proofpoint-ORIG-GUID: qF3c-QNtj10nWJIaJ1VbAtkWH8K5J07O
X-Proofpoint-GUID: qF3c-QNtj10nWJIaJ1VbAtkWH8K5J07O
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 10/21/22 16:36, James Houghton wrote:
> These functions are used to allocate new PTEs below the hstate PTE. This
> will be used by hugetlb_walk_step, which implements stepping forwards in
> a HugeTLB high-granularity page table walk.
> 
> The reasons that we don't use the standard pmd_alloc/pte_alloc*
> functions are:
>  1) This prevents us from accidentally overwriting swap entries or
>     attempting to use swap entries as present non-leaf PTEs (see
>     pmd_alloc(); we assume that !pte_none means pte_present and
>     non-leaf).
>  2) Locking hugetlb PTEs can different than regular PTEs. (Although, as
>     implemented right now, locking is the same.)
>  3) We can maintain compatibility with CONFIG_HIGHPTE. That is, HugeTLB
>     HGM won't use HIGHPTE, but the kernel can still be built with it,
>     and other mm code will use it.
> 
> When GENERAL_HUGETLB supports P4D-based hugepages, we will need to
> implement hugetlb_pud_alloc to implement hugetlb_walk_step.
> 
> Signed-off-by: James Houghton <jthoughton@google.com>
> ---
>  include/linux/hugetlb.h |  5 +++
>  mm/hugetlb.c            | 94 +++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 99 insertions(+)
> 
> diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
> index d30322108b34..003255b0e40f 100644
> --- a/include/linux/hugetlb.h
> +++ b/include/linux/hugetlb.h
> @@ -119,6 +119,11 @@ void hugetlb_pte_copy(struct hugetlb_pte *dest, const struct hugetlb_pte *src)
>  
>  bool hugetlb_pte_present_leaf(const struct hugetlb_pte *hpte, pte_t pte);
>  
> +pmd_t *hugetlb_pmd_alloc(struct mm_struct *mm, struct hugetlb_pte *hpte,
> +		unsigned long addr);
> +pte_t *hugetlb_pte_alloc(struct mm_struct *mm, struct hugetlb_pte *hpte,
> +		unsigned long addr);
> +
>  struct hugepage_subpool {
>  	spinlock_t lock;
>  	long count;
> diff --git a/mm/hugetlb.c b/mm/hugetlb.c
> index a0e46d35dabc..e3733388adee 100644
> --- a/mm/hugetlb.c
> +++ b/mm/hugetlb.c
> @@ -341,6 +341,100 @@ static bool has_same_uncharge_info(struct file_region *rg,
>  #endif
>  }
>  
> +pmd_t *hugetlb_pmd_alloc(struct mm_struct *mm, struct hugetlb_pte *hpte,
> +		unsigned long addr)

A little confused as there are no users yet ... Is hpte the 'hstate PTE'
that we are trying to allocate ptes under?  For example, in the case of
a hugetlb_pmd_alloc caller hpte would be a PUD or CONT_PMD size pte?

> +{
> +	spinlock_t *ptl = hugetlb_pte_lockptr(mm, hpte);
> +	pmd_t *new;
> +	pud_t *pudp;
> +	pud_t pud;
> +
> +	if (hpte->level != HUGETLB_LEVEL_PUD)
> +		return ERR_PTR(-EINVAL);

Ah yes, it is PUD level.  However, I guess CONT_PMD would also be valid
on arm64?

> +
> +	pudp = (pud_t *)hpte->ptep;
> +retry:
> +	pud = *pudp;

We might want to consider a READ_ONCE here.  I am not an expert on such
things, but recall a similar as pointed out in the now obsolete commit
27ceae9833843.

-- 
Mike Kravetz

> +	if (likely(pud_present(pud)))
> +		return unlikely(pud_leaf(pud))
> +			? ERR_PTR(-EEXIST)
> +			: pmd_offset(pudp, addr);
> +	else if (!huge_pte_none(huge_ptep_get(hpte->ptep)))
> +		/*
> +		 * Not present and not none means that a swap entry lives here,
> +		 * and we can't get rid of it.
> +		 */
> +		return ERR_PTR(-EEXIST);
> +
> +	new = pmd_alloc_one(mm, addr);
> +	if (!new)
> +		return ERR_PTR(-ENOMEM);
> +
> +	spin_lock(ptl);
> +	if (!pud_same(pud, *pudp)) {
> +		spin_unlock(ptl);
> +		pmd_free(mm, new);
> +		goto retry;
> +	}
> +
> +	mm_inc_nr_pmds(mm);
> +	smp_wmb(); /* See comment in pmd_install() */
> +	pud_populate(mm, pudp, new);
> +	spin_unlock(ptl);
> +	return pmd_offset(pudp, addr);
> +}
> +
> +pte_t *hugetlb_pte_alloc(struct mm_struct *mm, struct hugetlb_pte *hpte,
> +		unsigned long addr)
> +{
> +	spinlock_t *ptl = hugetlb_pte_lockptr(mm, hpte);
> +	pgtable_t new;
> +	pmd_t *pmdp;
> +	pmd_t pmd;
> +
> +	if (hpte->level != HUGETLB_LEVEL_PMD)
> +		return ERR_PTR(-EINVAL);
> +
> +	pmdp = (pmd_t *)hpte->ptep;
> +retry:
> +	pmd = *pmdp;
> +	if (likely(pmd_present(pmd)))
> +		return unlikely(pmd_leaf(pmd))
> +			? ERR_PTR(-EEXIST)
> +			: pte_offset_kernel(pmdp, addr);
> +	else if (!huge_pte_none(huge_ptep_get(hpte->ptep)))
> +		/*
> +		 * Not present and not none means that a swap entry lives here,
> +		 * and we can't get rid of it.
> +		 */
> +		return ERR_PTR(-EEXIST);
> +
> +	/*
> +	 * With CONFIG_HIGHPTE, calling `pte_alloc_one` directly may result
> +	 * in page tables being allocated in high memory, needing a kmap to
> +	 * access. Instead, we call __pte_alloc_one directly with
> +	 * GFP_PGTABLE_USER to prevent these PTEs being allocated in high
> +	 * memory.
> +	 */
> +	new = __pte_alloc_one(mm, GFP_PGTABLE_USER);
> +	if (!new)
> +		return ERR_PTR(-ENOMEM);
> +
> +	spin_lock(ptl);
> +	if (!pmd_same(pmd, *pmdp)) {
> +		spin_unlock(ptl);
> +		pgtable_pte_page_dtor(new);
> +		__free_page(new);
> +		goto retry;
> +	}
> +
> +	mm_inc_nr_ptes(mm);
> +	smp_wmb(); /* See comment in pmd_install() */
> +	pmd_populate(mm, pmdp, new);
> +	spin_unlock(ptl);
> +	return pte_offset_kernel(pmdp, addr);
> +}
> +
>  static void coalesce_file_region(struct resv_map *resv, struct file_region *rg)
>  {
>  	struct file_region *nrg, *prg;
> -- 
> 2.38.0.135.g90850a2211-goog
>