Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 15 Jun 2022 10:27:25 -0700
From:   Mike Kravetz <mike.kravetz@oracle.com>
To:     Peter Xu <peterx@redhat.com>
Cc:     linux-kernel@vger.kernel.org, linux-mm@kvack.org,
        Muchun Song <songmuchun@bytedance.com>,
        Michal Hocko <mhocko@suse.com>,
        Naoya Horiguchi <naoya.horiguchi@linux.dev>,
        James Houghton <jthoughton@google.com>,
        Mina Almasry <almasrymina@google.com>,
        "Aneesh Kumar K . V" <aneesh.kumar@linux.vnet.ibm.com>,
        Anshuman Khandual <anshuman.khandual@arm.com>,
        Paul Walmsley <paul.walmsley@sifive.com>,
        Christian Borntraeger <borntraeger@linux.ibm.com>,
        Andrew Morton <akpm@linux-foundation.org>
Subject: Re: [RFC PATCH 1/3] hugetlb: skip to end of PT page mapping when pte
 not present
Message-ID: <YqoWfcZXJ7NfJaYJ@monkey>
References: <20220527225849.284839-1-mike.kravetz@oracle.com>
 <20220527225849.284839-2-mike.kravetz@oracle.com>
 <YpUhe6BSfflOVz7b@xz-m1.local>
 <e9456223-c26a-4d49-2920-9a597a817190@oracle.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <e9456223-c26a-4d49-2920-9a597a817190@oracle.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?1UsL6DzOB1zlKZZSUoTk87MTIl5WxC1QoL74KhO5hz0Gf++Tf8pTYV/NMoaS?=
 =?us-ascii?Q?Wya7kahxzuFxcBs6nm0kMk8rILjkUbzbMyCeYEtQuarLIkY/TokUzFp8cFj5?=
 =?us-ascii?Q?twijoS6315CwmYA8Tp2faN9aiV/M5d1yA0NynvDdM/DiPmlYoqfEZbzRGWjS?=
 =?us-ascii?Q?Lu9WrjmNgvANDSiWbX2kSKt/cTvERXRsuB4ymkgm0Zrxz90t3XwJPa60Dqt9?=
 =?us-ascii?Q?+7h1O0zyWm40q5qRc1/EbS9At3vdC8CO6SMuQC83yi2YiD5lJlz3zwZWWMXB?=
 =?us-ascii?Q?p3YlnHR35dy5JNaw138S4n78p/oUB1C+S05Sa9oKlG5CQb5UtwCfUfKMUYYf?=
 =?us-ascii?Q?yponUW0JoJCtJyq5SByrYglsp1SRuC5Wn7IIKxl109cJwtZ9jTMmA/L0UDn7?=
 =?us-ascii?Q?pwe2GZ7UnKD4Uw0UoqYPLqwcYDlWOM58bKtgbxGrLNakKLC7qP+wxIjhPayb?=
 =?us-ascii?Q?goSAuSGwG68y8EMPcFrCXVJDoinxkfXGobavQ2CK27tLyRoMPH2GId2Ofy83?=
 =?us-ascii?Q?EI33cOVXEbRwi4IkuT+ONq+O0+y0Hkb4dP1F/H55BX64FmtOHzroPwaYrXSk?=
 =?us-ascii?Q?I7UUwA55MAZo7A5xfqIrRDHeGA0DPwlLwZaOYEDnr3HDy312M2Yhmxmq/Qpi?=
 =?us-ascii?Q?72LCTthn2RKvLBrpQLylASW6G52iiTKOkGIT30LZxRBnt6RlO6xQR9xLF3JL?=
 =?us-ascii?Q?CEAncfYydoLrjeyKKcpP3f5ogaB81Eij5gfCjQMGtQNQiGMr3ltyj/YY5/6o?=
 =?us-ascii?Q?qlgQE3M6OUqNwi+uSaUZW0Zb+EFpNqe7q1aqlZfkXuSRKeOK5nZZgqSu6eGB?=
 =?us-ascii?Q?7iQyWX4l7Ae5BNa//AKapE0s5hdm5tcDnCLCZ+4TEA/en5S3LMSQHK3M0WwV?=
 =?us-ascii?Q?Wu9OrSOVV9oek94CaBCueyeNCmoPOw90fENEYKgJ1QtHubm/wl1yMW4itYj4?=
 =?us-ascii?Q?QaSjCJdZjc1RhYteZd9nIOz57k3DwffmTmhqqV+YSzoIysQF/auLUbiqaASk?=
 =?us-ascii?Q?UCU7Zpk9nGbecJ5zNbjlps44Rqb3R2WQD5S5F6ij6Q55ULtOv6mEa28gel9R?=
 =?us-ascii?Q?4xtbFDDnEtJBxJsa3BDzqZNSpNGgCXyYERjFLiusNA/NcI2hn4krwHQFPMoE?=
 =?us-ascii?Q?VvsU3jgXLCWkeVtg+zo6GojLmT0HUFaCxRU3m8AfxCAw3WqargCJcYol+k5J?=
 =?us-ascii?Q?4+detQ/MrS+xmr2JzdvzjuAucnGCSBgnJlWm8Qjm4Kwa13Fxr6KXF3Z/tVhi?=
 =?us-ascii?Q?v3KhuypLNNqcXiu+ylOEleUcHqVgaNbcihyGxrZ+mTDN5EQLK9lLyGZyPeq6?=
 =?us-ascii?Q?bq77UilaIjvZvPArLlY85b8rhxtjZ3S3y28tFO8umhpGcj2BLx0Cz9Hjr5pP?=
 =?us-ascii?Q?rX4M8L7RMXHbcKJ/ApljKk/0EU3HyN3ARtkJRui3356Fwz4I+yDANW6+6JOL?=
 =?us-ascii?Q?BU0DB3ApY0mTomOnycIth0tiJo9XWZUgzVDKlfztWVwTaCuJMBOeMRomof0+?=
 =?us-ascii?Q?ZjspG6rK2ROSq3QOFs9LeM1ZZPGc46IQwrKqoNHPGi3J+e43iPq3cYCPyKES?=
 =?us-ascii?Q?1RodpWShPgwNeBHSD0GXPB3nrsTO1idA+jADG0Ed3D6pmg3Yr+2ge9mpodxG?=
 =?us-ascii?Q?c3fkAl1UmwZbFjVJgvPiFYmPGqRH6mJeEDT8/3D8oo0JPRWNofDKsRzPO5xt?=
 =?us-ascii?Q?9TXFw8R1EyoOSNX4fOUpaBBdAQlCVDTAmO8s9cVNdbi81V6xuIdDKk7r5JvY?=
 =?us-ascii?Q?NebjJ65rOvxy18lurwQa3IwIt0JZP6U=3D?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 43c79dd4-8f70-47fd-78e2-08da4ef452b4
X-MS-Exchange-CrossTenant-AuthSource: BY5PR10MB4196.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Jun 2022 17:27:29.5235
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Bo+djK/44jHDEDTRELL3XR5bVvKyPdNbRpflUgBsjxTepm16EBfv1B0hXThdASLdbHaPYCAum0RBwak2IdXMbQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR10MB3136
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.517,18.0.874
 definitions=2022-06-15_05:2022-06-15,2022-06-15 signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 spamscore=0 adultscore=0
 mlxlogscore=687 phishscore=0 malwarescore=0 bulkscore=0 suspectscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2204290000
 definitions=main-2206150064
X-Proofpoint-ORIG-GUID: oO3p406MJsDf6rQsaIRKxLMSBrkt3Qj6
X-Proofpoint-GUID: oO3p406MJsDf6rQsaIRKxLMSBrkt3Qj6
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 05/31/22 10:00, Mike Kravetz wrote:
> On 5/30/22 12:56, Peter Xu wrote:
> > Hi, Mike,
> > 
> > On Fri, May 27, 2022 at 03:58:47PM -0700, Mike Kravetz wrote:
> >> +unsigned long hugetlb_mask_last_hp(struct hstate *h)
> >> +{
> >> +	unsigned long hp_size = huge_page_size(h);
> >> +
> >> +	if (hp_size == P4D_SIZE)
> >> +		return PGDIR_SIZE - P4D_SIZE;
> >> +	else if (hp_size == PUD_SIZE)
> >> +		return P4D_SIZE - PUD_SIZE;
> >> +	else if (hp_size == PMD_SIZE)
> >> +		return PUD_SIZE - PMD_SIZE;
> >> +
> >> +	return ~(0);
> >> +}
> > 
> > How about:
> > 
> > unsigned long hugetlb_mask_last_hp(struct hstate *h)
> > {
> > 	unsigned long hp_size = huge_page_size(h);
> > 
> > 	return hp_size * (PTRS_PER_PTE - 1);
> > }
> > 
> > ?

As mentioned in a followup e-mail, I am a little worried about this
calculation not being accurate for all configurations.  Today,
PTRS_PER_PTE == PTRS_PER_PMD == PTRS_PER_PUD == PTRS_PER_P4D in all
architectures that CONFIG_ARCH_WANT_GENERAL_HUGETLB.  However, if we
code things as above and that changes the bug might be hard to find.

In the next version, I will leave this as above but move to a switch
statement for better readability.

> > 
> > This is definitely a good idea, though I'm wondering the possibility to go
> > one step further to make hugetlb pgtable walk just like the normal pages.
> > 
> > Say, would it be non-trivial to bring some of huge_pte_offset() into the
> > walker functions, so that we can jump over even larger than PTRS_PER_PTE
> > entries (e.g. when p4d==NULL for 2m huge pages)?  It's very possible I
> > overlooked something, though.

I briefly looked at this.  To make it work, the walker zapping functions
such as zap_*_range would need to have a 'is_vm_hugetlb_page(vma)', and
if true use hugetlb specific page table routines instead of the generic
routines.

In many cases, the hugetlb specific page table routines are the same as
the generic routines.  But, there are a few exceptions.  IMO, it would
be better to first try to cleanup and unify those routines.  That would
make changes to the walker routines less invasive and easier to
maintain.  I believe is other code that would benefit from such a
cleanup.  Unless there are strong objections, I suggest we move forward
with the optimization here and move the cleanup and possible walker
changes to a later series.
-- 
Mike Kravetz