Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 19 Aug 2022 09:55:19 -0700
From:   Mike Kravetz <mike.kravetz@oracle.com>
To:     Michael Ellerman <mpe@ellerman.id.au>
Cc:     Andrew Morton <akpm@linux-foundation.org>,
        "Wang, Haiyue" <haiyue.wang@intel.com>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "david@redhat.com" <david@redhat.com>,
        "apopple@nvidia.com" <apopple@nvidia.com>,
        "linmiaohe@huawei.com" <linmiaohe@huawei.com>,
        "Huang, Ying" <ying.huang@intel.com>,
        "songmuchun@bytedance.com" <songmuchun@bytedance.com>,
        "naoya.horiguchi@linux.dev" <naoya.horiguchi@linux.dev>,
        "alex.sierra@amd.com" <alex.sierra@amd.com>,
        Heiko Carstens <hca@linux.ibm.com>,
        Vasily Gorbik <gor@linux.ibm.com>,
        Alexander Gordeev <agordeev@linux.ibm.com>,
        Christian Borntraeger <borntraeger@linux.ibm.com>,
        Sven Schnelle <svens@linux.ibm.com>,
        linuxppc-dev@lists.ozlabs.org
Subject: Re: [PATCH v6 1/2] mm: migration: fix the FOLL_GET failure on
 following huge page
Message-ID: <Yv/AdwdID9rr9xOL@monkey>
References: <20220812084921.409142-1-haiyue.wang@intel.com>
 <20220816022102.582865-1-haiyue.wang@intel.com>
 <20220816022102.582865-2-haiyue.wang@intel.com>
 <20220816175838.211a1b1e85bc68c439101995@linux-foundation.org>
 <BYAPR11MB3495F747CBF95E079E8FC8A5F76A9@BYAPR11MB3495.namprd11.prod.outlook.com>
 <20220816224322.33e0dfbcbf522fcdc2026f0e@linux-foundation.org>
 <Yv0ku1mn4LBzg/zG@monkey>
 <875yiomq9z.fsf@mpe.ellerman.id.au>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <875yiomq9z.fsf@mpe.ellerman.id.au>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Ui6v/EM8N1RDnRm1YgvoCLL/qqyscLqN1CpUvhVGdni0ypdPGImRwarTLRx2?=
 =?us-ascii?Q?m8OSwNkIIOVYRzJpWkPfqvduJb2w096Enp4/9Zjo3LJqgjSQmPHmSUxei1Ds?=
 =?us-ascii?Q?eJVrW3oQjBXdaKM7o5aQ1u7Vv3PHZzH+0Yd64JEWNpLBkKk2hMuOTQfHTEyp?=
 =?us-ascii?Q?SGPeHPmfL4w8Jgan50vOLmjGOiDxW5p9YyQyUc/DcjQdPWBC1mfe4eLcLnFK?=
 =?us-ascii?Q?42Zz/uVAC+6boLJrhmnxBJ7hrEb2BwzX4fvZHRWh+CoSfB+EeetQ9DqnfvU5?=
 =?us-ascii?Q?z9Yn6ud5SBKs+59G0qsA99VKK6zkCNOj+ykH3fDYIj00H/1j2i7sTCnQFcU0?=
 =?us-ascii?Q?9HBtQ4pWFyWlAwyTpajqdLa1p+WjInkfi7ndizZSzcNPPniEw2cd6RRb4X6H?=
 =?us-ascii?Q?nV6dCw0fVtZ7xFsIe6BPP5507NnsBi3rHkaFkGvI2wBRDNAOD01KDquImWxn?=
 =?us-ascii?Q?ZyMY5ugSvnjw5Y6W4IdhXnnqz9AKD1LgdEvI4kL2L/uL58GKndj3SFoUOGVP?=
 =?us-ascii?Q?KzCop7Y/OFmfA063LVRiWjh1P0he4faK/uOHhn+ovehDL1iPZW3R9EImm+H9?=
 =?us-ascii?Q?+Y9vcWTBti25CoEOWOAfbd1LILz4m2l1ALXy2r9SCIHCcAzbhF5LpuNJ27JJ?=
 =?us-ascii?Q?Q1VPR40FAsR4JvMew6Ep3kTy/SR9pxquM0iczbFnAbZy1WdNgoKNrtVJpCIW?=
 =?us-ascii?Q?VJZ06rn6PxuHXcT/Q2/DMIKQWQlqtsLiPb4dJYu59gGboyvXrFWYzQ6XoW0T?=
 =?us-ascii?Q?pltHZM9zRlRjxm8TeWuTZSlsiv2MdUaXozGZrbgcuGtt2gjzdMAo2Qnc9BNP?=
 =?us-ascii?Q?qlAPZquX/A4Nroq11gqB5d05K3PzpF2dNHAp5/9y2jhy4m77YYm8iE/BJZLC?=
 =?us-ascii?Q?cz2ms6pNj++wvg/Dv/fmnJPNRTPUyUe72rT3qXxK+/pBh/YGuusaxXMGfGGw?=
 =?us-ascii?Q?e1v0JlcTYIJjY1+DrldmJFw/330hkPExjDUfT+YIkJxrcjD9kMtejBvcQxQO?=
 =?us-ascii?Q?kBgkoI+6Y1Y8gZLYveFLNC2fJAksMhYJBATlibEr37+hJU7Am6Vrc8zT1ebC?=
 =?us-ascii?Q?Z9Q7gUnwnloqOoSc6RWribk4rsKQr9PwreUPqGOltCQVB3nPL2ClTqzRoa8W?=
 =?us-ascii?Q?W6gi+DaYEfInQgt7wRDEN8erHpFXOAHoLZPBk+h6wYSFtYkjQgriMZPG2phk?=
 =?us-ascii?Q?wK0y2WrpsayhIf0vdFzCRIAFg0lXlzwvQfLiayzMLkFGig9Zt38iv1gXw9AO?=
 =?us-ascii?Q?gcMoTF9tX9BiwZpWGT/1oA31apU77ICjV7vsUMvEQLHOWNvdgvqpyzktaRqe?=
 =?us-ascii?Q?Y/pGYuW5zQ2/TUevvnMTLgn/4M2edxouS3znwQUYppXVitClkQZJgSRpKKaq?=
 =?us-ascii?Q?Q0zZEYpDD4kX1cUoqG1IFUuLhnRnSdI4mzhWN+W2zyhoYaS40uEp8IwjowLn?=
 =?us-ascii?Q?s4Fv4VlijBNmsnW9mYU5dR33EI/97ZE+sI0vAS1Zvw2nYil4WB5N+y13F2Lk?=
 =?us-ascii?Q?jDkE828erq5wK3J3vpQnCXh+4kQXdolcbJknnQNI+LnQSp0wRsgIu0mEl4Rv?=
 =?us-ascii?Q?Ta/vL4lOz5euxebn4xtlKUuLh9QdJXuGWYX32M0BCxljmOAl6bWe8cQQb6xg?=
 =?us-ascii?Q?wg=3D=3D?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 40268e7d-b4c4-4926-769e-08da82039b2d
X-MS-Exchange-CrossTenant-AuthSource: DM6PR10MB4201.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Aug 2022 16:55:23.0046
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 0BrDQ7EDmcc7JCHymPjOrIeTJHio/st3QaOVb4j1dFWa/U0nOU1EaT8RR7ZcG1bTfepWEF7tiDD8zhkRZuOVLg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR10MB2447
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.517,FMLib:17.11.122.1
 definitions=2022-08-19_08,2022-08-18_01,2022-06-22_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 mlxlogscore=999
 phishscore=0 mlxscore=0 malwarescore=0 suspectscore=0 adultscore=0
 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2207270000 definitions=main-2208190062
X-Proofpoint-ORIG-GUID: bmEWJnTJNwoXZrZg0WAT_wl1a4wRkwBg
X-Proofpoint-GUID: bmEWJnTJNwoXZrZg0WAT_wl1a4wRkwBg
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 08/19/22 21:22, Michael Ellerman wrote:
> Mike Kravetz <mike.kravetz@oracle.com> writes:
> > On 08/16/22 22:43, Andrew Morton wrote:
> >> On Wed, 17 Aug 2022 03:31:37 +0000 "Wang, Haiyue" <haiyue.wang@intel.com> wrote:
> >>
> >> > > >  		}
> >> > >
> >> > > I would be better to fix this for real at those three client code sites?
> >> >
> >> > Then 5.19 will break for a while to wait for the final BIG patch ?
> >>
> >> If that's the proposal then your [1/2] should have had a cc:stable and
> >> changelog words describing the plan for 6.0.
> >>
> >> But before we do that I'd like to see at least a prototype of the final
> >> fixes to s390 and hugetlb, so we can assess those as preferable for
> >> backporting.  I don't think they'll be terribly intrusive or risky?
> >
> > I will start on adding follow_huge_pgd() support.  Although, I may need
> > some help with verification from the powerpc folks, as that is the only
> > architecture which supports hugetlb pages at that level.
> >
> > mpe any suggestions?
> 
> I'm happy to test.
> 
> I have a system where I can allocate 1GB huge pages.
> 
> I'm not sure how to actually test this path though. I hacked up the
> vm/migration.c test to allocate 1GB hugepages, but I can't see it going
> through follow_huge_pgd() (using ftrace).

I thing you needed to use 16GB to trigger this code path.  Anshuman introduced
support for page offline (and migration) at this level in commit 94310cbcaa3c
("mm/madvise: enable (soft|hard) offline of HugeTLB pages at PGD level").
When asked about the use case, he mentioned:

"Yes, its in the context of 16GB pages on POWER8 system where all the
 gigantic pages are pre allocated from the platform and passed on to
 the kernel through the device tree. We dont allocate these gigantic
 pages on runtime."

-- 
Mike Kravetz

> 
> Maybe I hacked it up badly, I'll have a closer look on Monday. But if
> you have any tips on how to trigger that path let me know :)
> 
> cheers