Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Tue, 12 Jul 2022 10:51:06 -0700
From:   Mike Kravetz <mike.kravetz@oracle.com>
To:     "Dr. David Alan Gilbert" <dgilbert@redhat.com>
Cc:     James Houghton <jthoughton@google.com>,
        Muchun Song <songmuchun@bytedance.com>,
        Peter Xu <peterx@redhat.com>,
        David Hildenbrand <david@redhat.com>,
        David Rientjes <rientjes@google.com>,
        Axel Rasmussen <axelrasmussen@google.com>,
        Mina Almasry <almasrymina@google.com>,
        Jue Wang <juew@google.com>,
        Manish Mishra <manish.mishra@nutanix.com>, linux-mm@kvack.org,
        linux-kernel@vger.kernel.org
Subject: Re: [RFC PATCH 07/26] hugetlb: add hugetlb_pte to track HugeTLB page
 table entries
Message-ID: <Ys20iusvF1An6BLs@monkey>
References: <20220624173656.2033256-1-jthoughton@google.com>
 <20220624173656.2033256-8-jthoughton@google.com>
 <YsyzGMS+MS0kZoP8@monkey>
 <Ys1B+fXo2uSPd46B@work-vm>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Ys1B+fXo2uSPd46B@work-vm>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?dSuBmNDnZ39BWD+eScz+af7af5dZV0uZbVFODL+BsTdIedaRt9ki69k2/xBD?=
 =?us-ascii?Q?SevlqEsn5DdrOU5+cNUp4n7AbyBMzarwQnVWxZ3qw4IHWYzuHTKNrsBNBgRr?=
 =?us-ascii?Q?rCY8v49uXs11FP3qp8Um6Jy2zzZmEgnxMmJpYVuTqkqbmglrRxzAgys0jkBF?=
 =?us-ascii?Q?9vrJDuH9/NM5g1k2NZdD4sWwUw/XcxZQg+XlHzW+8KoOtyr0Xn/YFzXuCcvJ?=
 =?us-ascii?Q?g5TPPxvpxgf8OgD5iBznO9xQyxU0Aahz9MvgXfdckfHMwqIeYVkodf0XVZLC?=
 =?us-ascii?Q?wVteCxe6XiTkwV4LVTvrjsFLwjG/sAfqnw2tRPhqsaurL4FjaZbPCYs8+R72?=
 =?us-ascii?Q?z381OsbKvI3vCXLWrL3wGxlotFMF3zvLEx4LxlcU3lXVKbKcfiBtAO5UllR6?=
 =?us-ascii?Q?0knZw7+ocx8ElD+hd7rscJA4tEt+gLUNqkEgN97R4E0mSdYxxoiy2pU4hr8e?=
 =?us-ascii?Q?prm785YoyhzsRucl/TIxmnQBTQAKiuwIEzuF4wNLHXC3FRUOWBDkITw2GLRg?=
 =?us-ascii?Q?YCDRFBeVyCzpS4TDcPRE81dj4b7/hxDBsPS/vVE88tZXhndMW8o/OZjNIXGC?=
 =?us-ascii?Q?L41lMz9OFI0+oyAQ2AbwrlM/xFMp3vhKGEkbmqyySHmCkQ9wX2QjuSPvS8bg?=
 =?us-ascii?Q?V8k1ee9Kw0zSBFh1j3OTPiIibVMPNDvq0ihmHIW2SYq5kIAwxlbtSaZtdQk+?=
 =?us-ascii?Q?Jyh3n2ERUWXn+tFuPasP4wTzhOj9SImuNjkTbIF69Mv6aLfAht2YlasGJApY?=
 =?us-ascii?Q?7Zswy3XcXFlmdq3fkgNNIlE/GMC+pLPB+PqCuYN5yE0dB6T5m+zxj+Y7QGpm?=
 =?us-ascii?Q?TDb3okFq9oaproewELGYhKD42OaZhS+GMPd5hbkFLn7jgGmAlPoe70WIV4vH?=
 =?us-ascii?Q?G9DISIbmNZhfO71oNZHo1CL/I8EjzmTJXEL1sVxnRtSF31tvbxiUvZ+nLcxc?=
 =?us-ascii?Q?DeR/5g5lKNiQZbFEp5hHZ1Y6McLsKT5O+rkv01Rxs0ZNJKxBnP6lmCqMNd4E?=
 =?us-ascii?Q?uBom+uh9MsdViJoCaG8xhUWiml5QCVoF+oQe/UahWlpf1/QChNrtnOhFXoy8?=
 =?us-ascii?Q?pg5/9lJiUSsc1mHkjz1KffOulOwuBtVg9BLfZSTV+3XhweP1EMiSIdK4zVzZ?=
 =?us-ascii?Q?3cEMKF8pwxpcH+eg2I18+1F79zFb5AJ4nwPoWXZXc/+Nww2NNP/v0ZL11fBa?=
 =?us-ascii?Q?eY5Kc+LNwgnPO5sPMXsJTEtBgp3Q7uB903yypeIvZcJoA5DjXc9RSt/LXDeN?=
 =?us-ascii?Q?VdXV9+VkgnIUyUA7wNIjaXL/MDc8xXV3ciWZScSDnL2iYQPxXdIxi1m++zQ3?=
 =?us-ascii?Q?P5qLDUOW2CgTNqB6Wn1IfhYPq5h63GxaSmRidSZYlhrDoUoo3L5Ebl1LO+L/?=
 =?us-ascii?Q?NC1L+aWSmqOWl5Q2YoylRn+XnSK0HVAH3KKKvGa2pjT3IvTGl1sksaHuCI8j?=
 =?us-ascii?Q?vyKLu1k2YVUMkVuc4X2oZH5BRL++cO9nReZ9WSAL0DS32dM6tltMoZin5Tt6?=
 =?us-ascii?Q?4bVAq9LKC9fIzOFocBvYQujkVNlruFQc6eL2dgNKz3/DmENcBPGYWQaTUBGO?=
 =?us-ascii?Q?Yd35gWPWkLXaH1ieKaZCkgw1WH3LVEtsXZoWgK0c8bDWJoris7lbQtJm1yYI?=
 =?us-ascii?Q?Xw=3D=3D?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0d6f3035-d0b2-409b-f1ad-08da642f19e5
X-MS-Exchange-CrossTenant-AuthSource: BY5PR10MB4196.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Jul 2022 17:51:08.8649
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 2+HtlVt8Ma+560z1guLma8HXpoHW9gv0NmbgY33kythX86QeNVnYssn2w9lwkIU9XIXlvWA5AjIxXy0bkloYYg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR10MB2832
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.517,18.0.883
 definitions=2022-07-12_12:2022-07-12,2022-07-12 signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 phishscore=0
 suspectscore=0 mlxlogscore=999 mlxscore=0 spamscore=0 malwarescore=0
 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2206140000 definitions=main-2207120071
X-Proofpoint-ORIG-GUID: zQf2yuDN7MUI2GCTtbeHlW7aLIU2Q_vi
X-Proofpoint-GUID: zQf2yuDN7MUI2GCTtbeHlW7aLIU2Q_vi
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 07/12/22 10:42, Dr. David Alan Gilbert wrote:
> * Mike Kravetz (mike.kravetz@oracle.com) wrote:
> > On 06/24/22 17:36, James Houghton wrote:
> > > After high-granularity mapping, page table entries for HugeTLB pages can
> > > be of any size/type. (For example, we can have a 1G page mapped with a
> > > mix of PMDs and PTEs.) This struct is to help keep track of a HugeTLB
> > > PTE after we have done a page table walk.
> > 
> > This has been rolling around in my head.
> > 
> > Will this first use case (live migration) actually make use of this
> > 'mixed mapping' model where hugetlb pages could be mapped at the PUD,
> > PMD and PTE level all within the same vma?  I only understand the use
> > case from a high level.  But, it seems that we would want to only want
> > to migrate PTE (or PMD) sized pages and not necessarily a mix.
> 
> I suspect we would pick one size and use that size for all transfers
> when in postcopy; not sure if there are any side cases though.
> 
> > The only reason I ask is because the code might be much simpler if all
> > mappings within a vma were of the same size.  Of course, the
> > performance/latency of converting a large mapping may be prohibitively
> > expensive.
> 
> Imagine we're migrating a few TB VM, backed by 1GB hugepages, I'm guessing it
> would be nice to clean up the PTE/PMDs for split 1GB pages as they're
> completed rather than having thousands of them for the whole VM.
> (I'm not sure if that is already doable)

Seems that would be doable by calling MADV_COLLAPSE for 1GB pages as
they are completed.

Thanks for information on post copy.
-- 
Mike Kravetz