Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <7fa5b2b2-dcef-f264-7932-c4fdbb9619d0@intel.com>
Date:   Thu, 18 Aug 2022 16:32:13 +0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Firefox/91.0 Thunderbird/91.12.0
Subject: Re: [PATCH 4/6] mm: hugetlb_vmemmap: add missing smp_wmb() before
 set_pte_at()
Content-Language: en-US
To:     Muchun Song <muchun.song@linux.dev>,
        Miaohe Lin <linmiaohe@huawei.com>
CC:     Andrew Morton <akpm@linux-foundation.org>,
        Mike Kravetz <mike.kravetz@oracle.com>,
        Muchun Song <songmuchun@bytedance.com>,
        Linux MM <linux-mm@kvack.org>, <linux-kernel@vger.kernel.org>
References: <20220816130553.31406-1-linmiaohe@huawei.com>
 <20220816130553.31406-5-linmiaohe@huawei.com>
 <D8C00BDA-160D-40CE-AFBD-9488F85E76CE@linux.dev>
 <ea67ab10-667e-f361-b80f-dafb13da4808@huawei.com>
 <0EAF1279-6A1C-41FA-9A32-414C36B3792A@linux.dev>
 <019c1272-9d01-9d51-91a0-2d656b25c318@intel.com>
 <18adbf89-473e-7ba6-9a2b-522e1592bdc6@huawei.com>
 <9c791de0-b702-1bbe-38a4-30e87d9d1b95@intel.com>
 <931536E2-3948-40AB-88A7-E36F67954AAA@linux.dev>
 <7be98c64-88a1-3bee-7f92-67bb1f4f495b@huawei.com>
 <3B1463C2-9DC4-43D0-93EC-2D2334A20502@linux.dev>
From:   "Yin, Fengwei" <fengwei.yin@intel.com>
In-Reply-To: <3B1463C2-9DC4-43D0-93EC-2D2334A20502@linux.dev>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?SmEwN2VJU3ZyR0xRWWN5MTRNR1RhRmczY2haTWpkZ3hkT2RKRlpFODVvcXdM?=
 =?utf-8?B?aUEvYWZJVnNqanN3a01OaFdqOTBnWEtEWU55ekpCSE9LZHNpQlU1SC9MRHNs?=
 =?utf-8?B?aVl4QUN4VWorTUYrK0kzTFRsWStkN0V2SFVMTXdMWG5wWVMxQkFGRngyRXJl?=
 =?utf-8?B?K1RxRnpnU0YvNzQ3QTZrdDhVZGxpeWtPL29rR0tUejMrcWRncXVXMnNJN2FG?=
 =?utf-8?B?K3MwWGVkeUFyQitNOGs1c3U4UXFwcVozVWYzdEpsQ0tveGdZZTFoU2ZIT2hV?=
 =?utf-8?B?S0wrVTUvTUQ4LzJoTW1pandNcGJvQmU3cUZSdmM1cGxmYjNpMWtmUHBYR3dU?=
 =?utf-8?B?SFNpK1BIOVU1ZWVZQ2VOeThHalA3dnpuQTJTMUFWYTAzNGJEN3RtSDhyaEdC?=
 =?utf-8?B?eVAwc3VoUnpUYmxVRkY1SVYvWnJZQjJaak5BL0d3bWhzQ2hoaDd3RUU1MGpK?=
 =?utf-8?B?UVRXdnEwMDRVbENZMlRxWks1ODZ6V21jdFk4V01zWEpHdkZwTzlVWUNyYysz?=
 =?utf-8?B?bTRFai9zclM3NW9oS1pwRmF2WkZ6Q1h3MzhYVW5xTDhsYnN3OHltUzR3MGI0?=
 =?utf-8?B?dStMeENjSEJYM3IyT21uemc1UXdkSU84WkI1OEthdW1nZXZhWnhvSGQwUFlJ?=
 =?utf-8?B?cFBEcTNYbkVnY1ByQ05ncTQvWU5aOGxPSTBMMUdYRzJIZ1cwUVlqWnJkMUg1?=
 =?utf-8?B?Rzdkcko0ZDFVT0RXTWFVT3RndzNPeUFPVGp1S3BUOTNWUG5nWVRMMzF6aSsy?=
 =?utf-8?B?cGkxSmhkL3lCTkN6VGR1ay9CRzViZXg3eGduRlJQU3ViTDFvV3VRaWlhZytv?=
 =?utf-8?B?NzUxYmRRdzFXTDBZRHZlYnoycS92T05DMHF0ZGRIZmtwT25MYTl4WDcvRUtL?=
 =?utf-8?B?ekQ1REc4TmVFTDRFK2NmR25OaWRCdDY0ejJqL1IycEM3WjhMcDlkdUhmc1ly?=
 =?utf-8?B?ME9VNXd3aDBEdFdhUEl4bVlKY0I1Y3V1SGJzdnBhc1BseVRBVkM3K3dzSTQ2?=
 =?utf-8?B?L3NzenJFcTQ1NGtROHlHN1dHKzNLSXVsYTRrK210bTFLRE5MK0F1SFVLS1Zy?=
 =?utf-8?B?S3NLdHRpdUpuWVg4YlIrSlBJb3VFLzdYdFlrSk5mOVdxOEVUclllNlVHUG5j?=
 =?utf-8?B?ZTlhV2RueE5HdTE2Ri82SVhsYkFKMTUwcW1VYXFrQS90THJtMkFjYkl1eGxC?=
 =?utf-8?B?ZlB2SU5sQS9XeGorcW9uZFNiRFl6cm5aWHdVdFliVFI0R2p5cCs2aWVCWDhi?=
 =?utf-8?B?MU9oRHpKOGJ0by9NRFBTMkxra1pwWkNmd3pDR1V6Vk9UT0hUUGZXSVFzY21s?=
 =?utf-8?B?MDIyWW0zLy9WOGxPVG5uK01leHdrL0g2WThQaEdNbnBJbjBwSXM2dUtQZEF0?=
 =?utf-8?B?TVZ6YVlBM1VzeDFrRzdGbS9tdWEzUEpReDdhMDIvZHpHVEFWcTNpSGJ6ZGNE?=
 =?utf-8?B?TkRHK1loNHBrNERKdE5xQnF1S245OXg1Ymt4dEF2L2R3anZ0OXJrc2F5K1U5?=
 =?utf-8?B?YjR1d05VQWNyTnIxMVZRZHRHRTNyRmlNcmt6YTVoVHg1b1Q3UkF3eC9CVjND?=
 =?utf-8?B?Y1puNk5ZVHpKVm16YWQ1MG0xTDM1allpUENHZFZJYkxyZytZemNYQ3FKWGkx?=
 =?utf-8?B?R3ZaNlNxVUMzRGJ3bWhaeTB6WkNRSUJWNE83Y2hXOS90bEhyU3ZVUmRRUWtS?=
 =?utf-8?B?b21wbnpWRE5GZ3pES2xRS01VNVg1RzA2YTRwZnVQSnpGMlh0Zk5XVUVPcmJt?=
 =?utf-8?B?andKUkFkTGJFajZEWmJpeCtjSTlOZWFSZ01nQ1pPY3E1b1FhZlI5WGFWcFJ0?=
 =?utf-8?B?cGQyTk11Zy9kMkoycW1IcjBuV1pnK1JjRE9uWTNHNGlVN1lpK2NGS25ua1FR?=
 =?utf-8?B?YkFFM0R6QjJKbkFPdHJLZUViQ3NIbDhMZnU0WEJ0bzZnNysyL0twMk1kSWNN?=
 =?utf-8?B?M1dWbDNtRS9adG9FS2k4R3VmeEdrUWc0cnAyMjQ2Z1R1bjlaSEphc2Z2TTE1?=
 =?utf-8?B?Mnp2TFZUbWpLMkhlcFYxa0Y0bmRXZGVGSlZRakxRYXVjWTBGb0FwR01ONk5N?=
 =?utf-8?B?ODM1UXplZkx0MTVYSlMvQmFEUmdSTHlyQnh4N0xlRk9SWXdTbEFBU2hUUm9O?=
 =?utf-8?Q?fQfPRwbX45KEC/5BEx0cYzzMX?=
X-MS-Exchange-CrossTenant-Network-Message-Id: a5ab24d3-b673-4262-4d61-08da80f42bcd
X-MS-Exchange-CrossTenant-AuthSource: CO1PR11MB4820.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Aug 2022 08:32:22.4252
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: PHAJcSqTUfwbFoIVxdGa1BgDpXQEW47lLkdPgceG22ne9VGaVx55IvBmtNXFTifKf8fhEwX2IdCTCUoLKGYQyQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA2PR11MB5019
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,
        RCVD_IN_DNSWL_HI,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,
        SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 8/18/2022 3:59 PM, Muchun Song wrote:
> 
> 
>> On Aug 18, 2022, at 15:52, Miaohe Lin <linmiaohe@huawei.com> wrote:
>>
>> On 2022/8/18 10:47, Muchun Song wrote:
>>>
>>>
>>>> On Aug 18, 2022, at 10:00, Yin, Fengwei <fengwei.yin@intel.com> wrote:
>>>>
>>>>
>>>>
>>>> On 8/18/2022 9:55 AM, Miaohe Lin wrote:
>>>>>>>> 	/*
>>>>>>>> 	 * The memory barrier inside __SetPageUptodate makes sure that
>>>>>>>> 	 * preceding stores to the page contents become visible before
>>>>>>>> 	 * the set_pte_at() write.
>>>>>>>> 	 */
>>>>>>>> 	__SetPageUptodate(page);
>>>>>>> IIUC, the case here we should make sure others (CPUs) can see new page’s
>>>>>>> contents after they have saw PG_uptodate is set. I think commit 0ed361dec369
>>>>>>> can tell us more details.
>>>>>>>
>>>>>>> I also looked at commit 52f37629fd3c to see why we need a barrier before
>>>>>>> set_pte_at(), but I didn’t find any info to explain why. I guess we want
>>>>>>> to make sure the order between the page’s contents and subsequent memory
>>>>>>> accesses using the corresponding virtual address, do you agree with this?
>>>>>> This is my understanding also. Thanks.
>>>>> That's also my understanding. Thanks both.
>>>> I have an unclear thing (not related with this patch directly): Who is response
>>>> for the read barrier in the read side in this case?
>>>>
>>>> For SetPageUptodate, there are paring write/read memory barrier.
>>>>
>>>
>>> I have the same question. So I think the example proposed by Miaohe is a little
>>> difference from the case (hugetlb_vmemmap) here.
>>
>> Per my understanding, memory barrier in PageUptodate() is needed because user might access the
>> page contents using page_address() (corresponding pagetable entry already exists) soon. But for
>> the above proposed case, if user wants to access the page contents, the corresponding pagetable
>> should be visible first or the page contents can't be accessed. So there should be a data dependency
>> acting as memory barrier between pagetable entry is loaded and page contents is accessed.
>> Or am I miss something?
> 
> Yep, it is a data dependency. The difference between hugetlb_vmemmap and PageUptodate() is that
> the page table (a pointer to the mapped page frame) is loaded by MMU while PageUptodate() is
> loaded by CPU. Seems like the data dependency should be inserted between the MMU access and the CPU
> access. Maybe it is hardware’s guarantee?
I just found the comment in pmd_install() explained why most arch has no read
side memory barrier except alpha which has read side memory barrier.


Regards
Yin, Fengwei

> 
>>
>> Thanks,
>> Miaohe Lin
>