Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <89af9b3f-1ab9-15db-d476-574271ce8292@oracle.com>
Date:   Mon, 7 Mar 2022 11:07:32 -0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.5.0
Subject: Re: [PATCH 1/4] mm/memory-failure.c: fix race with changing page
 compound again
Content-Language: en-US
To:     =?UTF-8?B?SE9SSUdVQ0hJIE5BT1lBKOWggOWPoyDnm7TkuZ8p?= 
        <naoya.horiguchi@nec.com>, Miaohe Lin <linmiaohe@huawei.com>
Cc:     "akpm@linux-foundation.org" <akpm@linux-foundation.org>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
References: <20220228140245.24552-1-linmiaohe@huawei.com>
 <20220228140245.24552-2-linmiaohe@huawei.com>
 <20220304082624.GA3778609@hori.linux.bs1.fc.nec.co.jp>
 <79f79813-bb9a-bfff-0837-a24d14da0b19@oracle.com>
 <999a0f88-9465-0c93-e922-8aa80fa64f81@huawei.com>
 <20220307070155.GA591088@hori.linux.bs1.fc.nec.co.jp>
From:   Mike Kravetz <mike.kravetz@oracle.com>
In-Reply-To: <20220307070155.GA591088@hori.linux.bs1.fc.nec.co.jp>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?SHIrMW82Q05DZHpkS0RaSzIyc0p5VjBEakNXQWdxZjhndGxNajI1VGhGLzZs?=
 =?utf-8?B?QVRrLzBsY3pHNFcyQXBXREtKNDRIcTVyREpLTldSMmtNMStHblBaN1FDUGRD?=
 =?utf-8?B?alBZTkFwTHJnaEthTElNemtxZEFXb1lvNklGZDdTbFpkMzgwaFkrT3hkQnF6?=
 =?utf-8?B?UGFYVXJ6VzRYTnlVZzQ2MUJnSys5bWNsUUF5d3NpSlBUM2l1WDJUelNoN0Uy?=
 =?utf-8?B?UUhuUGdOZ2hwVTI0UjBNTm1hWXdrMys3ejBHY29GYWZZODhXTVRBaDdxTWpC?=
 =?utf-8?B?YWtxcWVPdjRZc25mYVpzWEpWdkFpb0VUdDFOYlh0MVZqUGcvbTJJZS85clF1?=
 =?utf-8?B?WDgyNHNzUWdCV1lIdGFqRTlTTUwyQndOTFFBdGZoZDdBeGdjSEFwMm41NTRQ?=
 =?utf-8?B?NHJrSWtjQ2VUdjRZSlFvNHROWVBxV0ZQNFFlaEdTckJjVkV4Vmp5a0h6SW4y?=
 =?utf-8?B?alBuSG9GS3VlK21WaTZlaG4yci9lUmdwbzRNdDBEczZLSW80TFhlRE9OTEhl?=
 =?utf-8?B?TU9yaU0yT04rN1E2MmNZaWJwdWNxTFJDTnZtZTZnMkNVRzgxcUFNbVRGK29v?=
 =?utf-8?B?RGNWK3dqSlBPbWFuT2R4bkhCWUdDeEQ1YmU5NFl6a3RwaFJQOUFwZDBrb1M2?=
 =?utf-8?B?ZjFJZS96MjV5REE5Z2NQZ3h0SkVUTEg1akppWEJoUGx3aHA0aDlvQi9TS3ln?=
 =?utf-8?B?WitxSUt4d3hkQW5waitxdFRCSzNMdFRHTmVtRzltUFFPbElrd1pBdDRTbTll?=
 =?utf-8?B?YkpBeGdwWEJWSDJXNllQWGJKcXRsdkZFYjJsVUJQWTg4QnRvdVI0cGJJQ1dV?=
 =?utf-8?B?MlBOWUxGR0lIVmVLZG5RU1JheHp3MjlZZUdrT0NjOC9LTmk2aEV5S2E5RU9D?=
 =?utf-8?B?MDhVQzNkcXpzOEFEcDFIS3BOMGpqcStHK2Mzb00zNFlLRCtTeXJsa2FTWnli?=
 =?utf-8?B?T291N0c1T1BEV2JlUHNRRUJvWU9IM09tS0ZzR3lWTVNCb3RNaEJhYjc0VjN2?=
 =?utf-8?B?NjIxWXI4WGo2KzZwcFBWU1ZIRlZJd2dXanl1WDQwSlVOYnMrMEc3TFd6VW0y?=
 =?utf-8?B?SnVJQkdOcHVVUFZ5WGlXUGQxYXZYVUxJd0hCSmR4SUhiVHFVOXg5Q0RWdDZo?=
 =?utf-8?B?WGdJNVF6c3NucmZ3dHN3N2xTNCtQeEY4WDJMU3F5N1JLYkdWNkxLWUE5Q1pi?=
 =?utf-8?B?VXhkK3F5L1lmZHBOdGdQdWNZSSsyY1F6bk9iZjJMTm9JS1dhdkdzd0wrcFNy?=
 =?utf-8?B?ZExwSVhBYUc1RXBzWGpwWHkzT1hPSmkwY2Yva1VoVFFOSmZJMGRLN1pHQWtB?=
 =?utf-8?B?OWdSRDJsWmRjSG5RY2tGTnVLOW4xd1J3UXBrMlZDWk1Qb3IrbC8wSElhZ0Nk?=
 =?utf-8?B?dWdiVVkvTUdUMFV2Yyt5UDZYbmNVU2VMbS9sSldxdjN2bnhtZ09zZEFjeDgy?=
 =?utf-8?B?d2xjc3J1YkdNT2drTi82ZjZpL0NaQWh5Tk5ZOEZhNjVtY3g3V3NTUGd1WnBw?=
 =?utf-8?B?a1RHQXhxVEUxcUJ6QmFmcXRXZ2xRL2ZSdDJmdVlocmoxVFBnd2VaVUNQcnpH?=
 =?utf-8?B?SSs3N3RjN1RocXlvZnhoOEpNWkFaSmwzdyt3d2hsWHhaTkFCUjMxd3NmQ3ov?=
 =?utf-8?B?RTlpSEphdFU3ZGxWNmgzVzNYRDBPQVA0eTdkSGtSeEdKRkpVVjVXQ2RHQzRG?=
 =?utf-8?B?VGM3YmpYWDRKUTVmUzBiMWhzaGphYlZaUVZWaEJQdDI4WDhUcTRnSDdkbFFM?=
 =?utf-8?B?YW83TndleEdsd09oZ3VqV1pqTkJJaUFMQjd5QnVYNitOVndlSGFqWjVHRDJM?=
 =?utf-8?B?TVVtblp6Z1dyN0dXcXRrYlppUVVaQXZ6NVZYRW9JaFd1LzRjUWhqY2tHQ1Vy?=
 =?utf-8?B?cWo1RHl0ekhjV0tWSWVMaEFjWEZJUDdKV0pybUJ3TjFvcnYvZERVbDZscTlF?=
 =?utf-8?B?cXVoeHJDTzVQTnR3eXRaVktLWGthcVdqTENIemxWdUVsbGlNMEpuVE5RZ0Zh?=
 =?utf-8?B?b3VoczJINEl0Y1RwVDV5dHVtMUE1SXMySzBuNk4wSCsxbC9kMXNXNnRBalZm?=
 =?utf-8?B?bm9sRmJ4T1RGWXlRdyszVTZwZ1duNm8wNDNEUHowcGdkMWRKYW1jOFRHL0FR?=
 =?utf-8?B?NjJncWE3bjR3NFlGWjcxcGNndzE1WnZNM29lZm1adEpWdUdhMGt2elVxaGRq?=
 =?utf-8?Q?Q0hMu1BxpiARbUx2o1D5sU4=3D?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 82d84d8d-20a7-4c6f-f98d-08da006dbd2d
X-MS-Exchange-CrossTenant-AuthSource: BY5PR10MB4196.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Mar 2022 19:07:35.3875
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Mqc7Ht+pJjv1G6slfunZyKhEfr6FHmcgT6dUbgSVJb7C0W7tslSuTiAwumaLfd3JAUqCrxYeZCj9Udopa5UHDQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR10MB3282
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10279 signatures=690470
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 mlxscore=0
 suspectscore=0 bulkscore=0 mlxlogscore=999 adultscore=0 spamscore=0
 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2202240000 definitions=main-2203070106
X-Proofpoint-ORIG-GUID: TBOGPDenfjI5LumsYn5aHj1gVWs6iYPE
X-Proofpoint-GUID: TBOGPDenfjI5LumsYn5aHj1gVWs6iYPE
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 3/6/22 23:01, HORIGUCHI NAOYA(堀口 直也) wrote:
> On Mon, Mar 07, 2022 at 11:44:20AM +0800, Miaohe Lin wrote:
>> On 2022/3/5 3:32, Mike Kravetz wrote:
>>> On 3/4/22 00:26, HORIGUCHI NAOYA(堀口 直也) wrote:
>>>> On Mon, Feb 28, 2022 at 10:02:42PM +0800, Miaohe Lin wrote:
>>>>> There is a race window where we got the compound_head, the hugetlb page
>>>>> could be freed to buddy, or even changed to another compound page just
>>>>> before we try to get hwpoison page. If this happens, just bail out.
>>>>
>>>> I think that when some hugetlb page is about to change into other type/size
>>>> of compound page, it has to go through buddy allocator because hugetlb pages
>>>> are maintained in separate memory allocator and they never change into other
>>>> normal state directly.  memory_failure_hugetlb() takes refcount before
>>>> lock_page(), so the hugetlb page seems not change between get_hwpoison_page()
>>>> and lock_page(). So it this new check really necessary?
>>>
>>> A hugetlb page could change size without going through buddy via the new
>>> demote functionality [1].  Only hugetlb pages on the hugetlb free list can
>>> be demoted.  
>>>
>>> We should not demote a page if poison is set.  However, there is no check in
>>> the demote code.  IIUC, poison is set early in the memory error handling
>>> process, even before taking ref on page.  Demote code needs to be fixed so
>>> that poisoned pages are not demoted.  I can do that.
>>>
>>> With this change in place, then I think Naoya's statement that hugetlb pages
>>> can not change state is correct and this patch is not necessary.
>>>
>>
>> Sorry for my confusing commit words. What I mean to tell is indeed the below race:
>>   CPU 1							CPU 2
>>   memory_failure_hugetlb
>>   struct page *head = compound_head(p);
>> 							hugetlb page is freed to buddy, or
>> 							even changed to another compound page
>> 							as we haven't held the page refcnt now
>>   get_hwpoison_page -- page is not what we want now...
>>
>> Does this make sense for both of you? Many thanks for comment and reply! :)
> 
> Thanks for elaboration, I agree with you (I simply overlooked this race, sorry).

Yes, thank you.

> And please add this in the commit log.
> 
>> +
>> +       /**
>> +        * The page could have changed compound pages due to race window.
>> +        * If this happens just bail out.
>> +        */
>> +       if (!PageHuge(p) || compound_head(p) != head) {
>> +               action_result(pfn, MF_MSG_DIFFERENT_COMPOUND, MF_IGNORED);
>> +               res = -EBUSY;
>> +               goto out;
>> +       }
> 
> Let me have one comment on the diff. The result code MF_MSG_DIFFERENT_COMPOUND
> might not fit when PageHuge is false in the check (because it's no longer a
> compound page).  Maybe you may invent another result code, or changes
> MF_MSG_DIFFERENT_COMPOUND (for example) to MF_MSG_DIFFERENT_PAGE_SIZE?
> 

Suppose we do encounter this race.  Also, suppose p != head.
At the beginning of memory_failure_hugetlb, we do:

struct page *head = compound_head(p);
...
if (TestSetPageHWPoison(head))

So, it could be that we set Poison in the 'head' page but the error was really
in another page.  Is that correct?

Now with the race, head is not a huge page and the pages could even be on
buddy.  Does this mean we could have poison set on the wrong page in buddy?

-- 
Mike Kravetz