Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) client-ip=2620:137:e000::1:18;
Message-ID: <89009285-c75d-0f09-5b08-d133c42a18f9@nokia.com>
Date:   Tue, 5 Apr 2022 12:20:15 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.7.0
Content-Language: en-US
To:     Nicholas Piggin <npiggin@gmail.com>,
        Alexander Duyck <alexanderduyck@fb.com>,
        Matthew Wilcox <willy@infradead.org>,
        Michal Hocko <mhocko@suse.com>,
        Hugh Dickins <hughd@google.com>, Yu Zhao <yuzhao@google.com>,
        Mel Gorman <mgorman@techsingularity.net>,
        Lee Schermerhorn <lee.schermerhorn@hp.com>,
        Hisashi Hifumi <hifumi.hisashi@oss.ntt.co.jp>,
        Sasha Levin <sashal@kernel.org>,
        Andrew Morton <akpm@linux-foundation.org>, linux-mm@kvack.org
Cc:     linux-kernel@vger.kernel.org
From:   Alexander Sverdlin <alexander.sverdlin@nokia.com>
Subject: mm: swap: locking in release_pages()
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZHNWWjNUS045TUVyOHV0dnFia1BtdElTV3BzcFdkeFpPUXRHVzM4VFJFWmRp?=
 =?utf-8?B?cGN1dEhGam9ZREtGVXJHcDdnRDF4SHpNc0JWT3lIOHdSRVNzSWtwN2FFVitM?=
 =?utf-8?B?U2F0UklXL1BhUjhSSHJXclpEVW4wQXdvNHdNYkErMU05ZEF2YjdQNEw0bG0w?=
 =?utf-8?B?MUNNRGxieFA1V0NrS0ZvQS9yR0dUV1phQ3QrQXE2Vk9DZ3dxMU1QczVpck1N?=
 =?utf-8?B?a3Q3WEp4U0R0Tk0wYnM5My9iTitKUDFPb2JCMmJ2NXFkNjhTZWNaTzFpSE92?=
 =?utf-8?B?MkpzeWkwQ29LMlZhbUVwTHlkYnNOZE15V0FHK2hRVjdaYnBqNkYwWDBSMlY2?=
 =?utf-8?B?TmNqRU9ScmJ1QmxzK1VzMXB1MXNBS3l4aklCNGNDeVF5RTI2L0VrZlVuUmgy?=
 =?utf-8?B?Q3BRd29WempxV2tuODZhcWExVGN4OEc2T3dIVVF1ODdWckRuTFFCTlZXTXZV?=
 =?utf-8?B?enl0RHlpaHVzNitWNW5LOGc5Sk9ZZkpSNitmMnowQ3dKbjhXYmhyV2xYaVAy?=
 =?utf-8?B?ZzdGTVhxL1RJMDhieUtoMHQ4UDBLbkNiSXpnVzFPT1piOEZzT0didDZXTVEx?=
 =?utf-8?B?RFRTMDNjQ2RZcWxrMWJ4TjlnRGI5MjZ3VjlPODlTdS9GWEphbG5TS2NkQmZJ?=
 =?utf-8?B?MDg4RzVEdll5SXpxa3diLzJ2WjBFOE1ucW0yUkdVNHNrekI4NzNPckJITGtx?=
 =?utf-8?B?WVZSVEZIcWVRcktDdjRNOTlPbEUvcnRzTlpRald1anQ2R09Td3NZck5VZU1S?=
 =?utf-8?B?b1E1MlBmRnA4WXRMeVByaU1XZ1NnWXNzUU9iNENGWUtqK0Z1N2Q4NHVmdm94?=
 =?utf-8?B?SjhQNHBSVHplY0RMWXNCZDk4bTVJMlRRY05UaGNlK3NYSWF2Qk9TSUNYN2tO?=
 =?utf-8?B?aUZPR3NSUHppc25Yb1AwTWNtMEhGRmpZWVNnYy93Vm5LQmMyelFDNkpOL3Iw?=
 =?utf-8?B?WC9uRlZuMENidiszbC9yZWk5U2tIMVVTdGtnMnlBcEwyRWJ4UlROV0xwY0tB?=
 =?utf-8?B?eFVZZHljc3ZRSHh3MVNSRUtsU2RxbmZjRUJIbTRCZ3cySHNkaDYzZHRRQVZF?=
 =?utf-8?B?UGhOd1BrWWNlSC9tME0rNEZBaHI0L2hvODk1akNlS0FsYUQwSFlUdWZMdkVy?=
 =?utf-8?B?amdvYVluMWZmY3hid1hMc2pWRFJnMjI3WlltK2NrTGJCS05EVnNHTjRLNEs4?=
 =?utf-8?B?c0MyZHhZRS9BMkJOcUpRdTFVbjFFbWR4QzdNVm92WHN5UmVIQjMwRmJWL0FN?=
 =?utf-8?B?MEJ4cXorbDJzaDl5UWhCemwwdm1TUU1neWQrUXpHVVBCSFBDa1kvTHZRdnhp?=
 =?utf-8?B?RG83eTJYa0RMRDhXVFJ4V1Y2cVZUUlN1SkcyZE0zUXpXWHd4TmxoQ25OTkVH?=
 =?utf-8?B?WGg3NlJNY2I0cTB4MFVpN1ZnMER2RkZGT3pyczRBNllHbXBVdEtYdVRPWXVG?=
 =?utf-8?B?Y0FBOUlyVE5ieHJ2WGhEd3ZMcDFteE1JVGZuVTF3SUFtUG5ZL283dUhBQjRs?=
 =?utf-8?B?OFZrU3lPeFNseEZaL1F1YmZSNFVhWU50NERoNWdZTmJyL3VhN2dCdWpUVExC?=
 =?utf-8?B?VHIxa096WU5qN0NabVBmOHRwOVFHLytYTkNWUVhWL0JmejF6Z2VkNzY0RTBJ?=
 =?utf-8?B?SXNUVzgvNTAxaW9rMVZaZ282UWJJSWUrSmZXNm5nRmkxV1FRdW8wZGxsc2dQ?=
 =?utf-8?B?KzRkRGdPK21jU2tyd21EUURLQ0IzSnlPb1RXZmtuamNFZFpEMFNETnRTSnV0?=
 =?utf-8?B?T0ZyNXlQSGExUGhtOHFhci9uY1BldlBpN3daa2Q1Vm9UQ2lreHZ6aENUS1RE?=
 =?utf-8?B?U2xrc1dxb2MvMFl2bmxKRE9xZGpLVTZrYThPVTUzZ0gzbjdLdWswbW4vMEhD?=
 =?utf-8?B?MDh1QWI2c0VsazhVUnBqUWtaVXJidUlZbTJ5WDJSVHkrelZSZDZLdnpKVDZp?=
 =?utf-8?B?dlozamxJMnlLbWFFaHVHMXNXWFEvQmdXTkMrMGd0aHhuQ0ZPS0RmNDRzbkN6?=
 =?utf-8?B?WmNZcVRIYzg0cEovQ3R6TURhOERDU0E4WC9rblRKQS9EYXZNWkhmSXM3VUZt?=
 =?utf-8?B?UG9LMzhySzBuMmxORnpma3hUc3lVcGdhMXFCYWRuTk56ZXE1Um1WcUFvTUho?=
 =?utf-8?B?bTJLK243MlU1cmQxYkhZV3hhQVlsZDEwbjRiTEJwa3NvKzN6QjRySGd4UW1Y?=
 =?utf-8?B?VTBPMktpdDkxc0ZML1hFVERaejN1WTFHUVVtd0ltZWxHRHpGSDhqMVl1NUNj?=
 =?utf-8?B?ZlJkdnRDc3NIM1BqWUpHaEhiLzluUEpzeEFsbnM0TFROYXN3NFlObTRvWFdn?=
 =?utf-8?B?VCs4TlI2WmN0ZDdkRVNjQ3kxa1lmcVZnY2hESmgrTlFzYkxEMUM2WUtIZFlS?=
 =?utf-8?Q?/n+kaOjG+hPpN3Is=3D?=
X-OriginatorOrg: nokia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1d842f29-8ca0-4c4e-3ed8-08da16ede3f7
X-MS-Exchange-CrossTenant-AuthSource: AM0PR07MB4531.eurprd07.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Apr 2022 10:20:21.6225
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 5d471751-9675-428d-917b-70f44f9630b0
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: dx6GnbJDps3qKdGb+J2FaTQktI9REaCZpna2Lgpdadgu0J43jPucsVZp068LibsR0wq0XcoviBAXZNm0VNDj8nyJRLT3XgN3GX0kFMLG6YI=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PAXPR07MB8793
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
	DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,RDNS_NONE,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE
	autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
	lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Dear mm developers!

After experiencing a crash in release_pages() [1] I'm trying to understand the locking in the release_pages():

No matter if we consider v5.17 or v5.4 (as in my case), they both have similar locking patterns:

v5.17:

if (PageLRU(page)) {
        struct lruvec *prev_lruvec = lruvec;

        lruvec = folio_lruvec_relock_irqsave(folio, lruvec,
                                                        &flags);
        if (prev_lruvec != lruvec)
                lock_batch = 0;

        del_page_from_lru_list(page, lruvec);
        __clear_page_lru_flags(page);
}

v5.4:

if (PageLRU(page)) {
        struct pglist_data *pgdat = page_pgdat(page);

        if (pgdat != locked_pgdat) {
                if (locked_pgdat)
                        spin_unlock_irqrestore(&locked_pgdat->lru_lock,
                                                        flags);
                lock_batch = 0;
                locked_pgdat = pgdat;
                spin_lock_irqsave(&locked_pgdat->lru_lock, flags);
        }

        lruvec = mem_cgroup_page_lruvec(page, locked_pgdat);
        VM_BUG_ON_PAGE(!PageLRU(page), page);
        __ClearPageLRU(page);
        del_page_from_lru_list(page, lruvec, page_off_lru(page));
}

What I don't understand here is, what guarantees us that "if (PageLRU(page))" condition
is still valid after we swap the locks in "if (pgdat != locked_pgdat)" case?
If we check under one lock and VM_BUG_ON_PAGE() under another lock, what actually stops
it from crashing as below or BUG() from time to time?

1. Crash of v5.4.170 on an ARM32 machine:

Unable to handle kernel NULL pointer dereference at virtual address 00000104
pgd = e138149d
[00000104] *pgd=84d2fd003, *pmd=8ffd6f003
Internal error: Oops: a07 [#1] PREEMPT SMP ARM
...
CPU: 1 PID: 6172 Comm: AaSysInfoRColle Tainted: G    B      O      5.4.170-... #1
Hardware name: Keystone
PC is at release_pages+0x194/0x358
LR is at release_pages+0x10c/0x358
pc : [<c03367a4>]    lr : [<c033671c>]    psr: 600c0093
sp : ccec7da8  ip : 00000000  fp : 00000002
r10: 600c0013  r9 : c0e054a0  r8 : 00000001
r7 : 00000000  r6 : ccec7e68  r5 : 00000000  r4 : d435e280
r3 : 00000122  r2 : 00000100  r1 : 00000001  r0 : 600c0013
Flags: nZCv  IRQs off  FIQs on  Mode SVC_32  ISA ARM  Segment user
Control: 30c5387d  Table: 4d2b2cc0  DAC: 55555555
Process AaSysInfoRColle (pid: 6172, stack limit = 0x36239b43)
Stack: (0xccec7da8 to 0xccec8000)
7da0:                   c0a11970 c0ea3380 00000000 c0e04e48 cb8fbef8 ccec7dbc
7dc0: ccec7dbc c0e04e48 d435e280 ccec7e64 d435e280 00000000 ffffffff 00000000
7de0: 00000000 cb8fbef8 00000000 c0338654 00000001 c0345f0c ccec7e28 d0d0d0d0
7e00: 7fffffff c03af3e0 000000fc fffffe00 00000000 00000000 cb8fbdf8 00000000
7e20: 00000000 2f2f2f2f 00000000 c03ac448 cefdc02d c03af738 cefdc02d fefefeff
7e40: ccec7ea0 00000002 ccec7f60 c03ad6e8 00000010 00000000 00000000 ffffffff
7e60: ccec7ea0 00000101 d435e280 0000000a fffffffe c03ad6f0 00000000 ccec7ea0
7e80: ccec7f60 c03bf24c ccec7ef0 c03b9738 cb8fbdf8 c03bf24c ccec7f68 c03b1eac
7ea0: 62395b18 c0e04e48 20d0cab6 ccec7ee0 cb8fbdf8 ced8e580 ccec7f68 ffffffff
7ec0: ffffffff cb8fbdf8 fffffffe c0346468 ffffffff ffffffff 00000000 c03ee290
7ee0: 00000000 c03d16e8 cb8fbdf8 cb8fbe7c 00000007 00000000 00000000 cbb26580
7f00: c0268e64 ccec7f04 ccec7f04 c0e04e48 cb8fbdf8 c0e04e48 c0a108e4 cb8fbdf8
7f20: cb8fbeb8 c0a108e4 ccec7f68 00000000 ffffff9c c03bf790 c9cced48 00000000
7f40: cefdc000 c03b3950 ccec7f68 ccec7f58 cd2a2540 00000002 00000000 00000000
7f60: c3043a90 cd768440 79377e71 00000047 cefdc02d c03c55dc 0000000a c0e04e48
7f80: b0a9b7e8 b0a9b7e8 b6d695a1 aaa0fd60 0000000a c0201204 ccec6000 0000000a
7fa0: 000002a4 c0201000 b0a9b7e8 b6d695a1 b0a9b7e8 b07056b0 a5c2b128 00000000
7fc0: b0a9b7e8 b6d695a1 aaa0fd60 0000000a a5c2a344 a5c2a348 a5c2a474 000002a4
7fe0: b6e5191c a5c2a2bc b68922d3 b68e15d8 600c0030 b0a9b7e8 00000000 00000000
[<c03367a4>] (release_pages) from [<c0338654>] (__pagevec_release+0x28/0x44)
[<c0338654>] (__pagevec_release) from [<c0345f0c>] (shmem_undo_range+0x358/0x714)
[<c0345f0c>] (shmem_undo_range) from [<c0346468>] (shmem_evict_inode+0x11c/0x2ac)
[<c0346468>] (shmem_evict_inode) from [<c03bf790>] (evict+0xa0/0x1b8)
[<c03bf790>] (evict) from [<c03b3950>] (do_unlinkat+0x1d0/0x2a0)
[<c03b3950>] (do_unlinkat) from [<c0201000>] (ret_fast_syscall+0x0/0x4c)
Exception stack(0xccec7fa8 to 0xccec7ff0)
7fa0:                   b0a9b7e8 b6d695a1 b0a9b7e8 b07056b0 a5c2b128 00000000
7fc0: b0a9b7e8 b6d695a1 aaa0fd60 0000000a a5c2a344 a5c2a348 a5c2a474 000002a4
7fe0: b6e5191c a5c2a2bc b68922d3 b68e15d8
Code: e5933000 e3130020 1a000065 e1c420d4 (e5823004)
Kernel panic - not syncing: Fatal exception
 

-- 
Best regards,
Alexander Sverdlin.