Received-SPF: pass (google.com: domain of linux-kernel+bounces-155645-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Date: Tue, 23 Apr 2024 13:11:29 -0400
From: "Liam R. Howlett" <Liam.Howlett@oracle.com>
To: Yajun Deng <yajun.deng@linux.dev>
Cc: David Hildenbrand <david@redhat.com>, akpm@linux-foundation.org,
        linux-mm@kvack.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH] mm/rmap: remove unnecessary page_table_lock
Message-ID: <3v6xmyzmnw6go45riwlu7qv4c4phiexpqxldnlbgwsjhppe4oi@xdcqm4xupl4k>
Mail-Followup-To: "Liam R. Howlett" <Liam.Howlett@oracle.com>, 
	Yajun Deng <yajun.deng@linux.dev>, David Hildenbrand <david@redhat.com>, akpm@linux-foundation.org, 
	linux-mm@kvack.org, linux-kernel@vger.kernel.org
References: <20240422105212.1485788-1-yajun.deng@linux.dev>
 <b848c431-deca-42e4-925c-673b3fa1f251@redhat.com>
 <3c452d5db5b3d5879160ab62a9e0ac4481a6298a@linux.dev>
 <fd7fde90-21ea-4617-be17-ba387b44feaf@redhat.com>
 <b6ea1fb5bc6c06d2855e41b4034656b0a76b58f5@linux.dev>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <b6ea1fb5bc6c06d2855e41b4034656b0a76b58f5@linux.dev>
User-Agent: NeoMutt/20231103
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?us-ascii?Q?kEw6UagGWIalBTrmCvG36eHxHlV184U0qs6zO5y7d6z/tlTWroSVsPNpwprt?=
 =?us-ascii?Q?HT97qQvEdXThChvrAJdMiNmad9KvVTSGK5SOf4Kx2KaOW3sIv8EW31hFgS6K?=
 =?us-ascii?Q?QEtHBmpvrgkeC6bbyVPu8JXA3Gk+2NeuN9rk6xSbCbZg06bVZtKeH1q7RXWa?=
 =?us-ascii?Q?vwKK663aJFtB6BsWwmlwd61HIufA8Pv3167PPo/9INWhshGtJoxEB8SDW6uM?=
 =?us-ascii?Q?PrtwJJHaMEWjQ8KvSsIkeJTY0/LjCPsMwEf2Zh1eT8NGr96ncMs4zRDc+vMa?=
 =?us-ascii?Q?mYN8yyYLlsJVCg1v1MvzRMAuvMSwa1zabEcf7iSSkMSOCIBZYwkEmRrZgIcx?=
 =?us-ascii?Q?3zv5PMBeTYkPEm9TdOegn7mfttDapiuJjHOcFEPem5X3SM4cf2Hh0Sk7Kbdo?=
 =?us-ascii?Q?TMM0BmenoInbwY+CbkhfkYftreA95Aqg/xm9wFGck3vg514X8pr0KChhTFch?=
 =?us-ascii?Q?tn8qM33Utp5u4fpBQE6xhV3bvGACDjYxugELGPSRlEu4+xEjg2eGOz9H/HIh?=
 =?us-ascii?Q?aUCIFSdTss29/H7HrjgPptlb8jD25ono3Z5TzMxGRuTUdBZqlp7G4SStz7d3?=
 =?us-ascii?Q?cub+5QSH0Monf8URQ/jQrhML9v7RPA32CjzBFAiuPu5Hk3uh2mdl9hH5+H65?=
 =?us-ascii?Q?w0kIV5XxdeCpa8oIdX6u7NoZS9vARmdZgvmgErZkCUkAZxk9ct7oerJ+lI+K?=
 =?us-ascii?Q?0twXcYXEmel8DNB6VsPdmq+pY9MKXZkiL2Te5kLe4GuB5vqOb0GdOpOe+pR5?=
 =?us-ascii?Q?t9xxCu0cu0hgfYUT1uJzt+OspOqsHE0eLxqA7OkhBxFOvD6++mufBMwD/9wv?=
 =?us-ascii?Q?5DtwlrpEEqTgkgMTrRJivF2KR9I15mzKiA42FyozUVeAq2VmDUx/even9TZ+?=
 =?us-ascii?Q?PuMaqgyU+iqupBWDMRIAeWWUSTyTtgNGZPURUcGLn1UGId+zmMxSRBKmreoK?=
 =?us-ascii?Q?hehvrTJCn2fwausn5mks4H2S7jrBfp1faH5lsTgmCu2oZy0V3fExFOCoEJ5h?=
 =?us-ascii?Q?clhS/8S+gDDGLt2oM/MOnclKt19v4vRvA9a2Ef0eAVRy//QuVyc1A+nXcYon?=
 =?us-ascii?Q?QSso/Bs4iiskWmW7JPQUII+Ua61fhHAByZcEBFYG0OkS0Wqq7g+cxaTuY17s?=
 =?us-ascii?Q?pjOGlqFcFEuHMpg5XJFDVh+XoNPbexIgu20GooVybtqOjic2wuEwuKtvwD7E?=
 =?us-ascii?Q?9Hx9q2GdOzD2kk1ASi10uhOAdS8Xn2ckxUgh6EuQAboOAEoAthjq4AdNQYVk?=
 =?us-ascii?Q?UNWrA5aWVUgdcSBi+pLWUfCQuxfOF6nDQVedHnfvYiAjerv5eLQ+rtjBVxs9?=
 =?us-ascii?Q?18VkWasf5z4YLVui6/F/4PFC+y/02V3/anDZV/BxTzDBi3LoR3OYbZUBq+UL?=
 =?us-ascii?Q?9PMUDH+TG4hFTD+KpmHqwk7BmGba2wI1dAdYyHAHUcybtk0lTlJ7lLupCbFY?=
 =?us-ascii?Q?eY4rBlpWUgVxTFljZoaWNRrnE8MQxLB326tpHR1Fh/gX3tK+X/toXZL/h5pn?=
 =?us-ascii?Q?ajpNaomD8EswNVGSz2FUId7BMKUh8k314h9QOcBfacTMCIl5W1QSYTHqrd4x?=
 =?us-ascii?Q?R5zp3zHnh6IXT+M8lA/3lTTaTeB09D61+PqS+1Ft?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	Chu8H9OKnwQKDYXUmifH2hWvNUsWwhPI9EGiEl1B7Jkm9poFaqjKyVX5GrKUneTJ+OE3ROuHOeOM5S4MaMC+gJphO5/bf4xEi376StFToPoSV+aUE5sguNRes91bkj8CmC0gbUMtRiINtFRakx2mhNnit9ek/V7YPlNPwUYNbAeKw2UdbIiLH2SaeoCYabZTvp8D3hNWVPm5ab7cC/WasEzsMu5+XSI04WqkTNwsw86Ylp8HpcUJ2Tsx/dk7/X3Pv2TI1OGKt+mPAJFYRpkx2zhuUYB4J66MuSId8MgOA8DTzTT62oIlqgq0edz5CnhLd/YyDaXlBhX9HjiDXMp19FOBSTIFrY2vhB6m796ymY6HG9ZZxVAYyBXtdreXVv8XfsJoHAlajGu5TIZRaQilvreNMzlyTkX1w+aQrvTELdUMUzInC7UbvqwmMjnvVQD6aKhp2UfBOt79hQECFwcg38TmxYqcFjgVhXRnYl+mR695Jyjk0UuF/AB4/iyo8H/uHSjonZiINOStvZZ7gPI8d5LyWN1/HEEiE9osJ5EWPBMzA16/QAsz7GEi1ySwdd32eO1i63HFrKh7Bcs81hrwt1aZbj8JICo/03hRknpy1cY=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b9f9c37c-574e-484f-5cce-08dc63b86bbd
X-MS-Exchange-CrossTenant-AuthSource: DS0PR10MB7933.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Apr 2024 17:11:31.3924
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: zI7YcEw8Qm1A1eT/OBp2Ee/RxMvyqbg1NrR3jeJjJlE9hhrIYOC7uTFfoKPYXWS5puTgbiOC5sVHnjiWuh777Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB4431
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1011,Hydra:6.0.650,FMLib:17.11.176.26
 definitions=2024-04-23_14,2024-04-23_02,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999
 mlxscore=0 phishscore=0 spamscore=0 malwarescore=0 adultscore=0
 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2404010000 definitions=main-2404230039
X-Proofpoint-ORIG-GUID: eJOTqgqxq4T91lLC4dvKtOdzoP5aKEbU
X-Proofpoint-GUID: eJOTqgqxq4T91lLC4dvKtOdzoP5aKEbU

* Yajun Deng <yajun.deng@linux.dev> [240423 04:35]:
> April 23, 2024 at 4:18 PM, "David Hildenbrand" <david@redhat.com> wrote:
> 
> 
> 
> > 
> > On 23.04.24 09:53, Yajun Deng wrote:
> > 
> > > 
> > > April 22, 2024 at 7:24 PM, "David Hildenbrand" <david@redhat.com> wrote:
> > > 
> > >  > >>
> > > 
> > > > 
> > > > On 22.04.24 12:52, Yajun Deng wrote:
> > > > 
> > > 
> > >  page_table_lock is a lock that for page table, we won't change page
> > > 
> > >  table in __anon_vma_prepare(). As we can see, it works well in
> > > 
> > >  anon_vma_clone(). They do the same operation.
> > > 
> > > > 
> > > > We are reusing mm->page_table_lock to serialize, not the *actual* low-level page table locks that really protect PTEs.
> > > > 
> > > >  With that locking gone, there would be nothing protection vma->anon_vma.
> > > > 
> > > >  Note that anon_vma_clone() is likely called with the mmap_lock held in write mode, which is not the case for __anon_vma_prepare() ...
> > > > 
> > > 
> > >  Yes, anon_vma_clone() is called with the mmap_lock held. I added mmap_assert_write_locked(dst->vm_mm) to prove it.
> > > 
> > >  I added mmap_assert_write_locked(vma->vm_mm) in __anon_vma_prepare() at the same time, it shows __anon_vma_prepare()
> > > 
> > >  is also called with the mmap_lock held too.
> > > 
> > 
> > Make sure you actually have lockdep built in and enabled.
> > 
> 
> This is my config.
> CONFIG_LOCKDEP=n
> CONFIG_DEBUG_VM=y
> 
> I did another test.
> I put mmap_assert_write_locked(mm) before 'set_bit(MMF_OOM_SKIP, &mm->flags)' in mmap.c, it's outside the lock.
> It will crash when on boot. I think mmap_assert_write_locked() works.

If you are changing locks, then please test with lockdep on.

> 
> 
> > __anon_vma_prepare() is for example called from do_anonymous_page() where we might only hold the mmap_lock in read mode (or not at all IIRC with VMA in read mode).

Consider two concurrent readers getting to this function with the same
vma.  There is no mergeable anon vma, so both create a new anon_vma.

You take the anon_vma lock to parallelize the linking to the vma - but
they are different locks because they are both new anon_vma structs
allocated by concurrent readers.

You now need a lock that you know cannot allow this to happen. Looking
at the top of mm/rmap.c and see which one works.  The next in line is
the page table lock, so that one is used here.

What if we reverse the locks?  We can deadlock.

What if we don't take the anon_vma lock?  We can have two writers to the
anon_vma.

Thanks,
Liam