Received-SPF: pass (google.com: domain of linux-kernel+bounces-210387-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
From: "Liam R. Howlett" <Liam.Howlett@oracle.com>
To: linux-mm@kvack.org, Andrew Morton <akpm@linux-foundation.org>
Cc: linux-kernel@vger.kernel.org, Vlastimil Babka <vbabka@suse.cz>,
        Lorenzo Stoakes <lstoakes@gmail.com>,
        Suren Baghdasaryan <surenb@google.com>,
        "Liam R. Howlett" <Liam.Howlett@oracle.com>
Subject: [PATCH 0/8] Avoid MAP_FIXED gap exposure
Date: Tue, 11 Jun 2024 14:01:52 -0400
Message-ID: <20240611180200.711239-1-Liam.Howlett@oracle.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?us-ascii?Q?Ys0u8qXLaJc8iaPaR/x6gBjjcV7KhHlVTi2bLGsNbp0Fil51GtCc+5HMcCjj?=
 =?us-ascii?Q?VkGzAQaCsX3M7+m5Ugqc8lATnMfabTN6TqJc5BO6jFATU2HFpMvoYttd+uJn?=
 =?us-ascii?Q?a8UFqxbSTrRJbz3gZVGFR6qeiNPh+sR0WRhOXZQBF/SbQRDTvmnrn0C4w/wK?=
 =?us-ascii?Q?WCNGKyyTmBf+QjkM89mfssOt3NYN2NB0D/E3MzM5AmARbNZ398MWmRayPuIL?=
 =?us-ascii?Q?0M+f6hUsyT8ZwmzTvRPOoS+nZSKTUz+KJs20J1/W8m7ceACfMxu1LfERWb4X?=
 =?us-ascii?Q?TVFIoUp37jzADO4GkVVuuTLVYVs526R/W8uXKCbhYmTG4Ki0VDoht7Nalfw5?=
 =?us-ascii?Q?McACLtKftSbSuGaf9Ft0QIR5dwNwY1LXXXVOErqqkrWVKKaGdl0wfQBrww+v?=
 =?us-ascii?Q?XSg5AR59Eo0AWRUqoMtFOfUY+6Pud0RRBybiBaJgueQii28Buc4WtUkrqLBn?=
 =?us-ascii?Q?L9FWY3BjR0erfkP1sX/ehuSb0Vu4UOq3S/5o8vsRPCkttTlyUkSSmcPLG30c?=
 =?us-ascii?Q?Vk8lUVYm8XV2CPHrithyqWum5XmlT1S4ErYzOSYf9xgxvd3Fcn6E3rt859eg?=
 =?us-ascii?Q?2xgLaehzMvIieN7RDx3orGlfYnaC2+/6Z0R5pemfXRMCneC0YeiHBSRkA7+p?=
 =?us-ascii?Q?b1ZVa8CJ4om/haj8haTLzVbuKm6SG0vQBNDzX/qsp3cuqUD68ReaHncD1oc2?=
 =?us-ascii?Q?5+ANiwZhNdq987M+9ILiHp2nH7dlqrixDQEuHaI87o4PautcXlbRQ2fuhGZT?=
 =?us-ascii?Q?pR+QeS3N6JXEqz4DoJJOmxaTev0zLKtdlg2ob/Rn9yj0AG9oNqljbh5b4EpP?=
 =?us-ascii?Q?kscJD1ZsK706w2opQR8mY8/WytHiVHfLNn/WS/JK1sT1O6HuMRk4+E87IDzA?=
 =?us-ascii?Q?uMeZaoylmM/mX970eWaRf4HM+cCliFtR/JZL+mJ+rthV5YWsyjp3KitN/Z80?=
 =?us-ascii?Q?KpNqkWVJUJSlzCHjS9HOUAVoEaa8h0wrJFH2n1GqFkbd77u0L+ayhlwhYFm8?=
 =?us-ascii?Q?2eFXpYT33XaIHzuwW8cEor90OWPDDPpqH8Sz+CFPJlJG8xvH/PRKYFdbtomb?=
 =?us-ascii?Q?Fr8IcL8Q5kYa+GHzXPHuv3h63Bd8HSVzCvIAyY0wW2/D3ev0orcK7DEqbtkZ?=
 =?us-ascii?Q?Ijh+omph9A1b73FAJ/m0mYZp1XQff66DpDpZVgHEvWts4MRyNmQikzQwn54n?=
 =?us-ascii?Q?kw8VHMiudzylxv20seDlVhPGNiGTnYXXaZ3j/S060sJvLManGkSGGCPoXDkI?=
 =?us-ascii?Q?Reto5prtisAIlpw48ywcVC+ZNQXZy8xKRKhC5KfbganjUptW9C8Io6diapXE?=
 =?us-ascii?Q?YdDYIg+V9+mPR15ft85gfeX+WZd1BgGhSJpJ46O2AGo9hQIxQs8Yos5LS1IO?=
 =?us-ascii?Q?yW0/JSEsozR3tEneCxJIaAZe9p/F1yFK042A0JfVrlvCqEGVHIqAnLHzTw98?=
 =?us-ascii?Q?iWhzDUNTyFZj5kdMiRkRU/VDMP2mD2eOTxzy27mHjTcFS8aKaKMU7MKv6lz/?=
 =?us-ascii?Q?N1m3TBXr1zXp7nzeLH3W2RMSWVoZJxKL+wtbDERIQkQYyp67LhtfFNnX88dg?=
 =?us-ascii?Q?mb1IcXBSkTVRSWB18hqcMV0oF8YpYpL/kxUa2HUa?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	O2iUZDrUSqoadf2vEQ2c9+cMS5WfhnCXVVMSZmEdg+SzMLyUr4M/ToVNgvQL+ucEae1Xpq1nQ1ER4bTJy3aAcRsMWBIP3ivs63vU5YqsUvS91bUJ8DPGBH4qcI9vBW7bw5h7uDQIT+3n5kJUOC9cxyGBU2mJIga4rmKYPHis6g5G7DbKfwqpp23jRXk0+3z7gW9WR1Q4QtxLDbQf/J/428G/X+O28cVdMf6iLBLIuC2CTEqDCeOYcsKNl1VYib1pfSPmyo5/muihegtCz5TVtuNJ9jz/nJG7ZPL574sEYOQmnF8uNc+364CkbK19KBtcJ7OMeJOkLiHgILjzNiUxpaA4anQv1iKp+neafqwtgVZuT4TEWvP4LdsvZ4ZyOrMzZr0eRF5sNDMmJMdtIDRzxYOdWAIOKgDkGZ8fh6sLFcr0lgTN4vzMReMspIGGOWQRayfLnlqjRJjsQJtn2/oCA3FRNRCbnZvco8N9Ux/jzNK01HW/do5blTdWPmo+3+3lVIcFpRg+WPxu8zZFhc9ePVtXz5EjF8MPBnjinUe2HDhY+noBd3+kUcCu5vyO9uJrKwQ9m8lcVw8wPi22wLmAcXu/SE5p8JbHRUAiqFRiAek=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1ad352a3-83d3-4d0a-d840-08dc8a409e70
X-MS-Exchange-CrossTenant-AuthSource: DS0PR10MB7933.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Jun 2024 18:02:12.1564
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: BD/UoemPrRqFXoX7eWYIR2JSiOkmsaEWbTZf58nG2wMhEcYjL4ptzwR+e7hLAGBtdIs4gxAtxk/TxGwHmydicw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR10MB6154
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16
 definitions=2024-06-11_09,2024-06-11_01,2024-05-17_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 phishscore=0 adultscore=0
 mlxscore=0 bulkscore=0 malwarescore=0 mlxlogscore=920 suspectscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2405010000
 definitions=main-2406110123
X-Proofpoint-ORIG-GUID: HCkbwLsS4uWeTtarT9W2oJvBZUvLI9Xo
X-Proofpoint-GUID: HCkbwLsS4uWeTtarT9W2oJvBZUvLI9Xo

It is now possible to walk the vma tree using the rcu read locks and is
beneficial to do so to reduce lock contention.  Doing so while a
MAP_FIXED mapping is executing means that a reader may see a gap in the
vma tree that should never logically exist - and does not when using the
mmap lock in read mode.  The temporal gap exists because mmap_region()
calls munmap() prior to installing the new mapping.

This patch set stops rcu readers from seeing the temporal gap by
splitting up the munmap() function into two parts.  The first part
prepares the vma tree for modifications by doing the necessary splits
and tracks the vmas marked for removal in a side tree.  The second part
completes the munmapping of the vmas after the vma tree has been
overwritten (either by a MAP_FIXED replacement vma or by a NULL in the
munmap() case).

Please note that rcu walkers will still be able to see a temporary state
of split vmas that may be in the process of being removed, but the
temporal gap will not be exposed.  vma_start_write() are called on both
parts of the split vma, so this state is detectable.

RFC: https://lore.kernel.org/linux-mm/20240531163217.1584450-1-Liam.Howlett@oracle.com/

Changes since RFC:
- Fixed comment on __split_vma() - Thanks Lorenzo & Suren
- Split out abort & complete stages of munmap_vmas() to reduce
  complexity of review - Thanks Suren
- Correct accidental removal of validate_mm() and split the extraction
  of the validate_mm() to its own patch - Thanks Suren
- Fixed merge error in comments - Thanks Lorenzo
- Added reviewers, but I didn't add Suren's review of patch 2 as it
  significantly changed into 3 patches to make reviewing easier as he
  suggested.


Liam R. Howlett (8):
  mm/mmap: Correctly position vma_iterator in __split_vma()
  mm/mmap: Introduce abort_munmap_vmas()
  mm/mmap: Introduce vmi_complete_munmap_vmas()
  mm/mmap: Extract the gathering of vmas from do_vmi_align_munmap()
  mm/mmap: Introduce vma_munmap_struct for use in munmap operations
  mm/mmap: Change munmap to use vma_munmap_struct() for accounting and
    surrounding vmas
  mm/mmap: Extract validate_mm() from vma_complete()
  mm/mmap: Use split munmap calls for MAP_FIXED

 mm/internal.h |  22 +++
 mm/mmap.c     | 399 +++++++++++++++++++++++++++++++-------------------
 2 files changed, 270 insertions(+), 151 deletions(-)

-- 
2.43.0