Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 24 Oct 2022 14:55:27 -0700
From:   Mike Kravetz <mike.kravetz@oracle.com>
To:     linux-mm@kvack.org, linux-kernel@vger.kernel.org
Cc:     Naoya Horiguchi <naoya.horiguchi@linux.dev>,
        David Hildenbrand <david@redhat.com>,
        Axel Rasmussen <axelrasmussen@google.com>,
        Mina Almasry <almasrymina@google.com>,
        Peter Xu <peterx@redhat.com>, Rik van Riel <riel@surriel.com>,
        Vlastimil Babka <vbabka@suse.cz>,
        Matthew Wilcox <willy@infradead.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        Wei Chen <harperchen1110@gmail.com>, stable@vger.kernel.org
Subject: Re: [PATCH v2] hugetlb: don't delete vma_lock in hugetlb
 MADV_DONTNEED processing
Message-ID: <Y1cJz6i5YMNfSeAm@monkey>
References: <20221023025047.470646-1-mike.kravetz@oracle.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20221023025047.470646-1-mike.kravetz@oracle.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?ShRYfgJUWxxHRuoM0nxGIVHXpns36+pFRUAI65GmNPqbu+eUwmi2BaAtr5ZH?=
 =?us-ascii?Q?XFzp1+5IiXabiw5NZmpjF3Y3fcbWQ72vJYVEhWANVu2IR6P2whVvlEaRCEwe?=
 =?us-ascii?Q?JlJK+K+yTeJgehVqJYc5eh8N2W1+N6hFh75PwFyP//CNJYSIkSc65VOmtsqM?=
 =?us-ascii?Q?uih3dklADgYkDhEzmPIOqZj56l5SKjV+FQWrbX22jgESxIhWgsqr9TAHE1Cn?=
 =?us-ascii?Q?VIZ9rdBRSL8uANTVkDeYG+gy4twCRK5JV0RE7m6mA9PavvVgubCvhRI1eCv7?=
 =?us-ascii?Q?NMuaaDNAhJkbpMskDmCljgeHr/icvo5yt9CT7WC02rBbjaQb5hUVjqXVQSzT?=
 =?us-ascii?Q?aH+jXWxqXtSte4WpvzhpF7/F6v2YshZtOJxoD9ud4nuQQEQEZZjBil+uLWQm?=
 =?us-ascii?Q?W5QSnerWXGsyCmMkxkJNgHZ8QRvEXJujlvjhzgS2WCgNNvs/8LxJ19SZjJu/?=
 =?us-ascii?Q?xjaXC6x7kuABJXsnDFwFcNImzqGfN4R8hZtV27VcKp7GGTNgpy0J6WJQxDRW?=
 =?us-ascii?Q?upLYarOVNBwkAeA4D8kF9qto3cKbnEQo143ABo9Dp3DaiYP2ZDNLkIGu4Z+k?=
 =?us-ascii?Q?+aaunxE4otLA1wAU6Zh1mqCcb9HL+NHM/gJxMABNMeOh9DncC7FaV8MNwYgV?=
 =?us-ascii?Q?pgnf0cN3GkAh8pUAZMwuUwcSHNwGnBf8QPeu18dmrXsUNi6DsnmdoDGEF5Yh?=
 =?us-ascii?Q?iBZSBCMcOB6vCySF8sd0YbrkLenofx4fxA/2CozF8SzGz+D+emIdBdb1XiMB?=
 =?us-ascii?Q?V6HxcX89OvJ5lO+3Jbad+5nyPmdJaRD/hbUN/67EipY0PO3uw4vV0T/tSIp7?=
 =?us-ascii?Q?lzclqOKOxIytpJDpkGGjNaHIKH34T1E0As1On6kcsGDMw2mFRQLuIGkHp6wh?=
 =?us-ascii?Q?i4dskMz8qaGDZOGDc3LrUV3OY0quZt7RQwVAiO2sV1mZ/I9anISZsSrKlOrz?=
 =?us-ascii?Q?8QstvmofPpiD+odg53gS/QA8A+8TjYgdrlzf7XlxDWX/vi0ubdpWxzGv1aJ+?=
 =?us-ascii?Q?ylNNV8Veht/jbMBHARptfJ4tw1eaUHxR+TKHuHIJjMMa4yYIPvguY7PIsr8t?=
 =?us-ascii?Q?79kwBFfGOjZOgmoJ7CYwDXWxYD7w0Ug5+Rq2UhpCwox3Bg+uLt2F4TRC5Yu6?=
 =?us-ascii?Q?KEIJXjHmf5VK7zpwM7+TR5msLg3tAkLTWpsMZYqKKlSfGs9chxY+E+xJkTdC?=
 =?us-ascii?Q?R1bzhQ+w3wV77xxI/PvLigWvHgSubN3KOCHNWnYfz6vgm/vkiibFj1rjY+U4?=
 =?us-ascii?Q?P2lnk7itvlLDI+uLgvJMbUNAzhUhNW6qmryb24cicoDf12DSw015k2nBXFSZ?=
 =?us-ascii?Q?3bFLX3nyb4jd13iso5EmCcHLNbUykfQZ8cJMwOkvqjt57Xa5p55R3d4O3oWO?=
 =?us-ascii?Q?ggjpA2z234mBbM+vahdgGTOZCb0vcoP5+cBR9L0GdPkGJweusXFH+ozE5EM5?=
 =?us-ascii?Q?Mz45FWMpPdPcmOAWGCP5w8yP8MVnTTin8YqVd6EbIXvhn1ujXo5/G98g6kH6?=
 =?us-ascii?Q?7vYWMEjyN3LPLp/HWu1P2vrH4Cs2fsmbIVahx7CWfq+vA2i/FzP4sCSTyGM9?=
 =?us-ascii?Q?yPvfL7znhmQITNiJ8ibleRC9GWZlE8bdDCrLARCRuX/Eji/kn8As2ykPOW5t?=
 =?us-ascii?Q?Wg=3D=3D?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 292a4c87-daf2-4cc0-00ea-08dab60a77ef
X-MS-Exchange-CrossTenant-AuthSource: BY5PR10MB4196.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Oct 2022 21:55:30.6926
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ePuJkS3IEx7Zni+VU07HA47PSzOvvQ04C4eZqoGAAlxcv4QEYVhhpTtq/ixRgNC/HGL8y5kZwXFjXQ+6Eu4ZQQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR10MB6652
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1
 definitions=2022-10-24_07,2022-10-21_01,2022-06-22_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=739 bulkscore=0
 suspectscore=0 malwarescore=0 phishscore=0 mlxscore=0 spamscore=0
 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2209130000 definitions=main-2210240131
X-Proofpoint-GUID: l_YoxmKm-Bu4kgo4Ywe88O45d5dxRjGL
X-Proofpoint-ORIG-GUID: l_YoxmKm-Bu4kgo4Ywe88O45d5dxRjGL
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 10/22/22 19:50, Mike Kravetz wrote:
> madvise(MADV_DONTNEED) ends up calling zap_page_range() to clear the
> page tables associated with the address range.  For hugetlb vmas,
> zap_page_range will call __unmap_hugepage_range_final.  However,
> __unmap_hugepage_range_final assumes the passed vma is about to be
> removed and deletes the vma_lock to prevent pmd sharing as the vma is
> on the way out.  In the case of madvise(MADV_DONTNEED) the vma remains,
> but the missing vma_lock prevents pmd sharing and could potentially
> lead to issues with truncation/fault races.
> 
> This issue was originally reported here [1] as a BUG triggered in
> page_try_dup_anon_rmap.  Prior to the introduction of the hugetlb
> vma_lock, __unmap_hugepage_range_final cleared the VM_MAYSHARE flag to
> prevent pmd sharing.  Subsequent faults on this vma were confused as
> VM_MAYSHARE indicates a sharable vma, but was not set so page_mapping
> was not set in new pages added to the page table.  This resulted in
> pages that appeared anonymous in a VM_SHARED vma and triggered the BUG.
> 
> Create a new routine clear_hugetlb_page_range() that can be called from
> madvise(MADV_DONTNEED) for hugetlb vmas.  It has the same setup as
> zap_page_range, but does not delete the vma_lock.

After seeing a syzbot use after free report [2] that is also addressed by
this patch, I started thinking ... 

When __unmap_hugepage_range_final was created, the only time unmap_single_vma
was called for hugetlb vmas was during process exit time via exit_mmap.  I got
in trouble when I added a call via madvise(MADV_DONTNEED) which calls
zap_page_range.  This patch takes care of that calling path by having
madvise(MADV_DONTNEED) call a new routine clear_hugetlb_page_range instead of
zap_page_range for hugetlb vmas.  The use after free bug had me auditing code
paths to make sure __unmap_hugepage_range_final was REALLY only called at
process exit time.  If not, and we could fault on a vma after calling
__unmap_hugepage_range_final we would be in trouble.

My thought was, what if we had __unmap_hugepage_range_final check mm->mm_users
to determine if it was being called in the process exit path?  If !mm_users,
then we can delete the vma_lock to prevent pmd sharing as we know the process
is exiting.  If not, we do not delete the lock.  That seems to be more robust
and would prevent issues if someone accidentally introduces a new code path
where __unmap_hugepage_range_final (unmap_single_vma for a hugetlb vma)
could be called outside process exit context.

Thoughts?

[2] https://lore.kernel.org/linux-mm/000000000000d5e00a05e834962e@google.com/
-- 
Mike Kravetz