Received: by 2002:ab2:7903:0:b0:1fb:b500:807b with SMTP id a3csp1330770lqj; Mon, 3 Jun 2024 19:23:38 -0700 (PDT) X-Forwarded-Encrypted: i=3; AJvYcCUGKtjeqIWeCR1BfuQlK8YTXmsb4Tm4KVlCXptDKWNYn+IE8P5+ulJ36ThrZZ0x2NKDsyJEoOv1EZn7/PYhQ8XIhzMZtpXXo+FbRGxZhg== X-Google-Smtp-Source: AGHT+IFdZUrmeewe5/OHvgiz2w8r5D4dybiqA8RZFgfRAeyac79zq63SQ5jeyGDl1fo13KHLQ5qk X-Received: by 2002:a67:f61a:0:b0:48b:ae47:c11 with SMTP id ada2fe7eead31-48bc1f70286mr10740203137.4.1717467817938; Mon, 03 Jun 2024 19:23:37 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1717467817; cv=pass; d=google.com; s=arc-20160816; b=bNwRR7Iv4mTGzUDZGXMiu1XUGn9FL3xWqUvqk/Qtf6kl9dOpDbqndB827DwWqav3lZ Nzxrv39w5TOU0X80b3PcqtcG+bpnekXsXfQzhjqlBzJtrZjlBt/YHMRJZSfbLXmNNxmb voDw35zk/mRPTNQG77T6Xun6CC/ar4/fifpt+ZDRmUqsUcX15KCtsnxIGztbnhfhwvut 0vJeFgybhBlldj+Dv6I06Hk6PI6oXPcPOKkC7JmnN+XV/GUxtHVmWBwUwrlqzqu/kAlc r0daZ5yfYrDpNPK3tTzvzvAHUarVWeVSiRoVQtatqjezBuXUgEnt1sgSSV//tN5VO35v JVlg== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:message-id:date:subject:to:from; bh=5OzgKsiYClhl2hthTs2jo0H8/UT1fZHyBIgFqykUrxg=; fh=jhil3a7FRkga0QTzFsfEAYwrAVZnRmNgrjY9b9TN0Nc=; b=gIs+adxnGzMgEmFe0JVQDAQFBda+UoIkeH8M8RNm0IgwS/HahJV09QeZnSVAEtJDXv jnQzycL0pyvWnEZCfltqSGyWejKvcoC3J6vRoGm76gK4foKOju2lhBJ+ca+PdX/ReC1p ub7MW0YlGCsnOBMuFB8I02m9BrjrZyuoioLAicwC8qzkLGsuNPe+0nNa3HAuBlTHB9vW wmX1oB/xC2W7gAs5RlLDwoJuH9QTM85f1InNGXfPsEL48SHZBNVA8JGazkaNrUHQuzWM QfwMmcfhSTxRDy8nXSQctuJj5RIyABCeaB9KOUxne9eEsmwIDTDl3Z6/uOLDf3658sYA gwFw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=unisoc.com); spf=pass (google.com: domain of linux-kernel+bounces-199963-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-199963-linux.lists.archive=gmail.com@vger.kernel.org" Return-Path: Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id af79cd13be357-794f32a2e77si953776785a.548.2024.06.03.19.23.37 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 03 Jun 2024 19:23:37 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel+bounces-199963-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=unisoc.com); spf=pass (google.com: domain of linux-kernel+bounces-199963-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-199963-linux.lists.archive=gmail.com@vger.kernel.org" Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 826851C23B99 for ; Tue, 4 Jun 2024 02:23:01 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D435113D27C; Tue, 4 Jun 2024 02:22:55 +0000 (UTC) Received: from SHSQR01.spreadtrum.com (unknown [222.66.158.135]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 53D6E53BE for ; Tue, 4 Jun 2024 02:22:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=222.66.158.135 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717467775; cv=none; b=A8eq2wMTiKzxKGkFQWiUYbi2dIDBQrb7R9a7I4ceSrEJsSY9xvRFfGSKqGsi23g88cAIn/TCQfbtgsbBM2utxKFYDIW5PQrTO2PJO8PgN45ip6MBa9n+VuU7ayedNQIbbjac9WLOVO51W1X+Gr0DjNf7xI+CV2XcdG5WByhH0ZE= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717467775; c=relaxed/simple; bh=j7SmNnf2IJ7HtGtXQrqOdQgWBu66WrwLsS5J2Vr5gCE=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type; b=IHQu+XDxfx5JQhSmBwwUTk5zqTozbGnzhjX8yl1GPu0ogF7cktFHWANhWQD9D76G3cgg5B3JEQDdrM4mftIIME+vDnJN/6cnnw2LNFtZHiqKe+pFsJ3B7kWnj//8+a6yO0W0/a/lbN9s3rUDTiaOLL2ZHEJGSniY5BH3r/aTXsI= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=unisoc.com; spf=pass smtp.mailfrom=unisoc.com; arc=none smtp.client-ip=222.66.158.135 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=unisoc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=unisoc.com Received: from dlp.unisoc.com ([10.29.3.86]) by SHSQR01.spreadtrum.com with ESMTP id 4542Mg2p074077; Tue, 4 Jun 2024 10:22:42 +0800 (+08) (envelope-from zhaoyang.huang@unisoc.com) Received: from SHDLP.spreadtrum.com (bjmbx01.spreadtrum.com [10.0.64.7]) by dlp.unisoc.com (SkyGuard) with ESMTPS id 4VtZ5c3Zc2z2R4pwY; Tue, 4 Jun 2024 10:18:44 +0800 (CST) Received: from bj03382pcu01.spreadtrum.com (10.0.73.40) by BJMBX01.spreadtrum.com (10.0.64.7) with Microsoft SMTP Server (TLS) id 15.0.1497.23; Tue, 4 Jun 2024 10:22:40 +0800 From: "zhaoyang.huang" To: Andrew Morton , Uladzislau Rezki , Christoph Hellwig , Lorenzo Stoakes , Baoquan He , Thomas Gleixner , hailong liu , , , Zhaoyang Huang , Subject: [PATCHv4 1/1] mm: fix incorrect vbq reference in purge_fragmented_block Date: Tue, 4 Jun 2024 10:22:32 +0800 Message-ID: <20240604022232.1669983-1-zhaoyang.huang@unisoc.com> X-Mailer: git-send-email 2.25.1 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: SHCAS03.spreadtrum.com (10.0.1.207) To BJMBX01.spreadtrum.com (10.0.64.7) X-MAIL:SHSQR01.spreadtrum.com 4542Mg2p074077 From: Zhaoyang Huang vmalloc area runs out in our ARM64 system during an erofs test as vm_map_ram failed[1]. By following the debug log, we find that vm_map_ram()->vb_alloc() will allocate new vb->va which corresponding to 4MB vmalloc area as list_for_each_entry_rcu returns immediately when vbq->free->next points to vbq->free. That is to say, 65536 times of page fault after the list's broken will run out of the whole vmalloc area. This should be introduced by one vbq->free->next point to vbq->free which makes list_for_each_entry_rcu can not iterate the list and find the BUG. [1] PID: 1 TASK: ffffff80802b4e00 CPU: 6 COMMAND: "init" #0 [ffffffc08006afe0] __switch_to at ffffffc08111d5cc #1 [ffffffc08006b040] __schedule at ffffffc08111dde0 #2 [ffffffc08006b0a0] schedule at ffffffc08111e294 #3 [ffffffc08006b0d0] schedule_preempt_disabled at ffffffc08111e3f0 #4 [ffffffc08006b140] __mutex_lock at ffffffc08112068c #5 [ffffffc08006b180] __mutex_lock_slowpath at ffffffc08111f8f8 #6 [ffffffc08006b1a0] mutex_lock at ffffffc08111f834 #7 [ffffffc08006b1d0] reclaim_and_purge_vmap_areas at ffffffc0803ebc3c #8 [ffffffc08006b290] alloc_vmap_area at ffffffc0803e83fc #9 [ffffffc08006b300] vm_map_ram at ffffffc0803e78c0 Fixes: fc1e0d980037 ("mm/vmalloc: prevent stale TLBs in fully utilized blocks") For detailed reason of broken list, please refer to below URL https://lore.kernel.org/all/20240531024820.5507-1-hailong.liu@oppo.com/ Suggested-by: Hailong.Liu Signed-off-by: Zhaoyang Huang --- v2: introduce cpu in vmap_block to record the right CPU number v3: use get_cpu/put_cpu to prevent schedule between core v4: replace get_cpu/put_cpu by another API to avoid disabling preemption --- --- mm/vmalloc.c | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 22aa63f4ef63..89eb034f4ac6 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -2458,6 +2458,7 @@ struct vmap_block { struct list_head free_list; struct rcu_head rcu_head; struct list_head purge; + unsigned int cpu; }; /* Queue of free and dirty vmap blocks, for allocation and flushing purposes */ @@ -2585,8 +2586,15 @@ static void *new_vmap_block(unsigned int order, gfp_t gfp_mask) free_vmap_area(va); return ERR_PTR(err); } - - vbq = raw_cpu_ptr(&vmap_block_queue); + /* + * list_add_tail_rcu could happened in another core + * rather than vb->cpu due to task migration, which + * is safe as list_add_tail_rcu will ensure the list's + * integrity together with list_for_each_rcu from read + * side. + */ + vb->cpu = raw_smp_processor_id(); + vbq = per_cpu_ptr(&vmap_block_queue, vb->cpu); spin_lock(&vbq->lock); list_add_tail_rcu(&vb->free_list, &vbq->free); spin_unlock(&vbq->lock); @@ -2614,9 +2622,10 @@ static void free_vmap_block(struct vmap_block *vb) } static bool purge_fragmented_block(struct vmap_block *vb, - struct vmap_block_queue *vbq, struct list_head *purge_list, - bool force_purge) + struct list_head *purge_list, bool force_purge) { + struct vmap_block_queue *vbq = &per_cpu(vmap_block_queue, vb->cpu); + if (vb->free + vb->dirty != VMAP_BBMAP_BITS || vb->dirty == VMAP_BBMAP_BITS) return false; @@ -2664,7 +2673,7 @@ static void purge_fragmented_blocks(int cpu) continue; spin_lock(&vb->lock); - purge_fragmented_block(vb, vbq, &purge, true); + purge_fragmented_block(vb, &purge, true); spin_unlock(&vb->lock); } rcu_read_unlock(); @@ -2801,7 +2810,7 @@ static void _vm_unmap_aliases(unsigned long start, unsigned long end, int flush) * not purgeable, check whether there is dirty * space to be flushed. */ - if (!purge_fragmented_block(vb, vbq, &purge_list, false) && + if (!purge_fragmented_block(vb, &purge_list, false) && vb->dirty_max && vb->dirty != VMAP_BBMAP_BITS) { unsigned long va_start = vb->va->va_start; unsigned long s, e; -- 2.25.1