From:   Lei Lei2 Yin <yinlei2@lenovo.com>
To:     "kbusch@kernel.org" <kbusch@kernel.org>,
        "axboe@fb.com" <axboe@fb.com>, "hch@lst.de" <hch@lst.de>,
        "sagi@grimberg.me" <sagi@grimberg.me>
CC:     "linux-nvme@lists.infradead.org" <linux-nvme@lists.infradead.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "cybeyond@foxmail.com" <cybeyond@foxmail.com>
Subject: [PATCH] nvme: fix heap-use-after-free and oops in bio_endio for nvme
 multipath
Thread-Topic: [PATCH] nvme: fix heap-use-after-free and oops in bio_endio for
 nvme multipath
Thread-Index: Adlb4qYZRCgfva2qSAaoRzaKM2NfXg==
Date:   Tue, 21 Mar 2023 10:50:46 +0000
Message-ID: <PS1PR03MB493948DD1ECFAF150EE7B2F688819@PS1PR03MB4939.apcprd03.prod.outlook.com>
Accept-Language: zh-CN, en-US
Content-Language: zh-CN
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?Vj4oq0ec7cnLJotY8n/xkSQiy179X3btyORAzMg2q0CXPAZAav8PNXfKYiPO?=
 =?us-ascii?Q?vC83pTHGjZCoj6mO02CDAJjIrpAhkZfB5XIMF+qNY6FB4vzIyS/uGipqV8Ew?=
 =?us-ascii?Q?KB6FrumRgwhpSt0MaSKfM2W20xghZz+sSJl0Bex5SOgzEIEbP3XDfZCYKHS3?=
 =?us-ascii?Q?VQKPUB1javFqu6mLYFEHWn4XQhNsK5lEUb0xvaUL4sFSvPuY7BNTfChXkrZs?=
 =?us-ascii?Q?enj0E4MS4Efx+ARqJBdyxu1RM5Sscf4OGfpb0JkKNtLC3CtNa/uddfrn4FVc?=
 =?us-ascii?Q?XqOkeLsn/ixa1Zg1VjEXN/cr21lyCGSTvBolk+NHIIdn8JktDOmV3sXQ+HmY?=
 =?us-ascii?Q?y5tXDDhKzDj2GOWQNHgeJAI06PiEeuEGqTw9f070vsNUVNisqc26WbDjBJQ9?=
 =?us-ascii?Q?+ZvcAEcb5kwyzvUspNlzGtBSHa2Cz80MTwnq29JuRVNpGMAdutzUhQ9xYxU0?=
 =?us-ascii?Q?9rEg9JBo0BS62eZOF32/lzLrOV11qC7ZbYk/C5OTVttimt+1NHiv88RFs+dM?=
 =?us-ascii?Q?Dl8cDpvhUPxgt4AJ73BecKiziSAruwszk8e27eLCqcxC2rzmnmcnqCuVDg2V?=
 =?us-ascii?Q?LiV4CL0zcGdOvgSKWgnsPAjjVnLbA2LbvFDE/MEsnTa/8OfuzQC0vhAu5+ep?=
 =?us-ascii?Q?eIGqqR71OBkBaa3Douvs5MPwi389Pay5xpdQBBfzmlCXAsyZN74zCHwjiuau?=
 =?us-ascii?Q?EIkV5h3pX91WUmJ7hB4rKIk5SeduKaeIFqElucFdISEjvkI+3XHJoqq94Mau?=
 =?us-ascii?Q?ioyWGPVSYNRg7G2PbfjRbMOu8qbmVUwtVDU+nlSRqT5nHUZSvERHIb/M8Od1?=
 =?us-ascii?Q?PHmREh/cj1gCDuvl7A0focSNNoykyhfMvlep7p6jW0kCTyaPNIRc+XPsnzMS?=
 =?us-ascii?Q?2h6OsYoehNMcsEgVZcfVyrAbBQLkEgYQDsNZQp3L5qCaiIvVGLiBFhVf8CAd?=
 =?us-ascii?Q?K7WzG6ROOgrvRAjJUQJAjDoGFNy21ssVf1aX/ok4qa65Yc4NsOF5LABV77Px?=
 =?us-ascii?Q?K+WU3Yx9TLPWh3NzgU6h1zkyDZZphHw6ReW0DQbuybX5Hg0DrYzXl0RkoxHm?=
 =?us-ascii?Q?HN3sytN/yUe8iphKbRjUAOPZSFQwErUWs1dri/AwNLiXIAZkbvUNcTVm9ICF?=
 =?us-ascii?Q?cR8jewSkLtY9DiLHA4vGkzqKpEe2lBd8Cgh9dkVycRzo0zgmfOfj8gauEGOU?=
 =?us-ascii?Q?CEpdRylfgR5xDVpuaZFsTcSFMeAuksQVFvr22F928qN/3SJ9AoHeLBsxKiYt?=
 =?us-ascii?Q?jgaEwHxR6rN6yxELZOJb6apL/GwgP2pNhxHf58G7Mqa+nLH2/CN3KnEHTO7b?=
 =?us-ascii?Q?jViAyZCK2PSV0Q1FvGTmTkP4vtDf0dPywmfvBJFeCqnKVy9CMwi9jvWAj71b?=
 =?us-ascii?Q?1bE2rWYEU11l33c1Tuc0Kduc9f3H8nlQKJ2/tqDjzgKZIVzyffD/XolhlOb2?=
 =?us-ascii?Q?z7Q9hiv5plNF2fnosdIphf9v8n6I/SMjL0w6veO/0H240oJTER0F9I7PaZkf?=
 =?us-ascii?Q?xFJRCtc/fpAyoCJhzo9pVL/oT0MHuYM/WwDQhWzGWaMuXBYbZSbMTeZ2ijYY?=
 =?us-ascii?Q?lm5AMZdn+ZKJvEm6uHU=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: lenovo.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: PS1PR03MB4939.apcprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e8559bdb-68ab-420f-f6fd-08db29fa20a1
X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Mar 2023 10:50:46.8949
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 5c7d0b28-bdf8-410c-aa93-4df372b16203
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: TT57+ehdZhmBjM/59+0HS8uCskkAKFIwfpPyJ6yf5oOkTBptgG1VLt4ffWTlW46PO246cwc9F7u0rSdagb/w/w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SI2PR03MB6110
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

From b134e7930b50679ce48e5522ddd37672b1802340 Mon Sep 17 00:00:00 2001
From: Lei Yin <yinlei2@lenovo.com>
Date: Tue, 21 Mar 2023 16:09:08 +0800
Subject: [PATCH] nvme: fix heap-use-after-free and oops in bio_endio for nv=
me
 multipath

When blk_queue_split works in nvme_ns_head_submit_bio, input bio will be
splited to two bios. If parent bio is completed first, and the bi_disk
in parent bio is kfreed by nvme_free_ns, child will access this freed
bi_disk in bio_endio. This will trigger heap-use-after-free or null
pointer oops.

The following is kasan report:

BUG: KASAN: use-after-free in bio_endio+0x477/0x500
Read of size 8 at addr ffff888106f2e3a8 by task kworker/1:1H/241

CPU: 1 PID: 241 Comm: kworker/1:1H Kdump: loaded Tainted: G           O
      5.10.167 #1
Hardware name: Red Hat KVM, BIOS 0.5.1 01/01/2011
Workqueue: kblockd nvme_requeue_work [nvme_core]
Call Trace:
 dump_stack+0x92/0xc4
 ? bio_endio+0x477/0x500
 print_address_description.constprop.7+0x1e/0x230
 ? record_print_text.cold.40+0x11/0x11
 ? _raw_spin_trylock_bh+0x120/0x120
 ? blk_throtl_bio+0x225/0x3050
 ? bio_endio+0x477/0x500
 ? bio_endio+0x477/0x500
 kasan_report.cold.9+0x37/0x7c
 ? bio_endio+0x477/0x500
 bio_endio+0x477/0x500
 nvme_ns_head_submit_bio+0x950/0x1130 [nvme_core]
 ? nvme_find_path+0x7f0/0x7f0 [nvme_core]
 ? __kasan_slab_free+0x11a/0x150
 ? bio_endio+0x213/0x500
 submit_bio_noacct+0x2a4/0xd10
 ? _dev_info+0xcd/0xff
 ? _dev_notice+0xff/0xff
 ? blk_queue_enter+0x6c0/0x6c0
 ? _raw_spin_lock_irq+0x81/0xd5
 ? _raw_spin_lock+0xd0/0xd0
 nvme_requeue_work+0x144/0x18c [nvme_core]
 process_one_work+0x878/0x13e0
 worker_thread+0x87/0xf70
 ? __kthread_parkme+0x8f/0x100
 ? process_one_work+0x13e0/0x13e0
 kthread+0x30f/0x3d0
 ? kthread_parkme+0x80/0x80
 ret_from_fork+0x1f/0x30

Allocated by task 52:
 kasan_save_stack+0x19/0x40
 __kasan_kmalloc.constprop.11+0xc8/0xd0
 __alloc_disk_node+0x5c/0x320
 nvme_alloc_ns+0x6e9/0x1520 [nvme_core]
 nvme_validate_or_alloc_ns+0x17c/0x370 [nvme_core]
 nvme_scan_work+0x2d4/0x4d0 [nvme_core]
 process_one_work+0x878/0x13e0
 worker_thread+0x87/0xf70
 kthread+0x30f/0x3d0
 ret_from_fork+0x1f/0x30

Freed by task 54:
 kasan_save_stack+0x19/0x40
 kasan_set_track+0x1c/0x30
 kasan_set_free_info+0x1b/0x30
 __kasan_slab_free+0x108/0x150
 kfree+0xa7/0x300
 device_release+0x98/0x210
 kobject_release+0x109/0x3a0
 nvme_free_ns+0x15e/0x1f7 [nvme_core]
 nvme_remove_namespaces+0x22f/0x390 [nvme_core]
 nvme_do_delete_ctrl+0xac/0x106 [nvme_core]
 process_one_work+0x878/0x13e0
 worker_thread+0x87/0xf70
 kthread+0x30f/0x3d0
 ret_from_fork+0x1f/0x30

Signed-off-by: Lei Yin <yinlei2@lenovo.com>
---
 drivers/nvme/host/nvme.h | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index c3e4d9b6f9c0..b441c5ce4157 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -749,8 +749,17 @@ static inline void nvme_trace_bio_complete(struct requ=
est *req,
 {
 	struct nvme_ns *ns =3D req->q->queuedata;
=20
-	if ((req->cmd_flags & REQ_NVME_MPATH) && req->bio)
+	if ((req->cmd_flags & REQ_NVME_MPATH) && req->bio) {
 		trace_block_bio_complete(ns->head->disk->queue, req->bio);
+
+		/* Point bio->bi_disk to head disk.
+		 * This bio maybe as other bio's parent in bio chain. If this bi_disk
+		 * is kfreed by nvme_free_ns, other bio may get this bio by __bio_chain_=
endio
+		 * in bio_endio, and access this bi_disk. This will trigger heap-use-aft=
er-free
+		 * or null pointer oops.
+		 */
+		req->bio->bi_disk =3D ns->head->disk;
+	}
 }
=20
 extern struct device_attribute dev_attr_ana_grpid;
--=20
2.39.1