Hello.
We are Ubisectech Sirius Team, the vulnerability lab of China ValiantSec. Recently, our team has discovered a issue in Linux kernel 6.8.0-ge5e038b7ae9d. Attached to the email were a POC file of the issue.
Stack dump:
==================================================================
BUG: KASAN: null-ptr-deref in instrument_atomic_read_write include/linux/instrumented.h:96 [inline]
BUG: KASAN: null-ptr-deref in llist_del_all include/linux/llist.h:266 [inline]
BUG: KASAN: null-ptr-deref in tctx_task_work_run+0x7d/0x330 io_uring/io_uring.c:1267
Write of size 8 at addr 00000000000001c0 by task iou-sqp-215603/215604
CPU: 0 PID: 215604 Comm: iou-sqp-215603 Not tainted 6.8.0-ge5e038b7ae9d #40
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0x116/0x1b0 lib/dump_stack.c:114
kasan_report+0xbd/0xf0 mm/kasan/report.c:601
check_region_inline mm/kasan/generic.c:183 [inline]
kasan_check_range+0xf4/0x1a0 mm/kasan/generic.c:189
instrument_atomic_read_write include/linux/instrumented.h:96 [inline]
llist_del_all include/linux/llist.h:266 [inline]
tctx_task_work_run+0x7d/0x330 io_uring/io_uring.c:1267
io_sq_tw+0x12a/0x1d0 io_uring/sqpoll.c:245
io_sq_thread+0x8d7/0x18a0 io_uring/sqpoll.c:308
ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:243
</TASK>
==================================================================
Kernel panic - not syncing: KASAN: panic_on_warn set ...
CPU: 0 PID: 215604 Comm: iou-sqp-215603 Not tainted 6.8.0-ge5e038b7ae9d #40
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0x3d/0x1b0 lib/dump_stack.c:114
panic+0x6d2/0x780 kernel/panic.c:344
check_panic_on_warn+0xb1/0xc0 kernel/panic.c:237
end_report+0x107/0x150 mm/kasan/report.c:226
kasan_report+0xcd/0xf0 mm/kasan/report.c:603
check_region_inline mm/kasan/generic.c:183 [inline]
kasan_check_range+0xf4/0x1a0 mm/kasan/generic.c:189
instrument_atomic_read_write include/linux/instrumented.h:96 [inline]
llist_del_all include/linux/llist.h:266 [inline]
tctx_task_work_run+0x7d/0x330 io_uring/io_uring.c:1267
io_sq_tw+0x12a/0x1d0 io_uring/sqpoll.c:245
io_sq_thread+0x8d7/0x18a0 io_uring/sqpoll.c:308
ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:243
</TASK>
Kernel Offset: disabled
Rebooting in 86400 seconds..
Thank you for taking the time to read this email and we look forward to working with you further.
On 3/17/24 6:59 PM, Ubisectech Sirius wrote:
> Hello.
> We are Ubisectech Sirius Team, the vulnerability lab of China ValiantSec. Recently, our team has discovered a issue in Linux kernel 6.8.0-ge5e038b7ae9d. Attached to the email were a POC file of the issue.
>
> Stack dump:
>
> ==================================================================
> BUG: KASAN: null-ptr-deref in instrument_atomic_read_write include/linux/instrumented.h:96 [inline]
> BUG: KASAN: null-ptr-deref in llist_del_all include/linux/llist.h:266 [inline]
> BUG: KASAN: null-ptr-deref in tctx_task_work_run+0x7d/0x330 io_uring/io_uring.c:1267
> Write of size 8 at addr 00000000000001c0 by task iou-sqp-215603/215604
>
> CPU: 0 PID: 215604 Comm: iou-sqp-215603 Not tainted 6.8.0-ge5e038b7ae9d #40
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
> Call Trace:
> <TASK>
> __dump_stack lib/dump_stack.c:88 [inline]
> dump_stack_lvl+0x116/0x1b0 lib/dump_stack.c:114
> kasan_report+0xbd/0xf0 mm/kasan/report.c:601
> check_region_inline mm/kasan/generic.c:183 [inline]
> kasan_check_range+0xf4/0x1a0 mm/kasan/generic.c:189
> instrument_atomic_read_write include/linux/instrumented.h:96 [inline]
> llist_del_all include/linux/llist.h:266 [inline]
> tctx_task_work_run+0x7d/0x330 io_uring/io_uring.c:1267
> io_sq_tw+0x12a/0x1d0 io_uring/sqpoll.c:245
> io_sq_thread+0x8d7/0x18a0 io_uring/sqpoll.c:308
> ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
> ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:243
> </TASK>
> ==================================================================
> Kernel panic - not syncing: KASAN: panic_on_warn set ...
> CPU: 0 PID: 215604 Comm: iou-sqp-215603 Not tainted 6.8.0-ge5e038b7ae9d #40
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
I think you snipped the fault injection that came before this. It looks
like an allocation failure, so we don't get tsk->io_uring setup for the
SQPOLL thread. Not a great way to handle this, but can you try the
below? Would be nicer if we could just prune the task rather than wake
it and have it error.
diff --git a/io_uring/sqpoll.c b/io_uring/sqpoll.c
index 363052b4ea76..db7b0fdfe1cb 100644
--- a/io_uring/sqpoll.c
+++ b/io_uring/sqpoll.c
@@ -274,6 +274,10 @@ static int io_sq_thread(void *data)
char buf[TASK_COMM_LEN];
DEFINE_WAIT(wait);
+ /* offload context creation failed, just exit */
+ if (!current->io_uring) {
+ goto err_out;
+
snprintf(buf, sizeof(buf), "iou-sqp-%d", sqd->task_pid);
set_task_comm(current, buf);
@@ -371,7 +375,7 @@ static int io_sq_thread(void *data)
atomic_or(IORING_SQ_NEED_WAKEUP, &ctx->rings->sq_flags);
io_run_task_work();
mutex_unlock(&sqd->lock);
-
+err_out:
complete(&sqd->exited);
do_exit(0);
}
--
Jens Axboe
> I think you snipped the fault injection that came before this. It looks
> like an allocation failure, so we don't get tsk->io_uring setup for the
> SQPOLL thread. Not a great way to handle this, but can you try the
> below? Would be nicer if we could just prune the task rather than wake
> it and have it error.
Hi.
The issue does not appear again when I apply the patch to the Linux kernel.
On 3/17/24 6:59 PM, Ubisectech Sirius wrote:
> Hello.
> We are Ubisectech Sirius Team, the vulnerability lab of China ValiantSec. Recently, our team has discovered a issue in Linux kernel 6.8.0-ge5e038b7ae9d. Attached to the email were a POC file of the issue.
>
> Stack dump:
>
> ==================================================================
> BUG: KASAN: null-ptr-deref in instrument_atomic_read_write include/linux/instrumented.h:96 [inline]
> BUG: KASAN: null-ptr-deref in llist_del_all include/linux/llist.h:266 [inline]
> BUG: KASAN: null-ptr-deref in tctx_task_work_run+0x7d/0x330 io_uring/io_uring.c:1267
> Write of size 8 at addr 00000000000001c0 by task iou-sqp-215603/215604
>
> CPU: 0 PID: 215604 Comm: iou-sqp-215603 Not tainted 6.8.0-ge5e038b7ae9d #40
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
> Call Trace:
> <TASK>
> __dump_stack lib/dump_stack.c:88 [inline]
> dump_stack_lvl+0x116/0x1b0 lib/dump_stack.c:114
> kasan_report+0xbd/0xf0 mm/kasan/report.c:601
> check_region_inline mm/kasan/generic.c:183 [inline]
> kasan_check_range+0xf4/0x1a0 mm/kasan/generic.c:189
> instrument_atomic_read_write include/linux/instrumented.h:96 [inline]
> llist_del_all include/linux/llist.h:266 [inline]
> tctx_task_work_run+0x7d/0x330 io_uring/io_uring.c:1267
> io_sq_tw+0x12a/0x1d0 io_uring/sqpoll.c:245
> io_sq_thread+0x8d7/0x18a0 io_uring/sqpoll.c:308
> ret_from_fork+0x45/0x80 arch/x86/kernel/process.c:147
> ret_from_fork_asm+0x1a/0x30 arch/x86/entry/entry_64.S:243
> </TASK>
> ==================================================================
> Kernel panic - not syncing: KASAN: panic_on_warn set ...
> CPU: 0 PID: 215604 Comm: iou-sqp-215603 Not tainted 6.8.0-ge5e038b7ae9d #40
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
I think you snipped the fault injection that came before this. It looks
like an allocation failure, so we don't get tsk->io_uring setup for the
SQPOLL thread. Not a great way to handle this, but can you try the
below? Would be nicer if we could just prune the task rather than wake
it and have it error.
diff --git a/io_uring/sqpoll.c b/io_uring/sqpoll.c
index 363052b4ea76..db7b0fdfe1cb 100644
--- a/io_uring/sqpoll.c
+++ b/io_uring/sqpoll.c
@@ -274,6 +274,10 @@ static int io_sq_thread(void *data)
char buf[TASK_COMM_LEN];
DEFINE_WAIT(wait);
+ /* offload context creation failed, just exit */
+ if (!current->io_uring) {
+ goto err_out;
+
snprintf(buf, sizeof(buf), "iou-sqp-%d", sqd->task_pid);
set_task_comm(current, buf);
@@ -371,7 +375,7 @@ static int io_sq_thread(void *data)
atomic_or(IORING_SQ_NEED_WAKEUP, &ctx->rings->sq_flags);
io_run_task_work();
mutex_unlock(&sqd->lock);
-
+err_out:
complete(&sqd->exited);
do_exit(0);
}
--
Jens Axboe