Recently we found an ext4 performance regression problem between 4.18
and 5.10 by following test command on a x86 physical machine with nvme:
fio -direct=1 -iodepth=128 -rw=randwrite -ioengine=libaio -bs=4k
-size=2G -numjobs=1 -time_based -runtime=60 -group_reporting
-filename=/test/test -name=Rand_write_Testing --cpus_allowed=1
4.18: 288k IOPS 5.10: 234k IOPS
After anlayzing the changes between above two versions, we found that
the endio context changed since commit 378f32bab3714("ext4: introduce
direct I/O write using iomap infrastructure"), in aio DIO overwriting
case. And the problem still exist in latest mainline.
4.18: endio is processed under irq context
dio_bio_end_aio
defer_completion = dio->defer_completion || ... // defer_completion is
// false for overwrite
if (defer_completion) {
queue_work
} else {
dio_complete // endio in irq context
}
mainline: endio is processed in kworker
iomap_dio_bio_end_io
if (dio->flags & IOMAP_DIO_INLINE_COMP) // false, only for read
if (dio->flags & IOMAP_DIO_CALLER_COMP) // false, only for io_uring
queue_work // endio in kworker context
Assume that nvme irq registers on cpu 1, and fio runs on cpu 1, it could
be possible(Actually we did reproduce it easily) that fio, nvme irq and
kworker all run on cpu 1. Firstly, compared with 4.18, there are extra
operations(eg. queue work and wake up kworker) while processing endio,
which is slower than processing endio under nvme irq context. Secondly,
fio and kworker will race to get cpu resource, which will slow down fio.
There is no need to do
ext4_convert_unwritten_extents/ext4_handle_inode_extension under
ext4_dio_write_end_io in overwriting case, so we can put ext4 dio endio
under irq context.
It is worth noting that iomap_dio_complete shouldn't be executed under
irq context if datasync is required(IOMAP_DIO_NEED_SYNC), so overwriting
endio is still done in kworker for this situation.
Zhihao Cheng (2):
iomap: Add a IOMAP_DIO_MAY_INLINE_COMP flag
ext4: Optimize endio process for DIO overwrites
fs/ext4/file.c | 8 ++++++--
fs/iomap/direct-io.c | 10 ++++++++--
include/linux/iomap.h | 6 ++++++
3 files changed, 20 insertions(+), 4 deletions(-)
--
2.39.2