From: Zheng Liu Subject: [RFC][PATCH 0/3] ext4: dio overwrite nolock Date: Sat, 28 Apr 2012 11:39:03 +0800 Message-ID: <1335584346-8070-1-git-send-email-wenqing.lz@taobao.com> Cc: Zheng Liu To: linux-ext4@vger.kernel.org Return-path: Received: from mail-pb0-f46.google.com ([209.85.160.46]:40095 "EHLO mail-pb0-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752046Ab2D1DcZ (ORCPT ); Fri, 27 Apr 2012 23:32:25 -0400 Received: by pbbrp8 with SMTP id rp8so1232888pbb.19 for ; Fri, 27 Apr 2012 20:32:24 -0700 (PDT) Sender: linux-ext4-owner@vger.kernel.org List-ID: Hi list, Currently, in ext4, write dio is serialized because i_mutex is locked in generic_file_aio_write. But, when we overwrite some data without changing metadata, these dios can be parallelized. So this patch set aims to make overwrite dio paralleled. When we overwrite some data, the metadata of this file doesn't need to be modified. Thus, we can try to lock i_data_sem directly to synchronized all dio write operations. First of all, a new wrapper function is defined instead of genereic_file_aio_write in order to avoid to lock i_mutex. Then we need to define a new get_block function and a new flag for dio overwrite nolock feature so that we can avoid nested lock and deadlock. In ext4_map_blocks, i_data_sem is acquired to do a lookup. But after adding this new feature, this lock will be acquired in high level. Obviouslyi, here is a nested lock and we need to avoid it. Now, in ext4, we always start a new journal firstly, and then try to acquire i_data_sem. When we do a overwrite dio, journal doesn't need to be created in order to avoid a deadlock. In new wrapper function, called ext4_file_dio_write, it checks whether conditions are satisfied or not. If these are met, we lock i_data_sem directly and parallelize all write operations. In first patch, two functions are defined in order to split into buffered IO and direct IO because we can keep buffered IO that still uses vfs path, and add new feature into dio path. In second patch, we add a new flag and a new function for get_block. This get_block function only does a lookup without any locks. In last patch, dio overwrite nolock is added. This feature also need to use dioread_nolock option. When a filesystem is mounted with dioread_nolock, this feature is enabled. I have run some benchmarks in my desktop to test this feature. In my desktop, it has a Intel(R) Core(TM)2 Duo CPU E8400 @ 3.00GHz, 4G memory and a Intel X-25 160G SSD. I use fio to run my benchmarks and I compare dio overwrite nolock with w/o dioread_nolock and w/ dioread_nolock. = case 1 = == config file == [global] ioengine=psync direct=1 bs=4k size=32G runtime=60 directory=/mnt/ext4/ filename=testfile group_reporting thread [file1] numjobs=1 # 4 8 16 rw=randwrite == result (iops) == write 1 4 8 16 lock 7233 8612 9102 9165 dioread_nolock 8217 8228 8673 8755 diooverwrite_nolock 7740 15446 14563 17749 = case 2 = == config file == [global] ioengine=sync direct=1 bs=4k size=32G runtime=60 directory=/mnt/ext4/ filename=testfile group_reporting thread [file1] numjobs=1 # 2 4 8 rw=randread [file2] numjobs=1 # 2 4 8 rw=randwrite == result (iops) == read/write 2 4 8 16 lock 614/4343 1346/3124 1271/3930 1386/3904 dioread_nolock 1040/1963 2162/1243 3980/1479 13716/924 diooverwrite_nolock 1006/1913 1973/2602 3683/4515 6966/7260 Regards, Zheng Zheng Liu (3): ext4: split ext4_file_write into buffered IO and direct IO ext4: add a new flag for ext4_map_blocks ext4: add dio overwrite nolock fs/ext4/ext4.h | 2 + fs/ext4/file.c | 200 +++++++++++++++++++++++++++++++++++++++++++++++++------ fs/ext4/inode.c | 46 ++++++++++--- 3 files changed, 215 insertions(+), 33 deletions(-)