Received: by 2002:a25:ad19:0:0:0:0:0 with SMTP id y25csp3385970ybi; Fri, 19 Jul 2019 02:25:33 -0700 (PDT) X-Google-Smtp-Source: APXvYqxvhHtrGBPv3QkpDSEDFCYNIHKKn9AowcZNl2z8hBWIHYE9JUKvCKk8iW75cXOullE72gA2 X-Received: by 2002:a17:90a:9f0b:: with SMTP id n11mr19113413pjp.98.1563528333678; Fri, 19 Jul 2019 02:25:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1563528333; cv=none; d=google.com; s=arc-20160816; b=ltwhW4PMhN/9qfN3ivDLD7pSa4JLfsgjDaAvd7mH07f+EIH8W8LJj3OeZdz/IodgUK ymNhfSz6JdctZs1OlztipjPCOT1Siq+zlUILRcgy+QgKS0SvTok3kq5M7hpeGwRTwFNc mRs94Iba6HUKKZxXWiwBMzMt+p5R2p74PcgpqxR1hauwlNRy2ZznmjLvAYp8Yxgq/eg0 Bs/3koarbcGC0ODxYQMGiRwvVCW7qszh0QN9eEOyFtMjWl0WclzmcQIzJM78G/N6Exel 7poicYpxEhQm1ErhDutkeDe6jQlkhCJVkxUOSzIizobtqKZc5Hf0NqqGaxS1MJUUuftP l9qA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding :content-language:mime-version:user-agent:date:message-id:subject :from:cc:to; bh=jN7TKCz+2xc5AoKl/PubUi58CDWx7D4KDratO2ryoiA=; b=0QWTrYsUOGbjqYQpYkBprCusH+ZEvl2B/J+xMN56NsCCeTPNBj3E/WheFr5oTWW0rW di0GjJVdPuiWjrFi6Xpsci0yCgtTJPMFJqnvWGYyswYXQO4yopUORekfPSg08R2LNRrT gdjCAtMLyNRMotOzgAjpbFWZAPLhSS7nv8o3e1dmCn7lu/LNnx2T71v1NOtmz5C4WhZy 81xlcg8WUVRldjEFoomXlQs1IlC0JF2lXbpZV9vkPNe9v/DpkwuZXO3G82wS5xIGTWp7 8hzlw+NGriuGbh6tPFc6x7J7n/ZS9xkoK1LBqUOeGmRMtzWCy4CW5T0fQ7+2QCQvC9j6 UHrg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-ext4-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-ext4-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f4si1705693pgg.334.2019.07.19.02.25.10; Fri, 19 Jul 2019 02:25:33 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-ext4-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-ext4-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-ext4-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725794AbfGSJW3 (ORCPT + 99 others); Fri, 19 Jul 2019 05:22:29 -0400 Received: from out30-57.freemail.mail.aliyun.com ([115.124.30.57]:45286 "EHLO out30-57.freemail.mail.aliyun.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726036AbfGSJW2 (ORCPT ); Fri, 19 Jul 2019 05:22:28 -0400 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R121e4;CH=green;DM=||false|;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04394;MF=joseph.qi@linux.alibaba.com;NM=1;PH=DS;RN=4;SR=0;TI=SMTPD_---0TXH-bb4_1563528144; Received: from JosephdeMacBook-Pro.local(mailfrom:joseph.qi@linux.alibaba.com fp:SMTPD_---0TXH-bb4_1563528144) by smtp.aliyun-inc.com(127.0.0.1); Fri, 19 Jul 2019 17:22:25 +0800 To: Theodore Ts'o , Jan Kara Cc: linux-ext4@vger.kernel.org, Xiaoguang Wang From: Joseph Qi Subject: [RFC] performance regression with "ext4: Allow parallel DIO reads" Message-ID: Date: Fri, 19 Jul 2019 17:22:24 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Sender: linux-ext4-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org Hi Ted & Jan, I've observed an significant performance regression with the following commit in my Intel P3600 NVMe SSD. 16c54688592c ext4: Allow parallel DIO reads From my initial investigation, it may be because of the inode_lock_shared (down_read) consumes more than inode_lock (down_write) in mixed random read write workload. Here is my test result. ioengine=psync direct=1 rw=randrw iodepth=1 numjobs=8 size=20G runtime=600 w/ parallel dio reads : kernel 5.2.0 w/o parallel dio reads: kernel 5.2.0, then revert the following commits: 1d39834fba99 ext4: remove EXT4_STATE_DIOREAD_LOCK flag (related) e5465795cac4 ext4: fix off-by-one error when writing back pages before dio read (related) 16c54688592c ext4: Allow parallel DIO reads bs=4k: ------------------------------------------------------------------------------------------- w/ parallel dio reads | READ 30898KB/s, 7724, 555.00us | WRITE 30875KB/s, 7718, 479.70us ------------------------------------------------------------------------------------------- w/o parallel dio reads| READ 117915KB/s, 29478, 248.18us | WRITE 117854KB/s，29463, 21.91us ------------------------------------------------------------------------------------------- bs=16k: ------------------------------------------------------------------------------------------- w/ parallel dio reads | READ 58961KB/s, 3685, 835.28us | WRITE 58877KB/s, 3679, 1335.98us ------------------------------------------------------------------------------------------- w/o parallel dio reads| READ 218409KB/s, 13650, 554.46us | WRITE 218257KB/s，13641, 29.22us ------------------------------------------------------------------------------------------- bs=64k: ------------------------------------------------------------------------------------------- w/ parallel dio reads | READ 119396KB/s, 1865, 1759.38us | WRITE 119159KB/s, 1861, 2532.26us ------------------------------------------------------------------------------------------- w/o parallel dio reads| READ 422815KB/s, 6606, 1146.05us | WRITE 421619KB/s, 6587, 60.72us ------------------------------------------------------------------------------------------- bs=512k: ------------------------------------------------------------------------------------------- w/ parallel dio reads | READ 392973KB/s, 767, 5046.35us | WRITE 393165KB/s, 767, 5359.86us ------------------------------------------------------------------------------------------- w/o parallel dio reads| READ 590266KB/s, 1152, 4312.01us | WRITE 590554KB/s, 1153, 2606.82us ------------------------------------------------------------------------------------------- bs=1M: ------------------------------------------------------------------------------------------- w/ parallel dio reads | READ 487779KB/s, 476, 8058.55us | WRITE 485592KB/s, 474, 8630.51us ------------------------------------------------------------------------------------------- w/o parallel dio reads| READ 593927KB/s, 580, 7623.63us | WRITE 591265KB/s, 577, 6163.42us ------------------------------------------------------------------------------------------- Thanks, Joseph