Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755539Ab0GGRuS (ORCPT ); Wed, 7 Jul 2010 13:50:18 -0400 Received: from mx1.redhat.com ([209.132.183.28]:1026 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754241Ab0GGRuQ (ORCPT ); Wed, 7 Jul 2010 13:50:16 -0400 Date: Wed, 7 Jul 2010 13:50:11 -0400 From: Vivek Goyal To: Jeff Moyer Cc: Corrado Zoccolo , Jens Axboe , Linux-Kernel Subject: Re: [PATCH 0/2] cfq-iosched: fixing RQ_NOIDLE handling. Message-ID: <20100707175010.GA7842@redhat.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-12-10) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2881 Lines: 69 On Wed, Jul 07, 2010 at 01:03:08PM -0400, Jeff Moyer wrote: > Corrado Zoccolo writes: > > > Hi Jens, > > patch 8e55063 "cfq-iosched: fix corner cases in idling logic", is > > suspected for some regressions on high end hardware. > > The two patches from this series: > > - [PATCH 1/2] cfq-iosched: fix tree-wide handling of rq_noidle > > - [PATCH 2/2] cfq-iosched: RQ_NOIDLE enabled for SYNC_WORKLOAD > > fix two issues that I have identified, related to how RQ_NOIDLE is > > used by the upper layers. > > First patch makes sure that a RQ_NOIDLE coming after a sequence of > > possibly idling requests from the same queue on the no-idle tree will > > clear the noidle_tree_requires_idle flag. > > Second patch enables RQ_NOIDLE for queues in the idling tree, > > restoring the behaviour pre-8e55063 patch. > > Hi, Corrado, > > I ran your kernel through my tests. Here are the results, up against > vanilla, deadline, and the blk_yield patch set: > > just just > fs_mark fio mixed > -------------------------------+-------------- > deadline 529.44 151.4 | 450.0 78.2 > vanilla cfq 107.88 164.4 | 6.6 137.2 > blk_yield cfq 530.82 158.7 | 113.2 78.6 > corrado cfq 80.82 138.1 | 4.5 130.7 > > fs_mark results are in files/second, fio results are in MB/s. All > results are the average of 5 runs. In order to get results for the > mixed workload for both vanilla and Corrado's kernels, I had to extend > the runtime from 30s to 300s. > > So, the changes proposed in this thread actually make performance worse > across the board. This is really surprising. It should have atleast helped in just fs_mark case. I think what is happening is that we are idling on the fsync queue (because it is last queue in the group). After some time jbd thread will submit some IO and we will not preempt the fsync thread. That's why I had also implemented the logic of allowing preemption in case of group idle and that had helped. > > I re-ran my tests against a RHEL 5 kernel (which is based on 2.6.18), > and it shows that fs_mark performance is much better than stock CFQ in > 2.6.35-rc3, and the mixed workload results are much the same as they are > now (which is to say, the fs_mark process is completely starved by the > sequential reader). So, that problem has existed for a long time. If we just stop idling on WRITE_SYNC, we will should be back to almost 2.6.18 CFQ behavior. > > I'm still in the process of collecting data from production servers and > will report back with my findings there. That would be great. Vivek -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/