Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751773AbXB0TQe (ORCPT ); Tue, 27 Feb 2007 14:16:34 -0500 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1751774AbXB0TQe (ORCPT ); Tue, 27 Feb 2007 14:16:34 -0500 Received: from relay.2ka.mipt.ru ([194.85.82.65]:40733 "EHLO 2ka.mipt.ru" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751773AbXB0TQd (ORCPT ); Tue, 27 Feb 2007 14:16:33 -0500 Date: Tue, 27 Feb 2007 22:08:31 +0300 From: Evgeniy Polyakov To: Jens Axboe Cc: Suparna Bhattacharya , Ingo Molnar , linux-kernel@vger.kernel.org, Linus Torvalds , Arjan van de Ven , Christoph Hellwig , Andrew Morton , Alan Cox , Ulrich Drepper , Zach Brown , "David S. Miller" , Davide Libenzi , Thomas Gleixner Subject: Re: A quick fio test (was Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3) Message-ID: <20070227190831.GA29173@2ka.mipt.ru> References: <20070223171348.GA27838@in.ibm.com> <20070226135736.GF3822@kernel.dk> <20070226141315.GA15631@in.ibm.com> <20070226144548.GH3822@kernel.dk> <20070227043331.GA29942@in.ibm.com> <20070227094211.GR3822@kernel.dk> <20070227111258.GA19125@2ka.mipt.ru> <20070227112908.GB4271@kernel.dk> <20070227121958.GB31597@2ka.mipt.ru> <20070227184541.GA3733@kernel.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline In-Reply-To: <20070227184541.GA3733@kernel.dk> User-Agent: Mutt/1.5.9i X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (2ka.mipt.ru [0.0.0.0]); Tue, 27 Feb 2007 22:09:29 +0300 (MSK) Sender: linux-kernel-owner@vger.kernel.org X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2282 Lines: 57 On Tue, Feb 27, 2007 at 07:45:41PM +0100, Jens Axboe (jens.axboe@oracle.com) wrote: > > Deadline shows this: > > > > sync: > > READ: io=1,024MiB, aggrb=38,212KiB/s, minb=38,212KiB/s, > > maxb=38,212KiB/s, mint=28099msec, maxt=28099msec > > > > libaio: > > READ: io=1,024MiB, aggrb=37,933KiB/s, minb=37,933KiB/s, > > maxb=37,933KiB/s, mint=28306msec, maxt=28306msec > > > > syslet-rw: > > READ: io=1,024MiB, aggrb=34,759KiB/s, minb=34,759KiB/s, > > maxb=34,759KiB/s, mint=30891msec, maxt=30891msec > > > > There were about 10k async schedulings. > > I think the issue here is pretty simple - when fio gets a queue full > like condition (it reaches the depth you set, 32), it commits them and > starts queuing again. Since that'll likely block, it'll get issued by > another process. So you suddenly have a nice sequence of reads from one > process (pending, only one is actually committed since it's serialized), > and then a read further down the line that goes behind those you already > committed. Then result is seeky, where it should have been sequential. > > Do you get expected results if you set iodepth_low=1? That'll make fio > drain the queue before building it up again, should get you a sequential > access pattern with syslets. With such a change results should be better - not only because seek is removed with sequential read, but also number of working threads decreases with time - until queue is filled again. So, syslet-rw has increased to 37mb/sec out of 39/sync and 38/libaio, the latter two did not changed. With iodepth of 10k, I get the same performance for libaio and syslets - about 36mb/sec, it does not depend on iodepth_low being set to 1 or default (full). So syslets have small problems with small number of iodepth - its performance is about 34mb/sec and then increases to 36 with iodepth grow. While libaio decreases from 38 down to 36 mb/sec. iodepth_low=1 helps syslets to have 37mb/sec with iodepth=32, with 3200 and 10k it does not play any role. > -- > Jens Axboe -- Evgeniy Polyakov - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/