Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933295Ab0LTXYw (ORCPT ); Mon, 20 Dec 2010 18:24:52 -0500 Received: from straum.hexapodia.org ([207.7.131.186]:57476 "EHLO straum.hexapodia.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S933143Ab0LTXYv (ORCPT ); Mon, 20 Dec 2010 18:24:51 -0500 Date: Mon, 20 Dec 2010 15:24:51 -0800 From: Andy Isaacson To: linux-btrfs@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: btrfs: 21 minutes to read 1.2M file directory Message-ID: <20101220232451.GA9755@hexapodia.org> References: <20101220222446.GA8629@hexapodia.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20101220222446.GA8629@hexapodia.org> X-GPG-Fingerprint: 1914 0645 FD53 C18E EEEF C402 4A69 B1F3 68D2 A63F X-GPG-Key-URL: http://web.hexapodia.org/~adi/gpg.txt X-Domestic-Surveillance: money launder bomb tax evasion User-Agent: Mutt/1.5.18 (2008-05-17) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3562 Lines: 84 Sigh, wrong btrfs address on the original. Apologies for the double-post. On Mon, Dec 20, 2010 at 02:24:46PM -0800, Andy Isaacson wrote: > I have a directory with 1.2M files in it, which makes readdir very slow > on btrfs with cold caches (although it's reasonably fast with hot caches > as in the first example below): > > % time find /btr/foo > /btr/foo.list > find /btr/foo > /btr/foo.list 4.10s user 7.97s system 36% cpu 33.275 total > % head /btr/foo.list > /btr/foo > /btr/foo/1281373625.777.fg.jpg > /btr/foo/1281373625.777.bg.jpg > /btr/foo/1281373625.948.fg.jpg > /btr/foo/1281373625.948.bg.jpg > /btr/foo/1281373626.096.fg.jpg > /btr/foo/1281373626.096.bg.jpg > /btr/foo/1281373626.218.fg.jpg > /btr/foo/1281373626.218.bg.jpg > /btr/foo/1281373626.350.fg.jpg > % wc !$ > wc /btr/foo.list > 12166666 12166666 401499940 /btr/foo.list > % wc -l /btr/foo.list > 12166666 /btr/foo.list > % sudo sysctl -w vm.drop_caches=3 vm.drop_caches=0 > vm.drop_caches = 3 > vm.drop_caches = 0 > % time find /btr/foo > /btr/foo.list.2 > find /btr/foo > /btr/foo.list.2 5.62s user 24.54s system 2% cpu 21:40.90 total > % uname -a > Linux pyron 2.6.36-rc7-00149-g29979aa #71 SMP Wed Oct 13 09:42:57 PDT 2010 x86_64 GNU/Linux > > Interestingly, while readdir is busy I'm only seeing IO on sdb even > though the btrfs is on 3 targets: > > Label: btr uuid: 1271de53-b3d2-4d68-9d48-b19487e1c982 > Total devices 3 FS bytes used 555.13GB > devid 1 size 18.65GB used 18.64GB path /dev/sda2 > devid 3 size 512.00GB used 44.13GB path /dev/sdc1 > devid 2 size 512.00GB used 511.76GB path /dev/sdb1 > > "iostat -k 1 | grep sdb" tells me: > > Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn > > sdb 173.00 692.00 0.00 692 0 > sdb 185.00 740.00 0.00 740 0 > sdb 198.00 792.00 0.00 792 0 > sdb 177.00 712.00 0.00 712 0 > > I updated to a recent git and it's still slow (my test hasn't completed > yet 19 minutes in): > > Linux pyron 2.6.37-rc6-11882-g55ec86f #72 SMP Mon Dec 20 13:34:38 PST 2010 x86_64 GNU/Linux > > The devices are: > > [ 1.834527] ata1.00: ATA-7: INTEL SSDSA2M040G2GC, 2CV102HD, max UDMA/133 > [ 1.834816] ata1.00: 78165360 sectors, multi 1: LBA48 NCQ (depth 31/32) > [ 1.835369] ata1.00: configured for UDMA/133 > [ 1.835776] scsi 0:0:0:0: Direct-Access ATA INTEL SSDSA2M040 2CV1 PQ: 0 ANSI: 5 > ... > [ 2.904919] ata3.00: ATA-8: ST31500341AS, CC1H, max UDMA/133 > [ 2.905206] ata3.00: 2930277168 sectors, multi 0: LBA48 NCQ (depth 31/32) > [ 2.947393] ata3.00: configured for UDMA/133 > [ 2.947850] scsi 2:0:0:0: Direct-Access ATA ST31500341AS CC1H PQ: 0 ANSI: 5 > ... > [ 3.989664] ata5: SATA link up 3.0 Gbps (SStatus 123 SControl 300) > [ 4.018524] ata5.00: ATA-8: ST31500341AS, CC1H, max UDMA/133 > [ 4.018811] ata5.00: 2930277168 sectors, multi 0: LBA48 NCQ (depth 31/32) > [ 4.060838] ata5.00: configured for UDMA/133 > [ 4.061205] scsi 4:0:0:0: Direct-Access ATA ST31500341AS CC1H PQ: 0 ANSI: 5 > > The host is a "Intel(R) Core(TM) i7 CPU 930 @2.80GHz" with 12GB RAM. > > Thanks, > -andy -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/