Date: Tue, 19 Feb 2008 14:14:12 +0100
From: Paul Slootman <paul@wurtel.net>
To: linux-kernel@vger.kernel.org
Subject: Re: very poor ext3 write performance on big filesystems?
Message-ID: <20080219131412.GA2255@msgid.wurtel.net>
Mail-Followup-To: linux-kernel@vger.kernel.org
References: <9YdLC-75W-51@gated-at.bofh.it> <9YeRh-Gq-39@gated-at.bofh.it> <9Yf0W-SX-19@gated-at.bofh.it> <9YfNi-2da-23@gated-at.bofh.it> <9YfWL-2pZ-1@gated-at.bofh.it> <9Yg6H-2DJ-23@gated-at.bofh.it>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <9Yg6H-2DJ-23@gated-at.bofh.it>
User-Agent: Mutt/1.5.17 (2007-11-01)
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 1196
Lines: 26

On Mon 18 Feb 2008, Andi Kleen wrote:
> On Mon, Feb 18, 2008 at 10:16:32AM -0500, Theodore Tso wrote:
> > On Mon, Feb 18, 2008 at 04:02:36PM +0100, Tomasz Chmielewski wrote:
> > > I tried to copy that filesystem once (when it was much smaller) with "rsync 
> > > -a -H", but after 3 days, rsync was still building an index and didn't copy 
> > > any file.
> > 
> > If you're going to copy the whole filesystem don't use rsync! 
> 
> Yes, I managed to kill systems (drive them really badly into oom and
> get very long swap storms) with rsync -H in the past too. Something is very 
> wrong with the rsync implementation of this.

Note that the soon-to-be-released version 3.0.0 of rsync has very much
improved performance here, both in speed and memory usage, thanks to a
new incremental transfer protocol (before it read in the complete file
list, first on the source, then on the target, and then only started to
do the actual work).


Paul Slootman
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/