From: Theodore Tso Subject: Re: High CPU Utilization When Copying to Ext4 Date: Tue, 28 Jun 2011 16:14:07 -0400 Message-ID: References: <341DAA96EE3A8444B6E4657BE8A846EA4B3DA126FE@NDJSSCC06.ndc.nasa.gov>,<20110627030539.GF3064@thunk.org> <341DAA96EE3A8444B6E4657BE8A846EA4B3DA12708@NDJSSCC06.ndc.nasa.gov> Mime-Version: 1.0 (Apple Message framework v1084) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 8BIT Cc: "linux-ext4@vger.kernel.org" To: "Mccauliff, Sean D. (ARC-PX)[Lockheed Martin Space OPNS]" Return-path: Received: from DMZ-MAILSEC-SCANNER-2.MIT.EDU ([18.9.25.13]:55782 "EHLO dmz-mailsec-scanner-2.mit.edu" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751711Ab1F1UOQ convert rfc822-to-8bit (ORCPT ); Tue, 28 Jun 2011 16:14:16 -0400 In-Reply-To: <341DAA96EE3A8444B6E4657BE8A846EA4B3DA12708@NDJSSCC06.ndc.nasa.gov> Sender: linux-ext4-owner@vger.kernel.org List-ID: On Jun 28, 2011, at 2:37 PM, Mccauliff, Sean D. (ARC-PX)[Lockheed Martin Space OPNS] wrote: > uname -a > Linux sasr200-2.arc.nasa.gov 2.6.38.7-30.fc15.x86_64 #1 SMP Fri May 27 05:15:53 UTC 2011 x86_64 x86_64 x86_64 GNU/Linux > > There are about 10M files. Many are small. There are about 2M files that are sparse files. It's hen the copy program gets to these files that the cpu usage gets very high. There are no links of any kind. > > The copy program is written in Java, but uses the fiemap to get the logical address ranges that have actually been allocated. It merges any contiguous logical address ranges when it reads and writes to the new file. Fiemap?!? What kind of copy algorithm are you using? Why aren't you just doing a "read 10 megs from ext3", "write 10 megs to ext4"? How does fiemap figure into this? -- Ted