Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id ; Tue, 13 Nov 2001 01:36:05 -0500 Received: (majordomo@vger.kernel.org) by vger.kernel.org id ; Tue, 13 Nov 2001 01:35:46 -0500 Received: from vindaloo.ras.ucalgary.ca ([136.159.55.21]:63369 "EHLO vindaloo.ras.ucalgary.ca") by vger.kernel.org with ESMTP id ; Tue, 13 Nov 2001 01:35:41 -0500 Date: Mon, 12 Nov 2001 23:34:44 -0700 Message-Id: <200111130634.fAD6YiM19519@vindaloo.ras.ucalgary.ca> From: Richard Gooch To: Mike Fedyk Cc: Andrew Morton , Ben Israel , linux-kernel@vger.kernel.org Subject: Re: File System Performance In-Reply-To: <20011112172832.F32099@mikef-linux.matchmail.com> In-Reply-To: <00b201c16b81$9d7aaba0$5101a8c0@pbc.adelphia.net> <3BEFF9D1.3CC01AB3@zip.com.au> <00da01c16ba2$96aeda00$5101a8c0@pbc.adelphia.net> <3BF02702.34C21E75@zip.com.au> <200111121959.fACJxsj08462@vindaloo.ras.ucalgary.ca> <20011112150740.B32099@mikef-linux.matchmail.com> <200111130004.fAD04v912703@vindaloo.ras.ucalgary.ca> <20011112160822.E32099@mikef-linux.matchmail.com> <200111130026.fAD0QVK13232@vindaloo.ras.ucalgary.ca> <20011112172832.F32099@mikef-linux.matchmail.com> Sender: linux-kernel-owner@vger.kernel.org X-Mailing-List: linux-kernel@vger.kernel.org [In the interests of brevity, I'm going to trim this to respond to one point only, which seems to be the cause of the confusion] Mike Fedyk writes: > On Mon, Nov 12, 2001 at 05:26:31PM -0700, Richard Gooch wrote: > > Mike Fedyk writes: > > > On Mon, Nov 12, 2001 at 05:04:57PM -0700, Richard Gooch wrote: > > > > > > Here's an idea: add a "--compact" option to tar, so that it creates > > > > > > *all* inodes (files and directories alike) in the base directory, and > > > > > > then renames newly created entries to shuffle them into their correct > > > > > > positions. That should limit the number of block groups that are used, > > > > > > right? > > > > > > > > Currently, without any patching, any new directory will be put in a > different block group from its parent. Your statement seems inconsistent with the comment above fs/ext2/ialloc.c:ext2_new_inode(): /* * There are two policies for allocating an inode. If the new inode is * a directory, then a forward search is made for a block group with both * free space and a low directory-to-inode ratio; if that fails, then of * the groups with above-average free space, that group with the fewest * directories already is chosen. * * For other inodes, search forward from the parent directory\'s block * group to find a free inode. */ So N successive calls to mkdir(2), with the same parent, should result in those directories being stored in the same block group as each other. And, furthermore, if the parent directory block group is mostly empty, then the child directories are placed adjacent to the parent's block group. > So, if you create the dirs in the same dir and then shuffle them > around, you gain nothing. If my reading of the comment above is correct, then the shuffling will provide a gain. > > > > > > It would probably also be a good idea to do that for cp as well, so > > > > > > that when I do a "cp -al" of a virgin kernel tree, I can keep all the > > > > > > directory inodes together. It will make a cold diff even faster. > > > > > > > This doesn't fix all fast growth type apps, only tar and cp... Sure. But it would provide me with a way of getting a better layout for my kernel trees, and it might prove useful for other applications and kernels. And it will provide a good fallback if good in-kernel heuristic isn't forthcoming. > > > > Mike Fedyk writes: > > > > > I don't think that would help at all... With the current file/dir > > > > > allocator it will choose a new block group for each directory no > > > > > matter what the parent is... > > > > > > Now does this make sence? It would, if you were correct about the current implementation. Which I think isn't the case. > > > On Mon, Nov 12, 2001 at 05:04:57PM -0700, Richard Gooch wrote: > > > > I thought the current implementation was that when creating a > > > > directory, ext2fs searches forward from the block group the parent > > > > directory is in, looking for a "relatively free" block group. So, a > > > > number of successive calls to mkdir(2) with the same parent directory > > > > will result in the child directories being in the same block group. > > > > > > Not currently, but the patch that is out will do this. I think it currently *does*. Check the comment. Straight from the 2.4.14 sources. Regards, Richard.... Permanent: rgooch@atnf.csiro.au Current: rgooch@ras.ucalgary.ca - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/