2001-04-12 16:34:17

by Alexander Viro

[permalink] [raw]
Subject: [PATCH][CFT] ext2 directories in pagecache

Folks, IMO ext2-dir-patch got to the stable stage. Currently
it's against 2.4.4-pre2, but it should apply to anything starting with
2.4.2 or so.

Ted, could you review it for potential inclusion into 2.4 once
it gets enough testing? It's ext2-only (the only change outside of
ext2 is exporting waitfor_one_page()), it doesn't change fs layout,
it seriously simplifies ext2/dir.c and ext2/namei.c and it gives better
VM behaviour.

Patch is on ftp.math.psu.edu/pub/viro/ext2-dir-patch.gz

Folks, please give it a good beating - it works here, but I'd
really like it to get wide testing. Help would be very welcome.
Al


2001-04-18 12:42:44

by James Lewis Nance

[permalink] [raw]
Subject: Re: [PATCH][CFT] ext2 directories in pagecache

On Thu, Apr 12, 2001 at 12:33:42PM -0400, Alexander Viro wrote:
> Folks, IMO ext2-dir-patch got to the stable stage. Currently
> it's against 2.4.4-pre2, but it should apply to anything starting with
> 2.4.2 or so.

Have you had any feedback about this patch? I applied it last night to
2.4.3. It seemed to work. When I booted my computer this morning fsck
complained about problems with the directory on one of my ext2 file systems.
Since fsck does not run on every boot I dont really have a way of knowing if
this has anything to do with your patch or not. I'm running the patched
kernel again right now. Ill shutdown and force an fsck later today to see
if anything shows up.

Thanks,

Jim

2001-04-18 13:00:06

by Anton Altaparmakov

[permalink] [raw]
Subject: Re: [PATCH][CFT] ext2 directories in pagecache

At 13:44 18/04/2001, James Lewis Nance wrote:
>On Thu, Apr 12, 2001 at 12:33:42PM -0400, Alexander Viro wrote:
> > Folks, IMO ext2-dir-patch got to the stable stage. Currently
> > it's against 2.4.4-pre2, but it should apply to anything starting with
> > 2.4.2 or so.
>
>Have you had any feedback about this patch? I applied it last night to
>2.4.3. It seemed to work. When I booted my computer this morning fsck
>complained about problems with the directory on one of my ext2 file systems.
>Since fsck does not run on every boot I dont really have a way of knowing if
>this has anything to do with your patch or not. I'm running the patched
>kernel again right now. Ill shutdown and force an fsck later today to see
>if anything shows up.

Well, here is some feedback. I have been using this patch for quite a while
now without any problems what so ever. Including "make -j bzImage" on a
dual cpu machine and normal use of a productions system. - Frequent crashes
during ntfs development (and hence frequent reboots + fsck) have not shown
up any problems in ext2 + this patch, either. - It seems to be stable all
right on two different PCs (one SMP, celeron and one UP, pentium 133s).

Best regards,

Anton


--
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Linux NTFS Maintainer / WWW: http://sourceforge.net/projects/linux-ntfs/
ICQ: 8561279 / WWW: http://www-stu.christs.cam.ac.uk/~aia21/

2001-04-18 13:09:36

by Alexander Viro

[permalink] [raw]
Subject: Re: [PATCH][CFT] ext2 directories in pagecache



On Wed, 18 Apr 2001, James Lewis Nance wrote:

> On Thu, Apr 12, 2001 at 12:33:42PM -0400, Alexander Viro wrote:
> > Folks, IMO ext2-dir-patch got to the stable stage. Currently
> > it's against 2.4.4-pre2, but it should apply to anything starting with
> > 2.4.2 or so.
>
> Have you had any feedback about this patch? I applied it last night to
> 2.4.3. It seemed to work. When I booted my computer this morning fsck
> complained about problems with the directory on one of my ext2 file systems.

Anything prior to 2.4.4-pre2 has known metadata-corrupting bugs on ext2.
Whether they show up or not depends on the load, phase of moon, etc. but
they are there.

> Since fsck does not run on every boot I dont really have a way of knowing if
> this has anything to do with your patch or not. I'm running the patched
> kernel again right now. Ill shutdown and force an fsck later today to see
> if anything shows up.

Please, upgrade to 2.4.4-pre2 or later. Or, at least, replace bforget()
call in ext2_get_block() with brelse() - that's was the worst one
(and last to be fixed).
Al

2001-04-19 01:02:18

by Daniel Phillips

[permalink] [raw]
Subject: Re: [PATCH][CFT] ext2 directories in pagecache

Al Viro wrote:
> Folks, IMO ext2-dir-patch got to the stable stage. Currently
> it's against 2.4.4-pre2, but it should apply to anything starting with
> 2.4.2 or so.
>
> Ted, could you review it for potential inclusion into 2.4 once
> it gets enough testing? It's ext2-only (the only change outside of
> ext2 is exporting waitfor_one_page()), it doesn't change fs layout,
> it seriously simplifies ext2/dir.c and ext2/namei.c and it gives better
> VM behaviour.
>
> Patch is on ftp.math.psu.edu/pub/viro/ext2-dir-patch.gz
>
> Folks, please give it a good beating - it works here, but I'd
> really like it to get wide testing. Help would be very welcome.

I'm pretty familiar with this code since I hacked on it pretty extensively
last month and it does what it tries to do pretty well. However it relies
heavily on the assumption that directory blocks can be grouped into page-sized
units. This assumption doesn't hold in my directory indexing code, and I
don't see any clean way to extend the approach you use in this patch to my
index design.

On the other hand, there is an alternative approach, suggested to me by
Stephen Tweedie, that will run just as fast as this code and have a lot less
cruft in it, namely - perform buffer operations on the underlying buffers in
the page cache. Now perhaps we should discuss that idea and see if it goes
anywhere.

Aside from the part that's tied to the page cache, your patch is generally a
whole lot nicer to read that the original, and I have already incorporated
parts of it in my directory index patch.

--
Daniel

2001-04-23 22:23:58

by Alexander Viro

[permalink] [raw]
Subject: [PATCH][CFT] (updated) ext2 directories in pagecache



On Thu, 12 Apr 2001, Alexander Viro wrote:

> Folks, IMO ext2-dir-patch got to the stable stage. Currently
> it's against 2.4.4-pre2, but it should apply to anything starting with
> 2.4.2 or so.
>
> Ted, could you review it for potential inclusion into 2.4 once
> it gets enough testing? It's ext2-only (the only change outside of
> ext2 is exporting waitfor_one_page()), it doesn't change fs layout,
> it seriously simplifies ext2/dir.c and ext2/namei.c and it gives better
> VM behaviour.

Previous variant left junk in ->d_type of directory entries
on "old" filesystems (i.e. ones where it should be zeroed). Harmless
(on these filesystems readdir() returned DT_UNKNOWN anyway), but
it PO'd fsck and was the wrong thing anyway.

Fixed and rediffed against current tree (2.4.4-pre6). Folks,
please help with testing.

Patch is on ftp.math.psu.edu/pub/viro/ext2-dir-patch-S4-pre6.gz

Cheers,
Al

2001-04-28 18:17:02

by Alexander Viro

[permalink] [raw]
Subject: Re: [PATCH][CFT] (updated) ext2 directories in pagecache

Patch rediffed to 2.4.4, otherwise - no changes (2.4.4 has a
fix for ext2 race, but it's unrelated to the thing).

Patch is on ftp.math.psu.edu/pub/viro/ext2-dir-patch-S4.gz
Please, help with testing.
Al

2001-04-29 20:49:13

by Daniel Phillips

[permalink] [raw]
Subject: Re: [PATCH][CFT] (updated) ext2 directories in pagecache

> Patch is on ftp.math.psu.edu/pub/viro/ext2-dir-patch-S4.gz

Here is my ext2 directory index as a patch against your patch:

http://kernelnewbies.org/~phillips/htree/dx.pcache-2.4.4

Changes:

- COMBSORT macro replaced by custom sort code
- Most #ifdef CONFIG_EXT2_INDEX's changed to if (<constant>)

To do:

- Split up the split code
- Finalize hash function
- Test/debug big endian
- Fall back to linear search if bad index detected
- Fail gracefully on random data
- Remove the tracing and test options

To apply:

cd source/tree
zcat ext2-dir-patch-S4.gz | patch -p1
cat dx.pcache-2.4.4 | patch -p0

To create an indexed directory:

mount /dev/hdxxx /test -o index
mkdir /test/foo