I've released e2fsprogs 1.46.0 in all of the usual places; it's tagged
in the git trees on git.kernel.org, github, and sourceforge, and
available for download at:
http://www.kernel.org/pub/linux/kernel/people/tytso/e2fsprogs/v1.46.0
and
http://prdownloads.sourceforge.net/e2fsprogs/e2fsprogs-1.46.0.tar.gz
The release notes for 1.46.0 can be found below. Note that there is a
know build failulre if the Fuse libraries are not available (which is
fixed in git), and a regression test failure (j_recover_fast_commit)
on Big Endian systems which we are still investigating. So there will
probably be a v1.46.1 release relatively soon....
Cheers,
- Ted
E2fsprogs 1.46.0 (January 29, 2021)
===================================
Updates/Fixes since v1.45.7:
UI and Features
---------------
E2fsprogs now supports the fast_commit (COMPAT_FAST_COMMIT) feature.
This feature, first available in Linux version 5.10, adds a fine-grained
journalling which improves the latency of the fsync(2) system call. It
should also improve the performance of ext4 file systems exported via
NFS.
E2fsprogs now supports the stable_inodes (COMPAT_STABLE_INODES) feature.
This needed to support the siphash file system encryption algorithm,
which calculates the initial vector (IV) for encryption based on the
UUID and the inode number. This means that we can't renumber inodes
(for example, when shrinking a file system) and the UUID can't be
changed without breaking the ability to decrypt the encryption.
E2fsprogs now supports file systems which have both file system
encryption and the casefold feature enabled. This requires Linux
version 5.10.
E2fsck now will check file names on file systems with case folding
enabled to make sure the characters are valid UTF-8 characters. This is
done for file systems which enforce strict encodings, and optionally if
the extended "check_encoding" option is requested.
The fuse2fs program now supports the "-o norecovery" option, which will
suppress any journal replay that might be necessary, and mounts the file
system read-only.
E2fsck will now find and fix file system corruptions when the encrypted
files have a different policy from their containing directory.
The "htree" command in debugfs now displays the metadata checksums for
hash tree index blocks.
Dumpe2fs will print the error code that Linux kernels newer than v5.6
will save to indicate the class of error which triggered the ext4_error
event.
E2fsprogs programs (in particular, fuse2fs) can now update htree
directories without clearing the htree index.
Mke2fs now sets the s_overhead_cluster field, so that the kernel doesn't
need to calculate it at mount time. This speeds up mounting very large
file systems.
Fixes
-----
E2fsck will properly handle checking for duplicated file names when case
folding is enabled.
Fix various bugs where a maliciously corrupted file systems could case
e2fsck and other e2fsprogs programs to crash.
Tune2fs will properly recalculate directory block checksums when
clearing the dir_index feature.
Fix a bug in e2fsck directory rehashing which could fail with ENOSPC
because it doesn't take into account the space needed for the metadata
checksum, and doesn't create a sufficiently deep index tree.
Clarify the e2fsck messages when it resets the directory link count when
it is set to the overflow value but it is no longer needed.
The filefrag program can now request the kernel to display the extent
status cache by using "filefrag -E". (This requires Linux version 5.4
or newer.)
Performance, Internal Implementation, Development Support etc.
--------------------------------------------------------------
Speed up mke2fs when creating large bigalloc file systems by optimizing
ext2fs_convert_subcluster_bitmap().
Bitmap blocks are now read using multiple threads (for systems with
pthread support). This speeds up dumpe2fs, e2fsck, and debugfs for very
large file systems.
The dumpe2fs and tune2fs will now avoiding to read the block group
descriptors when they are not needed, which speeds up these program when
operating on very large file systems.
Drop use of the sysctl(2) system call, which is deprecated in Linux.
Add support for "configure --enable-developer-features" which enables
features only meant for developer. The first such feature is "e2fsck -E
clear_all_uninit_bits", which clears the uninitialized bit on all
extents for all inodes. Note that this can end up exposing uninitialized
data to userspace, and should only used in very specialized situations.
The e2fsck/revoke.c and e2fsck/recovery.c files are now kept idential
with the fs/jbd2 versions of these files in the kernel.
Fix various compiler and Coverity warnings.
Update to use gettext 0.19.8. This also removes the built-in "intl"
directory as this is now considered deprecated by gettext. This means
that if the system doesn't have gettext installed on the build system,
we will simply disable NLS support.
On Tue, Feb 02, 2021 at 12:32:07PM -0500, Theodore Ts'o wrote:
> E2fsprogs now supports the stable_inodes (COMPAT_STABLE_INODES) feature.
> This needed to support the siphash file system encryption algorithm,
> which calculates the initial vector (IV) for encryption based on the
> UUID and the inode number. This means that we can't renumber inodes
> (for example, when shrinking a file system) and the UUID can't be
> changed without breaking the ability to decrypt the encryption.
Note that in the new encryption formats which stable_inodes allows, the
encryption algorithm is still AES-256-XTS, not SipHash. (SipHash is a hash
function, not an encryption algorithm.) It's the key derivation and IV
generation method that change.
> E2fsprogs now supports file systems which have both file system
> encryption and the casefold feature enabled. This requires Linux
> version 5.10.
The kernel patches for encrypt + casefold on ext4 haven't been merged yet. So
this combination actually won't be supported until Linux 5.12 at the earliest.
- Eric
On Feb 2, 2021, at 10:36, Theodore Ts'o <[email protected]> wrote:
>
> I've released e2fsprogs 1.46.0 in all of the usual places;
>
> E2fsprogs now supports file systems which have both file system
> encryption and the casefold feature enabled. This requires Linux
> version 5.10.
Gah! I just noticed these patches being submitted to the kernel, but
they break the dirdata format in an incompatible manner. I hadn't been
paying close attention to casefold + encryption since I thought it was
only changing the hashing function and not the actual dirent format,
and I didn't see these patches to e2fsprogs being submitted to the
list (I couldn't find them in patchworks either).
Since this hasn't been landed to the kernel yet, and 1.46.0 was *just*
released, can we roll this back and instead make the extra hash stored
in a dirdata field? Preferably #3, since #1 is the Lustre FID, and #2 was
proposed for the high 32 bits of the 64-bit inode number (though it was
never landed and I don't think it is in use anywhere).
Having just looked at these patches for the first time, I can't say for sure,
but it looks like only a small on-disk format change would be needed to
make both features compatible with each other.
There would need to be a bit set in the dirent type field to indicate that
there is a hash stored after the name:
#define EXT2_DIRENT_HASH 0x40
and the length of the hash field would be stored as the first byte. Since the
hash is stored aligned on a 4-byte boundary, the length is variable
(8+alignment), but storing the length at the start would not add any more
space for 3/4 of the names due to the existing padding for alignment.
Cheers, Andreas