From: Andreas Dilger <adilger@sun.com>
Subject: Re: Questions for article
Date: Tue, 03 Jun 2008 16:07:10 -0600
Message-ID: <20080603220710.GM2961@webber.adilger.int>
References: <27337.143.166.226.57.1212443437.squirrel@tomslinux.homelinux.org>
 <20080602225942.GQ2961@webber.adilger.int>
 <61365.143.166.255.40.1212505833.squirrel@tomslinux.homelinux.org>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7BIT
Cc: linux-ext4@vger.kernel.org
To: Thomas King <kingttx@tomslinux.homelinux.org>
In-reply-to: <61365.143.166.255.40.1212505833.squirrel@tomslinux.homelinux.org>
Content-disposition: inline
Sender: linux-ext4-owner@vger.kernel.org

On Jun 03, 2008  10:10 -0500, Thomas King wrote:
> > The mballoc code also does efficient block allocations (multi-MB at a
> > time), BUT there is no userspace interface for this yet, except O_DIRECT.
> > The delayed allocation (delalloc) patches for ext4 are still in the unstable
> > part of the patch series...  What Henry is misunderstanding here is that
> > the filesystem blocksize isn't necessarily the maximum unit for space
> > allocation.  I agree we could do this more efficiently (e.g. allocate an
> > entire 128MB block group at a time for large files), but we haven't gotten
> > there yet.
>
> Can I assume this (large block size) is a possibility later?

Well, anything is a possibility later.  There are no plans to implement it.

> > I'd personally tend to keep quiet until we CAN show that ext4
> > runs well on a 100TB filesystem, that e2fsck time isn't fatal, etc.
>
> What will be the largest theoretical filesystem for ext4?

In theory, it could be 2^64 bytes in size, though common architectures
would currently be limited to 2^60 bytes due to 4kB PAGE_SIZE == blocksize.
I'm not at all interested in "theoretical filesystem size", however, since
theory != practise and a 2^64-byte filesystem that takes 10 weeks to format
or fsck wouldn't be very useful...  Not that I think ext4 is that bad, but
I don't like to make claims based on complete guesswork.

> Here are three other features he thought necessary for massive filesystems in
> Linux:
> -T10 DIF (block protect?) aware file system

- DIF support is underway, though I'm not aware of filesystem support for it

> -NFSv4.1 support

- in progress

> -Support for proposed POSIX relaxation extensions for HPC

- nothing more than a proposal, it wouldn't even begin to see Linux
  implementation until there is something more than a few emails on
  the list.  These are mostly meaningless outside of the context of
  a cluster.

Don't get me wrong, these ARE things that Linux will want to implement
as filesystems and clusters get huge, and it is also my job to work on
such large file system deployments.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.