Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <CAE1WUT7xJyx_gbxJu3r9DJGbqSkWZa-moieiDWC0bue2CxwAwg@mail.gmail.com>
 <BYAPR04MB4965F2E2624369B34346CC5686BC9@BYAPR04MB4965.namprd04.prod.outlook.com>
 <YBD72wlZC323yhqZ@mit.edu>
In-Reply-To: <YBD72wlZC323yhqZ@mit.edu>
From:   Amy Parker <enbyamy@gmail.com>
Date:   Wed, 27 Jan 2021 19:21:13 -0800
Message-ID: <CAE1WUT65q1RZjths-EoKtMLNKUX17A9vzfpUXcYtS5dxTf6AbA@mail.gmail.com>
Subject: Re: Getting a new fs in the kernel
To:     "Theodore Ts'o" <tytso@mit.edu>
Cc:     Chaitanya Kulkarni <Chaitanya.Kulkarni@wdc.com>,
        "linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Tue, Jan 26, 2021 at 9:36 PM Theodore Ts'o <tytso@mit.edu> wrote:
>
> On Tue, Jan 26, 2021 at 07:06:55PM +0000, Chaitanya Kulkarni wrote:
> > From what I've seen you can post the long patch-series as an RFC and get the
> >
> > discussion started.
> >
> > The priority should be ease of review and not the total patch-count.
>
> File systems are also complicated enough that it's useful to make the
> patches available via a git repo, and it's highly recommended that you
> are rebasing it against the latest kernel on a regular basis.

Was already setting up some local git infrastructure for this.

>
> I also strongly recommend that once you get something that mostly
> works, that you start doing regression testing of the file system.

"'Regression testing? What's that? If it compiles, it is good; if it
boots up, it is perfect."

In all seriousness, though, yeah, already been planning for stuff like that.

> Most of the major file systems in Linux use xfstests for their
> testing.

Decently familiar with xfstests, used it for some previous change
testing I had to do.

> One of the things that I've done is to package up xfstests
> as a test appliance, suitable for running under KVM or using Google
> Compute Engine, as a VM, to make it super easy for people to run
> regression tests.  (One of my original goals for packaging it up was
> to make it easy for graduate students who were creating research file
> systems to try running regression tests so they could find potential
> problems --- and understand how hard it is to make a robust,
> production-ready file system, by giving them a realtively well
> documented, turn-key system for running file system regression tests.)
>
> For more information, see:
>
>     https://thunk.org/gce-xfstests
>     https://github.com/tytso/xfstests-bld/blob/master/Documentation/kvm-quickstart.md
>     https://github.com/tytso/xfstests-bld/blob/master/Documentation/kvm-xfstests.md
>     https://github.com/tytso/xfstests-bld/blob/master/Documentation/gce-xfstests.md

Thank you so much for that!

>
> The final thing I'll point out is that file system development is a
> team sport.  Industry estimates are that it takes between 50 and 200
> person-years to create a production-ready, general purpose enterprise
> file system.  For example, ZFS took seven years to develop, starting
> with a core team of 4, and growing to over 14 developers by the time
> it was announced.  And that didn't include all of the QA, release
> engineering, testers, performance engineers, to get it integrated into
> the Solaris product.  Even after it was announced, it was a good four
> years before customers trusted it for production workloads.

Wasn't expecting to do that completely solo, I get that it takes a
significant amount of people time to build something as important as a
production filesystem. Once I get some basic stuff lined out for it,
if I decide to continue, already working on getting people to help
assist with its development.

>
> If you look at the major file systems in Linux: ext4, xfs, btrfs,
> f2fs, etc., you'll find that none of them are solo endeavors, and all
> of them have multiple companies who are employing the developers who
> work on them.  Figuring out how to convince companies that there are
> good business reasons for them to support the developers of your file
> system is important, since in order to keep things going for the long
> haul, it really needs to be more than a single person's hobby.

Yeah, got that.

>
> Good luck!
>
>                                         - Ted

Thank you!

Best regards,
Amy Parker
(she/her/hers)