Received-SPF: pass (google.com: domain of linux-kernel+bounces-28520-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161;
Date: Tue, 16 Jan 2024 23:41:25 -0500
From: Kent Overstreet <kent.overstreet@linux.dev>
To: Greg KH <greg@kroah.com>
Cc: Mark Brown <broonie@kernel.org>, Neal Gompa <neal@gompa.dev>, 
	Kees Cook <keescook@chromium.org>, Linus Torvalds <torvalds@linux-foundation.org>, 
	linux-bcachefs@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, 
	linux-hardening@vger.kernel.org, Nikolai Kondrashov <spbnick@gmail.com>, 
	Philip Li <philip.li@intel.com>, Luis Chamberlain <mcgrof@kernel.org>
Subject: Re: [GIT PULL] bcachefs updates for 6.8
Message-ID: <lr2wz4hos4pcavyrmswpvokiht5mmcww2e7eqyc2m7x5k6nbgf@6zwehwujgez3>
References: <202401101625.3664EA5B@keescook>
 <xlynx7ydht5uixtbkrg6vgt7likpg5az76gsejfgluxkztukhf@eijjqp4uxnjk>
 <be2fa62f-f4d3-4b1c-984d-698088908ff3@sirena.org.uk>
 <gaxigrudck7pr3iltgn3fp5cdobt3ieqjwohrnkkmmv67fctla@atcpcc4kdr3o>
 <f8023872-662f-4c3f-9f9b-be73fd775e2c@sirena.org.uk>
 <olmilpnd7jb57yarny6poqnw6ysqfnv7vdkc27pqxefaipwbdd@4qtlfeh2jcri>
 <CAEg-Je8=RijGLavvYDvw3eOf+CtvQ_fqdLZ3DOZfoHKu34LOzQ@mail.gmail.com>
 <40bcbbe5-948e-4c92-8562-53e60fd9506d@sirena.org.uk>
 <2uh4sgj5mqqkuv7h7fjlpigwjurcxoo6mqxz7cjyzh4edvqdhv@h2y6ytnh37tj>
 <2024011532-mortician-region-8302@gregkh>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <2024011532-mortician-region-8302@gregkh>

On Mon, Jan 15, 2024 at 09:13:01PM +0100, Greg KH wrote:
> On Mon, Jan 15, 2024 at 01:42:53PM -0500, Kent Overstreet wrote:
> > > That sounds more like a "(reproducible) tests don't exist" complaint
> > > which is a different thing again to people going off and NIHing fancy
> > > frameworks.
> > 
> > No, it's a leadership/mentorship thing.
> > 
> > And this is something that's always been lacking in kernel culture.
> > Witness the kind of general grousing that goes on at maintainer summits;
> > maintainers complain about being overworked and people not stepping up
> > to help with the grungy responsibilities, while simultaneously we still
> > very much have a "fuck off if you haven't proven yourself" attitude
> > towards newcomers. Understandable given the historical realities (this
> > shit is hard and the penalties of fucking up are high, so there does
> > need to be a barrier to entry), but it's left us with some real gaps.
> > 
> > We don't have enough a people in the senier engineer role who lay out
> > designs and organise people to take on projects that are bigger than one
> > single person can do, or that are necessary but not "fun".
> > 
> > Tests and test infrastructure fall into the necessary but not fun
> > category, so they languish.
> 
> No, they fall into the "no company wants to pay someone to do the work"
> category, so it doesn't get done.
> 
> It's not a "leadership" issue, what is the "leadership" supposed to do
> here, refuse to take any new changes unless someone ponys up and does
> the infrastructure and testing work first?  That's not going to fly, for
> valid reasons.
> 
> And as proof of this, we have had many real features, that benefit
> everyone, called out as "please, companies, pay for this to be done, you
> all want it, and so do we!" and yet, no one does it.  One real example
> is the RT work, it has a real roadmap, people to do the work, a tiny
> price tag, yet almost no one sponsoring it.  Yes, for that specific
> issue it's slowly getting there and better, but it is one example of how
> you view of this might not be all that correct.

Well, what's so special about any of those features? What's special
about the RT work? The list of features and enhancements we want is
never ending.

But good tools are important beacuse they affect the rate of everyday
development; they're a multiplier on the money everone is spending on
salaries.

In everyday development, the rate at which we can run tests and verify
the corectness of the code we're working on is more often than not _the_
limiting factor on rate of development. It's a particularly big deal for
getting new people up to speed, and for work that crosses subsystems.


> And you will see that we now have the infrastructure in places for this.
> The great kunit testing framework, the kselftest framework, and the
> stuff tying it all together is there.  All it takes is people actually
> using it to write their tests, which is slowly happening.
> 
> So maybe, the "leadership" here is working, but in a nice organic way of
> "wouldn't it be nice if you cleaned that out-of-tree unit test framework
> up and get it merged" type of leadership, not mandates-from-on-high that
> just don't work.  So organic you might have missed it :)

Things are moving in the right direction; the testing track at Plumber's
was exciting to see.

Kselftests is not there yet, though. Those tests could all be runnable
with a single command - and _most_ of what's needed is there, the kernel
config dependencies are listed out, but we're still lacking a
testrunner.

I've been trying to get someone interested in hooking them up to ktest
(my ktest, not that other thing), so that we'd have one common
testrunner for running anything that can be a VM test. Similarly with
blktests, mmtests, et cetera.

Having one common way of running all our functional VM tests, and a
common collection of those tests would be a huge win for productivity
because _way_ too many developers are still using slow ad hoc testing
methods, and a good test runner (ktest) gets the edit/compile/test cycle
down to < 1 minute, with the same tests framework for local development
and automated testing in the big test cloud...