Received-SPF: pass (google.com: domain of linux-ext4-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5;
Date:   Thu, 19 Oct 2023 12:18:54 +0200
From:   Jan Kara <jack@suse.cz>
To:     Andy Shevchenko <andriy.shevchenko@intel.com>
Cc:     Jan Kara <jack@suse.cz>, Ferry Toth <ftoth@exalondelft.nl>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org
Subject: Re: [GIT PULL] ext2, quota, and udf fixes for 6.6-rc1
Message-ID: <20231019101854.yb5gurasxgbdtui5@quack3>
References: <ZS5i1cWZF1fLurLz@smile.fi.intel.com>
 <ZS50DI8nw9oSc4Or@smile.fi.intel.com>
 <20231017133245.lvadrhbgklppnffv@quack3>
 <ZS6PRdhHRehDC+02@smile.fi.intel.com>
 <ZS6fIkTVtIs-UhFI@smile.fi.intel.com>
 <ZS6k7nLcbdsaxUGZ@smile.fi.intel.com>
 <ZS6pmuofSP3uDMIo@smile.fi.intel.com>
 <ZS6wLKrQJDf1_TUe@smile.fi.intel.com>
 <20231018184613.tphd3grenbxwgy2v@quack3>
 <ZTDtAiDRuPcS2Vwd@smile.fi.intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ZTDtAiDRuPcS2Vwd@smile.fi.intel.com>
Precedence: bulk

On Thu 19-10-23 11:46:58, Andy Shevchenko wrote:
> On Wed, Oct 18, 2023 at 08:46:13PM +0200, Jan Kara wrote:
> > On Tue 17-10-23 19:02:52, Andy Shevchenko wrote:
> > > On Tue, Oct 17, 2023 at 06:34:50PM +0300, Andy Shevchenko wrote:
> > > > On Tue, Oct 17, 2023 at 06:14:54PM +0300, Andy Shevchenko wrote:
> > > > > On Tue, Oct 17, 2023 at 05:50:10PM +0300, Andy Shevchenko wrote:
> > > > > > On Tue, Oct 17, 2023 at 04:42:29PM +0300, Andy Shevchenko wrote:
> > > > > > > On Tue, Oct 17, 2023 at 03:32:45PM +0200, Jan Kara wrote:
> > > > > > > > On Tue 17-10-23 14:46:20, Andy Shevchenko wrote:
> > > > > > > > > On Tue, Oct 17, 2023 at 01:32:53PM +0300, Andy Shevchenko wrote:
> > > > > > > > > > On Tue, Oct 17, 2023 at 01:29:27PM +0300, Andy Shevchenko wrote:
> > > > > > > > > > > On Tue, Oct 17, 2023 at 01:27:19PM +0300, Andy Shevchenko wrote:
> > > > > > > > > > > > On Wed, Aug 30, 2023 at 12:24:34PM +0200, Jan Kara wrote:
> > > > > > > > > > > > >   Hello Linus,
> 
> ...
> 
> > > > > > > > > > > > This merge commit (?) broke boot on Intel Merrifield.
> > > > > > > > > > > > It has earlycon enabled and only what I got is watchdog
> > > > > > > > > > > > trigger without a bit of information printed out.
> > > > > > > > > 
> > > > > > > > > Okay, seems false positive as with different configuration it
> > > > > > > > > boots. It might be related to the size of the kernel itself.
> > > > > > > > 
> > > > > > > > Ah, ok, that makes some sense.
> > > > > > > 
> > > > > > > I should have mentioned that it boots with the configuration say "A",
> > > > > > > while not with "B", where "B" = "A" + "C" and definitely the kernel
> > > > > > > and initrd sizes in the "B" case are bigger.
> > > > > > 
> > > > > > If it's a size (which is only grew from 13M->14M), it's weird.
> > > > > > 
> > > > > > Nevertheless, I reverted these in my local tree
> > > > > > 
> > > > > > 85515a7f0ae7 (HEAD -> topic/mrfld) Revert "defconfig: enable DEBUG_SPINLOCK"
> > > > > > 786e04262621 Revert "defconfig: enable DEBUG_ATOMIC_SLEEP"
> > > > > > 76ad0a0c3f2d Revert "defconfig: enable DEBUG_INFO"
> > > > > > f8090166c1be Revert "defconfig: enable DEBUG_LIST && DEBUG_OBJECTS_RCU_HEAD"
> > > > > > 
> > > > > > and it boots again! So, after this merge something affects one of this?
> > > > > > 
> > > > > > I'll continuing debugging which one is a culprit, just want to share
> > > > > > the intermediate findings.
> > > > > 
> > > > > CONFIG_DEBUG_LIST with this merge commit somehow triggers this issue.
> > > > > Any ideas?
> > > 
> > > > Dropping CONFIG_QUOTA* helps as well.
> > > 
> > > More precisely it's enough to drop either from CONFIG_DEBUG_LIST and CONFIG_QUOTA
> > > to make it boot again.
> > > 
> > > And I'm done for today.
> > 
> > OK, thanks for debugging! So can you perhaps enable CONFIG_DEBUG_LIST
> > permanently in your kernel config and then bisect through the quota changes
> > in the merge? My guess is commit dabc8b20756 ("quota: fix dqput() to follow
> > the guarantees dquot_srcu should provide") might be the culprit given your
> > testing but I fail to see how given I don't expect any quotas to be used
> > during boot of your platform... BTW, there's also fixup: 869b6ea160
> > ("quota: Fix slow quotaoff") merged last week so you could try testing a
> > kernel after this fix to see whether it changes anything.
> 
> It's exactly what my initial report is about, CONFIG_DEBUG_LIST was there
> always with CONFIG_QUOTA as well.

Ah, ok.

> Two bisections (v6.5 .. v6.6-rc1 & something...v6.6-rc6) pointed out to
> merge commit!

I thought CONFIG_DEBUG_LIST arrived through one path, some problematic
quota change arrived through another path and because they cause problems
only together, then bisecting to the merge would be exactly the outcome.
Alas that doesn't seem to be the case :-|.

> I _had_ tried to simply revert the quota changes (I haven't
> said about that before) and it didn't help. I'm so puzzled with all this.

Aha, OK. If even reverting quota changes doesn't help, then it's really
weird...

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR