Received-SPF: pass (google.com: domain of linux-ext4-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Wed, 7 Jul 2021 19:52:41 -0400
From:   "Theodore Ts'o" <tytso@mit.edu>
To:     Ivan Zahariev <famzah@famzah.net>
Cc:     linux-ext4@vger.kernel.org
Subject: Re: jbd2: fix deadlock while checkpoint thread waits commit thread
 to finish (backport to 4.14)
Message-ID: <YOY+SXgPShxGzyJO@mit.edu>
References: <3221ced0-e8f3-53da-3474-28367272f35d@famzah.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <3221ced0-e8f3-53da-3474-28367272f35d@famzah.net>
Precedence: bulk

On Wed, Jul 07, 2021 at 09:42:25PM +0300, Ivan Zahariev wrote:
> Hello,
> 
> We're running Linux kernel 4.14.x and our systems occasionally suffer a bug
> which is already fixed: https://github.com/torvalds/linux/commit/53cf978457325d8fb2cdecd7981b31a8229e446e
> 
> This bugfix hasn't been ported to Linux kernels 4.14 or 4.19. The patch
> applies cleanly. The two files "fs/jbd2/checkpoint.c" and
> "fs/jbd2/journal.c" seem pretty identical in the affected sections compared
> to kernel 5.4 where we have this bugfix already applied.
> 
> Is it on purpose that this bugfix hasn't been ported to 4.14? Is it safe
> that we backport it manually in our kernel 4.14 builds? Or is the "ext4"
> system in 4.14 and 5.4 fundamentally different and this would lead to data
> loss or other problems?

The commit was over two years ago, so my memory is not going to be
perfect.  However, Jan had made a comment suggesting the approach in
this commit because it should be easier to backport into older stble
kernels[1].

   "Since proper locking change is going to be a bit more involved, can you
    perhaps fix this deadlock by just dropping j_checkpoint_mutex in
    log_do_checkpoint() when we are going to wait for transaction commit. I've
    checked and that should be fine and that is going to be much easier change
    to backport into stable kernels..."

[1] https://marc.info/?l=linux-ext4&m=154212553014669&w=2

So I suspect it was just that I failed to remember to add a "Cc:
stable@kernel.org" and so it was never automatically backported into
4.14 or 4.19.

Do you have a reliable reproduction which is triggering the deadlock
on your kernels?  If so, have you tried applying the patch and does it
make the problem go away for you?

Cheers,

						- Ted