Subject: Re: Write I/O queue hangup at random on recent Linus' kernels
To:     "Kenneth R. Crudup" <kenny@panix.com>
Cc:     linux-bcache@vger.kernel.org, linux-block@vger.kernel.org,
        linux-ext4@vger.kernel.org, linux-nvme@lists.infradead.org,
        Shinichiro Kawasaki <shinichiro.kawasaki@wdc.com>
References: <b3ba57a7-d363-9c17-c4be-9dbe86875@panix.com>
 <b9c2681f-e63a-4d3b-913d-d8a75e2c2ea0@kernel.dk>
 <be6a783-97db-c3bf-b16f-e8c62b14755d@panix.com>
 <17206ea6-506d-b1de-09e8-c935ff308bd6@kernel.dk>
 <903be817-4118-f34e-1b35-a0108045590f@kernel.dk>
 <986e942b-d430-783b-5b1c-4525d4a94e48@panix.com>
From:   Jens Axboe <axboe@kernel.dk>
Message-ID: <ddc41b84-c414-006a-0840-250281caf1e5@kernel.dk>
Date:   Fri, 26 Nov 2021 11:10:09 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <986e942b-d430-783b-5b1c-4525d4a94e48@panix.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 11/26/21 10:58 AM, Kenneth R. Crudup wrote:
> 
> On Fri, 26 Nov 2021, Jens Axboe wrote:
> 
>> Can you apply this on top of 5.16-rc2 or current -git and see if it fixes
>> it for you?
> 
> Since it's related to writeback throttling, is there anything I can try to
> taunt it? I was thinking "iozone" or "bonnie++", but if there's something
> better, let me know.

I'd just do what you usually do, that's usually the best way to gain
confidence in the fix. But it's related to writeback throttling _and_
racing with eg iostats grabbing a reference to the request, so it would
be more likely to trigger with a heavier combination of those two.

That said, I'm pretty confident in the fix, so if you'd just use it for
a day or two, then with the previous amount of hangs you saw on -rc2 it
should provide a good level of confidence in it.

-- 
Jens Axboe