Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit
Date:   Thu, 21 Nov 2019 08:13:27 +0100
From:   Oleksandr Natalenko <oleksandr@natalenko.name>
To:     linux-kernel@vger.kernel.org
Cc:     linux-block@vger.kernel.org, paolo.valente@linaro.org
Subject: Injecting delays into block layer
User-Agent: Roundcube Webmail/1.4.0
Message-ID: <d7ee69fc368db16fa96a05643083674a@natalenko.name>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Hi Paolo et al.

I have a strong suspect that something is going wrong when the 
underlying block device responds with a large delay. What makes me 
thinking so is that I use a VM on some cloud provider, and they have 
substantial block device latency resulting in permanently high (~20%) 
iowait. It spikes occasionally when their cluster is overloaded, and 
when that happens, the I/O in my VM may stop and never recover. This is 
a rare occasion, but it really happens.

What's worse, so far I've seen such a behaviour with BFQ only. I'm still 
testing other schedulers though.

Important note: I have no strict evidences that this is *the* case, thus 
I'm asking for some suggestions. My idea is to fire up a local VM and 
inject delays to a block device while performing some I/O from within 
the VM.

So the question is: how can those delays be injected? Using dm-delay? 
Can those delays be random?

Thanks in advance.

-- 
   Oleksandr Natalenko (post-factum)