2017-11-21 18:28:31

by Jens Axboe

[permalink] [raw]
Subject: Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On 11/21/2017 11:12 AM, Christian Borntraeger wrote:
>
>
> On 11/21/2017 07:09 PM, Jens Axboe wrote:
>> On 11/21/2017 10:27 AM, Jens Axboe wrote:
>>> On 11/21/2017 03:14 AM, Christian Borntraeger wrote:
>>>> Bisect points to
>>>>
>>>> 1b5a7455d345b223d3a4658a9e5fce985b7998c1 is the first bad commit
>>>> commit 1b5a7455d345b223d3a4658a9e5fce985b7998c1
>>>> Author: Christoph Hellwig <[email protected]>
>>>> Date: Mon Jun 26 12:20:57 2017 +0200
>>>>
>>>> blk-mq: Create hctx for each present CPU
>>>>
>>>> commit 4b855ad37194f7bdbb200ce7a1c7051fecb56a08 upstream.
>>>>
>>>> Currently we only create hctx for online CPUs, which can lead to a lot
>>>> of churn due to frequent soft offline / online operations. Instead
>>>> allocate one for each present CPU to avoid this and dramatically simplify
>>>> the code.
>>>>
>>>> Signed-off-by: Christoph Hellwig <[email protected]>
>>>> Reviewed-by: Jens Axboe <[email protected]>
>>>> Cc: Keith Busch <[email protected]>
>>>> Cc: [email protected]
>>>> Cc: [email protected]
>>>> Link: http://lkml.kernel.org/r/[email protected]
>>>> Signed-off-by: Thomas Gleixner <[email protected]>
>>>> Cc: Oleksandr Natalenko <[email protected]>
>>>> Cc: Mike Galbraith <[email protected]>
>>>> Signed-off-by: Greg Kroah-Hartman <[email protected]>
>>>
>>> I wonder if we're simply not getting the masks updated correctly. I'll
>>> take a look.
>>
>> Can't make it trigger here. We do init for each present CPU, which means
>> that if I offline a few CPUs here and register a queue, those still show
>> up as present (just offline) and get mapped accordingly.
>>
>> From the looks of it, your setup is different. If the CPU doesn't show
>> up as present and it gets hotplugged, then I can see how this condition
>> would trigger. What environment are you running this in? We might have
>> to re-introduce the cpu hotplug notifier, right now we just monitor
>> for a dead cpu and handle that.
>
> I am not doing a hot unplug and the replug, I use KVM and add a previously
> not available CPU.
>
> in libvirt/virsh speak:
> <vcpu placement='static' current='1'>4</vcpu>

So that's why we run into problems. It's not present when we load the device,
but becomes present and online afterwards.

Christoph, we used to handle this just fine, your patch broke it.

I'll see if I can come up with an appropriate fix.

--
Jens Axboe


From 1584700363431132393@xxx Tue Nov 21 18:13:55 +0000 2017
X-GM-THRID: 1584670276912512570
X-Gmail-Labels: Inbox,Category Forums,HistoricalUnread