Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Content-Type: text/plain;
        charset=utf-8
Mime-Version: 1.0 (1.0)
Subject: Re: [PATCH] nvme: default to 0 poll queues
From:   Jens Axboe <axboe@kernel.dk>
In-Reply-To: <4ad5653b-1cd4-a770-2290-ca032eeb7072@roeck-us.net>
Date:   Sat, 8 Dec 2018 23:31:48 -0700
Cc:     Christoph Hellwig <hch@lst.de>,
        Keith Busch <keith.busch@intel.com>,
        Sagi Grimberg <sagi@grimberg.me>,
        linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <C7D155CA-8DA8-4DF2-BBC2-14E6AF436F24@kernel.dk>
References: <20181209004953.GA11638@roeck-us.net> <fa7aabad-774b-4e5b-686f-3696762291c1@kernel.dk> <4ad5653b-1cd4-a770-2290-ca032eeb7072@roeck-us.net>
To:     Guenter Roeck <linux@roeck-us.net>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Dec 8, 2018, at 11:22 PM, Guenter Roeck <linux@roeck-us.net> wrote:
>=20
>> On 12/8/18 9:38 PM, Jens Axboe wrote:
>>> On 12/8/18 5:49 PM, Guenter Roeck wrote:
>>> Hi,
>>>=20
>>>> On Mon, Nov 19, 2018 at 08:18:24AM -0700, Jens Axboe wrote:
>>>> We need a better way of configuring this, and given that polling is
>>>> (still) a bit niche, let's default to using 0 poll queues. That way
>>>> we'll have the same read/write/poll behavior as 4.20, and users that
>>>> want to test/use polling are required to do manual configuration of the=

>>>> number of poll queues.
>>>>=20
>>>> Reviewed-by: Christoph Hellwig <hch@lst.de>
>>>> Signed-off-by: Jens Axboe <axboe@kernel.dk>
>>>> ---
>>>=20
>>> This patch results in a boot stall when booting parisc (hppa) images
>>> from nvme in qemu.
>>>=20
>>> ...
>>> Fusion MPT SAS Host driver 3.04.20
>>> rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
>>> rcu:    (detected by 0, t=3D5252 jiffies, g=3D141, q=3D22)
>>> rcu: All QSes seen, last rcu_sched kthread activity 5252 (-66742--71994)=
, jiffies_till_next_fqs=3D1, root ->qsmask 0x0
>>> kworker/u8:3    R  running task        0    85      2 0x00000004
>>> Workqueue: nvme-reset-wq nvme_reset_work
>>> Backtrace:
>>>  [<10190d20>] show_stack+0x28/0x38
>>>  [<101dd1e0>] sched_show_task.part.3+0xc4/0x144
>>>  [<101dd290>] sched_show_task+0x30/0x38
>>>  [<10221e18>] rcu_check_callbacks+0x760/0x7a4
>>>=20
>>> rcu: rcu_sched kthread starved for 5252 jiffies! g141 f0x2 RCU_GP_WAIT_FQ=
S(5) ->state=3D0x0 ->cpu=3D0
>>> rcu: RCU grace-period kthread stack dump:
>>> rcu_sched       R  running task        0    10      2 0x00000000
>>> Backtrace:
>>>  [<10995b1c>] __schedule+0x214/0x648
>>>  [<10995f94>] schedule+0x44/0xa8
>>>  [<1099a7c4>] schedule_timeout+0x114/0x1a0
>>>  [<10220e70>] rcu_gp_kthread+0x744/0x968
>>>  [<101d5438>] kthread+0x154/0x15c
>>>  [<1019501c>] ret_from_kernel_thread+0x1c/0x24
>>>=20
>>> [ continued ]
>>>=20
>>> This is only seen in SMP configurations; non-SMP configurations are ok.
>>> Reverting the patch fixes the problem. v4.20-rcX and earlier kernels
>>> also boot without problems.
>>>=20
>>> For reference, here is the qemu command line. This is with qemu 3.0.
>>>=20
>>> qemu-system-hppa -kernel vmlinux -no-reboot \
>>>    -snapshot \
>>>    -device nvme,serial=3Dfoo,drive=3Dd0 \
>>>    -drive file=3Drootfs.ext2,if=3Dnone,format=3Draw,id=3Dd0 \
>>>    -append 'root=3D/dev/nvme0n1 rw rootwait panic=3D-1 console=3DttyS0,1=
15200 ' \
>>>    -nographic -monitor null
>>>=20
>>> Please let me know if you need additional information.
>> Hmm, I think the queue reduction case has a logic error. Actually there
>> are two bugs:
>> 1) Ensure we don't keep overwriting the queue count we ask for
>> 2) Don't include poll_queues in the vectors we need
>> Untested... And not super pretty. But does this work for you?
>=20
> It solves the boot problem on parisc/hppa. I didn't test with any other ar=
chitectures.
> Should I run a complete test sequence ?

That=E2=80=99d be great, thanks.=20