Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
IronPort-SDR: wLRr6LamsSd8y30YAAPRxsdQETpyYJ3RPiflzsOqOjnYW+A5aHZ+IvzeUucGYu2J7nJ7S5Cnq/
 gObFJOY68JaL/Z0mkENmxRCJJT9aZLMhlP26iS6hXvAqXOyr73VkyHyY80kCvl5LWrJSgscg0E
 /sMILGSJDSi8DibGNyNPnTkNuliIesTdjy4rq69h4csMoVmtozjH7bSNZnHVKSD1l/FJZ1mfFU
 tKZ0X3K7ZjjjCGpVvbHaBNpbSYf2Cg7tnUjjaR7t30Zw/oJWV4ZDcAzZzV9zen+pyCoJwCFM+t
 Mxo=
From:   Chaitanya Kulkarni <Chaitanya.Kulkarni@wdc.com>
To:     Josh Triplett <josh@joshtriplett.org>,
        Keith Busch <kbusch@kernel.org>, Jens Axboe <axboe@fb.com>,
        Christoph Hellwig <hch@lst.de>,
        Sagi Grimberg <sagi@grimberg.me>
CC:     "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-nvme@lists.infradead.org" <linux-nvme@lists.infradead.org>
Subject: Re: [PATCH] nvme: Check for readiness more quickly, to speed up boot
 time
Thread-Topic: [PATCH] nvme: Check for readiness more quickly, to speed up boot
 time
Thread-Index: AQHV7qtY5m6g7a4cOEmKX4VCzFI0fA==
Date:   Sun, 1 Mar 2020 02:01:05 +0000
Message-ID: <BYAPR04MB5749363E3AC8C583F5CB076786E60@BYAPR04MB5749.namprd04.prod.outlook.com>
References: <20200229025228.GA203607@localhost>
Accept-Language: en-US
Content-Language: en-US
wdcipoutbound: EOP-TRUE
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: faea1525-f000-4ef5-7505-08d7bd84670a
X-MS-Exchange-CrossTenant-originalarrivaltime: 01 Mar 2020 02:01:05.7226
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b61c8803-16f3-4c35-9b17-6f65f441df86
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: C+G+8ygP+IHfT7dOLobsy5gXLo8UoZqP07G1Ml5JpqlrwuNy3XVVOMxlwO4hVN2rKYIjbiskEG0tqZPGMG7UWY9opqco6DbRCKSowX7Kv5I=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR04MB5333
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Nit:- please have a look at the patch subject line and make=0A=
sure it is not exceeding the required length.=0A=
=0A=
One question though, have you seen similar kind of performance =0A=
improvements when system is booted ?=0A=
=0A=
I took some numbers and couldn't see similar benefit. See [1] :-=0A=
=0A=
Without :-=0A=
=0A=
714.532560-714.456099 =3D .076461=0A=
721.189886-721.110845 =3D .079041=0A=
727.836938-727.765572 =3D .071366=0A=
734.589886-734.519779 =3D .070107=0A=
741.244296-741.173503 =3D .070793=0A=
=0A=
With this patch :-=0A=
=0A=
805.549656-805.461924 =3D .087732=0A=
812.199549-812.124364 =3D .075185=0A=
818.868111-818.793779 =3D .074332=0A=
825.636130-825.554311 =3D .081819=0A=
832.287043-832.205882 =3D .081161=0A=
=0A=
Regards,=0A=
Chaitanya=0A=
=0A=
[1] Detail log :-=0A=
=0A=
Without this patch :-=0A=
=0A=
[  714.456099] nvme_init 3133=0A=
[  714.458501] nvme nvme0: pci function 0000:61:00.0=0A=
[  714.532560] nvme nvme0: 32/0/0 default/read/poll queues=0A=
[  721.110845] nvme_init 3133=0A=
[  721.114112] nvme nvme0: pci function 0000:61:00.0=0A=
[  721.189886] nvme nvme0: 32/0/0 default/read/poll queues=0A=
[  727.765572] nvme_init 3133=0A=
[  727.767814] nvme nvme0: pci function 0000:61:00.0=0A=
[  727.836938] nvme nvme0: 32/0/0 default/read/poll queues=0A=
[  734.519779] nvme_init 3133=0A=
[  734.522099] nvme nvme0: pci function 0000:61:00.0=0A=
[  734.589886] nvme nvme0: 32/0/0 default/read/poll queues=0A=
[  741.173503] nvme_init 3133=0A=
[  741.176089] nvme nvme0: pci function 0000:61:00.0=0A=
[  741.244296] nvme nvme0: 32/0/0 default/read/poll queues=0A=
=0A=
With this patch :-=0A=
=0A=
[  805.461924] nvme_init 3133=0A=
[  805.464521] nvme nvme0: pci function 0000:61:00.0=0A=
[  805.549656] nvme nvme0: 32/0/0 default/read/poll queues=0A=
[  812.124364] nvme_init 3133=0A=
[  812.126975] nvme nvme0: pci function 0000:61:00.0=0A=
[  812.199549] nvme nvme0: 32/0/0 default/read/poll queues=0A=
[  818.793779] nvme_init 3133=0A=
[  818.796581] nvme nvme0: pci function 0000:61:00.0=0A=
[  818.868111] nvme nvme0: 32/0/0 default/read/poll queues=0A=
[  825.554311] nvme_init 3133=0A=
[  825.556551] nvme nvme0: pci function 0000:61:00.0=0A=
[  825.636130] nvme nvme0: 32/0/0 default/read/poll queues=0A=
[  832.205882] nvme_init 3133=0A=
[  832.208934] nvme nvme0: pci function 0000:61:00.0=0A=
[  832.287043] nvme nvme0: 32/0/0 default/read/poll queues=0A=
=0A=
=0A=
On 02/28/2020 06:52 PM, Josh Triplett wrote:=0A=
> After initialization, nvme_wait_ready checks for readiness every 100ms,=
=0A=
> even though the drive may be ready far sooner than that. This delays=0A=
> system boot by hundreds of milliseconds. Reduce the delay, checking for=
=0A=
> readiness every millisecond instead.=0A=
>=0A=
> Boot-time tests on an AWS c5.12xlarge:=0A=
>=0A=
> Before:=0A=
> [    0.546936] initcall nvme_init+0x0/0x5b returned 0 after 37 usecs=0A=
> ...=0A=
> [    0.764178] nvme nvme0: 2/0/0 default/read/poll queues=0A=
> [    0.768424]  nvme0n1: p1=0A=
> [    0.774132] EXT4-fs (nvme0n1p1): mounted filesystem with ordered data =
mode. Opts: (null)=0A=
> [    0.774146] VFS: Mounted root (ext4 filesystem) on device 259:1.=0A=
> ...=0A=
> [    0.788141] Run /sbin/init as init process=0A=
>=0A=
> After:=0A=
> [    0.537088] initcall nvme_init+0x0/0x5b returned 0 after 37 usecs=0A=
> ...=0A=
> [    0.543457] nvme nvme0: 2/0/0 default/read/poll queues=0A=
> [    0.548473]  nvme0n1: p1=0A=
> [    0.554339] EXT4-fs (nvme0n1p1): mounted filesystem with ordered data =
mode. Opts: (null)=0A=
> [    0.554344] VFS: Mounted root (ext4 filesystem) on device 259:1.=0A=
> ...=0A=
> [    0.567931] Run /sbin/init as init process=0A=
>=0A=
> Signed-off-by: Josh Triplett <josh@joshtriplett.org>=0A=
> ---=0A=
>   drivers/nvme/host/core.c | 2 +-=0A=
>   1 file changed, 1 insertion(+), 1 deletion(-)=0A=
>=0A=
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c=0A=
> index a4d8c90ee7cc..04174a40b9b0 100644=0A=
> --- a/drivers/nvme/host/core.c=0A=
> +++ b/drivers/nvme/host/core.c=0A=
> @@ -2074,7 +2074,7 @@ static int nvme_wait_ready(struct nvme_ctrl *ctrl, =
u64 cap, bool enabled)=0A=
>   		if ((csts & NVME_CSTS_RDY) =3D=3D bit)=0A=
>   			break;=0A=
>=0A=
> -		msleep(100);=0A=
> +		usleep_range(1000, 2000);=0A=
>   		if (fatal_signal_pending(current))=0A=
>   			return -EINTR;=0A=
>   		if (time_after(jiffies, timeout)) {=0A=
>=0A=
=0A=