Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
IronPort-SDR: P4IOIpyGx9el8iX1nFq7xBrJXrtIJSJsnyXfPJNG+Dhtva4PpEIxTH59c9FhxQ5ZtWjuxI3gqf
 A6YsArfYYeR3dm0bqUtjzxxN9vJr908COeKY2Fg8o7/rgevZl0K7cTkfKpDN3HXgELHw595JDd
 x4GqBB9le2YuuyD8mlgGa9UfroeAm7C/SIMZtcheQqvoin95KnDuLRFGwgRwvSpGMpYIcVtclu
 NjV1yD60ivweXFpFOC4LpRk4/sY1us/z8Yy00To3j5TGcBBfIVBSQ3zoXt7wjtoAKN1TtTVXMB
 LmQ=
From:   Damien Le Moal <Damien.LeMoal@wdc.com>
To:     Kanchan Joshi <joshiiitr@gmail.com>
CC:     Christoph Hellwig <hch@lst.de>,
        Kanchan Joshi <joshi.k@samsung.com>,
        Jens Axboe <axboe@kernel.dk>,
        "sagi@grimberg.me" <sagi@grimberg.me>,
        Johannes Thumshirn <Johannes.Thumshirn@wdc.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-nvme@lists.infradead.org" <linux-nvme@lists.infradead.org>,
        Keith Busch <kbusch@kernel.org>,
        Selvakumar S <selvakuma.s1@samsung.com>,
        Javier Gonzalez <javier.gonz@samsung.com>,
        Nitesh Shetty <nj.shetty@samsung.com>
Subject: Re: [PATCH 1/2] nvme: set io-scheduler requirement for ZNS
Thread-Topic: [PATCH 1/2] nvme: set io-scheduler requirement for ZNS
Thread-Index: AQHWdSEIMmCxi6h3l0CfpdyAr+vLMA==
Date:   Wed, 19 Aug 2020 11:17:29 +0000
Message-ID: <CY4PR04MB3751B077D152A80B19236C20E75D0@CY4PR04MB3751.namprd04.prod.outlook.com>
References: <20200818052936.10995-1-joshi.k@samsung.com>
 <CGME20200818053252epcas5p4ee61d64bba5f6a131105e40330984f5e@epcas5p4.samsung.com>
 <20200818052936.10995-2-joshi.k@samsung.com> <20200818071141.GA2544@lst.de>
 <CA+1E3rJg0QOX4YgwWJf8Sm=6C-un4TsRX00E31fbFFPT9LYhXg@mail.gmail.com>
 <CY4PR04MB375148979AA403CF47EADE0BE75D0@CY4PR04MB3751.namprd04.prod.outlook.com>
 <CA+1E3r+C2KQENu=fO_+FZoUEvqZrAQcxziwSGt=FVidv85KQxA@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
wdcipoutbound: EOP-TRUE
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: 105a9491-3659-4dac-9636-08d844317647
X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Aug 2020 11:17:30.0115
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b61c8803-16f3-4c35-9b17-6f65f441df86
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 05sTsFfxoea8zZQ3g7SkRGx7BdHr5lCVm4iF5ZazoswAnp6zyevD31kzvNFGOe8+MpqQyz8IEFWps4M8EY2qpg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR04MB1175
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 2020/08/19 19:32, Kanchan Joshi wrote:=0A=
> On Wed, Aug 19, 2020 at 3:08 PM Damien Le Moal <Damien.LeMoal@wdc.com> wr=
ote:=0A=
>>=0A=
>> On 2020/08/19 18:27, Kanchan Joshi wrote:=0A=
>>> On Tue, Aug 18, 2020 at 12:46 PM Christoph Hellwig <hch@lst.de> wrote:=
=0A=
>>>>=0A=
>>>> On Tue, Aug 18, 2020 at 10:59:35AM +0530, Kanchan Joshi wrote:=0A=
>>>>> Set elevator feature ELEVATOR_F_ZBD_SEQ_WRITE required for ZNS.=0A=
>>>>=0A=
>>>> No, it is not.=0A=
>>>=0A=
>>> Are you saying MQ-Deadline (write-lock) is not needed for writes on ZNS=
?=0A=
>>> I see that null-block zoned and SCSI-ZBC both set this requirement. I=
=0A=
>>> wonder how it became different for NVMe.=0A=
>>=0A=
>> It is not required for an NVMe ZNS drive that has zone append native sup=
port.=0A=
>> zonefs and upcoming btrfs do not use regular writes, removing the requir=
ement=0A=
>> for zone write locking.=0A=
> =0A=
> I understand that if a particular user (zonefs, btrfs etc) is not=0A=
> sending regular-write and sending append instead, write-lock is not=0A=
> required.=0A=
> But if that particular user or some other user (say F2FS) sends=0A=
> regular write(s), write-lock is needed.=0A=
=0A=
And that can be trivially enabled by setting the drive elevator to mq-deadl=
ine.=0A=
=0A=
> Above block-layer, both the opcodes REQ_OP_WRITE and=0A=
> REQ_OP_ZONE_APPEND are available to be used by users. And I thought=0A=
> write-lock is taken or not is a per-opcode thing and not per-user (FS,=0A=
> MD/DM, user-space etc.), is not that correct? And MQ-deadline can=0A=
> cater to both the opcodes, while other schedulers cannot serve=0A=
> REQ_OP_WRITE well for zoned-device.=0A=
=0A=
mq-deadline ignores zone append commands. No zone lock is taken for these. =
In=0A=
scsi, the emulation takes the zone lock before transforming the zone append=
 into=0A=
a regular write. That locking is consistent with the mq-scheduler level loc=
king=0A=
since the same lock bitmap is used. So if the user only issues zone append=
=0A=
writes, mq-deadline is not needed and there is no reasons to force its use =
by=0A=
setting ELEVATOR_F_ZBD_SEQ_WRITE. E.g. the user may want to use kyber...=0A=
=0A=
>> In the context of your patch series, ELEVATOR_F_ZBD_SEQ_WRITE should be =
set only=0A=
>> and only if the drive does not have native zone append support.=0A=
> =0A=
> Sure I can keep it that way, once I get it right. If it is really not=0A=
> required for native-append drive, it should not be here at the place=0A=
> where I added.=0A=
> =0A=
>> And even in that=0A=
>> case, since for an emulated zone append the zone write lock is taken and=
=0A=
>> released by the emulation driver itself, ELEVATOR_F_ZBD_SEQ_WRITE is req=
uired=0A=
>> only if the user will also be issuing regular writes at high QD. And tha=
t is=0A=
>> trivially controllable by the user by simply setting the drive elevator =
to=0A=
>> mq-deadline. Conclusion: setting ELEVATOR_F_ZBD_SEQ_WRITE is not needed.=
=0A=
> =0A=
> Are we saying applications should switch schedulers based on the write=0A=
> QD  (use any-scheduler for QD1 and mq-deadline for QD-N).=0A=
> Even if it does that, it does not know what other applications would=0A=
> be doing. That seems hard-to-get-right and possible only in a=0A=
> tightly-controlled environment.=0A=
=0A=
Even for SMR, the user is free to set the elevator to none, which disables =
zone=0A=
write locking. Issuing writes correctly then becomes the responsibility of =
the=0A=
application. This can be useful for settings that for instance use NCQ I/O=
=0A=
priorities, which give better results when "none" is used.=0A=
=0A=
As far as I know, zoned drives are always used in tightly controlled=0A=
environments. Problems like "does not know what other applications would be=
=0A=
doing" are non-existent. Setting up the drive correctly for the use case at=
 hand=0A=
is a sysadmin/server setup problem, based on *the* application (singular)=
=0A=
requirements.=0A=
=0A=
=0A=
-- =0A=
Damien Le Moal=0A=
Western Digital Research=0A=