Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
IronPort-SDR: 0PXqT+G6rHw3hAEexG2swKVXHmT5xxz3+4i+y/wTGoHqjOdEPEeURylKCoAyx/UA9uiUMshYe6
 J51mWLi7cPfrN09ObD1E4w9d6sJxOYYXSJZOMS5RreSQud+7xj8NL/3BmjuPLYNkFEIXWz90DM
 Va3RpMoPHYTZ0icDpGR4iHw9K29hMjKGW7Dznq7CPI8jjIEtaplkgDzjCvFxPXbuZ8NXhGXzb5
 jQWvHq9oIfrbQ/n0BHT303cXfH2toUUPz2G7w1zupDi3rTlOCMM2QcdqaSYGtmMsEW1sFcOvS8
 yEg=
From:   Damien Le Moal <Damien.LeMoal@wdc.com>
To:     Kanchan Joshi <joshiiitr@gmail.com>
CC:     Christoph Hellwig <hch@lst.de>,
        Kanchan Joshi <joshi.k@samsung.com>,
        Jens Axboe <axboe@kernel.dk>,
        "sagi@grimberg.me" <sagi@grimberg.me>,
        Johannes Thumshirn <Johannes.Thumshirn@wdc.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-nvme@lists.infradead.org" <linux-nvme@lists.infradead.org>,
        Keith Busch <kbusch@kernel.org>,
        Selvakumar S <selvakuma.s1@samsung.com>,
        Javier Gonzalez <javier.gonz@samsung.com>,
        Nitesh Shetty <nj.shetty@samsung.com>
Subject: Re: [PATCH 1/2] nvme: set io-scheduler requirement for ZNS
Thread-Topic: [PATCH 1/2] nvme: set io-scheduler requirement for ZNS
Thread-Index: AQHWdSEIMmCxi6h3l0CfpdyAr+vLMA==
Date:   Mon, 7 Sep 2020 12:53:20 +0000
Message-ID: <CY4PR04MB375105E77F87B60E74E025BAE7280@CY4PR04MB3751.namprd04.prod.outlook.com>
References: <20200818052936.10995-1-joshi.k@samsung.com>
 <CGME20200818053252epcas5p4ee61d64bba5f6a131105e40330984f5e@epcas5p4.samsung.com>
 <20200818052936.10995-2-joshi.k@samsung.com> <20200818071141.GA2544@lst.de>
 <CA+1E3rJg0QOX4YgwWJf8Sm=6C-un4TsRX00E31fbFFPT9LYhXg@mail.gmail.com>
 <CY4PR04MB375148979AA403CF47EADE0BE75D0@CY4PR04MB3751.namprd04.prod.outlook.com>
 <CA+1E3r+C2KQENu=fO_+FZoUEvqZrAQcxziwSGt=FVidv85KQxA@mail.gmail.com>
 <CY4PR04MB3751B077D152A80B19236C20E75D0@CY4PR04MB3751.namprd04.prod.outlook.com>
 <CA+1E3r+0+jThD2KEu-d4rB=C4xL3Bb+cD=jUDbsGGBKR_GMpfQ@mail.gmail.com>
 <CY4PR04MB375151ABDF83FE0A9804C3EDE7280@CY4PR04MB3751.namprd04.prod.outlook.com>
 <CA+1E3r+sN8n6-ibJ1v_S5HEdfhbJ8yAYTm5ja=0M8ng2ujLZ4g@mail.gmail.com>
 <CY4PR04MB37516303D8C5FC32FB530FDBE7280@CY4PR04MB3751.namprd04.prod.outlook.com>
 <CA+1E3rJtGTt6gss-5uAjzQ+BAXWTOTcjUzyFToz-QWbEfkLkaA@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
wdcipoutbound: EOP-TRUE
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: cd12864b-20b7-4dce-e164-08d8532cffbc
X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Sep 2020 12:53:20.5930
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b61c8803-16f3-4c35-9b17-6f65f441df86
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: gb7ROj4DaiNcPiH76zqubvTKA0CAcVrsvqjD7V6/YX3SqoYaoK5LS12k1Xkhyvyq1F4UPw+/pw70MVVhYtvfug==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1PR04MB2314
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 2020/09/07 20:54, Kanchan Joshi wrote:=0A=
> On Mon, Sep 7, 2020 at 5:07 PM Damien Le Moal <Damien.LeMoal@wdc.com> wro=
te:=0A=
>>=0A=
>> On 2020/09/07 20:24, Kanchan Joshi wrote:=0A=
>>> On Mon, Sep 7, 2020 at 1:52 PM Damien Le Moal <Damien.LeMoal@wdc.com> w=
rote:=0A=
>>>>=0A=
>>>> On 2020/09/07 16:01, Kanchan Joshi wrote:=0A=
>>>>>> Even for SMR, the user is free to set the elevator to none, which di=
sables zone=0A=
>>>>>> write locking. Issuing writes correctly then becomes the responsibil=
ity of the=0A=
>>>>>> application. This can be useful for settings that for instance use N=
CQ I/O=0A=
>>>>>> priorities, which give better results when "none" is used.=0A=
>>>>>=0A=
>>>>> Was it not a problem that even if the application is sending writes=
=0A=
>>>>> correctly, scheduler may not preserve the order.=0A=
>>>>> And even when none is being used, re-queue can happen which may lead=
=0A=
>>>>> to different ordering.=0A=
>>>>=0A=
>>>> "Issuing writes correctly" means doing small writes, one per zone at m=
ost. In=0A=
>>>> that case, it does not matter if the block layer reorders writes. Per =
zone, they=0A=
>>>> will still be sequential.=0A=
>>>>=0A=
>>>>>> As far as I know, zoned drives are always used in tightly controlled=
=0A=
>>>>>> environments. Problems like "does not know what other applications w=
ould be=0A=
>>>>>> doing" are non-existent. Setting up the drive correctly for the use =
case at hand=0A=
>>>>>> is a sysadmin/server setup problem, based on *the* application (sing=
ular)=0A=
>>>>>> requirements.=0A=
>>>>>=0A=
>>>>> Fine.=0A=
>>>>> But what about the null-block-zone which sets MQ-deadline but does no=
t=0A=
>>>>> actually use write-lock to avoid race among multiple appends on a=0A=
>>>>> zone.=0A=
>>>>> Does that deserve a fix?=0A=
>>>>=0A=
>>>> In nullblk, commands are executed under a spinlock. So there is no con=
currency=0A=
>>>> problem. The spinlock serializes the execution of all commands. null_b=
lk zone=0A=
>>>> append emulation thus does not need to take the scheduler level zone w=
rite lock=0A=
>>>> like scsi does.=0A=
>>>=0A=
>>> I do not see spinlock for that. There is one "nullb->lock", but its=0A=
>>> scope is limited to memory-backed handling.=0A=
>>> For concurrent zone-appends on a zone, multiple threads may set the=0A=
>>> "same" write-pointer into incoming request(s).=0A=
>>> Are you referring to any other spinlock that can avoid "same wp being=
=0A=
>>> returned to multiple threads".=0A=
>>=0A=
>> Checking again, it looks like you are correct. nullb->lock is indeed onl=
y used=0A=
>> for processing read/write with memory backing turned on.=0A=
>> We either need to extend that spinlock use, or add one to protect the zo=
ne array=0A=
>> when doing zoned commands and checks of read/write against a zone wp.=0A=
>> Care to send a patch ? I can send one too.=0A=
> =0A=
> Sure, I can send.=0A=
> Do you think it is not OK to use zone write-lock (same like SCSI=0A=
> emulation) instead of introducing a new spinlock?=0A=
=0A=
zone write lock will not protect against read or zone management commands=
=0A=
executed concurrently with writes. Only concurrent writes to the same zone =
will=0A=
be serialized with the scheduler zone write locking, which may not be used =
at=0A=
all also if the user set the scheduler to none. A lock for exclusive access=
 and=0A=
changes to the zone array is needed.=0A=
=0A=
=0A=
> =0A=
> =0A=
=0A=
=0A=
-- =0A=
Damien Le Moal=0A=
Western Digital Research=0A=