Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
IronPort-SDR: +e7ithggeBRGBH45fYVTdDOswKGjeCoG1WFJ2AjD7phOZ/JyR20n0UyV3gCYhCs0qbaPiMycf3
 dpnDhKUFD9tXOl7jvRL9DIH1GPZRFjpkr5LFJR4bQhtliNEc16oO25YPjcF+sl2GCWGZVJ2MXb
 tZDM4xymnMRXx0hDWalrItn2liuXssaeaz7HJtDaUOvQcYk+b6R23/flsrwVAL0FNzJPplqBhw
 4FtwvmMk0AovW2gMT6TXLJKBTBVgO6RIM4TSUMwn7jXlDvIBeGg7C6OLkHI80oPZKKzDgvNbi+
 Ljo=
From:   Avri Altman <Avri.Altman@wdc.com>
To:     Can Guo <cang@codeaurora.org>
CC:     "James E . J . Bottomley" <jejb@linux.vnet.ibm.com>,
        "Martin K . Petersen" <martin.petersen@oracle.com>,
        "linux-scsi@vger.kernel.org" <linux-scsi@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
        Bart Van Assche <bvanassche@acm.org>,
        yongmyung lee <ymhungry.lee@samsung.com>,
        Daejun Park <daejun7.park@samsung.com>,
        "alim.akhtar@samsung.com" <alim.akhtar@samsung.com>,
        "asutoshd@codeaurora.org" <asutoshd@codeaurora.org>,
        Zang Leigang <zangleigang@hisilicon.com>,
        Avi Shchislowski <Avi.Shchislowski@wdc.com>,
        Bean Huo <beanhuo@micron.com>,
        "stanley.chu@mediatek.com" <stanley.chu@mediatek.com>
Subject: RE: [PATCH v5 06/10] scsi: ufshpb: Add hpb dev reset response
Thread-Topic: [PATCH v5 06/10] scsi: ufshpb: Add hpb dev reset response
Thread-Index: AQHXD2ex1y1f5g2rP02e5lIjKmzR26qIGY0AgAAGPJCAAA7hAIAAAEIAgAAbY4CAAAQzAIAACHQAgAAQ5TA=
Date:   Wed, 17 Mar 2021 15:46:52 +0000
Message-ID: <DM6PR04MB65757E84F80B654689FBAE05FC6A9@DM6PR04MB6575.namprd04.prod.outlook.com>
References: <20210302132503.224670-1-avri.altman@wdc.com>
 <20210302132503.224670-7-avri.altman@wdc.com>
 <59a62fc17ec9229a8498e696eb0474be@codeaurora.org>
 <DM6PR04MB6575006E0682C3D11F54965DFC6A9@DM6PR04MB6575.namprd04.prod.outlook.com>
 <1d0e3c5441ecf14b6614ec0af0d30af6@codeaurora.org>
 <DM6PR04MB65750C0AE1F1EDB41EDEE491FC6A9@DM6PR04MB6575.namprd04.prod.outlook.com>
 <37d0a4f115ad5d08ab12a76e6cbe17a5@codeaurora.org>
 <DM6PR04MB65755C69AD3D64BC5B1E93D8FC6A9@DM6PR04MB6575.namprd04.prod.outlook.com>
 <4562e78aee9c5fbb7bbff65930fc81cd@codeaurora.org>
In-Reply-To: <4562e78aee9c5fbb7bbff65930fc81cd@codeaurora.org>
Accept-Language: en-US
Content-Language: en-US
wdcipoutbound: EOP-TRUE
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: dedb5aec-07c6-481d-2d01-08d8e95be2b1
X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Mar 2021 15:46:52.6029
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b61c8803-16f3-4c35-9b17-6f65f441df86
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 71oC0RfV2+JRzfI3LhRQ1En/xYF947yrfWkeXOByEpcSMyNEDAoTSal6x7y3obYZ08s1t5P1ET0CAKnfhSYumg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR04MB7116
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

> >> >> >>
> >> >> >> Just curious, directly doing below things inside ufshpb_rsp_upiu=
()
> >> >> >> does
> >> >> >> not
> >> >> >> seem a problem to me, does this really deserve a separate work?
> >> >> > I don't know, I never even consider of doing this.
> >> >> > The active region list may contain up to few thousands of regions=
 -
> >> >> > It is not rare to see configurations that covers the entire devic=
e.
> >> >> >
> >> >>
> >> >> Yes, true, it can be a huge list. But what does the ops
> >> >> "HPB_RSP_DEV_RESET"
> >> >> really mean? The specs says "Device reset HPB Regions information",
> >> >> but
> >> >> I
> >> >> don't know what is really happening. Could you please elaborate?
> >> > It means that the device informs the host that the L2P cache is no
> >> > longer valid.
> >> > The spec doesn't say what to do in that case.
> >>
> >> Then it means that all the clean (without DIRTY flag set) HPB entries
> >> (ppns)
> >> in active rgns in host memory side may not be valid to the device
> >> anymore.
> >> Please correct me if I am wrong.
> >>
> >> > We thought that in host mode, it make sense to update all the active
> >> > regions.
> >>
> >> But current logic does not set the state of the sub-regions (in active
> >> regions) to
> >> INVALID, it only marks all active regions as UPDATE.
> >>
> >> Although one of subsequent read cmds shall put the sub-region back to
> >> activate_list,
> >> ufshpb_test_ppn_dirty() can still return false, thus these read cmds
> >> still think the
> >> ppns are valid and they shall move forward to send HPB Write Buffer
> >> (buffer id =3D 0x2,
> >> in case of HPB2.0) and HPB Read cmds.
> >>
> >> HPB Read cmds with invalid ppns will be treated as normal Read cmds by
> >> device as the
> >> specs says, but what would happen to HPB Write Buffer cmds (buffer id
> >> =3D
> >> 0x2, in case
> >> of HPB2.0) with invalid ppns? Can this be a real problem?
> > No need to control the ppn dirty / invalid state for this case.
> > The device send device reset so it is aware that all the L2P cache is
> > invalid.
> > Any HPB_READ is treated like normal READ10.
> >
> > Only once HPB-READ-BUFFER is completed,
> > the device will relate back to the physical address.
>=20
> What about HPB-WRITE-BUFFER (buffer id =3D 0x2) cmds?
Same.
Oper 0x2 is a relative simple case.
The device is expected to manage some versioning framework not to be "foole=
d" by erroneous ppn.
There are some more challenging races that the device should meet.

Thanks,
Avri
>=20
> Thanks,
> Can Guo.
>=20
> >
> >>
> >> >
> >> > I think I will go with your suggestion.
> >> > Effectively, in host mode, since it is deactivating "cold" regions,
> >> > the lru list is kept relatively small, and contains only "hot" regio=
ns.
> >>
> >> hmm... I don't really have a idea on this, please go with whatever you
> >> and Daejun think is fine here.
> > I will take your advice and remove the worker.
> >
> >
> > Thanks,
> > Avri
> >
> >>
> >> Thanks,
> >> Can Guo.
> >>
> >> >
> >> > Thanks,
> >> > Avri
> >> >
> >> >>
> >> >> Thanks,
> >> >> Can Guo.
> >> >>
> >> >> > But yes, I can do that.
> >> >> > Better to get ack from Daejun first.
> >> >> >
> >> >> > Thanks,
> >> >> > Avri
> >> >> >
> >> >> >>
> >> >> >> Thanks,
> >> >> >> Can Guo.
> >> >> >>
> >> >> >> > +{
> >> >> >> > +     struct ufshpb_lu *hpb;
> >> >> >> > +     struct victim_select_info *lru_info;
> >> >> >> > +     struct ufshpb_region *rgn;
> >> >> >> > +     unsigned long flags;
> >> >> >> > +
> >> >> >> > +     hpb =3D container_of(work, struct ufshpb_lu,
> >> ufshpb_lun_reset_work);
> >> >> >> > +
> >> >> >> > +     lru_info =3D &hpb->lru_info;
> >> >> >> > +
> >> >> >> > +     spin_lock_irqsave(&hpb->rgn_state_lock, flags);
> >> >> >> > +
> >> >> >> > +     list_for_each_entry(rgn, &lru_info->lh_lru_rgn, list_lru=
_rgn)
> >> >> >> > +             set_bit(RGN_FLAG_UPDATE, &rgn->rgn_flags);
> >> >> >> > +
> >> >> >> > +     spin_unlock_irqrestore(&hpb->rgn_state_lock, flags);
> >> >> >> > +}
> >> >> >> > +
> >> >> >> >  static void ufshpb_normalization_work_handler(struct work_str=
uct
> >> >> >> > *work)
> >> >> >> >  {
> >> >> >> >       struct ufshpb_lu *hpb;
> >> >> >> > @@ -1798,6 +1832,8 @@ static int ufshpb_alloc_region_tbl(struc=
t
> >> >> >> > ufs_hba *hba, struct ufshpb_lu *hpb)
> >> >> >> >               } else {
> >> >> >> >                       rgn->rgn_state =3D HPB_RGN_INACTIVE;
> >> >> >> >               }
> >> >> >> > +
> >> >> >> > +             rgn->rgn_flags =3D 0;
> >> >> >> >       }
> >> >> >> >
> >> >> >> >       return 0;
> >> >> >> > @@ -2012,9 +2048,12 @@ static int ufshpb_lu_hpb_init(struct
> >> ufs_hba
> >> >> >> > *hba, struct ufshpb_lu *hpb)
> >> >> >> >       INIT_LIST_HEAD(&hpb->list_hpb_lu);
> >> >> >> >
> >> >> >> >       INIT_WORK(&hpb->map_work, ufshpb_map_work_handler);
> >> >> >> > -     if (hpb->is_hcm)
> >> >> >> > +     if (hpb->is_hcm) {
> >> >> >> >               INIT_WORK(&hpb->ufshpb_normalization_work,
> >> >> >> >                         ufshpb_normalization_work_handler);
> >> >> >> > +             INIT_WORK(&hpb->ufshpb_lun_reset_work,
> >> >> >> > +                       ufshpb_reset_work_handler);
> >> >> >> > +     }
> >> >> >> >
> >> >> >> >       hpb->map_req_cache =3D
> kmem_cache_create("ufshpb_req_cache",
> >> >> >> >                         sizeof(struct ufshpb_req), 0, 0, NULL)=
;
> >> >> >> > @@ -2114,8 +2153,10 @@ static void
> ufshpb_discard_rsp_lists(struct
> >> >> >> > ufshpb_lu *hpb)
> >> >> >> >
> >> >> >> >  static void ufshpb_cancel_jobs(struct ufshpb_lu *hpb)
> >> >> >> >  {
> >> >> >> > -     if (hpb->is_hcm)
> >> >> >> > +     if (hpb->is_hcm) {
> >> >> >> > +             cancel_work_sync(&hpb->ufshpb_lun_reset_work);
> >> >> >> >               cancel_work_sync(&hpb->ufshpb_normalization_work=
);
> >> >> >> > +     }
> >> >> >> >       cancel_work_sync(&hpb->map_work);
> >> >> >> >  }
> >> >> >> >
> >> >> >> > diff --git a/drivers/scsi/ufs/ufshpb.h b/drivers/scsi/ufs/ufsh=
pb.h
> >> >> >> > index 84598a317897..37c1b0ea0c0a 100644
> >> >> >> > --- a/drivers/scsi/ufs/ufshpb.h
> >> >> >> > +++ b/drivers/scsi/ufs/ufshpb.h
> >> >> >> > @@ -121,6 +121,7 @@ struct ufshpb_region {
> >> >> >> >       struct list_head list_lru_rgn;
> >> >> >> >       unsigned long rgn_flags;
> >> >> >> >  #define RGN_FLAG_DIRTY 0
> >> >> >> > +#define RGN_FLAG_UPDATE 1
> >> >> >> >
> >> >> >> >       /* region reads - for host mode */
> >> >> >> >       spinlock_t rgn_lock;
> >> >> >> > @@ -217,6 +218,7 @@ struct ufshpb_lu {
> >> >> >> >       /* for selecting victim */
> >> >> >> >       struct victim_select_info lru_info;
> >> >> >> >       struct work_struct ufshpb_normalization_work;
> >> >> >> > +     struct work_struct ufshpb_lun_reset_work;
> >> >> >> >
> >> >> >> >       /* pinned region information */
> >> >> >> >       u32 lu_pinned_start;