2020-12-18 03:34:21

by Jaegeuk Kim

[permalink] [raw]
Subject: [PATCH] scsi: ufs: fix livelock of ufshcd_clear_ua_wluns

When gate_work/ungate_work gets an error during hibern8_enter or exit,
ufshcd_err_handler()
ufshcd_scsi_block_requests()
ufshcd_reset_and_restore()
ufshcd_clear_ua_wluns() -> stuck
ufshcd_scsi_unblock_requests()

In order to avoid it, ufshcd_clear_ua_wluns() can be called per recovery flows
such as suspend/resume, link_recovery, and error_handler.

Signed-off-by: Jaegeuk Kim <[email protected]>
---
drivers/scsi/ufs/ufshcd.c | 14 +++++++++-----
1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/drivers/scsi/ufs/ufshcd.c b/drivers/scsi/ufs/ufshcd.c
index e221add25a7e..e711def829cd 100644
--- a/drivers/scsi/ufs/ufshcd.c
+++ b/drivers/scsi/ufs/ufshcd.c
@@ -3963,6 +3963,8 @@ int ufshcd_link_recovery(struct ufs_hba *hba)
if (ret)
dev_err(hba->dev, "%s: link recovery failed, err %d",
__func__, ret);
+ else
+ ufshcd_clear_ua_wluns(hba);

return ret;
}
@@ -5968,6 +5970,8 @@ static void ufshcd_err_handler(struct work_struct *work)
ufshcd_scsi_unblock_requests(hba);
ufshcd_err_handling_unprepare(hba);
up(&hba->eh_sem);
+
+ ufshcd_clear_ua_wluns(hba);
}

/**
@@ -6908,14 +6912,11 @@ static int ufshcd_host_reset_and_restore(struct ufs_hba *hba)
ufshcd_set_clk_freq(hba, true);

err = ufshcd_hba_enable(hba);
- if (err)
- goto out;

/* Establish the link again and restore the device */
- err = ufshcd_probe_hba(hba, false);
if (!err)
- ufshcd_clear_ua_wluns(hba);
-out:
+ err = ufshcd_probe_hba(hba, false);
+
if (err)
dev_err(hba->dev, "%s: Host init failed %d\n", __func__, err);
ufshcd_update_evt_hist(hba, UFS_EVT_HOST_RESET, (u32)err);
@@ -8745,6 +8746,7 @@ static int ufshcd_suspend(struct ufs_hba *hba, enum ufs_pm_op pm_op)
ufshcd_resume_clkscaling(hba);
hba->clk_gating.is_suspended = false;
hba->dev_info.b_rpm_dev_flush_capable = false;
+ ufshcd_clear_ua_wluns(hba);
ufshcd_release(hba);
out:
if (hba->dev_info.b_rpm_dev_flush_capable) {
@@ -8855,6 +8857,8 @@ static int ufshcd_resume(struct ufs_hba *hba, enum ufs_pm_op pm_op)
cancel_delayed_work(&hba->rpm_dev_flush_recheck_work);
}

+ ufshcd_clear_ua_wluns(hba);
+
/* Schedule clock gating in case of no access to UFS device yet */
ufshcd_release(hba);

--
2.29.2.729.g45daf8777d-goog


2020-12-20 22:05:38

by Avri Altman

[permalink] [raw]
Subject: RE: [PATCH] scsi: ufs: fix livelock of ufshcd_clear_ua_wluns

Hi J,

>
> When gate_work/ungate_work gets an error during hibern8_enter or exit,
> ufshcd_err_handler()
> ufshcd_scsi_block_requests()
> ufshcd_reset_and_restore()
> ufshcd_clear_ua_wluns() -> stuck
> ufshcd_scsi_unblock_requests()
>
> In order to avoid it, ufshcd_clear_ua_wluns() can be called per recovery
> flows
> such as suspend/resume, link_recovery, and error_handler.
Not sure that suspend/resume are UAC events?

Also the 'fixes' tag is missing.

Thanks,
Avri

2020-12-21 04:28:03

by Can Guo

[permalink] [raw]
Subject: Re: [PATCH] scsi: ufs: fix livelock of ufshcd_clear_ua_wluns

On 2020-12-18 11:31, Jaegeuk Kim wrote:
> When gate_work/ungate_work gets an error during hibern8_enter or exit,
> ufshcd_err_handler()
> ufshcd_scsi_block_requests()
> ufshcd_reset_and_restore()
> ufshcd_clear_ua_wluns() -> stuck
> ufshcd_scsi_unblock_requests()
>
> In order to avoid it, ufshcd_clear_ua_wluns() can be called per
> recovery flows
> such as suspend/resume, link_recovery, and error_handler.
>
> Signed-off-by: Jaegeuk Kim <[email protected]>
> ---
> drivers/scsi/ufs/ufshcd.c | 14 +++++++++-----
> 1 file changed, 9 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/scsi/ufs/ufshcd.c b/drivers/scsi/ufs/ufshcd.c
> index e221add25a7e..e711def829cd 100644
> --- a/drivers/scsi/ufs/ufshcd.c
> +++ b/drivers/scsi/ufs/ufshcd.c
> @@ -3963,6 +3963,8 @@ int ufshcd_link_recovery(struct ufs_hba *hba)
> if (ret)
> dev_err(hba->dev, "%s: link recovery failed, err %d",
> __func__, ret);
> + else
> + ufshcd_clear_ua_wluns(hba);
>
> return ret;
> }
> @@ -5968,6 +5970,8 @@ static void ufshcd_err_handler(struct work_struct
> *work)
> ufshcd_scsi_unblock_requests(hba);
> ufshcd_err_handling_unprepare(hba);
> up(&hba->eh_sem);
> +

Maybe add a check like if (!err && needs_reset) as error handler
also handles non-fatal errors which do not require a full reset
and restore?

> + ufshcd_clear_ua_wluns(hba);
> }
>
> /**
> @@ -6908,14 +6912,11 @@ static int
> ufshcd_host_reset_and_restore(struct ufs_hba *hba)
> ufshcd_set_clk_freq(hba, true);
>
> err = ufshcd_hba_enable(hba);
> - if (err)
> - goto out;
>
> /* Establish the link again and restore the device */
> - err = ufshcd_probe_hba(hba, false);
> if (!err)
> - ufshcd_clear_ua_wluns(hba);
> -out:
> + err = ufshcd_probe_hba(hba, false);
> +
> if (err)
> dev_err(hba->dev, "%s: Host init failed %d\n", __func__, err);
> ufshcd_update_evt_hist(hba, UFS_EVT_HOST_RESET, (u32)err);
> @@ -8745,6 +8746,7 @@ static int ufshcd_suspend(struct ufs_hba *hba,
> enum ufs_pm_op pm_op)
> ufshcd_resume_clkscaling(hba);
> hba->clk_gating.is_suspended = false;
> hba->dev_info.b_rpm_dev_flush_capable = false;
> + ufshcd_clear_ua_wluns(hba);
> ufshcd_release(hba);
> out:
> if (hba->dev_info.b_rpm_dev_flush_capable) {
> @@ -8855,6 +8857,8 @@ static int ufshcd_resume(struct ufs_hba *hba,
> enum ufs_pm_op pm_op)
> cancel_delayed_work(&hba->rpm_dev_flush_recheck_work);
> }
>
> + ufshcd_clear_ua_wluns(hba);
> +
> /* Schedule clock gating in case of no access to UFS device yet */
> ufshcd_release(hba);

2020-12-21 15:18:11

by Jaegeuk Kim

[permalink] [raw]
Subject: Re: [PATCH] scsi: ufs: fix livelock of ufshcd_clear_ua_wluns

On 12/21, Can Guo wrote:
> On 2020-12-18 11:31, Jaegeuk Kim wrote:
> > When gate_work/ungate_work gets an error during hibern8_enter or exit,
> > ufshcd_err_handler()
> > ufshcd_scsi_block_requests()
> > ufshcd_reset_and_restore()
> > ufshcd_clear_ua_wluns() -> stuck
> > ufshcd_scsi_unblock_requests()
> >
> > In order to avoid it, ufshcd_clear_ua_wluns() can be called per recovery
> > flows
> > such as suspend/resume, link_recovery, and error_handler.
> >
> > Signed-off-by: Jaegeuk Kim <[email protected]>
> > ---
> > drivers/scsi/ufs/ufshcd.c | 14 +++++++++-----
> > 1 file changed, 9 insertions(+), 5 deletions(-)
> >
> > diff --git a/drivers/scsi/ufs/ufshcd.c b/drivers/scsi/ufs/ufshcd.c
> > index e221add25a7e..e711def829cd 100644
> > --- a/drivers/scsi/ufs/ufshcd.c
> > +++ b/drivers/scsi/ufs/ufshcd.c
> > @@ -3963,6 +3963,8 @@ int ufshcd_link_recovery(struct ufs_hba *hba)
> > if (ret)
> > dev_err(hba->dev, "%s: link recovery failed, err %d",
> > __func__, ret);
> > + else
> > + ufshcd_clear_ua_wluns(hba);
> >
> > return ret;
> > }
> > @@ -5968,6 +5970,8 @@ static void ufshcd_err_handler(struct work_struct
> > *work)
> > ufshcd_scsi_unblock_requests(hba);
> > ufshcd_err_handling_unprepare(hba);
> > up(&hba->eh_sem);
> > +
>
> Maybe add a check like if (!err && needs_reset) as error handler
> also handles non-fatal errors which do not require a full reset
> and restore?

I see. Let me add it in v2.

>
> > + ufshcd_clear_ua_wluns(hba);
> > }
> >
> > /**
> > @@ -6908,14 +6912,11 @@ static int
> > ufshcd_host_reset_and_restore(struct ufs_hba *hba)
> > ufshcd_set_clk_freq(hba, true);
> >
> > err = ufshcd_hba_enable(hba);
> > - if (err)
> > - goto out;
> >
> > /* Establish the link again and restore the device */
> > - err = ufshcd_probe_hba(hba, false);
> > if (!err)
> > - ufshcd_clear_ua_wluns(hba);
> > -out:
> > + err = ufshcd_probe_hba(hba, false);
> > +
> > if (err)
> > dev_err(hba->dev, "%s: Host init failed %d\n", __func__, err);
> > ufshcd_update_evt_hist(hba, UFS_EVT_HOST_RESET, (u32)err);
> > @@ -8745,6 +8746,7 @@ static int ufshcd_suspend(struct ufs_hba *hba,
> > enum ufs_pm_op pm_op)
> > ufshcd_resume_clkscaling(hba);
> > hba->clk_gating.is_suspended = false;
> > hba->dev_info.b_rpm_dev_flush_capable = false;
> > + ufshcd_clear_ua_wluns(hba);
> > ufshcd_release(hba);
> > out:
> > if (hba->dev_info.b_rpm_dev_flush_capable) {
> > @@ -8855,6 +8857,8 @@ static int ufshcd_resume(struct ufs_hba *hba,
> > enum ufs_pm_op pm_op)
> > cancel_delayed_work(&hba->rpm_dev_flush_recheck_work);
> > }
> >
> > + ufshcd_clear_ua_wluns(hba);
> > +
> > /* Schedule clock gating in case of no access to UFS device yet */
> > ufshcd_release(hba);

2020-12-21 15:26:04

by Jaegeuk Kim

[permalink] [raw]
Subject: Re: [PATCH] scsi: ufs: fix livelock of ufshcd_clear_ua_wluns

Hi,

On 12/20, Avri Altman wrote:
> Hi J,
>
> >
> > When gate_work/ungate_work gets an error during hibern8_enter or exit,
> > ufshcd_err_handler()
> > ufshcd_scsi_block_requests()
> > ufshcd_reset_and_restore()
> > ufshcd_clear_ua_wluns() -> stuck
> > ufshcd_scsi_unblock_requests()
> >
> > In order to avoid it, ufshcd_clear_ua_wluns() can be called per recovery
> > flows
> > such as suspend/resume, link_recovery, and error_handler.
> Not sure that suspend/resume are UAC events?

Could you elaborate a bit? The goal is to clear UAC after UFS reset happens.

>
> Also the 'fixes' tag is missing.

Added. Thanks,

>
> Thanks,
> Avri

2020-12-21 15:30:42

by Jaegeuk Kim

[permalink] [raw]
Subject: Re: [PATCH v2] scsi: ufs: fix livelock of ufshcd_clear_ua_wluns

When gate_work/ungate_work gets an error during hibern8_enter or exit,
ufshcd_err_handler()
ufshcd_scsi_block_requests()
ufshcd_reset_and_restore()
ufshcd_clear_ua_wluns() -> stuck
ufshcd_scsi_unblock_requests()

In order to avoid it, ufshcd_clear_ua_wluns() can be called per recovery flows
such as suspend/resume, link_recovery, and error_handler.

Fixes: 1918651f2d7e ("scsi: ufs: Clear UAC for RPMB after ufshcd resets")
Signed-off-by: Jaegeuk Kim <[email protected]>
---

Change log from v1:
- add condition check to call
- add Fixes tag

drivers/scsi/ufs/ufshcd.c | 15 ++++++++++-----
1 file changed, 10 insertions(+), 5 deletions(-)

diff --git a/drivers/scsi/ufs/ufshcd.c b/drivers/scsi/ufs/ufshcd.c
index e221add25a7e..29a62552f6f1 100644
--- a/drivers/scsi/ufs/ufshcd.c
+++ b/drivers/scsi/ufs/ufshcd.c
@@ -3963,6 +3963,8 @@ int ufshcd_link_recovery(struct ufs_hba *hba)
if (ret)
dev_err(hba->dev, "%s: link recovery failed, err %d",
__func__, ret);
+ else
+ ufshcd_clear_ua_wluns(hba);

return ret;
}
@@ -5968,6 +5970,9 @@ static void ufshcd_err_handler(struct work_struct *work)
ufshcd_scsi_unblock_requests(hba);
ufshcd_err_handling_unprepare(hba);
up(&hba->eh_sem);
+
+ if (!err && needs_reset)
+ ufshcd_clear_ua_wluns(hba);
}

/**
@@ -6908,14 +6913,11 @@ static int ufshcd_host_reset_and_restore(struct ufs_hba *hba)
ufshcd_set_clk_freq(hba, true);

err = ufshcd_hba_enable(hba);
- if (err)
- goto out;

/* Establish the link again and restore the device */
- err = ufshcd_probe_hba(hba, false);
if (!err)
- ufshcd_clear_ua_wluns(hba);
-out:
+ err = ufshcd_probe_hba(hba, false);
+
if (err)
dev_err(hba->dev, "%s: Host init failed %d\n", __func__, err);
ufshcd_update_evt_hist(hba, UFS_EVT_HOST_RESET, (u32)err);
@@ -8745,6 +8747,7 @@ static int ufshcd_suspend(struct ufs_hba *hba, enum ufs_pm_op pm_op)
ufshcd_resume_clkscaling(hba);
hba->clk_gating.is_suspended = false;
hba->dev_info.b_rpm_dev_flush_capable = false;
+ ufshcd_clear_ua_wluns(hba);
ufshcd_release(hba);
out:
if (hba->dev_info.b_rpm_dev_flush_capable) {
@@ -8855,6 +8858,8 @@ static int ufshcd_resume(struct ufs_hba *hba, enum ufs_pm_op pm_op)
cancel_delayed_work(&hba->rpm_dev_flush_recheck_work);
}

+ ufshcd_clear_ua_wluns(hba);
+
/* Schedule clock gating in case of no access to UFS device yet */
ufshcd_release(hba);

--
2.29.2.729.g45daf8777d-goog

2020-12-21 17:02:29

by Avri Altman

[permalink] [raw]
Subject: RE: [PATCH] scsi: ufs: fix livelock of ufshcd_clear_ua_wluns

> > > When gate_work/ungate_work gets an error during hibern8_enter or
> exit,
> > > ufshcd_err_handler()
> > > ufshcd_scsi_block_requests()
> > > ufshcd_reset_and_restore()
> > > ufshcd_clear_ua_wluns() -> stuck
> > > ufshcd_scsi_unblock_requests()
> > >
> > > In order to avoid it, ufshcd_clear_ua_wluns() can be called per recovery
> > > flows
> > > such as suspend/resume, link_recovery, and error_handler.
> > Not sure that suspend/resume are UAC events?
>
> Could you elaborate a bit? The goal is to clear UAC after UFS reset happens.
So why calling it on every suspend and resume?

>
> >
> > Also the 'fixes' tag is missing.
>
> Added. Thanks,
>
> >
> > Thanks,
> > Avri

2020-12-21 17:21:46

by Jaegeuk Kim

[permalink] [raw]
Subject: Re: [PATCH] scsi: ufs: fix livelock of ufshcd_clear_ua_wluns

On 12/21, Avri Altman wrote:
> > > > When gate_work/ungate_work gets an error during hibern8_enter or
> > exit,
> > > > ufshcd_err_handler()
> > > > ufshcd_scsi_block_requests()
> > > > ufshcd_reset_and_restore()
> > > > ufshcd_clear_ua_wluns() -> stuck
> > > > ufshcd_scsi_unblock_requests()
> > > >
> > > > In order to avoid it, ufshcd_clear_ua_wluns() can be called per recovery
> > > > flows
> > > > such as suspend/resume, link_recovery, and error_handler.
> > > Not sure that suspend/resume are UAC events?
> >
> > Could you elaborate a bit? The goal is to clear UAC after UFS reset happens.
> So why calling it on every suspend and resume?

1. If UAC was cleared, there's no impact.
2. ufshcd_link_recovery() can reset UFS directly by ufs_mtk_resume().
3. ufshcd_suspend can call ufshcd_host_reset_and_restore() as well.

>
> >
> > >
> > > Also the 'fixes' tag is missing.
> >
> > Added. Thanks,
> >
> > >
> > > Thanks,
> > > Avri

2020-12-21 18:08:09

by Avri Altman

[permalink] [raw]
Subject: RE: [PATCH] scsi: ufs: fix livelock of ufshcd_clear_ua_wluns

>
>
> On 12/21, Avri Altman wrote:
> > > > > When gate_work/ungate_work gets an error during hibern8_enter or
> > > exit,
> > > > > ufshcd_err_handler()
> > > > > ufshcd_scsi_block_requests()
> > > > > ufshcd_reset_and_restore()
> > > > > ufshcd_clear_ua_wluns() -> stuck
> > > > > ufshcd_scsi_unblock_requests()
> > > > >
> > > > > In order to avoid it, ufshcd_clear_ua_wluns() can be called per
> recovery
> > > > > flows
> > > > > such as suspend/resume, link_recovery, and error_handler.
> > > > Not sure that suspend/resume are UAC events?
> > >
> > > Could you elaborate a bit? The goal is to clear UAC after UFS reset
> happens.
> > So why calling it on every suspend and resume?
>
> 1. If UAC was cleared, there's no impact.
But the command is still sent.

> 2. ufshcd_link_recovery() can reset UFS directly by ufs_mtk_resume().
> 3. ufshcd_suspend can call ufshcd_host_reset_and_restore() as well.
Seems excessive IMO.
Why not selectively send when indeed required, e.g. on reset?

2020-12-21 19:39:43

by Jaegeuk Kim

[permalink] [raw]
Subject: Re: [PATCH] scsi: ufs: fix livelock of ufshcd_clear_ua_wluns

On 12/21, Avri Altman wrote:
> >
> >
> > On 12/21, Avri Altman wrote:
> > > > > > When gate_work/ungate_work gets an error during hibern8_enter or
> > > > exit,
> > > > > > ufshcd_err_handler()
> > > > > > ufshcd_scsi_block_requests()
> > > > > > ufshcd_reset_and_restore()
> > > > > > ufshcd_clear_ua_wluns() -> stuck
> > > > > > ufshcd_scsi_unblock_requests()
> > > > > >
> > > > > > In order to avoid it, ufshcd_clear_ua_wluns() can be called per
> > recovery
> > > > > > flows
> > > > > > such as suspend/resume, link_recovery, and error_handler.
> > > > > Not sure that suspend/resume are UAC events?
> > > >
> > > > Could you elaborate a bit? The goal is to clear UAC after UFS reset
> > happens.
> > > So why calling it on every suspend and resume?
> >
> > 1. If UAC was cleared, there's no impact.
> But the command is still sent.

No, ufshcd_clear_ua_wluns() will return by hba->wlun_dev_clr_ua.

>
> > 2. ufshcd_link_recovery() can reset UFS directly by ufs_mtk_resume().
> > 3. ufshcd_suspend can call ufshcd_host_reset_and_restore() as well.
> Seems excessive IMO.
> Why not selectively send when indeed required, e.g. on reset?

I think hba->wlun_dev_clr_ua is the indicator whether there was a reset or not.

2020-12-21 20:16:20

by Avri Altman

[permalink] [raw]
Subject: RE: [PATCH] scsi: ufs: fix livelock of ufshcd_clear_ua_wluns

> > > > > > > In order to avoid it, ufshcd_clear_ua_wluns() can be called per
> > > recovery
> > > > > > > flows
> > > > > > > such as suspend/resume, link_recovery, and error_handler.
> > > > > > Not sure that suspend/resume are UAC events?
> > > > >
> > > > > Could you elaborate a bit? The goal is to clear UAC after UFS reset
> > > happens.
> > > > So why calling it on every suspend and resume?
> > >
> > > 1. If UAC was cleared, there's no impact.
> > But the command is still sent.
>
> No, ufshcd_clear_ua_wluns() will return by hba->wlun_dev_clr_ua.
>
> >
> > > 2. ufshcd_link_recovery() can reset UFS directly by ufs_mtk_resume().
> > > 3. ufshcd_suspend can call ufshcd_host_reset_and_restore() as well.
> > Seems excessive IMO.
> > Why not selectively send when indeed required, e.g. on reset?
>
> I think hba->wlun_dev_clr_ua is the indicator whether there was a reset or
> not.
Ahha - I missed that. Thanks for clarifying it.
OK Then.

Thanks,
Avri

2021-01-06 03:47:12

by Martin K. Petersen

[permalink] [raw]
Subject: Re: [PATCH v2] scsi: ufs: fix livelock of ufshcd_clear_ua_wluns


Jaegeuk,

> When gate_work/ungate_work gets an error during hibern8_enter or exit,
> ufshcd_err_handler()
> ufshcd_scsi_block_requests()
> ufshcd_reset_and_restore()
> ufshcd_clear_ua_wluns() -> stuck
> ufshcd_scsi_unblock_requests()
>
> In order to avoid it, ufshcd_clear_ua_wluns() can be called per recovery flows
> such as suspend/resume, link_recovery, and error_handler.
>
> Fixes: 1918651f2d7e ("scsi: ufs: Clear UAC for RPMB after ufshcd resets")
> Signed-off-by: Jaegeuk Kim <[email protected]>

Please resubmit instead of replying to an existing patch. Both b4 and
patchwork get confused.

Thanks!

--
Martin K. Petersen Oracle Linux Engineering

2021-01-06 04:31:26

by Jaegeuk Kim

[permalink] [raw]
Subject: Re: [PATCH v2] scsi: ufs: fix livelock of ufshcd_clear_ua_wluns

On 01/05, Martin K. Petersen wrote:
>
> Jaegeuk,
>
> > When gate_work/ungate_work gets an error during hibern8_enter or exit,
> > ufshcd_err_handler()
> > ufshcd_scsi_block_requests()
> > ufshcd_reset_and_restore()
> > ufshcd_clear_ua_wluns() -> stuck
> > ufshcd_scsi_unblock_requests()
> >
> > In order to avoid it, ufshcd_clear_ua_wluns() can be called per recovery flows
> > such as suspend/resume, link_recovery, and error_handler.
> >
> > Fixes: 1918651f2d7e ("scsi: ufs: Clear UAC for RPMB after ufshcd resets")
> > Signed-off-by: Jaegeuk Kim <[email protected]>
>
> Please resubmit instead of replying to an existing patch. Both b4 and
> patchwork get confused.

Ok, I posted a new one. Thanks,

>
> Thanks!
>
> --
> Martin K. Petersen Oracle Linux Engineering