On 2020-10-21 12:52, [email protected] wrote:
> On 10/21, Can Guo wrote:
>> On 2020-10-21 03:52, Jaegeuk Kim wrote:
>> > The below call stack prevents clk_gating at every IO completion.
>> > We can remove the condition, ufshcd_any_tag_in_use(), since
>> > clkgating_work
>> > will check it again.
>> >
>>
>> I think checking ufshcd_any_tag_in_use() in either ufshcd_release() or
>> gate_work() can break UFS clk gating's functionality.
>>
>> ufshcd_any_tag_in_use() was introduced to replace hba->lrb_in_use.
>> However,
>> they are not exactly same - ufshcd_any_tag_in_use() returns true if
>> any tag
>> assigned from block layer is still in use, but tags are released
>> asynchronously
>> (through block softirq), meaning it does not reflect the real
>> occupation of
>> UFS host.
>> That is after UFS host finishes all tasks, ufshcd_any_tag_in_use() can
>> still
>> return true.
>>
>> This change only removes the check of ufshcd_any_tag_in_use() in
>> ufshcd_release(),
>> but having the check of it in gate_work() can still prevent gating
>> from
>> happening.
>> The current change works for you maybe because the tags are release
>> before
>> hba->clk_gating.delay_ms expires, but if hba->clk_gating.delay_ms is
>> shorter
>> or
>> somehow block softirq is retarded, gate_work() may have chance to see
>> ufshcd_any_tag_in_use()
>> returns true. What do you think?
>
> I don't think this breaks clkgating, but fix the wrong condition check
> which
> prevented gate_work at all. As you mentioned, even if this schedules
> gate_work
> by racy conditions, gate_work will handle it as a last resort.
>
If clocks cannot be gated after the last task is cleared from UFS host,
then clk gating
is broken, no? Assume UFS has completed the last task in its queue, as
this change says,
ufshcd_any_tag_in_use() is preventing ufshcd_release() from invoking
gate_work().
Similarly, ufshcd_any_tag_in_use() can prevent gate_work() from doing
its real work -
disabling the clocks. Do you agree?
if (hba->clk_gating.active_reqs
|| hba->ufshcd_state != UFSHCD_STATE_OPERATIONAL
|| ufshcd_any_tag_in_use(hba) || hba->outstanding_tasks
|| hba->active_uic_cmd || hba->uic_async_done)
goto rel_lock;
Thanks,
Can Guo.
>>
>> Thanks,
>>
>> Can Guo.
>>
>> In __ufshcd_transfer_req_compl
>> Ihba->lrb_in_use is cleared immediately when UFS driver
>> finishes all tasks
>>
>> > ufshcd_complete_requests(struct ufs_hba *hba)
>> > ufshcd_transfer_req_compl()
>> > __ufshcd_transfer_req_compl()
>> > __ufshcd_release(hba)
>> > if (ufshcd_any_tag_in_use() == 1)
>> > return;
>> > ufshcd_tmc_handler(hba);
>> > blk_mq_tagset_busy_iter();
>> >
>> > Cc: Alim Akhtar <[email protected]>
>> > Cc: Avri Altman <[email protected]>
>> > Cc: Can Guo <[email protected]>
>> > Signed-off-by: Jaegeuk Kim <[email protected]>
>> > ---
>> > drivers/scsi/ufs/ufshcd.c | 2 +-
>> > 1 file changed, 1 insertion(+), 1 deletion(-)
>> >
>> > diff --git a/drivers/scsi/ufs/ufshcd.c b/drivers/scsi/ufs/ufshcd.c
>> > index b5ca0effe636..cecbd4ace8b4 100644
>> > --- a/drivers/scsi/ufs/ufshcd.c
>> > +++ b/drivers/scsi/ufs/ufshcd.c
>> > @@ -1746,7 +1746,7 @@ static void __ufshcd_release(struct ufs_hba *hba)
>> >
>> > if (hba->clk_gating.active_reqs || hba->clk_gating.is_suspended ||
>> > hba->ufshcd_state != UFSHCD_STATE_OPERATIONAL ||
>> > - ufshcd_any_tag_in_use(hba) || hba->outstanding_tasks ||
>> > + hba->outstanding_tasks ||
>> > hba->active_uic_cmd || hba->uic_async_done)
>> > return;
On 10/21, Can Guo wrote:
> On 2020-10-21 12:52, [email protected] wrote:
> > On 10/21, Can Guo wrote:
> > > On 2020-10-21 03:52, Jaegeuk Kim wrote:
> > > > The below call stack prevents clk_gating at every IO completion.
> > > > We can remove the condition, ufshcd_any_tag_in_use(), since
> > > > clkgating_work
> > > > will check it again.
> > > >
> > >
> > > I think checking ufshcd_any_tag_in_use() in either ufshcd_release() or
> > > gate_work() can break UFS clk gating's functionality.
> > >
> > > ufshcd_any_tag_in_use() was introduced to replace hba->lrb_in_use.
> > > However,
> > > they are not exactly same - ufshcd_any_tag_in_use() returns true if
> > > any tag
> > > assigned from block layer is still in use, but tags are released
> > > asynchronously
> > > (through block softirq), meaning it does not reflect the real
> > > occupation of
> > > UFS host.
> > > That is after UFS host finishes all tasks, ufshcd_any_tag_in_use()
> > > can still
> > > return true.
> > >
> > > This change only removes the check of ufshcd_any_tag_in_use() in
> > > ufshcd_release(),
> > > but having the check of it in gate_work() can still prevent gating
> > > from
> > > happening.
> > > The current change works for you maybe because the tags are release
> > > before
> > > hba->clk_gating.delay_ms expires, but if hba->clk_gating.delay_ms is
> > > shorter
> > > or
> > > somehow block softirq is retarded, gate_work() may have chance to see
> > > ufshcd_any_tag_in_use()
> > > returns true. What do you think?
> >
> > I don't think this breaks clkgating, but fix the wrong condition check
> > which
> > prevented gate_work at all. As you mentioned, even if this schedules
> > gate_work
> > by racy conditions, gate_work will handle it as a last resort.
> >
>
> If clocks cannot be gated after the last task is cleared from UFS host, then
> clk gating
> is broken, no? Assume UFS has completed the last task in its queue, as this
> change says,
> ufshcd_any_tag_in_use() is preventing ufshcd_release() from invoking
> gate_work().
> Similarly, ufshcd_any_tag_in_use() can prevent gate_work() from doing its
> real work -
> disabling the clocks. Do you agree?
>
> if (hba->clk_gating.active_reqs
> || hba->ufshcd_state != UFSHCD_STATE_OPERATIONAL
> || ufshcd_any_tag_in_use(hba) || hba->outstanding_tasks
> || hba->active_uic_cmd || hba->uic_async_done)
> goto rel_lock;
I see the point, but this happens only when clkgate_delay_ms is too short
to give enough time for releasing tag. If it's correctly set, I think there'd
be no problem, unless softirq was delayed by other RT threads which is just
a corner case tho.
>
> Thanks,
>
> Can Guo.
>
> > >
> > > Thanks,
> > >
> > > Can Guo.
> > >
> > > In __ufshcd_transfer_req_compl
> > > Ihba->lrb_in_use is cleared immediately when UFS driver
> > > finishes all tasks
> > >
> > > > ufshcd_complete_requests(struct ufs_hba *hba)
> > > > ufshcd_transfer_req_compl()
> > > > __ufshcd_transfer_req_compl()
> > > > __ufshcd_release(hba)
> > > > if (ufshcd_any_tag_in_use() == 1)
> > > > return;
> > > > ufshcd_tmc_handler(hba);
> > > > blk_mq_tagset_busy_iter();
> > > >
> > > > Cc: Alim Akhtar <[email protected]>
> > > > Cc: Avri Altman <[email protected]>
> > > > Cc: Can Guo <[email protected]>
> > > > Signed-off-by: Jaegeuk Kim <[email protected]>
> > > > ---
> > > > drivers/scsi/ufs/ufshcd.c | 2 +-
> > > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > > >
> > > > diff --git a/drivers/scsi/ufs/ufshcd.c b/drivers/scsi/ufs/ufshcd.c
> > > > index b5ca0effe636..cecbd4ace8b4 100644
> > > > --- a/drivers/scsi/ufs/ufshcd.c
> > > > +++ b/drivers/scsi/ufs/ufshcd.c
> > > > @@ -1746,7 +1746,7 @@ static void __ufshcd_release(struct ufs_hba *hba)
> > > >
> > > > if (hba->clk_gating.active_reqs || hba->clk_gating.is_suspended ||
> > > > hba->ufshcd_state != UFSHCD_STATE_OPERATIONAL ||
> > > > - ufshcd_any_tag_in_use(hba) || hba->outstanding_tasks ||
> > > > + hba->outstanding_tasks ||
> > > > hba->active_uic_cmd || hba->uic_async_done)
> > > > return;
On 2020-10-23 08:53, Jaegeuk Kim wrote:
> On 10/21, Can Guo wrote:
>> On 2020-10-21 12:52, [email protected] wrote:
>> > On 10/21, Can Guo wrote:
>> > > On 2020-10-21 03:52, Jaegeuk Kim wrote:
>> > > > The below call stack prevents clk_gating at every IO completion.
>> > > > We can remove the condition, ufshcd_any_tag_in_use(), since
>> > > > clkgating_work
>> > > > will check it again.
>> > > >
>> > >
>> > > I think checking ufshcd_any_tag_in_use() in either ufshcd_release() or
>> > > gate_work() can break UFS clk gating's functionality.
>> > >
>> > > ufshcd_any_tag_in_use() was introduced to replace hba->lrb_in_use.
>> > > However,
>> > > they are not exactly same - ufshcd_any_tag_in_use() returns true if
>> > > any tag
>> > > assigned from block layer is still in use, but tags are released
>> > > asynchronously
>> > > (through block softirq), meaning it does not reflect the real
>> > > occupation of
>> > > UFS host.
>> > > That is after UFS host finishes all tasks, ufshcd_any_tag_in_use()
>> > > can still
>> > > return true.
>> > >
>> > > This change only removes the check of ufshcd_any_tag_in_use() in
>> > > ufshcd_release(),
>> > > but having the check of it in gate_work() can still prevent gating
>> > > from
>> > > happening.
>> > > The current change works for you maybe because the tags are release
>> > > before
>> > > hba->clk_gating.delay_ms expires, but if hba->clk_gating.delay_ms is
>> > > shorter
>> > > or
>> > > somehow block softirq is retarded, gate_work() may have chance to see
>> > > ufshcd_any_tag_in_use()
>> > > returns true. What do you think?
>> >
>> > I don't think this breaks clkgating, but fix the wrong condition check
>> > which
>> > prevented gate_work at all. As you mentioned, even if this schedules
>> > gate_work
>> > by racy conditions, gate_work will handle it as a last resort.
>> >
>>
>> If clocks cannot be gated after the last task is cleared from UFS
>> host, then
>> clk gating
>> is broken, no? Assume UFS has completed the last task in its queue, as
>> this
>> change says,
>> ufshcd_any_tag_in_use() is preventing ufshcd_release() from invoking
>> gate_work().
>> Similarly, ufshcd_any_tag_in_use() can prevent gate_work() from doing
>> its
>> real work -
>> disabling the clocks. Do you agree?
>>
>> if (hba->clk_gating.active_reqs
>> || hba->ufshcd_state != UFSHCD_STATE_OPERATIONAL
>> || ufshcd_any_tag_in_use(hba) ||
>> hba->outstanding_tasks
>> || hba->active_uic_cmd || hba->uic_async_done)
>> goto rel_lock;
>
> I see the point, but this happens only when clkgate_delay_ms is too
> short
> to give enough time for releasing tag. If it's correctly set, I think
> there'd
> be no problem, unless softirq was delayed by other RT threads which is
> just
> a corner case tho.
>
Yes, we are fixing corner cases, aren't we? I thought you would like to
address it since you are fixing clk gating.
Regards,
Can Guo.
>>
>> Thanks,
>>
>> Can Guo.
>>
>> > >
>> > > Thanks,
>> > >
>> > > Can Guo.
>> > >
>> > > In __ufshcd_transfer_req_compl
>> > > Ihba->lrb_in_use is cleared immediately when UFS driver
>> > > finishes all tasks
>> > >
>> > > > ufshcd_complete_requests(struct ufs_hba *hba)
>> > > > ufshcd_transfer_req_compl()
>> > > > __ufshcd_transfer_req_compl()
>> > > > __ufshcd_release(hba)
>> > > > if (ufshcd_any_tag_in_use() == 1)
>> > > > return;
>> > > > ufshcd_tmc_handler(hba);
>> > > > blk_mq_tagset_busy_iter();
>> > > >
>> > > > Cc: Alim Akhtar <[email protected]>
>> > > > Cc: Avri Altman <[email protected]>
>> > > > Cc: Can Guo <[email protected]>
>> > > > Signed-off-by: Jaegeuk Kim <[email protected]>
>> > > > ---
>> > > > drivers/scsi/ufs/ufshcd.c | 2 +-
>> > > > 1 file changed, 1 insertion(+), 1 deletion(-)
>> > > >
>> > > > diff --git a/drivers/scsi/ufs/ufshcd.c b/drivers/scsi/ufs/ufshcd.c
>> > > > index b5ca0effe636..cecbd4ace8b4 100644
>> > > > --- a/drivers/scsi/ufs/ufshcd.c
>> > > > +++ b/drivers/scsi/ufs/ufshcd.c
>> > > > @@ -1746,7 +1746,7 @@ static void __ufshcd_release(struct ufs_hba *hba)
>> > > >
>> > > > if (hba->clk_gating.active_reqs || hba->clk_gating.is_suspended ||
>> > > > hba->ufshcd_state != UFSHCD_STATE_OPERATIONAL ||
>> > > > - ufshcd_any_tag_in_use(hba) || hba->outstanding_tasks ||
>> > > > + hba->outstanding_tasks ||
>> > > > hba->active_uic_cmd || hba->uic_async_done)
>> > > > return;
On 10/26, Can Guo wrote:
> On 2020-10-23 08:53, Jaegeuk Kim wrote:
> > On 10/21, Can Guo wrote:
> > > On 2020-10-21 12:52, [email protected] wrote:
> > > > On 10/21, Can Guo wrote:
> > > > > On 2020-10-21 03:52, Jaegeuk Kim wrote:
> > > > > > The below call stack prevents clk_gating at every IO completion.
> > > > > > We can remove the condition, ufshcd_any_tag_in_use(), since
> > > > > > clkgating_work
> > > > > > will check it again.
> > > > > >
> > > > >
> > > > > I think checking ufshcd_any_tag_in_use() in either ufshcd_release() or
> > > > > gate_work() can break UFS clk gating's functionality.
> > > > >
> > > > > ufshcd_any_tag_in_use() was introduced to replace hba->lrb_in_use.
> > > > > However,
> > > > > they are not exactly same - ufshcd_any_tag_in_use() returns true if
> > > > > any tag
> > > > > assigned from block layer is still in use, but tags are released
> > > > > asynchronously
> > > > > (through block softirq), meaning it does not reflect the real
> > > > > occupation of
> > > > > UFS host.
> > > > > That is after UFS host finishes all tasks, ufshcd_any_tag_in_use()
> > > > > can still
> > > > > return true.
> > > > >
> > > > > This change only removes the check of ufshcd_any_tag_in_use() in
> > > > > ufshcd_release(),
> > > > > but having the check of it in gate_work() can still prevent gating
> > > > > from
> > > > > happening.
> > > > > The current change works for you maybe because the tags are release
> > > > > before
> > > > > hba->clk_gating.delay_ms expires, but if hba->clk_gating.delay_ms is
> > > > > shorter
> > > > > or
> > > > > somehow block softirq is retarded, gate_work() may have chance to see
> > > > > ufshcd_any_tag_in_use()
> > > > > returns true. What do you think?
> > > >
> > > > I don't think this breaks clkgating, but fix the wrong condition check
> > > > which
> > > > prevented gate_work at all. As you mentioned, even if this schedules
> > > > gate_work
> > > > by racy conditions, gate_work will handle it as a last resort.
> > > >
> > >
> > > If clocks cannot be gated after the last task is cleared from UFS
> > > host, then
> > > clk gating
> > > is broken, no? Assume UFS has completed the last task in its queue,
> > > as this
> > > change says,
> > > ufshcd_any_tag_in_use() is preventing ufshcd_release() from invoking
> > > gate_work().
> > > Similarly, ufshcd_any_tag_in_use() can prevent gate_work() from
> > > doing its
> > > real work -
> > > disabling the clocks. Do you agree?
> > >
> > > if (hba->clk_gating.active_reqs
> > > || hba->ufshcd_state != UFSHCD_STATE_OPERATIONAL
> > > || ufshcd_any_tag_in_use(hba) ||
> > > hba->outstanding_tasks
> > > || hba->active_uic_cmd || hba->uic_async_done)
> > > goto rel_lock;
> >
> > I see the point, but this happens only when clkgate_delay_ms is too
> > short
> > to give enough time for releasing tag. If it's correctly set, I think
> > there'd
> > be no problem, unless softirq was delayed by other RT threads which is
> > just
> > a corner case tho.
> >
>
> Yes, we are fixing corner cases, aren't we? I thought you would like to
> address it since you are fixing clk gating.
I think that can be fixed by a separate patch which controls delay_ms when
user tries to change it from default 150 ms?
>
> Regards,
>
> Can Guo.
>
> > >
> > > Thanks,
> > >
> > > Can Guo.
> > >
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Can Guo.
> > > > >
> > > > > In __ufshcd_transfer_req_compl
> > > > > Ihba->lrb_in_use is cleared immediately when UFS driver
> > > > > finishes all tasks
> > > > >
> > > > > > ufshcd_complete_requests(struct ufs_hba *hba)
> > > > > > ufshcd_transfer_req_compl()
> > > > > > __ufshcd_transfer_req_compl()
> > > > > > __ufshcd_release(hba)
> > > > > > if (ufshcd_any_tag_in_use() == 1)
> > > > > > return;
> > > > > > ufshcd_tmc_handler(hba);
> > > > > > blk_mq_tagset_busy_iter();
> > > > > >
> > > > > > Cc: Alim Akhtar <[email protected]>
> > > > > > Cc: Avri Altman <[email protected]>
> > > > > > Cc: Can Guo <[email protected]>
> > > > > > Signed-off-by: Jaegeuk Kim <[email protected]>
> > > > > > ---
> > > > > > drivers/scsi/ufs/ufshcd.c | 2 +-
> > > > > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > > > > >
> > > > > > diff --git a/drivers/scsi/ufs/ufshcd.c b/drivers/scsi/ufs/ufshcd.c
> > > > > > index b5ca0effe636..cecbd4ace8b4 100644
> > > > > > --- a/drivers/scsi/ufs/ufshcd.c
> > > > > > +++ b/drivers/scsi/ufs/ufshcd.c
> > > > > > @@ -1746,7 +1746,7 @@ static void __ufshcd_release(struct ufs_hba *hba)
> > > > > >
> > > > > > if (hba->clk_gating.active_reqs || hba->clk_gating.is_suspended ||
> > > > > > hba->ufshcd_state != UFSHCD_STATE_OPERATIONAL ||
> > > > > > - ufshcd_any_tag_in_use(hba) || hba->outstanding_tasks ||
> > > > > > + hba->outstanding_tasks ||
> > > > > > hba->active_uic_cmd || hba->uic_async_done)
> > > > > > return;
On 2020-10-22 17:53, Jaegeuk Kim wrote:
> On 10/21, Can Guo wrote:
>> On 2020-10-21 12:52, [email protected] wrote:
>> > On 10/21, Can Guo wrote:
>> > > On 2020-10-21 03:52, Jaegeuk Kim wrote:
>> > > > The below call stack prevents clk_gating at every IO completion.
>> > > > We can remove the condition, ufshcd_any_tag_in_use(), since
>> > > > clkgating_work
>> > > > will check it again.
>> > > >
>> > >
>> > > I think checking ufshcd_any_tag_in_use() in either ufshcd_release() or
>> > > gate_work() can break UFS clk gating's functionality.
>> > >
>> > > ufshcd_any_tag_in_use() was introduced to replace hba->lrb_in_use.
>> > > However,
>> > > they are not exactly same - ufshcd_any_tag_in_use() returns true if
>> > > any tag
>> > > assigned from block layer is still in use, but tags are released
>> > > asynchronously
>> > > (through block softirq), meaning it does not reflect the real
>> > > occupation of
>> > > UFS host.
>> > > That is after UFS host finishes all tasks, ufshcd_any_tag_in_use()
>> > > can still
>> > > return true.
>> > >
>> > > This change only removes the check of ufshcd_any_tag_in_use() in
>> > > ufshcd_release(),
>> > > but having the check of it in gate_work() can still prevent gating
>> > > from
>> > > happening.
>> > > The current change works for you maybe because the tags are release
>> > > before
>> > > hba->clk_gating.delay_ms expires, but if hba->clk_gating.delay_ms is
>> > > shorter
>> > > or
>> > > somehow block softirq is retarded, gate_work() may have chance to see
>> > > ufshcd_any_tag_in_use()
>> > > returns true. What do you think?
>> >
>> > I don't think this breaks clkgating, but fix the wrong condition check
>> > which
>> > prevented gate_work at all. As you mentioned, even if this schedules
>> > gate_work
>> > by racy conditions, gate_work will handle it as a last resort.
>> >
>>
>> If clocks cannot be gated after the last task is cleared from UFS
>> host, then
>> clk gating
>> is broken, no? Assume UFS has completed the last task in its queue, as
>> this
>> change says,
>> ufshcd_any_tag_in_use() is preventing ufshcd_release() from invoking
>> gate_work().
>> Similarly, ufshcd_any_tag_in_use() can prevent gate_work() from doing
>> its
>> real work -
>> disabling the clocks. Do you agree?
>>
>> if (hba->clk_gating.active_reqs
>> || hba->ufshcd_state != UFSHCD_STATE_OPERATIONAL
>> || ufshcd_any_tag_in_use(hba) ||
>> hba->outstanding_tasks
>> || hba->active_uic_cmd || hba->uic_async_done)
>> goto rel_lock;
>
> I see the point, but this happens only when clkgate_delay_ms is too
> short
> to give enough time for releasing tag. If it's correctly set, I think
> there'd
> be no problem, unless softirq was delayed by other RT threads which is
> just
> a corner case tho.
>
>>
>> Thanks,
>>
>> Can Guo.
>>
>> > >
>> > > Thanks,
>> > >
>> > > Can Guo.
>> > >
>> > > In __ufshcd_transfer_req_compl
>> > > Ihba->lrb_in_use is cleared immediately when UFS driver
>> > > finishes all tasks
>> > >
>> > > > ufshcd_complete_requests(struct ufs_hba *hba)
>> > > > ufshcd_transfer_req_compl()
>> > > > __ufshcd_transfer_req_compl()
>> > > > __ufshcd_release(hba)
>> > > > if (ufshcd_any_tag_in_use() == 1)
>> > > > return;
>> > > > ufshcd_tmc_handler(hba);
>> > > > blk_mq_tagset_busy_iter();
>> > > >
>> > > > Cc: Alim Akhtar <[email protected]>
>> > > > Cc: Avri Altman <[email protected]>
>> > > > Cc: Can Guo <[email protected]>
>> > > > Signed-off-by: Jaegeuk Kim <[email protected]>
>> > > > ---
>> > > > drivers/scsi/ufs/ufshcd.c | 2 +-
>> > > > 1 file changed, 1 insertion(+), 1 deletion(-)
>> > > >
>> > > > diff --git a/drivers/scsi/ufs/ufshcd.c b/drivers/scsi/ufs/ufshcd.c
>> > > > index b5ca0effe636..cecbd4ace8b4 100644
>> > > > --- a/drivers/scsi/ufs/ufshcd.c
>> > > > +++ b/drivers/scsi/ufs/ufshcd.c
>> > > > @@ -1746,7 +1746,7 @@ static void __ufshcd_release(struct ufs_hba *hba)
>> > > >
>> > > > if (hba->clk_gating.active_reqs || hba->clk_gating.is_suspended ||
>> > > > hba->ufshcd_state != UFSHCD_STATE_OPERATIONAL ||
>> > > > - ufshcd_any_tag_in_use(hba) || hba->outstanding_tasks ||
>> > > > + hba->outstanding_tasks ||
>> > > > hba->active_uic_cmd || hba->uic_async_done)
>> > > > return;
How about checking outstanding_reqs as well, say in
ufshcd_any_tag_in_use() ?
-asd