The rtt-upiu packets precede any data-out upiu packets, thus
synchronizing the data input to the device: this mostly applies to write
operations, but there are other operations that requires rtt as well.
There are several rules binding this rtt - data-out dialog, specifically
There can be at most outstanding bMaxNumOfRTT such packets. This might
have an effect on write performance (sequential write in particular), as
each data-out upiu must wait for its rtt sibling.
UFSHCI expects bMaxNumOfRTT to be min(bDeviceRTTCap, NORTT). However,
as of today, there does not appears to be no-one who sets it: not the
host controller nor the driver. It wasn't an issue up to now:
bMaxNumOfRTT is set to 2 after manufacturing, and wasn't limiting the
write performance.
UFS4.0, and specifically gear 5 changes this, and requires the device to
be more attentive. This doesn't come free - the device has to allocate
more resources to that end, but the sequential write performance
improvement is significant. Early measurements shows 25% gain when
moving from rtt 2 to 9. Therefore, set bMaxNumOfRTT to be
min(bDeviceRTTCap, NORTT) as UFSHCI expects.
v3 -> v4:
Allow bMaxNumOfRTT to be configured via sysfs (Bart)
v2 -> v3:
Allow platform vendors to take precedence having their own rtt
negotiation mechanism (Peter)
v1 -> v2:
bMaxNumOfRTT is a Persistent attribute - do not override if it was
written (Bean)
Avri Altman (3):
scsi: ufs: Allow RTT negotiation
scsi: ufs: Allow platform vendors to set rtt
scsi: ufs: sysfs: Make max_number_of_rtt read-write
Documentation/ABI/testing/sysfs-driver-ufs | 14 +++---
drivers/ufs/core/ufs-sysfs.c | 58 +++++++++++++++++++++-
drivers/ufs/core/ufshcd.c | 39 +++++++++++++++
include/ufs/ufshcd.h | 4 ++
include/ufs/ufshci.h | 1 +
5 files changed, 109 insertions(+), 7 deletions(-)
--
2.34.1
The rtt-upiu packets precede any data-out upiu packets, thus
synchronizing the data input to the device: this mostly applies to write
operations, but there are other operations that requires rtt as well.
There are several rules binding this rtt - data-out dialog, specifically
There can be at most outstanding bMaxNumOfRTT such packets. This might
have an effect on write performance (sequential write in particular), as
each data-out upiu must wait for its rtt sibling.
UFSHCI expects bMaxNumOfRTT to be min(bDeviceRTTCap, NORTT). However,
as of today, there does not appears to be no-one who sets it: not the
host controller nor the driver. It wasn't an issue up to now:
bMaxNumOfRTT is set to 2 after manufacturing, and wasn't limiting the
write performance.
UFS4.0, and specifically gear 5 changes this, and requires the device to
be more attentive. This doesn't come free - the device has to allocate
more resources to that end, but the sequential write performance
improvement is significant. Early measurements shows 25% gain when
moving from rtt 2 to 9. Therefore, set bMaxNumOfRTT to be
min(bDeviceRTTCap, NORTT) as UFSHCI expects.
Signed-off-by: Avri Altman <[email protected]>
Reviewed-by: Bean Huo <[email protected]>
---
drivers/ufs/core/ufshcd.c | 36 ++++++++++++++++++++++++++++++++++++
include/ufs/ufshcd.h | 2 ++
include/ufs/ufshci.h | 1 +
3 files changed, 39 insertions(+)
diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c
index 0819ddafe7a6..c472bfdf071e 100644
--- a/drivers/ufs/core/ufshcd.c
+++ b/drivers/ufs/core/ufshcd.c
@@ -102,6 +102,9 @@
/* Default RTC update every 10 seconds */
#define UFS_RTC_UPDATE_INTERVAL_MS (10 * MSEC_PER_SEC)
+/* bMaxNumOfRTT is equal to two after device manufacturing */
+#define DEFAULT_MAX_NUM_RTT 2
+
/* UFSHC 4.0 compliant HC support this mode. */
static bool use_mcq_mode = true;
@@ -2405,6 +2408,8 @@ static inline int ufshcd_hba_capabilities(struct ufs_hba *hba)
((hba->capabilities & MASK_TASK_MANAGEMENT_REQUEST_SLOTS) >> 16) + 1;
hba->reserved_slot = hba->nutrs - 1;
+ hba->nortt = FIELD_GET(MASK_NUMBER_OUTSTANDING_RTT, hba->capabilities) + 1;
+
/* Read crypto capabilities */
err = ufshcd_hba_init_crypto_capabilities(hba);
if (err) {
@@ -8119,6 +8124,35 @@ static void ufshcd_ext_iid_probe(struct ufs_hba *hba, u8 *desc_buf)
dev_info->b_ext_iid_en = ext_iid_en;
}
+static void ufshcd_rtt_set(struct ufs_hba *hba, u8 *desc_buf)
+{
+ struct ufs_dev_info *dev_info = &hba->dev_info;
+ u32 rtt = 0;
+ u32 dev_rtt = 0;
+
+ /* RTT override makes sense only for UFS-4.0 and above */
+ if (dev_info->wspecversion < 0x400)
+ return;
+
+ if (ufshcd_query_attr_retry(hba, UPIU_QUERY_OPCODE_READ_ATTR,
+ QUERY_ATTR_IDN_MAX_NUM_OF_RTT, 0, 0, &dev_rtt)) {
+ dev_err(hba->dev, "failed reading bMaxNumOfRTT\n");
+ return;
+ }
+
+ /* do not override if it was already written */
+ if (dev_rtt != DEFAULT_MAX_NUM_RTT)
+ return;
+
+ rtt = min_t(int, desc_buf[DEVICE_DESC_PARAM_RTT_CAP], hba->nortt);
+ if (rtt == dev_rtt)
+ return;
+
+ if (ufshcd_query_attr_retry(hba, UPIU_QUERY_OPCODE_WRITE_ATTR,
+ QUERY_ATTR_IDN_MAX_NUM_OF_RTT, 0, 0, &rtt))
+ dev_err(hba->dev, "failed writing bMaxNumOfRTT\n");
+}
+
void ufshcd_fixup_dev_quirks(struct ufs_hba *hba,
const struct ufs_dev_quirk *fixups)
{
@@ -8278,6 +8312,8 @@ static int ufs_get_device_desc(struct ufs_hba *hba)
if (hba->ext_iid_sup)
ufshcd_ext_iid_probe(hba, desc_buf);
+ ufshcd_rtt_set(hba, desc_buf);
+
/*
* ufshcd_read_string_desc returns size of the string
* reset the error value
diff --git a/include/ufs/ufshcd.h b/include/ufs/ufshcd.h
index bad88bd91995..d74bd2d67b06 100644
--- a/include/ufs/ufshcd.h
+++ b/include/ufs/ufshcd.h
@@ -819,6 +819,7 @@ enum ufshcd_mcq_opr {
* @capabilities: UFS Controller Capabilities
* @mcq_capabilities: UFS Multi Circular Queue capabilities
* @nutrs: Transfer Request Queue depth supported by controller
+ * @nortt - Max outstanding RTTs supported by controller
* @nutmrs: Task Management Queue depth supported by controller
* @reserved_slot: Used to submit device commands. Protected by @dev_cmd.lock.
* @ufs_version: UFS Version to which controller complies
@@ -957,6 +958,7 @@ struct ufs_hba {
u32 capabilities;
int nutrs;
+ int nortt;
u32 mcq_capabilities;
int nutmrs;
u32 reserved_slot;
diff --git a/include/ufs/ufshci.h b/include/ufs/ufshci.h
index 385e1c6b8d60..c50f92bf2e1d 100644
--- a/include/ufs/ufshci.h
+++ b/include/ufs/ufshci.h
@@ -68,6 +68,7 @@ enum {
/* Controller capability masks */
enum {
MASK_TRANSFER_REQUESTS_SLOTS = 0x0000001F,
+ MASK_NUMBER_OUTSTANDING_RTT = 0x0000FF00,
MASK_TASK_MANAGEMENT_REQUEST_SLOTS = 0x00070000,
MASK_EHSLUTRD_SUPPORTED = 0x00400000,
MASK_AUTO_HIBERN8_SUPPORT = 0x00800000,
--
2.34.1
Allow platform vendors to take precedence having their own rtt
negotiation mechanism. This makes sense because the host controller's
nortt characteristic may defer among vendors.
Signed-off-by: Avri Altman <[email protected]>
---
drivers/ufs/core/ufshcd.c | 5 ++++-
include/ufs/ufshcd.h | 2 ++
2 files changed, 6 insertions(+), 1 deletion(-)
diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c
index c472bfdf071e..0407d1064e74 100644
--- a/drivers/ufs/core/ufshcd.c
+++ b/drivers/ufs/core/ufshcd.c
@@ -8312,7 +8312,10 @@ static int ufs_get_device_desc(struct ufs_hba *hba)
if (hba->ext_iid_sup)
ufshcd_ext_iid_probe(hba, desc_buf);
- ufshcd_rtt_set(hba, desc_buf);
+ if (hba->vops && hba->vops->rtt_set)
+ hba->vops->rtt_set(hba, desc_buf);
+ else
+ ufshcd_rtt_set(hba, desc_buf);
/*
* ufshcd_read_string_desc returns size of the string
diff --git a/include/ufs/ufshcd.h b/include/ufs/ufshcd.h
index d74bd2d67b06..9237ea65bd26 100644
--- a/include/ufs/ufshcd.h
+++ b/include/ufs/ufshcd.h
@@ -329,6 +329,7 @@ struct ufs_pwr_mode_info {
* @get_outstanding_cqs: called to get outstanding completion queues
* @config_esi: called to config Event Specific Interrupt
* @config_scsi_dev: called to configure SCSI device parameters
+ * @rtt_set: negotiate rtt
*/
struct ufs_hba_variant_ops {
const char *name;
@@ -374,6 +375,7 @@ struct ufs_hba_variant_ops {
int (*get_outstanding_cqs)(struct ufs_hba *hba,
unsigned long *ocqs);
int (*config_esi)(struct ufs_hba *hba);
+ void (*rtt_set)(struct ufs_hba *hba, u8 *desc_buf);
};
/* clock gating state */
--
2.34.1
Given the importance of the RTT parameter, we want to be able to
configure it via sysfs. This is because UFS users should be discouraged
from change UFS device parameters without the UFSHCI driver being aware
of these changes.
Signed-off-by: Avri Altman <[email protected]>
---
Documentation/ABI/testing/sysfs-driver-ufs | 14 +++---
drivers/ufs/core/ufs-sysfs.c | 58 +++++++++++++++++++++-
2 files changed, 65 insertions(+), 7 deletions(-)
diff --git a/Documentation/ABI/testing/sysfs-driver-ufs b/Documentation/ABI/testing/sysfs-driver-ufs
index 5bf7073b4f75..fe943ce76c60 100644
--- a/Documentation/ABI/testing/sysfs-driver-ufs
+++ b/Documentation/ABI/testing/sysfs-driver-ufs
@@ -920,14 +920,16 @@ Description: This file shows whether the configuration descriptor is locked.
What: /sys/bus/platform/drivers/ufshcd/*/attributes/max_number_of_rtt
What: /sys/bus/platform/devices/*.ufs/attributes/max_number_of_rtt
-Date: February 2018
-Contact: Stanislav Nijnikov <[email protected]>
+Date: May 2024
+Contact: Avri Altman <[email protected]>
Description: This file provides the maximum current number of
- outstanding RTTs in device that is allowed. The full
- information about the attribute could be found at
- UFS specifications 2.1.
+ outstanding RTTs in device that is allowed. bMaxNumOfRTT is a
+ read-write persistent attribute and is equal to two after device
+ manufacturing. It shall not be set to a value greater than
+ bDeviceRTTCap value, and it may be set only when the hw queues are
+ empty.
- The file is read only.
+ The file is read write.
What: /sys/bus/platform/drivers/ufshcd/*/attributes/exception_event_control
What: /sys/bus/platform/devices/*.ufs/attributes/exception_event_control
diff --git a/drivers/ufs/core/ufs-sysfs.c b/drivers/ufs/core/ufs-sysfs.c
index 3d049967f6bc..8c6aeb836407 100644
--- a/drivers/ufs/core/ufs-sysfs.c
+++ b/drivers/ufs/core/ufs-sysfs.c
@@ -1340,6 +1340,63 @@ static const struct attribute_group ufs_sysfs_flags_group = {
.attrs = ufs_sysfs_device_flags,
};
+static ssize_t max_number_of_rtt_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct ufs_hba *hba = dev_get_drvdata(dev);
+ u32 rtt;
+ int ret;
+
+ down(&hba->host_sem);
+ if (!ufshcd_is_user_access_allowed(hba)) {
+ up(&hba->host_sem);
+ return -EBUSY;
+ }
+
+ ufshcd_rpm_get_sync(hba);
+ ret = ufshcd_query_attr(hba, UPIU_QUERY_OPCODE_READ_ATTR,
+ QUERY_ATTR_IDN_MAX_NUM_OF_RTT, 0, 0, &rtt);
+ ufshcd_rpm_put_sync(hba);
+
+ if (ret)
+ goto out;
+
+ ret = sysfs_emit(buf, "0x%08X\n", rtt);
+
+out:
+ up(&hba->host_sem);
+ return ret;
+}
+
+static ssize_t max_number_of_rtt_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t count)
+{
+ struct ufs_hba *hba = dev_get_drvdata(dev);
+ unsigned int rtt;
+ int ret;
+
+ if (kstrtouint(buf, 0, &rtt))
+ return -EINVAL;
+
+ down(&hba->host_sem);
+ if (!ufshcd_is_user_access_allowed(hba)) {
+ ret = -EBUSY;
+ goto out;
+ }
+
+ ufshcd_rpm_get_sync(hba);
+ ret = ufshcd_query_attr(hba, UPIU_QUERY_OPCODE_WRITE_ATTR,
+ QUERY_ATTR_IDN_MAX_NUM_OF_RTT, 0, 0, &rtt);
+ ufshcd_rpm_put_sync(hba);
+
+out:
+ up(&hba->host_sem);
+ return ret < 0 ? ret : count;
+}
+
+static DEVICE_ATTR_RW(max_number_of_rtt);
+
static inline bool ufshcd_is_wb_attrs(enum attr_idn idn)
{
return idn >= QUERY_ATTR_IDN_WB_FLUSH_STATUS &&
@@ -1387,7 +1444,6 @@ UFS_ATTRIBUTE(max_data_in_size, _MAX_DATA_IN);
UFS_ATTRIBUTE(max_data_out_size, _MAX_DATA_OUT);
UFS_ATTRIBUTE(reference_clock_frequency, _REF_CLK_FREQ);
UFS_ATTRIBUTE(configuration_descriptor_lock, _CONF_DESC_LOCK);
-UFS_ATTRIBUTE(max_number_of_rtt, _MAX_NUM_OF_RTT);
UFS_ATTRIBUTE(exception_event_control, _EE_CONTROL);
UFS_ATTRIBUTE(exception_event_status, _EE_STATUS);
UFS_ATTRIBUTE(ffu_status, _FFU_STATUS);
--
2.34.1
On 5/15/24 23:51, Avri Altman wrote:
> Allow platform vendors to take precedence having their own rtt
> negotiation mechanism. This makes sense because the host controller's
> nortt characteristic may defer among vendors.
defer -> vary?
> + void (*rtt_set)(struct ufs_hba *hba, u8 *desc_buf);
Please change "rtt_set" into "set_rtt" such that the word order matches
the regular word order in the English language.
Thanks,
Bart.
On 5/15/24 23:51, Avri Altman wrote:
> +static ssize_t max_number_of_rtt_store(struct device *dev,
> + struct device_attribute *attr,
> + const char *buf, size_t count)
> +{
> + struct ufs_hba *hba = dev_get_drvdata(dev);
> + unsigned int rtt;
> + int ret;
> +
> + if (kstrtouint(buf, 0, &rtt))
> + return -EINVAL;
> +
> + down(&hba->host_sem);
> + if (!ufshcd_is_user_access_allowed(hba)) {
> + ret = -EBUSY;
> + goto out;
> + }
> +
> + ufshcd_rpm_get_sync(hba);
> + ret = ufshcd_query_attr(hba, UPIU_QUERY_OPCODE_WRITE_ATTR,
> + QUERY_ATTR_IDN_MAX_NUM_OF_RTT, 0, 0, &rtt);
> + ufshcd_rpm_put_sync(hba);
> +
> +out:
> + up(&hba->host_sem);
> + return ret < 0 ? ret : count;
> +}
Since modifying RTT is only allowed while no commands are in progress,
shouldn't max_number_of_rtt_store() freeze and unfreeze all request
queues of all logical units?
Thanks,
Bart.
On 5/15/24 23:51, Avri Altman wrote:
> void ufshcd_fixup_dev_quirks(struct ufs_hba *hba,
> const struct ufs_dev_quirk *fixups)
> {
> @@ -8278,6 +8312,8 @@ static int ufs_get_device_desc(struct ufs_hba *hba)
> if (hba->ext_iid_sup)
> ufshcd_ext_iid_probe(hba, desc_buf);
>
> + ufshcd_rtt_set(hba, desc_buf);
> +
Why does this call occur in ufs_get_device_desc()? ufshcd_rtt_set() sets
a device parameter. Shouldn't this call be moved one level up into
ufshcd_device_params_init()?
Otherwise this patch looks good to me.
Thanks,
Bart.
> On 5/15/24 23:51, Avri Altman wrote:
> > void ufshcd_fixup_dev_quirks(struct ufs_hba *hba,
> > const struct ufs_dev_quirk *fixups)
> > {
> > @@ -8278,6 +8312,8 @@ static int ufs_get_device_desc(struct ufs_hba *hba)
> > if (hba->ext_iid_sup)
> > ufshcd_ext_iid_probe(hba, desc_buf);
> >
> > + ufshcd_rtt_set(hba, desc_buf);
> > +
>
> Why does this call occur in ufs_get_device_desc()? ufshcd_rtt_set() sets
> a device parameter. Shouldn't this call be moved one level up into
> ufshcd_device_params_init()?
Because otherwise bDeviceRTTCap is not available.
Please note several device configuration calls in ufs_get_device_desc - all requires some device descriptor fields.
Thanks,
Avri
>
> Otherwise this patch looks good to me.
>
> Thanks,
>
> Bart.
>
> On 5/15/24 23:51, Avri Altman wrote:
> > Allow platform vendors to take precedence having their own rtt
> > negotiation mechanism. This makes sense because the host controller's
> > nortt characteristic may defer among vendors.
>
> defer -> vary?
>
> > + void (*rtt_set)(struct ufs_hba *hba, u8 *desc_buf);
>
> Please change "rtt_set" into "set_rtt" such that the word order matches
> the regular word order in the English language.
Done.
Thanks,
Avri
>
> Thanks,
>
> Bart.
> On 5/15/24 23:51, Avri Altman wrote:
> > +static ssize_t max_number_of_rtt_store(struct device *dev,
> > + struct device_attribute *attr,
> > + const char *buf, size_t count)
> > +{
> > + struct ufs_hba *hba = dev_get_drvdata(dev);
> > + unsigned int rtt;
> > + int ret;
> > +
> > + if (kstrtouint(buf, 0, &rtt))
> > + return -EINVAL;
> > +
> > + down(&hba->host_sem);
> > + if (!ufshcd_is_user_access_allowed(hba)) {
> > + ret = -EBUSY;
> > + goto out;
> > + }
> > +
> > + ufshcd_rpm_get_sync(hba);
> > + ret = ufshcd_query_attr(hba, UPIU_QUERY_OPCODE_WRITE_ATTR,
> > + QUERY_ATTR_IDN_MAX_NUM_OF_RTT, 0, 0, &rtt);
> > + ufshcd_rpm_put_sync(hba);
> > +
> > +out:
> > + up(&hba->host_sem);
> > + return ret < 0 ? ret : count;
> > +}
>
> Since modifying RTT is only allowed while no commands are in progress,
> shouldn't max_number_of_rtt_store() freeze and unfreeze all request
> queues of all logical units?
Done.
Thanks,
Avri
>
> Thanks,
>
> Bart.
On 5/16/24 09:21, Avri Altman wrote:
>> On 5/15/24 23:51, Avri Altman wrote:
>>> void ufshcd_fixup_dev_quirks(struct ufs_hba *hba,
>>> const struct ufs_dev_quirk *fixups)
>>> {
>>> @@ -8278,6 +8312,8 @@ static int ufs_get_device_desc(struct ufs_hba *hba)
>>> if (hba->ext_iid_sup)
>>> ufshcd_ext_iid_probe(hba, desc_buf);
>>>
>>> + ufshcd_rtt_set(hba, desc_buf);
>>> +
>>
>> Why does this call occur in ufs_get_device_desc()? ufshcd_rtt_set() sets
>> a device parameter. Shouldn't this call be moved one level up into
>> ufshcd_device_params_init()?
> Because otherwise bDeviceRTTCap is not available.
> Please note several device configuration calls in ufs_get_device_desc - all requires some device descriptor fields.
Please make sure that the functionality of ufs_get_device_desc() and the
name of that function remain in sync.
I think that bDeviceRTTCap is available in ufshcd_device_params_init()
after ufs_get_device_desc() has returned?
Thanks,
Bart.
> On 5/16/24 09:21, Avri Altman wrote:
> >> On 5/15/24 23:51, Avri Altman wrote:
> >>> void ufshcd_fixup_dev_quirks(struct ufs_hba *hba,
> >>> const struct ufs_dev_quirk *fixups)
> >>> {
> >>> @@ -8278,6 +8312,8 @@ static int ufs_get_device_desc(struct ufs_hba
> *hba)
> >>> if (hba->ext_iid_sup)
> >>> ufshcd_ext_iid_probe(hba, desc_buf);
> >>>
> >>> + ufshcd_rtt_set(hba, desc_buf);
> >>> +
> >>
> >> Why does this call occur in ufs_get_device_desc()? ufshcd_rtt_set()
> >> sets a device parameter. Shouldn't this call be moved one level up
> >> into ufshcd_device_params_init()?
> > Because otherwise bDeviceRTTCap is not available.
> > Please note several device configuration calls in ufs_get_device_desc - all
> requires some device descriptor fields.
>
> Please make sure that the functionality of ufs_get_device_desc() and the name of
> that function remain in sync.
>
> I think that bDeviceRTTCap is available in ufshcd_device_params_init() after
> ufs_get_device_desc() has returned?
Will make the required change.
Thanks,
Avri
>
> Thanks,
>
> Bart.