We currently only enforce BW floors for a subset of nodes in a path.
All BCMs that need updating are queued in the pre_aggregate/aggregate
phase. The first set() commits all queued BCMs and subsequent set()
calls short-circuit without committing anything. Since the floor BW
isn't set in sum_avg/max_peak until set(), then some BCMs are committed
before their associated nodes reflect the floor.
Set the floor as each node is being aggregated. This ensures that all
all relevant floors are set before the BCMs are committed.
Fixes: 266cd33b5913 ("interconnect: qcom: Ensure that the floor bandwidth value is enforced")
Signed-off-by: Mike Tipton <[email protected]>
---
drivers/interconnect/qcom/icc-rpmh.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/drivers/interconnect/qcom/icc-rpmh.c b/drivers/interconnect/qcom/icc-rpmh.c
index bf01d09dba6c..f118f57eae37 100644
--- a/drivers/interconnect/qcom/icc-rpmh.c
+++ b/drivers/interconnect/qcom/icc-rpmh.c
@@ -57,6 +57,11 @@ int qcom_icc_aggregate(struct icc_node *node, u32 tag, u32 avg_bw,
qn->sum_avg[i] += avg_bw;
qn->max_peak[i] = max_t(u32, qn->max_peak[i], peak_bw);
}
+
+ if (node->init_avg || node->init_peak) {
+ qn->sum_avg[i] = max_t(u64, qn->sum_avg[i], node->init_avg);
+ qn->max_peak[i] = max_t(u64, qn->max_peak[i], node->init_peak);
+ }
}
*agg_avg += avg_bw;
@@ -90,11 +95,6 @@ int qcom_icc_set(struct icc_node *src, struct icc_node *dst)
qp = to_qcom_provider(node->provider);
qn = node->data;
- qn->sum_avg[QCOM_ICC_BUCKET_AMC] = max_t(u64, qn->sum_avg[QCOM_ICC_BUCKET_AMC],
- node->avg_bw);
- qn->max_peak[QCOM_ICC_BUCKET_AMC] = max_t(u64, qn->max_peak[QCOM_ICC_BUCKET_AMC],
- node->peak_bw);
-
qcom_icc_bcm_voter_commit(qp->voter);
return 0;
--
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project
Hi Mike,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on linus/master]
[also build test WARNING on v5.13-rc7 next-20210625]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Mike-Tipton/interconnect-Fix-sync-state-issues/20210626-053132
base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 55fcd4493da5ac8a0f7a0b3b5ae8448aee2041bb
config: arm64-allyesconfig (attached as .config)
compiler: aarch64-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/ad01b68a345bdae6c24bd45720f4a01a2f9be49b
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Mike-Tipton/interconnect-Fix-sync-state-issues/20210626-053132
git checkout ad01b68a345bdae6c24bd45720f4a01a2f9be49b
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=arm64
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <[email protected]>
All warnings (new ones prefixed by >>):
drivers/interconnect/qcom/icc-rpmh.c: In function 'qcom_icc_set':
>> drivers/interconnect/qcom/icc-rpmh.c:87:24: warning: variable 'qn' set but not used [-Wunused-but-set-variable]
87 | struct qcom_icc_node *qn;
| ^~
vim +/qn +87 drivers/interconnect/qcom/icc-rpmh.c
976daac4a1c581 David Dai 2020-02-28 76
976daac4a1c581 David Dai 2020-02-28 77 /**
976daac4a1c581 David Dai 2020-02-28 78 * qcom_icc_set - set the constraints based on path
976daac4a1c581 David Dai 2020-02-28 79 * @src: source node for the path to set constraints on
976daac4a1c581 David Dai 2020-02-28 80 * @dst: destination node for the path to set constraints on
976daac4a1c581 David Dai 2020-02-28 81 *
976daac4a1c581 David Dai 2020-02-28 82 * Return: 0 on success, or an error code otherwise
976daac4a1c581 David Dai 2020-02-28 83 */
976daac4a1c581 David Dai 2020-02-28 84 int qcom_icc_set(struct icc_node *src, struct icc_node *dst)
976daac4a1c581 David Dai 2020-02-28 85 {
976daac4a1c581 David Dai 2020-02-28 86 struct qcom_icc_provider *qp;
266cd33b591385 Georgi Djakov 2020-10-22 @87 struct qcom_icc_node *qn;
976daac4a1c581 David Dai 2020-02-28 88 struct icc_node *node;
976daac4a1c581 David Dai 2020-02-28 89
976daac4a1c581 David Dai 2020-02-28 90 if (!src)
976daac4a1c581 David Dai 2020-02-28 91 node = dst;
976daac4a1c581 David Dai 2020-02-28 92 else
976daac4a1c581 David Dai 2020-02-28 93 node = src;
976daac4a1c581 David Dai 2020-02-28 94
976daac4a1c581 David Dai 2020-02-28 95 qp = to_qcom_provider(node->provider);
266cd33b591385 Georgi Djakov 2020-10-22 96 qn = node->data;
266cd33b591385 Georgi Djakov 2020-10-22 97
976daac4a1c581 David Dai 2020-02-28 98 qcom_icc_bcm_voter_commit(qp->voter);
976daac4a1c581 David Dai 2020-02-28 99
976daac4a1c581 David Dai 2020-02-28 100 return 0;
976daac4a1c581 David Dai 2020-02-28 101 }
976daac4a1c581 David Dai 2020-02-28 102 EXPORT_SYMBOL_GPL(qcom_icc_set);
976daac4a1c581 David Dai 2020-02-28 103
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/[email protected]
On 2021-06-26 02:58, Mike Tipton wrote:
> We currently only enforce BW floors for a subset of nodes in a path.
> All BCMs that need updating are queued in the pre_aggregate/aggregate
> phase. The first set() commits all queued BCMs and subsequent set()
> calls short-circuit without committing anything. Since the floor BW
> isn't set in sum_avg/max_peak until set(), then some BCMs are committed
> before their associated nodes reflect the floor.
>
> Set the floor as each node is being aggregated. This ensures that all
> all relevant floors are set before the BCMs are committed.
>
> Fixes: 266cd33b5913 ("interconnect: qcom: Ensure that the floor
> bandwidth value is enforced")
> Signed-off-by: Mike Tipton <[email protected]>
> ---
> drivers/interconnect/qcom/icc-rpmh.c | 10 +++++-----
> 1 file changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/interconnect/qcom/icc-rpmh.c
> b/drivers/interconnect/qcom/icc-rpmh.c
> index bf01d09dba6c..f118f57eae37 100644
> --- a/drivers/interconnect/qcom/icc-rpmh.c
> +++ b/drivers/interconnect/qcom/icc-rpmh.c
> @@ -57,6 +57,11 @@ int qcom_icc_aggregate(struct icc_node *node, u32
> tag, u32 avg_bw,
> qn->sum_avg[i] += avg_bw;
> qn->max_peak[i] = max_t(u32, qn->max_peak[i], peak_bw);
> }
> +
> + if (node->init_avg || node->init_peak) {
> + qn->sum_avg[i] = max_t(u64, qn->sum_avg[i], node->init_avg);
> + qn->max_peak[i] = max_t(u64, qn->max_peak[i], node->init_peak);
> + }
Hi Mike,
Original problem is BCMs not getting added to commit_list for unused
nodes, right? that is solved by moving *_bcm_voter_add() to
pre_aggregate().
I could not get why we need to do above change, we are enforcing node
votes with floor votes in framework + below code snippet that you
removed.
How would adding this code in qcom_icc_aggregate() make difference? Is
there any other issue that i am not to able to get?
> }
>
> *agg_avg += avg_bw;
> @@ -90,11 +95,6 @@ int qcom_icc_set(struct icc_node *src, struct
> icc_node *dst)
> qp = to_qcom_provider(node->provider);
> qn = node->data;
>
> - qn->sum_avg[QCOM_ICC_BUCKET_AMC] = max_t(u64,
> qn->sum_avg[QCOM_ICC_BUCKET_AMC],
> - node->avg_bw);
> - qn->max_peak[QCOM_ICC_BUCKET_AMC] = max_t(u64,
> qn->max_peak[QCOM_ICC_BUCKET_AMC],
> - node->peak_bw);
> -
> qcom_icc_bcm_voter_commit(qp->voter);
>
> return 0;
On 7/1/2021 11:48 AM, [email protected] wrote:
> On 2021-06-26 02:58, Mike Tipton wrote:
>> We currently only enforce BW floors for a subset of nodes in a path.
>> All BCMs that need updating are queued in the pre_aggregate/aggregate
>> phase. The first set() commits all queued BCMs and subsequent set()
>> calls short-circuit without committing anything. Since the floor BW
>> isn't set in sum_avg/max_peak until set(), then some BCMs are committed
>> before their associated nodes reflect the floor.
>>
>> Set the floor as each node is being aggregated. This ensures that all
>> all relevant floors are set before the BCMs are committed.
>>
>> Fixes: 266cd33b5913 ("interconnect: qcom: Ensure that the floor
>> bandwidth value is enforced")
>> Signed-off-by: Mike Tipton <[email protected]>
>> ---
>> drivers/interconnect/qcom/icc-rpmh.c | 10 +++++-----
>> 1 file changed, 5 insertions(+), 5 deletions(-)
>>
>> diff --git a/drivers/interconnect/qcom/icc-rpmh.c
>> b/drivers/interconnect/qcom/icc-rpmh.c
>> index bf01d09dba6c..f118f57eae37 100644
>> --- a/drivers/interconnect/qcom/icc-rpmh.c
>> +++ b/drivers/interconnect/qcom/icc-rpmh.c
>> @@ -57,6 +57,11 @@ int qcom_icc_aggregate(struct icc_node *node, u32
>> tag, u32 avg_bw,
>> qn->sum_avg[i] += avg_bw;
>> qn->max_peak[i] = max_t(u32, qn->max_peak[i], peak_bw);
>> }
>> +
>> + if (node->init_avg || node->init_peak) {
>> + qn->sum_avg[i] = max_t(u64, qn->sum_avg[i], node->init_avg);
>> + qn->max_peak[i] = max_t(u64, qn->max_peak[i],
>> node->init_peak);
>> + }
> Hi Mike,
> Original problem is BCMs not getting added to commit_list for unused
> nodes, right? that is solved by moving *_bcm_voter_add() to
> pre_aggregate().
> I could not get why we need to do above change, we are enforcing node
> votes with floor votes in framework + below code snippet that you removed.
> How would adding this code in qcom_icc_aggregate() make difference? Is
> there any other issue that i am not to able to get?
This series fixes a couple of separate issues. One is not removing the
initial floor after sync_state for unvoted paths, which the change to
add BCMs to the commit list in pre_aggregate addresses. The other issue
is not properly enforcing the initial floor *before* sync_state, which
this patch addresses.
We only commit to HW what we've aggregated in our internal,
provider-specific buckets (AMC, WAKE, SLEEP) during
pre_aggregate/aggregate. All BCMs in the path are added to the commit
list during this stage. Everything in the commit list is voted on the
*first* set() callback for the path. The commit list is empty for every
subsequent set() callback, so nothing actually happens except in the
first set().
The original snippet below in qcom_icc_set() partially re-aggregates AMC
with BW values from the icc_node struct. This is to capture the floor
enforced by the framework itself. However, this is only for a single
node in the path, which means it's also only for a single BCM in the
path. Since we commit all BCMs on the first set(), this means most BCMs
in the path don't see the floor BW at the time we commit them. This can
lead to some BCMs being under-voted, or in some cases completely off.
So, instead of aggregating the floor for each node in qcom_icc_set(),
this patch moves it to qcom_icc_aggregate(). This means all floors are
captured internally before committing the BCMs.
>> }
>>
>> *agg_avg += avg_bw;
>> @@ -90,11 +95,6 @@ int qcom_icc_set(struct icc_node *src, struct
>> icc_node *dst)
>> qp = to_qcom_provider(node->provider);
>> qn = node->data;
>>
>> - qn->sum_avg[QCOM_ICC_BUCKET_AMC] = max_t(u64,
>> qn->sum_avg[QCOM_ICC_BUCKET_AMC],
>> - node->avg_bw);
>> - qn->max_peak[QCOM_ICC_BUCKET_AMC] = max_t(u64,
>> qn->max_peak[QCOM_ICC_BUCKET_AMC],
>> - node->peak_bw);
>> -
>> qcom_icc_bcm_voter_commit(qp->voter);
>>
>> return 0;