Hi,
Changes since v5
================
1. Rename compatible (and files) to qcom,msm8998-llcc-bwmon as Rajendra suggested.
Keep the reviews/acks as the change is not significant.
2. Update comment in DTS, update description in bindings and in Kconfig.
Changes since v4
================
1. Patch #1 (binding): Use qcom,msm8998-cpu-bwmon fallback compatible, only one
interconnect. Rename to qcom,msm8998-cpu-bwmon.yaml. This reflects
discussion with Bjorn, about the proper fallback compatible. Driver was
tested only on SDM845, so only that one compatible is actually implemented.
Keep the reviews/acks as the change is not significant.
2. Patch #4 (DTS): Use qcom,msm8998-cpu-bwmon fallback compatible, only one
interconnect, use the LLCC bandwidth in OPP.
remove unused irq_enable (kbuild robot);
Changes since v3
================
1. Patch #2 (bwmon): remove unused irq_enable (kbuild robot);
split bwmon_clear() into clearing counters and interrupts, so bwmon_start()
does not clear the counters twice.
Changes since v2
================
1. Spent a lot of time on benchmarking and learning the BWMON behavior.
2. Drop PM/OPP patch - applied.
3. Patch #1: drop opp-avg-kBps.
4. Patch #2: Add several comments explaining pieces of code and BWMON, extend
commit msg with measurements, extend help message, add new #defines to document
some magic values, reorder bwmon clear/disable/enable operations to match
downstream source and document this with comments, fix unit count from 1 MB
to 65 kB.
5. Patch #4: drop opp-avg-kBps.
6. Add accumulated Rb tags.
Changes since v1
================
1. Add defconfig change.
2. Fix missing semicolon in MODULE_AUTHOR.
3. Add original downstream (msm-4.9 tree) copyrights to the driver.
Description
===========
BWMON is a data bandwidth monitor providing throughput/bandwidth over certain
interconnect links in a SoC. It might be used to gather current bus usage and
vote for interconnect bandwidth, thus adjusting the bus speed based on actual
usage.
The work is built on top of Thara Gopinath's patches with several cleanups,
changes and simplifications.
Cc: Rajendra Nayak <[email protected]>
Best regards,
Krzysztof
Krzysztof Kozlowski (4):
dt-bindings: interconnect: qcom,msm8998-cpu-bwmon: add BWMON device
soc: qcom: icc-bwmon: Add bandwidth monitoring driver
arm64: defconfig: enable Qualcomm Bandwidth Monitor
arm64: dts: qcom: sdm845: Add CPU BWMON
.../interconnect/qcom,msm8998-llcc-bwmon.yaml | 85 ++++
MAINTAINERS | 7 +
arch/arm64/boot/dts/qcom/sdm845.dtsi | 37 ++
arch/arm64/configs/defconfig | 1 +
drivers/soc/qcom/Kconfig | 15 +
drivers/soc/qcom/Makefile | 1 +
drivers/soc/qcom/icc-bwmon.c | 421 ++++++++++++++++++
7 files changed, 567 insertions(+)
create mode 100644 Documentation/devicetree/bindings/interconnect/qcom,msm8998-llcc-bwmon.yaml
create mode 100644 drivers/soc/qcom/icc-bwmon.c
--
2.34.1
Enable the Qualcomm Bandwidth Monitor to allow scaling interconnects
depending on bandwidth usage between CPU and memory. This is used
already on Qualcomm SDM845 SoC.
Signed-off-by: Krzysztof Kozlowski <[email protected]>
---
arch/arm64/configs/defconfig | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
index 6906b83f5e45..6edbcfd3f4ca 100644
--- a/arch/arm64/configs/defconfig
+++ b/arch/arm64/configs/defconfig
@@ -1096,6 +1096,7 @@ CONFIG_QCOM_SOCINFO=m
CONFIG_QCOM_STATS=m
CONFIG_QCOM_WCNSS_CTRL=m
CONFIG_QCOM_APR=m
+CONFIG_QCOM_ICC_BWMON=m
CONFIG_ARCH_R8A77995=y
CONFIG_ARCH_R8A77990=y
CONFIG_ARCH_R8A77950=y
--
2.34.1
Add bindings for the Qualcomm Bandwidth Monitor device providing
performance data on interconnects. The bindings describe only BWMON CPU
(version 4), e.g. the instance which appeared for the first on Qualcomm
MSM8998 SoC and is also used on SDM845. This BWMON device sits between
CPU and Last Level Cache Controller.
Signed-off-by: Krzysztof Kozlowski <[email protected]>
Reviewed-by: Rob Herring <[email protected]>
Acked-by: Georgi Djakov <[email protected]>
---
.../interconnect/qcom,msm8998-llcc-bwmon.yaml | 85 +++++++++++++++++++
1 file changed, 85 insertions(+)
create mode 100644 Documentation/devicetree/bindings/interconnect/qcom,msm8998-llcc-bwmon.yaml
diff --git a/Documentation/devicetree/bindings/interconnect/qcom,msm8998-llcc-bwmon.yaml b/Documentation/devicetree/bindings/interconnect/qcom,msm8998-llcc-bwmon.yaml
new file mode 100644
index 000000000000..76e09658d615
--- /dev/null
+++ b/Documentation/devicetree/bindings/interconnect/qcom,msm8998-llcc-bwmon.yaml
@@ -0,0 +1,85 @@
+# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/interconnect/qcom,msm8998-llcc-bwmon.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Qualcomm Interconnect Bandwidth Monitor
+
+maintainers:
+ - Krzysztof Kozlowski <[email protected]>
+
+description: |
+ Bandwidth Monitor measures current throughput on buses between various NoC
+ fabrics and provides information when it crosses configured thresholds.
+
+ Certain SoCs might have more than one Bandwidth Monitors, for example on SDM845::
+ - Measuring the bandwidth between CPUs and Last Level Cache Controller -
+ called LLCC BWMON,
+ - Measuring the bandwidth between Last Level Cache Controller and memory (DDR).
+
+properties:
+ compatible:
+ oneOf:
+ - items:
+ - enum:
+ - qcom,sdm845-llcc-bwmon
+ - const: qcom,msm8998-llcc-bwmon
+ - const: qcom,msm8998-llcc-bwmon # BWMON v4
+
+ interconnects:
+ maxItems: 1
+
+ interrupts:
+ maxItems: 1
+
+ operating-points-v2: true
+ opp-table: true
+
+ reg:
+ # BWMON v4 (currently described) and BWMON v5 use one register address
+ # space. BWMON v2 uses two register spaces - not yet described.
+ maxItems: 1
+
+required:
+ - compatible
+ - interconnects
+ - interrupts
+ - operating-points-v2
+ - opp-table
+ - reg
+
+additionalProperties: false
+
+examples:
+ - |
+ #include <dt-bindings/interconnect/qcom,sdm845.h>
+ #include <dt-bindings/interrupt-controller/arm-gic.h>
+
+ pmu@1436400 {
+ compatible = "qcom,sdm845-llcc-bwmon", "qcom,msm8998-llcc-bwmon";
+ reg = <0x01436400 0x600>;
+ interrupts = <GIC_SPI 581 IRQ_TYPE_LEVEL_HIGH>;
+ interconnects = <&gladiator_noc MASTER_APPSS_PROC 3 &mem_noc SLAVE_LLCC 3>;
+
+ operating-points-v2 = <&llcc_bwmon_opp_table>;
+
+ llcc_bwmon_opp_table: opp-table {
+ compatible = "operating-points-v2";
+ opp-0 {
+ opp-peak-kBps = <4800000>;
+ };
+ opp-1 {
+ opp-peak-kBps = <9216000>;
+ };
+ opp-2 {
+ opp-peak-kBps = <15052800>;
+ };
+ opp-3 {
+ opp-peak-kBps = <20889600>;
+ };
+ opp-4 {
+ opp-peak-kBps = <25497600>;
+ };
+ };
+ };
--
2.34.1
Add device node for CPU-memory BWMON device (bandwidth monitoring) on
SDM845 measuring bandwidth between CPU (gladiator_noc) and Last Level
Cache (memnoc). Usage of this BWMON allows to remove fixed bandwidth
votes from cpufreq (CPU nodes) thus achieve high memory throughput even
with lower CPU frequencies.
Co-developed-by: Thara Gopinath <[email protected]>
Signed-off-by: Thara Gopinath <[email protected]>
Signed-off-by: Krzysztof Kozlowski <[email protected]>
---
arch/arm64/boot/dts/qcom/sdm845.dtsi | 37 ++++++++++++++++++++++++++++
1 file changed, 37 insertions(+)
diff --git a/arch/arm64/boot/dts/qcom/sdm845.dtsi b/arch/arm64/boot/dts/qcom/sdm845.dtsi
index 83e8b63f0910..1872fea04785 100644
--- a/arch/arm64/boot/dts/qcom/sdm845.dtsi
+++ b/arch/arm64/boot/dts/qcom/sdm845.dtsi
@@ -2026,6 +2026,43 @@ llcc: system-cache-controller@1100000 {
interrupts = <GIC_SPI 582 IRQ_TYPE_LEVEL_HIGH>;
};
+ pmu@1436400 {
+ compatible = "qcom,sdm845-llcc-bwmon", "qcom,msm8998-llcc-bwmon";
+ reg = <0 0x01436400 0 0x600>;
+ interrupts = <GIC_SPI 581 IRQ_TYPE_LEVEL_HIGH>;
+ interconnects = <&gladiator_noc MASTER_APPSS_PROC 3 &mem_noc SLAVE_LLCC 3>;
+
+ operating-points-v2 = <&llcc_bwmon_opp_table>;
+
+ llcc_bwmon_opp_table: opp-table {
+ compatible = "operating-points-v2";
+
+ /*
+ * The interconnect path bandwidth taken from
+ * cpu4_opp_table bandwidth for OSM L3
+ * interconnect. This also matches the OSM L3
+ * from bandwidth table of qcom,cpu4-l3lat-mon
+ * (qcom,core-dev-table, bus width: 16 bytes)
+ * from msm-4.9 downstream kernel.
+ */
+ opp-0 {
+ opp-peak-kBps = <4800000>;
+ };
+ opp-1 {
+ opp-peak-kBps = <9216000>;
+ };
+ opp-2 {
+ opp-peak-kBps = <15052800>;
+ };
+ opp-3 {
+ opp-peak-kBps = <20889600>;
+ };
+ opp-4 {
+ opp-peak-kBps = <25497600>;
+ };
+ };
+ };
+
pcie0: pci@1c00000 {
compatible = "qcom,pcie-sdm845";
reg = <0 0x01c00000 0 0x2000>,
--
2.34.1
Bandwidth monitoring (BWMON) sits between various subsytems like CPU,
GPU, Last Level caches and memory subsystem. The BWMON can be
configured to monitor the data throuhput between memory and other
subsytems. The throughput is measured within specified sampling window
and is used to vote for corresponding interconnect bandwidth.
Current implementation brings support for BWMON v4, used for example on
SDM845 to measure bandwidth between CPU (gladiator_noc) and Last Level
Cache (memnoc). Usage of this BWMON allows to remove fixed bandwidth
votes from cpufreq (CPU nodes) thus achieve high memory throughput even
with lower CPU frequencies.
Co-developed-by: Thara Gopinath <[email protected]>
Signed-off-by: Thara Gopinath <[email protected]>
Signed-off-by: Krzysztof Kozlowski <[email protected]>
---
MAINTAINERS | 7 +
drivers/soc/qcom/Kconfig | 15 ++
drivers/soc/qcom/Makefile | 1 +
drivers/soc/qcom/icc-bwmon.c | 421 +++++++++++++++++++++++++++++++++++
4 files changed, 444 insertions(+)
create mode 100644 drivers/soc/qcom/icc-bwmon.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 6157e706ed02..d6e4d2954868 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -16376,6 +16376,13 @@ S: Maintained
F: Documentation/devicetree/bindings/i2c/i2c-qcom-cci.txt
F: drivers/i2c/busses/i2c-qcom-cci.c
+QUALCOMM INTERCONNECT BWMON DRIVER
+M: Krzysztof Kozlowski <[email protected]>
+L: [email protected]
+S: Maintained
+F: Documentation/devicetree/bindings/interconnect/qcom,msm8998-llcc-bwmon.yaml
+F: drivers/soc/qcom/icc-bwmon.c
+
QUALCOMM IOMMU
M: Rob Clark <[email protected]>
L: [email protected]
diff --git a/drivers/soc/qcom/Kconfig b/drivers/soc/qcom/Kconfig
index e718b8735444..2c8091535bf7 100644
--- a/drivers/soc/qcom/Kconfig
+++ b/drivers/soc/qcom/Kconfig
@@ -228,4 +228,19 @@ config QCOM_APR
application processor and QDSP6. APR is
used by audio driver to configure QDSP6
ASM, ADM and AFE modules.
+
+config QCOM_ICC_BWMON
+ tristate "QCOM Interconnect Bandwidth Monitor driver"
+ depends on ARCH_QCOM || COMPILE_TEST
+ select PM_OPP
+ help
+ Sets up driver monitoring bandwidth on various interconnects and
+ based on that voting for interconnect bandwidth, adjusting their
+ speed to current demand.
+ Current implementation brings support for BWMON v4, used for example
+ on SDM845 to measure bandwidth between CPU (gladiator_noc) and Last
+ Level Cache (memnoc). Usage of this BWMON allows to remove some of
+ the fixed bandwidth votes from cpufreq (CPU nodes) thus achieve high
+ memory throughput even with lower CPU frequencies.
+
endmenu
diff --git a/drivers/soc/qcom/Makefile b/drivers/soc/qcom/Makefile
index 70d5de69fd7b..d66604aff2b0 100644
--- a/drivers/soc/qcom/Makefile
+++ b/drivers/soc/qcom/Makefile
@@ -28,3 +28,4 @@ obj-$(CONFIG_QCOM_LLCC) += llcc-qcom.o
obj-$(CONFIG_QCOM_RPMHPD) += rpmhpd.o
obj-$(CONFIG_QCOM_RPMPD) += rpmpd.o
obj-$(CONFIG_QCOM_KRYO_L2_ACCESSORS) += kryo-l2-accessors.o
+obj-$(CONFIG_QCOM_ICC_BWMON) += icc-bwmon.o
diff --git a/drivers/soc/qcom/icc-bwmon.c b/drivers/soc/qcom/icc-bwmon.c
new file mode 100644
index 000000000000..b4b1435704c2
--- /dev/null
+++ b/drivers/soc/qcom/icc-bwmon.c
@@ -0,0 +1,421 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2014-2018, The Linux Foundation. All rights reserved.
+ * Copyright (C) 2021-2022 Linaro Ltd
+ * Author: Krzysztof Kozlowski <[email protected]>, based on
+ * previous work of Thara Gopinath and msm-4.9 downstream sources.
+ */
+#include <linux/interconnect.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/of_device.h>
+#include <linux/platform_device.h>
+#include <linux/pm_opp.h>
+#include <linux/sizes.h>
+
+/*
+ * The BWMON samples data throughput within 'sample_ms' time. With three
+ * configurable thresholds (Low, Medium and High) gives four windows (called
+ * zones) of current bandwidth:
+ *
+ * Zone 0: byte count < THRES_LO
+ * Zone 1: THRES_LO < byte count < THRES_MED
+ * Zone 2: THRES_MED < byte count < THRES_HIGH
+ * Zone 3: THRES_HIGH < byte count
+ *
+ * Zones 0 and 2 are not used by this driver.
+ */
+
+/* Internal sampling clock frequency */
+#define HW_TIMER_HZ 19200000
+
+#define BWMON_GLOBAL_IRQ_STATUS 0x0
+#define BWMON_GLOBAL_IRQ_CLEAR 0x8
+#define BWMON_GLOBAL_IRQ_ENABLE 0xc
+#define BWMON_GLOBAL_IRQ_ENABLE_ENABLE BIT(0)
+
+#define BWMON_IRQ_STATUS 0x100
+#define BWMON_IRQ_STATUS_ZONE_SHIFT 4
+#define BWMON_IRQ_CLEAR 0x108
+#define BWMON_IRQ_ENABLE 0x10c
+#define BWMON_IRQ_ENABLE_ZONE1_SHIFT 5
+#define BWMON_IRQ_ENABLE_ZONE2_SHIFT 6
+#define BWMON_IRQ_ENABLE_ZONE3_SHIFT 7
+#define BWMON_IRQ_ENABLE_MASK (BIT(BWMON_IRQ_ENABLE_ZONE1_SHIFT) | \
+ BIT(BWMON_IRQ_ENABLE_ZONE3_SHIFT))
+
+#define BWMON_ENABLE 0x2a0
+#define BWMON_ENABLE_ENABLE BIT(0)
+
+#define BWMON_CLEAR 0x2a4
+#define BWMON_CLEAR_CLEAR BIT(0)
+
+#define BWMON_SAMPLE_WINDOW 0x2a8
+#define BWMON_THRESHOLD_HIGH 0x2ac
+#define BWMON_THRESHOLD_MED 0x2b0
+#define BWMON_THRESHOLD_LOW 0x2b4
+
+#define BWMON_ZONE_ACTIONS 0x2b8
+/*
+ * Actions to perform on some zone 'z' when current zone hits the threshold:
+ * Increment counter of zone 'z'
+ */
+#define BWMON_ZONE_ACTIONS_INCREMENT(z) (0x2 << ((z) * 2))
+/* Clear counter of zone 'z' */
+#define BWMON_ZONE_ACTIONS_CLEAR(z) (0x1 << ((z) * 2))
+
+/* Zone 0 threshold hit: Clear zone count */
+#define BWMON_ZONE_ACTIONS_ZONE0 (BWMON_ZONE_ACTIONS_CLEAR(0))
+
+/* Zone 1 threshold hit: Increment zone count & clear lower zones */
+#define BWMON_ZONE_ACTIONS_ZONE1 (BWMON_ZONE_ACTIONS_INCREMENT(1) | \
+ BWMON_ZONE_ACTIONS_CLEAR(0))
+
+/* Zone 2 threshold hit: Increment zone count & clear lower zones */
+#define BWMON_ZONE_ACTIONS_ZONE2 (BWMON_ZONE_ACTIONS_INCREMENT(2) | \
+ BWMON_ZONE_ACTIONS_CLEAR(1) | \
+ BWMON_ZONE_ACTIONS_CLEAR(0))
+
+/* Zone 3 threshold hit: Increment zone count & clear lower zones */
+#define BWMON_ZONE_ACTIONS_ZONE3 (BWMON_ZONE_ACTIONS_INCREMENT(3) | \
+ BWMON_ZONE_ACTIONS_CLEAR(2) | \
+ BWMON_ZONE_ACTIONS_CLEAR(1) | \
+ BWMON_ZONE_ACTIONS_CLEAR(0))
+/* Value for BWMON_ZONE_ACTIONS */
+#define BWMON_ZONE_ACTIONS_DEFAULT (BWMON_ZONE_ACTIONS_ZONE0 | \
+ BWMON_ZONE_ACTIONS_ZONE1 << 8 | \
+ BWMON_ZONE_ACTIONS_ZONE2 << 16 | \
+ BWMON_ZONE_ACTIONS_ZONE3 << 24)
+
+/*
+ * There is no clear documentation/explanation of BWMON_THRESHOLD_COUNT
+ * register. Based on observations, this is number of times one threshold has to
+ * be reached, to trigger interrupt in given zone.
+ *
+ * 0xff are maximum values meant to ignore the zones 0 and 2.
+ */
+#define BWMON_THRESHOLD_COUNT 0x2bc
+#define BWMON_THRESHOLD_COUNT_ZONE1_SHIFT 8
+#define BWMON_THRESHOLD_COUNT_ZONE2_SHIFT 16
+#define BWMON_THRESHOLD_COUNT_ZONE3_SHIFT 24
+#define BWMON_THRESHOLD_COUNT_ZONE0_DEFAULT 0xff
+#define BWMON_THRESHOLD_COUNT_ZONE2_DEFAULT 0xff
+
+/* BWMONv4 count registers use count unit of 64 kB */
+#define BWMON_COUNT_UNIT_KB 64
+#define BWMON_ZONE_COUNT 0x2d8
+#define BWMON_ZONE_MAX(zone) (0x2e0 + 4 * (zone))
+
+struct icc_bwmon_data {
+ unsigned int sample_ms;
+ unsigned int default_highbw_kbps;
+ unsigned int default_medbw_kbps;
+ unsigned int default_lowbw_kbps;
+ u8 zone1_thres_count;
+ u8 zone3_thres_count;
+};
+
+struct icc_bwmon {
+ struct device *dev;
+ void __iomem *base;
+ int irq;
+
+ unsigned int default_lowbw_kbps;
+ unsigned int sample_ms;
+ unsigned int max_bw_kbps;
+ unsigned int min_bw_kbps;
+ unsigned int target_kbps;
+ unsigned int current_kbps;
+};
+
+static void bwmon_clear_counters(struct icc_bwmon *bwmon)
+{
+ /*
+ * Clear counters. The order and barriers are
+ * important. Quoting downstream Qualcomm msm-4.9 tree:
+ *
+ * The counter clear and IRQ clear bits are not in the same 4KB
+ * region. So, we need to make sure the counter clear is completed
+ * before we try to clear the IRQ or do any other counter operations.
+ */
+ writel(BWMON_CLEAR_CLEAR, bwmon->base + BWMON_CLEAR);
+}
+
+static void bwmon_clear_irq(struct icc_bwmon *bwmon)
+{
+ /*
+ * Clear zone and global interrupts. The order and barriers are
+ * important. Quoting downstream Qualcomm msm-4.9 tree:
+ *
+ * Synchronize the local interrupt clear in mon_irq_clear()
+ * with the global interrupt clear here. Otherwise, the CPU
+ * may reorder the two writes and clear the global interrupt
+ * before the local interrupt, causing the global interrupt
+ * to be retriggered by the local interrupt still being high.
+ *
+ * Similarly, because the global registers are in a different
+ * region than the local registers, we need to ensure any register
+ * writes to enable the monitor after this call are ordered with the
+ * clearing here so that local writes don't happen before the
+ * interrupt is cleared.
+ */
+ writel(BWMON_IRQ_ENABLE_MASK, bwmon->base + BWMON_IRQ_CLEAR);
+ writel(BIT(0), bwmon->base + BWMON_GLOBAL_IRQ_CLEAR);
+}
+
+static void bwmon_disable(struct icc_bwmon *bwmon)
+{
+ /* Disable interrupts. Strict ordering, see bwmon_clear_irq(). */
+ writel(0x0, bwmon->base + BWMON_GLOBAL_IRQ_ENABLE);
+ writel(0x0, bwmon->base + BWMON_IRQ_ENABLE);
+
+ /*
+ * Disable bwmon. Must happen before bwmon_clear_irq() to avoid spurious
+ * IRQ.
+ */
+ writel(0x0, bwmon->base + BWMON_ENABLE);
+}
+
+static void bwmon_enable(struct icc_bwmon *bwmon, unsigned int irq_enable)
+{
+ /* Enable interrupts */
+ writel(BWMON_GLOBAL_IRQ_ENABLE_ENABLE,
+ bwmon->base + BWMON_GLOBAL_IRQ_ENABLE);
+ writel(irq_enable, bwmon->base + BWMON_IRQ_ENABLE);
+
+ /* Enable bwmon */
+ writel(BWMON_ENABLE_ENABLE, bwmon->base + BWMON_ENABLE);
+}
+
+static unsigned int bwmon_kbps_to_count(unsigned int kbps)
+{
+ return kbps / BWMON_COUNT_UNIT_KB;
+}
+
+static void bwmon_set_threshold(struct icc_bwmon *bwmon, unsigned int reg,
+ unsigned int kbps)
+{
+ unsigned int thres;
+
+ thres = mult_frac(bwmon_kbps_to_count(kbps), bwmon->sample_ms,
+ MSEC_PER_SEC);
+ writel_relaxed(thres, bwmon->base + reg);
+}
+
+static void bwmon_start(struct icc_bwmon *bwmon,
+ const struct icc_bwmon_data *data)
+{
+ unsigned int thres_count;
+ int window;
+
+ bwmon_clear_counters(bwmon);
+
+ window = mult_frac(bwmon->sample_ms, HW_TIMER_HZ, MSEC_PER_SEC);
+ /* Maximum sampling window: 0xfffff */
+ writel_relaxed(window, bwmon->base + BWMON_SAMPLE_WINDOW);
+
+ bwmon_set_threshold(bwmon, BWMON_THRESHOLD_HIGH,
+ data->default_highbw_kbps);
+ bwmon_set_threshold(bwmon, BWMON_THRESHOLD_MED,
+ data->default_medbw_kbps);
+ bwmon_set_threshold(bwmon, BWMON_THRESHOLD_LOW,
+ data->default_lowbw_kbps);
+
+ thres_count = data->zone3_thres_count << BWMON_THRESHOLD_COUNT_ZONE3_SHIFT |
+ BWMON_THRESHOLD_COUNT_ZONE2_DEFAULT << BWMON_THRESHOLD_COUNT_ZONE2_SHIFT |
+ data->zone1_thres_count << BWMON_THRESHOLD_COUNT_ZONE1_SHIFT |
+ BWMON_THRESHOLD_COUNT_ZONE0_DEFAULT;
+ writel_relaxed(thres_count, bwmon->base + BWMON_THRESHOLD_COUNT);
+ writel_relaxed(BWMON_ZONE_ACTIONS_DEFAULT,
+ bwmon->base + BWMON_ZONE_ACTIONS);
+ /* Write barriers in bwmon_clear_irq() */
+
+ bwmon_clear_irq(bwmon);
+ bwmon_enable(bwmon, BWMON_IRQ_ENABLE_MASK);
+}
+
+static irqreturn_t bwmon_intr(int irq, void *dev_id)
+{
+ struct icc_bwmon *bwmon = dev_id;
+ unsigned int status, max;
+ int zone;
+
+ status = readl(bwmon->base + BWMON_IRQ_STATUS);
+ status &= BWMON_IRQ_ENABLE_MASK;
+ if (!status) {
+ /*
+ * Only zone 1 and zone 3 interrupts are enabled but zone 2
+ * threshold could be hit and trigger interrupt even if not
+ * enabled.
+ * Such spurious interrupt might come with valuable max count or
+ * not, so solution would be to always check all
+ * BWMON_ZONE_MAX() registers to find the highest value.
+ * Such case is currently ignored.
+ */
+ return IRQ_NONE;
+ }
+
+ bwmon_disable(bwmon);
+
+ zone = get_bitmask_order(status >> BWMON_IRQ_STATUS_ZONE_SHIFT) - 1;
+ /*
+ * Zone max bytes count register returns count units within sampling
+ * window. Downstream kernel for BWMONv4 (called BWMON type 2 in
+ * downstream) always increments the max bytes count by one.
+ */
+ max = readl(bwmon->base + BWMON_ZONE_MAX(zone)) + 1;
+ max *= BWMON_COUNT_UNIT_KB;
+ bwmon->target_kbps = mult_frac(max, MSEC_PER_SEC, bwmon->sample_ms);
+
+ return IRQ_WAKE_THREAD;
+}
+
+static irqreturn_t bwmon_intr_thread(int irq, void *dev_id)
+{
+ struct icc_bwmon *bwmon = dev_id;
+ unsigned int irq_enable = 0;
+ struct dev_pm_opp *opp, *target_opp;
+ unsigned int bw_kbps, up_kbps, down_kbps;
+
+ bw_kbps = bwmon->target_kbps;
+
+ target_opp = dev_pm_opp_find_bw_ceil(bwmon->dev, &bw_kbps, 0);
+ if (IS_ERR(target_opp) && PTR_ERR(target_opp) == -ERANGE)
+ target_opp = dev_pm_opp_find_bw_floor(bwmon->dev, &bw_kbps, 0);
+
+ bwmon->target_kbps = bw_kbps;
+
+ bw_kbps--;
+ opp = dev_pm_opp_find_bw_floor(bwmon->dev, &bw_kbps, 0);
+ if (IS_ERR(opp) && PTR_ERR(opp) == -ERANGE)
+ down_kbps = bwmon->target_kbps;
+ else
+ down_kbps = bw_kbps;
+
+ up_kbps = bwmon->target_kbps + 1;
+
+ if (bwmon->target_kbps >= bwmon->max_bw_kbps)
+ irq_enable = BIT(BWMON_IRQ_ENABLE_ZONE1_SHIFT);
+ else if (bwmon->target_kbps <= bwmon->min_bw_kbps)
+ irq_enable = BIT(BWMON_IRQ_ENABLE_ZONE3_SHIFT);
+ else
+ irq_enable = BWMON_IRQ_ENABLE_MASK;
+
+ bwmon_set_threshold(bwmon, BWMON_THRESHOLD_HIGH, up_kbps);
+ bwmon_set_threshold(bwmon, BWMON_THRESHOLD_MED, down_kbps);
+ /* Write barriers in bwmon_clear_counters() */
+ bwmon_clear_counters(bwmon);
+ bwmon_clear_irq(bwmon);
+ bwmon_enable(bwmon, irq_enable);
+
+ if (bwmon->target_kbps == bwmon->current_kbps)
+ goto out;
+
+ dev_pm_opp_set_opp(bwmon->dev, target_opp);
+ bwmon->current_kbps = bwmon->target_kbps;
+
+out:
+ dev_pm_opp_put(target_opp);
+ if (!IS_ERR(opp))
+ dev_pm_opp_put(opp);
+
+ return IRQ_HANDLED;
+}
+
+static int bwmon_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct dev_pm_opp *opp;
+ struct icc_bwmon *bwmon;
+ const struct icc_bwmon_data *data;
+ int ret;
+
+ bwmon = devm_kzalloc(dev, sizeof(*bwmon), GFP_KERNEL);
+ if (!bwmon)
+ return -ENOMEM;
+
+ data = of_device_get_match_data(dev);
+
+ bwmon->base = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(bwmon->base)) {
+ dev_err(dev, "failed to map bwmon registers\n");
+ return PTR_ERR(bwmon->base);
+ }
+
+ bwmon->irq = platform_get_irq(pdev, 0);
+ if (bwmon->irq < 0) {
+ dev_err(dev, "failed to acquire bwmon IRQ\n");
+ return bwmon->irq;
+ }
+
+ ret = devm_pm_opp_of_add_table(dev);
+ if (ret)
+ return dev_err_probe(dev, ret, "failed to add OPP table\n");
+
+ bwmon->max_bw_kbps = UINT_MAX;
+ opp = dev_pm_opp_find_bw_floor(dev, &bwmon->max_bw_kbps, 0);
+ if (IS_ERR(opp))
+ return dev_err_probe(dev, ret, "failed to find max peak bandwidth\n");
+
+ bwmon->min_bw_kbps = 0;
+ opp = dev_pm_opp_find_bw_ceil(dev, &bwmon->min_bw_kbps, 0);
+ if (IS_ERR(opp))
+ return dev_err_probe(dev, ret, "failed to find min peak bandwidth\n");
+
+ bwmon->sample_ms = data->sample_ms;
+ bwmon->default_lowbw_kbps = data->default_lowbw_kbps;
+ bwmon->dev = dev;
+
+ bwmon_disable(bwmon);
+ ret = devm_request_threaded_irq(dev, bwmon->irq, bwmon_intr,
+ bwmon_intr_thread,
+ IRQF_ONESHOT, dev_name(dev), bwmon);
+ if (ret)
+ return dev_err_probe(dev, ret, "failed to request IRQ\n");
+
+ platform_set_drvdata(pdev, bwmon);
+ bwmon_start(bwmon, data);
+
+ return 0;
+}
+
+static int bwmon_remove(struct platform_device *pdev)
+{
+ struct icc_bwmon *bwmon = platform_get_drvdata(pdev);
+
+ bwmon_disable(bwmon);
+
+ return 0;
+}
+
+/* BWMON v4 */
+static const struct icc_bwmon_data sdm845_bwmon_data = {
+ .sample_ms = 4,
+ .default_highbw_kbps = 4800 * 1024, /* 4.8 GBps */
+ .default_medbw_kbps = 512 * 1024, /* 512 MBps */
+ .default_lowbw_kbps = 0,
+ .zone1_thres_count = 16,
+ .zone3_thres_count = 1,
+};
+
+static const struct of_device_id bwmon_of_match[] = {
+ { .compatible = "qcom,sdm845-llcc-bwmon", .data = &sdm845_bwmon_data },
+ {}
+};
+MODULE_DEVICE_TABLE(of, bwmon_of_match);
+
+static struct platform_driver bwmon_driver = {
+ .probe = bwmon_probe,
+ .remove = bwmon_remove,
+ .driver = {
+ .name = "qcom-bwmon",
+ .of_match_table = bwmon_of_match,
+ },
+};
+module_platform_driver(bwmon_driver);
+
+MODULE_AUTHOR("Krzysztof Kozlowski <[email protected]>");
+MODULE_DESCRIPTION("QCOM BWMON driver");
+MODULE_LICENSE("GPL");
--
2.34.1
On 30/06/2022 13:14, Rajendra Nayak wrote:
>
> On 6/29/2022 7:32 PM, Krzysztof Kozlowski wrote:
>> Add bindings for the Qualcomm Bandwidth Monitor device providing
>> performance data on interconnects. The bindings describe only BWMON CPU
>> (version 4), e.g. the instance which appeared for the first on Qualcomm
>> MSM8998 SoC and is also used on SDM845. This BWMON device sits between
>> CPU and Last Level Cache Controller.
>>
>> Signed-off-by: Krzysztof Kozlowski <[email protected]>
>> Reviewed-by: Rob Herring <[email protected]>
>> Acked-by: Georgi Djakov <[email protected]>
>> ---
>> .../interconnect/qcom,msm8998-llcc-bwmon.yaml | 85 +++++++++++++++++++
>> 1 file changed, 85 insertions(+)
>> create mode 100644 Documentation/devicetree/bindings/interconnect/qcom,msm8998-llcc-bwmon.yaml
>>
>> diff --git a/Documentation/devicetree/bindings/interconnect/qcom,msm8998-llcc-bwmon.yaml b/Documentation/devicetree/bindings/interconnect/qcom,msm8998-llcc-bwmon.yaml
>> new file mode 100644
>> index 000000000000..76e09658d615
>> --- /dev/null
>> +++ b/Documentation/devicetree/bindings/interconnect/qcom,msm8998-llcc-bwmon.yaml
>> @@ -0,0 +1,85 @@
>> +# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause
>> +%YAML 1.2
>> +---
>> +$id: http://devicetree.org/schemas/interconnect/qcom,msm8998-llcc-bwmon.yaml#
>> +$schema: http://devicetree.org/meta-schemas/core.yaml#
>> +
>> +title: Qualcomm Interconnect Bandwidth Monitor
>> +
>> +maintainers:
>> + - Krzysztof Kozlowski <[email protected]>
>> +
>> +description: |
>> + Bandwidth Monitor measures current throughput on buses between various NoC
>> + fabrics and provides information when it crosses configured thresholds.
>> +
>> + Certain SoCs might have more than one Bandwidth Monitors, for example on SDM845::
>> + - Measuring the bandwidth between CPUs and Last Level Cache Controller -
>> + called LLCC BWMON,
>> + - Measuring the bandwidth between Last Level Cache Controller and memory (DDR).
>> +
>> +properties:
>> + compatible:
>> + oneOf:
>> + - items:
>> + - enum:
>> + - qcom,sdm845-llcc-bwmon
>> + - const: qcom,msm8998-llcc-bwmon
>> + - const: qcom,msm8998-llcc-bwmon # BWMON v4
>> +
>> + interconnects:
>> + maxItems: 1
>> +
>> + interrupts:
>> + maxItems: 1
>> +
>> + operating-points-v2: true
>> + opp-table: true
>> +
>> + reg:
>> + # BWMON v4 (currently described) and BWMON v5 use one register address
>> + # space. BWMON v2 uses two register spaces - not yet described.
>> + maxItems: 1
>> +
>> +required:
>> + - compatible
>> + - interconnects
>> + - interrupts
>> + - operating-points-v2
>> + - opp-table
>> + - reg
>> +
>> +additionalProperties: false
>> +
>> +examples:
>> + - |
>> + #include <dt-bindings/interconnect/qcom,sdm845.h>
>> + #include <dt-bindings/interrupt-controller/arm-gic.h>
>> +
>> + pmu@1436400 {
>> + compatible = "qcom,sdm845-llcc-bwmon", "qcom,msm8998-llcc-bwmon";
>
> so with this compatible fallback scheme, I am trying to understand what
> do I need to do if I have to add support for another SoC for instance.
>
> I just update the binding with the new SoC compatible (lets say qcom,sc7280-llcc-bwmon)
> and in the device tree node use it as
> compatible = "qcom,sc7280-llcc-bwmon", "qcom,sdm845-llcc-bwmon", "qcom,msm8998-llcc-bwmon";
> without any updates in the driver?
I expect:
"qcom,sc7280-llcc-bwmon", "qcom,msm8998-llcc-bwmon";
and you need to add sc7280 compatible to the driver. The actual proper
solution in my patch would be to use msm8998 compatible in the driver,
but I did not test MSM8998.
Maybe we should switch to that anyway?
Best regards,
Krzysztof
On 6/30/2022 4:53 PM, Krzysztof Kozlowski wrote:
> On 30/06/2022 13:14, Rajendra Nayak wrote:
>>
>> On 6/29/2022 7:32 PM, Krzysztof Kozlowski wrote:
>>> Add bindings for the Qualcomm Bandwidth Monitor device providing
>>> performance data on interconnects. The bindings describe only BWMON CPU
>>> (version 4), e.g. the instance which appeared for the first on Qualcomm
>>> MSM8998 SoC and is also used on SDM845. This BWMON device sits between
>>> CPU and Last Level Cache Controller.
>>>
>>> Signed-off-by: Krzysztof Kozlowski <[email protected]>
>>> Reviewed-by: Rob Herring <[email protected]>
>>> Acked-by: Georgi Djakov <[email protected]>
>>> ---
>>> .../interconnect/qcom,msm8998-llcc-bwmon.yaml | 85 +++++++++++++++++++
>>> 1 file changed, 85 insertions(+)
>>> create mode 100644 Documentation/devicetree/bindings/interconnect/qcom,msm8998-llcc-bwmon.yaml
>>>
>>> diff --git a/Documentation/devicetree/bindings/interconnect/qcom,msm8998-llcc-bwmon.yaml b/Documentation/devicetree/bindings/interconnect/qcom,msm8998-llcc-bwmon.yaml
>>> new file mode 100644
>>> index 000000000000..76e09658d615
>>> --- /dev/null
>>> +++ b/Documentation/devicetree/bindings/interconnect/qcom,msm8998-llcc-bwmon.yaml
>>> @@ -0,0 +1,85 @@
>>> +# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause
>>> +%YAML 1.2
>>> +---
>>> +$id: http://devicetree.org/schemas/interconnect/qcom,msm8998-llcc-bwmon.yaml#
>>> +$schema: http://devicetree.org/meta-schemas/core.yaml#
>>> +
>>> +title: Qualcomm Interconnect Bandwidth Monitor
>>> +
>>> +maintainers:
>>> + - Krzysztof Kozlowski <[email protected]>
>>> +
>>> +description: |
>>> + Bandwidth Monitor measures current throughput on buses between various NoC
>>> + fabrics and provides information when it crosses configured thresholds.
>>> +
>>> + Certain SoCs might have more than one Bandwidth Monitors, for example on SDM845::
>>> + - Measuring the bandwidth between CPUs and Last Level Cache Controller -
>>> + called LLCC BWMON,
>>> + - Measuring the bandwidth between Last Level Cache Controller and memory (DDR).
>>> +
>>> +properties:
>>> + compatible:
>>> + oneOf:
>>> + - items:
>>> + - enum:
>>> + - qcom,sdm845-llcc-bwmon
>>> + - const: qcom,msm8998-llcc-bwmon
>>> + - const: qcom,msm8998-llcc-bwmon # BWMON v4
>>> +
>>> + interconnects:
>>> + maxItems: 1
>>> +
>>> + interrupts:
>>> + maxItems: 1
>>> +
>>> + operating-points-v2: true
>>> + opp-table: true
>>> +
>>> + reg:
>>> + # BWMON v4 (currently described) and BWMON v5 use one register address
>>> + # space. BWMON v2 uses two register spaces - not yet described.
>>> + maxItems: 1
>>> +
>>> +required:
>>> + - compatible
>>> + - interconnects
>>> + - interrupts
>>> + - operating-points-v2
>>> + - opp-table
>>> + - reg
>>> +
>>> +additionalProperties: false
>>> +
>>> +examples:
>>> + - |
>>> + #include <dt-bindings/interconnect/qcom,sdm845.h>
>>> + #include <dt-bindings/interrupt-controller/arm-gic.h>
>>> +
>>> + pmu@1436400 {
>>> + compatible = "qcom,sdm845-llcc-bwmon", "qcom,msm8998-llcc-bwmon";
>>
>> so with this compatible fallback scheme, I am trying to understand what
>> do I need to do if I have to add support for another SoC for instance.
>>
>> I just update the binding with the new SoC compatible (lets say qcom,sc7280-llcc-bwmon)
>> and in the device tree node use it as
>> compatible = "qcom,sc7280-llcc-bwmon", "qcom,sdm845-llcc-bwmon", "qcom,msm8998-llcc-bwmon";
>> without any updates in the driver?
>
> I expect:
> "qcom,sc7280-llcc-bwmon", "qcom,msm8998-llcc-bwmon";
> and you need to add sc7280 compatible to the driver. The actual proper
> solution in my patch would be to use msm8998 compatible in the driver,
> but I did not test MSM8998.
>
> Maybe we should switch to that anyway?
Right, looks like without it every new SoC compatible added would need a dummy
update in the driver even though you really don't need to do anything different
in the driver.
On 6/29/2022 7:32 PM, Krzysztof Kozlowski wrote:
> Add bindings for the Qualcomm Bandwidth Monitor device providing
> performance data on interconnects. The bindings describe only BWMON CPU
> (version 4), e.g. the instance which appeared for the first on Qualcomm
> MSM8998 SoC and is also used on SDM845. This BWMON device sits between
> CPU and Last Level Cache Controller.
>
> Signed-off-by: Krzysztof Kozlowski <[email protected]>
> Reviewed-by: Rob Herring <[email protected]>
> Acked-by: Georgi Djakov <[email protected]>
> ---
> .../interconnect/qcom,msm8998-llcc-bwmon.yaml | 85 +++++++++++++++++++
> 1 file changed, 85 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/interconnect/qcom,msm8998-llcc-bwmon.yaml
>
> diff --git a/Documentation/devicetree/bindings/interconnect/qcom,msm8998-llcc-bwmon.yaml b/Documentation/devicetree/bindings/interconnect/qcom,msm8998-llcc-bwmon.yaml
> new file mode 100644
> index 000000000000..76e09658d615
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/interconnect/qcom,msm8998-llcc-bwmon.yaml
> @@ -0,0 +1,85 @@
> +# SPDX-License-Identifier: GPL-2.0-only OR BSD-2-Clause
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/interconnect/qcom,msm8998-llcc-bwmon.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: Qualcomm Interconnect Bandwidth Monitor
> +
> +maintainers:
> + - Krzysztof Kozlowski <[email protected]>
> +
> +description: |
> + Bandwidth Monitor measures current throughput on buses between various NoC
> + fabrics and provides information when it crosses configured thresholds.
> +
> + Certain SoCs might have more than one Bandwidth Monitors, for example on SDM845::
> + - Measuring the bandwidth between CPUs and Last Level Cache Controller -
> + called LLCC BWMON,
> + - Measuring the bandwidth between Last Level Cache Controller and memory (DDR).
> +
> +properties:
> + compatible:
> + oneOf:
> + - items:
> + - enum:
> + - qcom,sdm845-llcc-bwmon
> + - const: qcom,msm8998-llcc-bwmon
> + - const: qcom,msm8998-llcc-bwmon # BWMON v4
> +
> + interconnects:
> + maxItems: 1
> +
> + interrupts:
> + maxItems: 1
> +
> + operating-points-v2: true
> + opp-table: true
> +
> + reg:
> + # BWMON v4 (currently described) and BWMON v5 use one register address
> + # space. BWMON v2 uses two register spaces - not yet described.
> + maxItems: 1
> +
> +required:
> + - compatible
> + - interconnects
> + - interrupts
> + - operating-points-v2
> + - opp-table
> + - reg
> +
> +additionalProperties: false
> +
> +examples:
> + - |
> + #include <dt-bindings/interconnect/qcom,sdm845.h>
> + #include <dt-bindings/interrupt-controller/arm-gic.h>
> +
> + pmu@1436400 {
> + compatible = "qcom,sdm845-llcc-bwmon", "qcom,msm8998-llcc-bwmon";
so with this compatible fallback scheme, I am trying to understand what
do I need to do if I have to add support for another SoC for instance.
I just update the binding with the new SoC compatible (lets say qcom,sc7280-llcc-bwmon)
and in the device tree node use it as
compatible = "qcom,sc7280-llcc-bwmon", "qcom,sdm845-llcc-bwmon", "qcom,msm8998-llcc-bwmon";
without any updates in the driver?
> + reg = <0x01436400 0x600>;
> + interrupts = <GIC_SPI 581 IRQ_TYPE_LEVEL_HIGH>;
> + interconnects = <&gladiator_noc MASTER_APPSS_PROC 3 &mem_noc SLAVE_LLCC 3>;
> +
> + operating-points-v2 = <&llcc_bwmon_opp_table>;
> +
> + llcc_bwmon_opp_table: opp-table {
> + compatible = "operating-points-v2";
> + opp-0 {
> + opp-peak-kBps = <4800000>;
> + };
> + opp-1 {
> + opp-peak-kBps = <9216000>;
> + };
> + opp-2 {
> + opp-peak-kBps = <15052800>;
> + };
> + opp-3 {
> + opp-peak-kBps = <20889600>;
> + };
> + opp-4 {
> + opp-peak-kBps = <25497600>;
> + };
> + };
> + };
On 30/06/2022 13:29, Rajendra Nayak wrote:
>>> I just update the binding with the new SoC compatible (lets say qcom,sc7280-llcc-bwmon)
>>> and in the device tree node use it as
>>> compatible = "qcom,sc7280-llcc-bwmon", "qcom,sdm845-llcc-bwmon", "qcom,msm8998-llcc-bwmon";
>>> without any updates in the driver?
>>
>> I expect:
>> "qcom,sc7280-llcc-bwmon", "qcom,msm8998-llcc-bwmon";
>> and you need to add sc7280 compatible to the driver. The actual proper
>> solution in my patch would be to use msm8998 compatible in the driver,
>> but I did not test MSM8998.
>>
>> Maybe we should switch to that anyway?
>
> Right, looks like without it every new SoC compatible added would need a dummy
> update in the driver even though you really don't need to do anything different
> in the driver.
OK, then v7 is coming :)
Best regards,
Krzysztof