2020-11-04 19:55:29

by Dmitry Osipenko

[permalink] [raw]
Subject: [PATCH v7 00/47] Introduce memory interconnect for NVIDIA Tegra SoCs

This series brings initial support for memory interconnect to Tegra20,
Tegra30 and Tegra124 SoCs.

For the starter only display controllers and devfreq devices are getting
interconnect API support, others could be supported later on. The display
controllers have the biggest demand for interconnect API right now because
dynamic memory frequency scaling can't be done safely without taking into
account bandwidth requirement from the displays. In particular this series
fixes distorted display output on T30 Ouya and T124 TK1 devices.

Changelog:

v7: - Corrected example in "dt-bindings: host1x: Document new interconnect
properties".

- Added "obj-$(CONFIG_ARCH_TEGRA_132_SOC) += clk-tegra124-emc.o"
to "memory: tegra124-emc: Make driver modular" in order to fix
ARM64 kernel compilation reported by kernel test robot.

- Fixed build error reported by kernel test robot which happened due to
a missing stub for tegra_sku_info for COMPILE_TEST drivers. Added this
new patch:

soc/tegra: fuse: Add stub for tegra_sku_info

- Fixed minor W=1 warning reported by kernel test robot for the
"drm/tegra: dc: Support memory bandwidth management" patch.

- Changed TEGRA_MC_ICC_TAG type from a enum to bitmask in order to have
tags more extensible in the future.

- Fixed dt_binding_check warning in "dt-bindings: memory: tegra124: mc:
Document new interconnect property", was reported by kernel bot.

- Added acks/r-b from Rob Herring, Thierry Reding and Krzysztof Kozlowski
to the reviewed patches.

- The EMC "core-supply" property is now explicitly optional in the DT
bindings. This change was suggested by Krzysztof Kozlowski.

- Removed unnecessary white-spaces in the device-tree example of the patch:

dt-bindings: memory: tegra20: emc: Document new interconnect property

This change was suggested by Krzysztof Kozlowski.

- Explicitly documented that core-supply is an optional property in patch:

dt-bindings: memory: tegra20: emc: Document new interconnect property

This change was suggested by Krzysztof Kozlowski.

- Replaced underscores with hyphens in the DT node names. Suggested
by Krzysztof Kozlowski.

- Added operating-points-v2 to the required properties of the patch:

dt-bindings: memory: tegra124: emc: Document OPP table and voltage regulator

This was missed by accident in the v6.

- Renamed get_memory_controller() to memory_controller_get(). Suggested
by Krzysztof Kozlowski.

- Changed internal ICC IDs like it was suggested by Thierry Reding in
review comment to "memory: tegra-mc: Add interconnect framework".

- The MC's ICC xlate_extended() now is a per-SoC hook. Suggested by
Thierry Reding.

- Corrected and improved the debugfs of the EMC drivers which
transitioned to OPP table usage. The debug code now uses OPP
API, which is must-have for voltage scaling, and min/max rate
limits aren't changed using the clk API because OPP API should
be used.

- The tegra20-devfeq driver is removed and replaced with the EMC_STAT
driver, which is now a part of the T20 EMC driver. Suggested by
Thierry Reding. Implemented in these patches:

memory: tegra20-emc: Add devfreq support
PM / devfreq: tegra20: Deprecate in a favor of emc-stat based driver

- Reordered patches, like it was suggested by Krzysztof Kozlowski by
having patches which make smaller improvements before patches which
add new features.

- The EMC drivers now query actual DRAM bus width from hardware
instead of hardcoding the default-expected value.

- The EMC drivers now properly account clock rate requests from
different internal sources (ICC, debugfs, etc) by selecting the
fastest clock rate.

- There are lots of miscellaneous minor clean-ups and improvements.
All patches were fully re-tested.

- Rebased patches on a recent linux-next. Fixed merge conflicts with
the recent Tegra DRM driver changes.

- Improved tegra30-devfeq patches by implementing suggestions from
Chanwoo Choi. Removed unnecessary EPROBE_DEFER handling, added
clarifying comment for the KHz conversions, improved code that
touches ARRAY_SIZE.

v6: - This series was massively reworked in comparison to v5, most of the
patches that previously got r-b need a new round of a review (!).

- Added missed clk-rounding to the set() callback of EMC ICC providers.
Now clk_set_min_rate() doesn't error out on rate overflow.

- Now peak bandwidth is properly taken into account by the set() callback
of EMC ICC providers.

- EMC runs at 2x of the DRAM bus only on Tegra20, this now taken in account
properly by the EMC ICC set() callbacks.

- ICC drivers use new icc_sync_state() and xlate_extended().

- ICC drivers support new TEGRA_MC_ICC_TAG_ISO for ICC paths, which
conveys to ICC driver that memory path uses isochronous transfers.

- Added support for memory latency scaling to Tegra30 ICC provider.
It's required for fixing display FIFO underflows on T30.

- Added basic interconnect support to Tegra124 drivers.

- Tegra20/30/124 EMC drivers now support voltage scaling using generic
OPP API.

- The display bandwidth management is reworked and improved. It now
supports both bandwidth and latency allocations. The nv-display is
now also taken into account properly, i.e. it's kept untouched.
The extra bandwidth reservation required for ISO clients is moved
from DC driver to the ICC drivers.

- Dropped patch that tuned T20 display controller memory client because
turned out that it kills ~30% of memory bandwidth. It should be possible
to support client tuning, but it's too complicated for now.

- Corrected display's cursor and winb-vfilter ICC clients.
The winb-vfilter was erroneously used in place of cursor's client
in device-trees.

- Added devm_tegra_get_memory_controller() and switched drivers to
use it.

- Device-tree OPP tables are now supported by memory and devfreq
drivers.

- Tegra20-devfeq driver is reworked and now uses EMC-stats instead
of IMC-stats (which are nearly identical modules) because previously
I failed to understand how EMC-stats work and succeeded this time,
thinking that it simply doesn't work. This removes a bit icky dependency
on using both EMC and MC drivers simultaneously by the devfreq driver.

- Tegra20-devfeq driver now is a sub-device of the EMC, it also now uses
interconnect API for driving memory bandwidth.

- Tegra30-devfreq got interconnect support.

- Devfreq patches now use dev_err_probe(), which was suggested by
Chanwoo Choi.

- Added acks from Chanwoo Choi and Rob Herring to the reviewed and
unchanged patches.

- Added tested-by from Peter Geis and Nicolas Chauvet, who tested this
series on Ouya and TK1 devices, reporting that it fixes display
corruption on these devices which happened due to insufficient memory
bandwidth.

- Added patches to fix T20 EMC registers size.

- Fixed missing LA entry for PTC in the Tegra MC drivers.

- New and updated patches in v6:

dt-bindings: memory: tegra20: emc: Correct registers range in example
dt-bindings: memory: tegra20: emc: Document nvidia,memory-controller property
dt-bindings: memory: tegra20: emc: Document OPP table and voltage regulator
dt-bindings: memory: tegra20: emc: Document mfd-simple compatible and statistics sub-device
dt-bindings: memory: tegra30: emc: Document OPP table and voltage regulator
dt-bindings: memory: tegra124: mc: Document new interconnect property
dt-bindings: memory: tegra124: emc: Document new interconnect property
dt-bindings: memory: tegra124: emc: Document OPP table and voltage regulator
dt-bindings: tegra30-actmon: Document OPP and interconnect properties
dt-bindings: memory: tegra124: Add memory client IDs
ARM: tegra: Correct EMC registers size in Tegra20 device-tree
ARM: tegra: Add interconnect properties to Tegra124 device-tree
ARM: tegra: Add nvidia,memory-controller phandle to Tegra20 EMC device-tree
ARM: tegra: Add DVFS properties to Tegra20 EMC device-tree node
ARM: tegra: Add DVFS properties to Tegra30 EMC and ACTMON device-tree nodes
ARM: tegra: Add DVFS properties to Tegra124 EMC and ACTMON device-tree nodes
memory: tegra: Add and use devm_tegra_get_memory_controller()
memory: tegra-mc: Add interconnect framework
memory: tegra20: Support interconnect framework
memory: tegra20-emc: Skip parsing of emc-stats DT sub-node
memory: tegra: Add missing latency allowness entry for Page Table Cache
memory: tegra: Add FIFO sizes to Tegra30 memory clients
memory: tegra30: Support interconnect framework
memory: tegra124-emc: Make driver modular
memory: tegra124: Support interconnect framework
memory: tegra: Remove superfluous error messages around platform_get_irq()
drm/tegra: dc: Support memory bandwidth management
drm/tegra: dc: Extend debug stats with total number of events
PM / devfreq: tegra20: Convert to EMC_STAT driver, support interconnect and device-tree
PM / devfreq: tegra30: Support interconnect and OPPs from device-tree
PM / devfreq: tegra30: Separate configurations per-SoC generation
opp: Put interconnect paths outside of opp_table_lock

v5: - The devfreq drivers now won't probe if memory timings aren't specified
in a device-tree, like was suggested by Chanwoo Choi in a review comment
to v4. Initially I wanted to always probe the driver even with a single
fixed memory freq, but after a closer look turned out it can't be done
easily for Tegra20 driver.

- The "interconnect: Relax requirement in of_icc_get_from_provider()"
patch was already applied, hence one less patch in comparison to v4.

- Renamed display interconnect paths in accordance to the names that
were used by Thierry Reding in one of his recent patches that supposed
to update the Host1x's DT binding.

- Added acks from Chanwoo Choi.

- Added clarifying comment to tegra_mc_icc_set() about why it's a dummy
function, this is done in a response to the review comment made by
Georgi Djakov to v4.

v4: - All drivers that use interconnect API now select it in the Kconfig in
order to properly express the build dependency.

- The IS_ENABLED(CONFIG_INTERCONNECT) is dropped now from all patches.

- Added MODULE_AUTHOR() to the modularized drivers, for completeness.

- Added missed TEGRA_MC Kconfig dependency for the Tegra20 EMC driver.

- Added more acks from Rob Herring that I accidentally missed to add in v3.

v3: - Added acks from Rob Herring that were given to some of the v2 patches.

- Specified name of the TRM documentation chapter in the patch
"dt-bindings: host1x: Document new interconnect properties", which was
suggested by Rob Herring in the review comment to v2.

- Added patches that allow EMC drivers to be compiled as a loadable kernel
modules. This came up during of the v2 review when Georgi Djakov pointed
out that interconnect-core could be compiled as a kernel module. Please
note that the Tegra124 EMC driver is compile-tested only, I don't have
Tegra124 HW.

- In the review comment to [1] Stephen Boyd suggested that it will be
better not to make changes to clk API, which was needed in order to
avoid clashing of the interconnect driver with the devfreq in regards
to memory clk-rate rounding.

[1] https://patchwork.ozlabs.org/project/linux-tegra/patch/[email protected]/

Stephen Boyd suggested that instead we should provide OPP table via DT.
I tried to investigate whether this could be done and turned out
it's a bit complicated. Technically it should be doable, but:

1. For now we don't fully support voltage scaling of the CORE regulator
and so OPP table in the DT isn't really needed today. We can
generate table from the memory timings, which is what Tegra devfreq
drivers already do.

2. The OPP table should be defined in the DT for the Memory Controller
node and then its usage somehow should be shared by both interconnect
and devfreq drivers. It's not obvious what's the best way to do it.

So, it will be much better to postpone the DT OPP table addition
until these questions are resolved. We can infer OPPs from the
memory timings and we could get the memory rates from the memory
driver directly, avoiding the problems induced by the clk API usage.
This idea is implemented in v3, see these patches:

PM / devfreq: tegra20: Use MC timings for building OPP table
PM / devfreq: tegra30: Use MC timings for building OPP table

v2: - Instead of a single dma-mem interconnect path, the paths are now
defined per memory client.

- The EMC provider now uses #interconnect-cells=<0>.

- Dropped Tegra124 because there is no enough information about how to
properly calculate required EMC clock rate for it and I don't have
hardware for testing. Somebody else will have to work on it.

- Moved interconnect providers code into drivers/memory/tegra/*.

- Added "Create tegra20-devfreq device" patch because interconnect
is not very usable without the devfreq memory auto-scaling since
memory freq will be fixed to the display's requirement.

Dmitry Osipenko (47):
clk: tegra: Export Tegra20 EMC kernel symbols
soc/tegra: fuse: Export tegra_read_ram_code()
soc/tegra: fuse: Add stub for tegra_sku_info
dt-bindings: memory: tegra20: emc: Correct registers range in example
dt-bindings: memory: tegra20: emc: Document nvidia,memory-controller
property
dt-bindings: memory: tegra20: mc: Document new interconnect property
dt-bindings: memory: tegra20: emc: Document new interconnect property
dt-bindings: memory: tegra20: emc: Document OPP table and voltage
regulator
dt-bindings: memory: tegra30: mc: Document new interconnect property
dt-bindings: memory: tegra30: emc: Document new interconnect property
dt-bindings: memory: tegra30: emc: Document OPP table and voltage
regulator
dt-bindings: memory: tegra124: mc: Document new interconnect property
dt-bindings: memory: tegra124: emc: Document new interconnect property
dt-bindings: memory: tegra124: emc: Document OPP table and voltage
regulator
dt-bindings: tegra30-actmon: Document OPP and interconnect properties
dt-bindings: host1x: Document new interconnect properties
dt-bindings: memory: tegra20: Add memory client IDs
dt-bindings: memory: tegra30: Add memory client IDs
dt-bindings: memory: tegra124: Add memory client IDs
ARM: tegra: Correct EMC registers size in Tegra20 device-tree
ARM: tegra: Add interconnect properties to Tegra20 device-tree
ARM: tegra: Add interconnect properties to Tegra30 device-tree
ARM: tegra: Add interconnect properties to Tegra124 device-tree
ARM: tegra: Add nvidia,memory-controller phandle to Tegra20 EMC
device-tree
ARM: tegra: Add DVFS properties to Tegra20 EMC device-tree node
ARM: tegra: Add DVFS properties to Tegra30 EMC and ACTMON device-tree
nodes
ARM: tegra: Add DVFS properties to Tegra124 EMC and ACTMON device-tree
nodes
memory: tegra: Add and use devm_tegra_memory_controller_get()
memory: tegra: Use devm_platform_ioremap_resource()
memory: tegra: Remove superfluous error messages around
platform_get_irq()
memory: tegra: Add missing latency allowness entry for Page Table
Cache
memory: tegra-mc: Add interconnect framework
memory: tegra20-emc: Make driver modular
memory: tegra20-emc: Continue probing if timings are missing in
device-tree
memory: tegra20: Support interconnect framework
memory: tegra20-emc: Add devfreq support
memory: tegra30: Add FIFO sizes to memory clients
memory: tegra30-emc: Make driver modular
memory: tegra30-emc: Continue probing if timings are missing in
device-tree
memory: tegra30: Support interconnect framework
memory: tegra124-emc: Make driver modular
memory: tegra124: Support interconnect framework
drm/tegra: dc: Support memory bandwidth management
drm/tegra: dc: Extend debug stats with total number of events
PM / devfreq: tegra30: Support interconnect and OPPs from device-tree
PM / devfreq: tegra30: Separate configurations per-SoC generation
PM / devfreq: tegra20: Deprecate in a favor of emc-stat based driver

.../arm/tegra/nvidia,tegra30-actmon.txt | 25 +
.../display/tegra/nvidia,tegra20-host1x.txt | 68 +++
.../nvidia,tegra124-emc.yaml | 19 +
.../nvidia,tegra124-mc.yaml | 5 +
.../memory-controllers/nvidia,tegra20-emc.txt | 22 +-
.../memory-controllers/nvidia,tegra20-mc.txt | 3 +
.../nvidia,tegra30-emc.yaml | 18 +
.../memory-controllers/nvidia,tegra30-mc.yaml | 5 +
arch/arm/boot/dts/tegra124-apalis-emc.dtsi | 8 +
.../arm/boot/dts/tegra124-jetson-tk1-emc.dtsi | 8 +
arch/arm/boot/dts/tegra124-nyan-big-emc.dtsi | 10 +
.../arm/boot/dts/tegra124-nyan-blaze-emc.dtsi | 10 +
.../boot/dts/tegra124-peripherals-opp.dtsi | 419 ++++++++++++++++
arch/arm/boot/dts/tegra124.dtsi | 31 ++
.../boot/dts/tegra20-acer-a500-picasso.dts | 7 +
arch/arm/boot/dts/tegra20-colibri.dtsi | 4 +
arch/arm/boot/dts/tegra20-paz00.dts | 6 +
.../arm/boot/dts/tegra20-peripherals-opp.dtsi | 92 ++++
arch/arm/boot/dts/tegra20.dtsi | 33 +-
.../tegra30-asus-nexus7-grouper-common.dtsi | 4 +
...30-asus-nexus7-grouper-memory-timings.dtsi | 12 +
.../arm/boot/dts/tegra30-peripherals-opp.dtsi | 383 +++++++++++++++
arch/arm/boot/dts/tegra30.dtsi | 33 +-
drivers/clk/tegra/Makefile | 3 +-
drivers/clk/tegra/clk-tegra124-emc.c | 41 +-
drivers/clk/tegra/clk-tegra124.c | 27 +-
drivers/clk/tegra/clk-tegra20-emc.c | 3 +
drivers/clk/tegra/clk.h | 16 +-
drivers/devfreq/Kconfig | 10 -
drivers/devfreq/Makefile | 1 -
drivers/devfreq/tegra20-devfreq.c | 210 --------
drivers/devfreq/tegra30-devfreq.c | 164 ++++---
drivers/gpu/drm/tegra/Kconfig | 1 +
drivers/gpu/drm/tegra/dc.c | 359 ++++++++++++++
drivers/gpu/drm/tegra/dc.h | 19 +
drivers/gpu/drm/tegra/drm.c | 14 +
drivers/gpu/drm/tegra/hub.c | 3 +
drivers/gpu/drm/tegra/plane.c | 121 +++++
drivers/gpu/drm/tegra/plane.h | 15 +
drivers/memory/tegra/Kconfig | 14 +-
drivers/memory/tegra/mc.c | 155 +++++-
drivers/memory/tegra/mc.h | 22 +
drivers/memory/tegra/tegra114.c | 6 +
drivers/memory/tegra/tegra124-emc.c | 378 +++++++++++++--
drivers/memory/tegra/tegra124.c | 88 +++-
drivers/memory/tegra/tegra20-emc.c | 458 ++++++++++++++++--
drivers/memory/tegra/tegra20.c | 77 +++
drivers/memory/tegra/tegra210-emc-core.c | 39 +-
drivers/memory/tegra/tegra30-emc.c | 353 ++++++++++++--
drivers/memory/tegra/tegra30.c | 245 +++++++++-
drivers/soc/tegra/fuse/tegra-apbmisc.c | 2 +
include/dt-bindings/memory/tegra124-mc.h | 68 +++
include/dt-bindings/memory/tegra20-mc.h | 53 ++
include/dt-bindings/memory/tegra30-mc.h | 67 +++
include/linux/clk/tegra.h | 8 +
include/soc/tegra/emc.h | 16 -
include/soc/tegra/fuse.h | 4 +
include/soc/tegra/mc.h | 27 ++
58 files changed, 3852 insertions(+), 460 deletions(-)
create mode 100644 arch/arm/boot/dts/tegra124-peripherals-opp.dtsi
create mode 100644 arch/arm/boot/dts/tegra20-peripherals-opp.dtsi
create mode 100644 arch/arm/boot/dts/tegra30-peripherals-opp.dtsi
delete mode 100644 drivers/devfreq/tegra20-devfreq.c
delete mode 100644 include/soc/tegra/emc.h

--
2.27.0


2020-11-04 19:56:32

by Dmitry Osipenko

[permalink] [raw]
Subject: [PATCH v7 36/47] memory: tegra20-emc: Add devfreq support

Add devfreq support to the Tegra20 EMC driver. Memory utilization
statistics will be periodically polled from the memory controller and
appropriate minimum clock rate will be selected by the devfreq governor.

Signed-off-by: Dmitry Osipenko <[email protected]>
---
drivers/memory/tegra/Kconfig | 2 +
drivers/memory/tegra/tegra20-emc.c | 92 ++++++++++++++++++++++++++++++
2 files changed, 94 insertions(+)

diff --git a/drivers/memory/tegra/Kconfig b/drivers/memory/tegra/Kconfig
index ac3dfe155505..76e9a3b10839 100644
--- a/drivers/memory/tegra/Kconfig
+++ b/drivers/memory/tegra/Kconfig
@@ -12,6 +12,8 @@ config TEGRA20_EMC
tristate "NVIDIA Tegra20 External Memory Controller driver"
default y
depends on TEGRA_MC && ARCH_TEGRA_2x_SOC
+ select DEVFREQ_GOV_SIMPLE_ONDEMAND
+ select PM_DEVFREQ
select PM_OPP
help
This driver is for the External Memory Controller (EMC) found on
diff --git a/drivers/memory/tegra/tegra20-emc.c b/drivers/memory/tegra/tegra20-emc.c
index 5e10aa97809f..9946b957bb01 100644
--- a/drivers/memory/tegra/tegra20-emc.c
+++ b/drivers/memory/tegra/tegra20-emc.c
@@ -8,6 +8,7 @@
#include <linux/clk.h>
#include <linux/clk/tegra.h>
#include <linux/debugfs.h>
+#include <linux/devfreq.h>
#include <linux/err.h>
#include <linux/interconnect-provider.h>
#include <linux/interrupt.h>
@@ -102,6 +103,10 @@

#define EMC_FBIO_CFG5_DRAM_WIDTH_X16 BIT(4)

+#define EMC_PWR_GATHER_CLEAR (1 << 8)
+#define EMC_PWR_GATHER_DISABLE (2 << 8)
+#define EMC_PWR_GATHER_ENABLE (3 << 8)
+
static const u16 emc_timing_registers[] = {
EMC_RC,
EMC_RFC,
@@ -157,6 +162,7 @@ struct emc_timing {
};

enum emc_rate_request_type {
+ EMC_RATE_DEVFREQ,
EMC_RATE_DEBUG,
EMC_RATE_ICC,
EMC_RATE_TYPE_MAX,
@@ -193,6 +199,9 @@ struct tegra_emc {

/* protect shared rate-change code path */
struct mutex rate_lock;
+
+ struct devfreq_simple_ondemand_data ondemand_data;
+ struct devfreq *devfreq;
};

static irqreturn_t tegra_emc_isr(int irq, void *data)
@@ -952,6 +961,88 @@ static int tegra_emc_opp_table_init(struct tegra_emc *emc)
return err;
}

+static int tegra_emc_devfreq_target(struct device *dev, unsigned long *freq,
+ u32 flags)
+{
+ struct tegra_emc *emc = dev_get_drvdata(dev);
+ struct dev_pm_opp *opp;
+ unsigned long rate;
+
+ opp = devfreq_recommended_opp(dev, freq, flags);
+ if (IS_ERR(opp)) {
+ dev_err(dev, "failed to find opp for %lu Hz\n", *freq);
+ return PTR_ERR(opp);
+ }
+
+ rate = dev_pm_opp_get_freq(opp);
+ dev_pm_opp_put(opp);
+
+ return emc_set_min_rate(emc, rate, EMC_RATE_DEVFREQ);
+}
+
+static int tegra_emc_devfreq_get_dev_status(struct device *dev,
+ struct devfreq_dev_status *stat)
+{
+ struct tegra_emc *emc = dev_get_drvdata(dev);
+
+ /* freeze counters */
+ writel_relaxed(EMC_PWR_GATHER_DISABLE, emc->regs + EMC_STAT_CONTROL);
+
+ /*
+ * busy_time: number of clocks EMC request was accepted
+ * total_time: number of clocks PWR_GATHER control was set to ENABLE
+ */
+ stat->busy_time = readl_relaxed(emc->regs + EMC_STAT_PWR_COUNT);
+ stat->total_time = readl_relaxed(emc->regs + EMC_STAT_PWR_CLOCKS);
+ stat->current_frequency = clk_get_rate(emc->clk);
+
+ /* clear counters and restart */
+ writel_relaxed(EMC_PWR_GATHER_CLEAR, emc->regs + EMC_STAT_CONTROL);
+ writel_relaxed(EMC_PWR_GATHER_ENABLE, emc->regs + EMC_STAT_CONTROL);
+
+ return 0;
+}
+
+static struct devfreq_dev_profile tegra_emc_devfreq_profile = {
+ .polling_ms = 30,
+ .target = tegra_emc_devfreq_target,
+ .get_dev_status = tegra_emc_devfreq_get_dev_status,
+};
+
+static int tegra_emc_devfreq_init(struct tegra_emc *emc)
+{
+ int err;
+
+ /*
+ * PWR_COUNT is 1/2 of PWR_CLOCKS at max, and thus, the up-threshold
+ * should be less than 50. Secondly, multiple active memory clients
+ * may cause over 20% of lost clock cycles due to stalls caused by
+ * competing memory accesses. This means that threshold should be
+ * set to a less than 30 in order to have a properly working governor.
+ */
+ emc->ondemand_data.upthreshold = 20;
+
+ /*
+ * Reset statistic gathers state, select global bandwidth for the
+ * statistics collection mode and set clocks counter saturation
+ * limit to maximum.
+ */
+ writel_relaxed(0x00000000, emc->regs + EMC_STAT_CONTROL);
+ writel_relaxed(0x00000000, emc->regs + EMC_STAT_LLMC_CONTROL);
+ writel_relaxed(0xffffffff, emc->regs + EMC_STAT_PWR_CLOCK_LIMIT);
+
+ emc->devfreq = devfreq_add_device(emc->dev, &tegra_emc_devfreq_profile,
+ DEVFREQ_GOV_SIMPLE_ONDEMAND,
+ &emc->ondemand_data);
+ if (IS_ERR(emc->devfreq)) {
+ err = PTR_ERR(emc->devfreq);
+ dev_err(emc->dev, "failed to initialize devfreq: %d", err);
+ return err;
+ }
+
+ return 0;
+}
+
static int tegra_emc_probe(struct platform_device *pdev)
{
struct device_node *np;
@@ -1019,6 +1110,7 @@ static int tegra_emc_probe(struct platform_device *pdev)
tegra_emc_rate_requests_init(emc);
tegra_emc_debugfs_init(emc);
tegra_emc_interconnect_init(emc);
+ tegra_emc_devfreq_init(emc);

/*
* Don't allow the kernel module to be unloaded. Unloading adds some
--
2.27.0

2020-11-04 20:02:08

by Dmitry Osipenko

[permalink] [raw]
Subject: [PATCH v7 39/47] memory: tegra30-emc: Continue probing if timings are missing in device-tree

EMC driver will become mandatory after turning it into interconnect
provider because interconnect users, like display controller driver, will
fail to probe using newer device-trees that have interconnect properties.
Thus make EMC driver to probe even if timings are missing in device-tree.

Signed-off-by: Dmitry Osipenko <[email protected]>
---
drivers/memory/tegra/tegra30-emc.c | 29 +++++++++++++++--------------
1 file changed, 15 insertions(+), 14 deletions(-)

diff --git a/drivers/memory/tegra/tegra30-emc.c b/drivers/memory/tegra/tegra30-emc.c
index d0926088360a..3488786da03b 100644
--- a/drivers/memory/tegra/tegra30-emc.c
+++ b/drivers/memory/tegra/tegra30-emc.c
@@ -988,6 +988,11 @@ static struct device_node *emc_find_node_by_ram_code(struct device *dev)
u32 value, ram_code;
int err;

+ if (of_get_child_count(dev->of_node) == 0) {
+ dev_info(dev, "device-tree doesn't have memory timings\n");
+ return NULL;
+ }
+
ram_code = tegra_read_ram_code();

for_each_child_of_node(dev->of_node, np) {
@@ -1057,6 +1062,9 @@ static long emc_round_rate(unsigned long rate,
struct tegra_emc *emc = arg;
unsigned int i;

+ if (!emc->num_timings)
+ return clk_get_rate(emc->clk);
+
min_rate = min(min_rate, emc->timings[emc->num_timings - 1].rate);

for (i = 0; i < emc->num_timings; i++) {
@@ -1262,16 +1270,6 @@ static int tegra_emc_probe(struct platform_device *pdev)
struct tegra_emc *emc;
int err;

- if (of_get_child_count(pdev->dev.of_node) == 0) {
- dev_info(&pdev->dev,
- "device-tree node doesn't have memory timings\n");
- return -ENODEV;
- }
-
- np = emc_find_node_by_ram_code(&pdev->dev);
- if (!np)
- return -EINVAL;
-
emc = devm_kzalloc(&pdev->dev, sizeof(*emc), GFP_KERNEL);
if (!emc) {
of_node_put(np);
@@ -1285,10 +1283,13 @@ static int tegra_emc_probe(struct platform_device *pdev)
emc->clk_nb.notifier_call = emc_clk_change_notify;
emc->dev = &pdev->dev;

- err = emc_load_timings_from_dt(emc, np);
- of_node_put(np);
- if (err)
- return err;
+ np = emc_find_node_by_ram_code(&pdev->dev);
+ if (np) {
+ err = emc_load_timings_from_dt(emc, np);
+ of_node_put(np);
+ if (err)
+ return err;
+ }

emc->regs = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(emc->regs))
--
2.27.0

2020-11-04 20:02:53

by Dmitry Osipenko

[permalink] [raw]
Subject: [PATCH v7 16/47] dt-bindings: host1x: Document new interconnect properties

Most of Host1x devices have at least one memory client. These clients
are directly connected to the memory controller. The new interconnect
properties represent the memory client's connection to the memory
controller.

Reviewed-by: Rob Herring <[email protected]>
Signed-off-by: Dmitry Osipenko <[email protected]>
---
.../display/tegra/nvidia,tegra20-host1x.txt | 68 +++++++++++++++++++
1 file changed, 68 insertions(+)

diff --git a/Documentation/devicetree/bindings/display/tegra/nvidia,tegra20-host1x.txt b/Documentation/devicetree/bindings/display/tegra/nvidia,tegra20-host1x.txt
index ac63ae4a3861..34d993338453 100644
--- a/Documentation/devicetree/bindings/display/tegra/nvidia,tegra20-host1x.txt
+++ b/Documentation/devicetree/bindings/display/tegra/nvidia,tegra20-host1x.txt
@@ -20,6 +20,10 @@ Required properties:
- reset-names: Must include the following entries:
- host1x

+Each host1x client module having to perform DMA through the Memory Controller
+should have the interconnect endpoints set to the Memory Client and External
+Memory respectively.
+
The host1x top-level node defines a number of children, each representing one
of the following host1x client modules:

@@ -36,6 +40,12 @@ of the following host1x client modules:
- reset-names: Must include the following entries:
- mpe

+ Optional properties:
+ - interconnects: Must contain entry for the MPE memory clients.
+ - interconnect-names: Must include name of the interconnect path for each
+ interconnect entry. Consult TRM documentation for information about
+ available memory clients, see MEMORY CONTROLLER section.
+
- vi: video input

Required properties:
@@ -113,6 +123,12 @@ of the following host1x client modules:
Required properties:
- remote-endpoint: phandle to vi port 'endpoint' node.

+ Optional properties:
+ - interconnects: Must contain entry for the VI memory clients.
+ - interconnect-names: Must include name of the interconnect path for each
+ interconnect entry. Consult TRM documentation for information about
+ available memory clients, see MEMORY CONTROLLER section.
+
- epp: encoder pre-processor

Required properties:
@@ -126,6 +142,12 @@ of the following host1x client modules:
- reset-names: Must include the following entries:
- epp

+ Optional properties:
+ - interconnects: Must contain entry for the EPP memory clients.
+ - interconnect-names: Must include name of the interconnect path for each
+ interconnect entry. Consult TRM documentation for information about
+ available memory clients, see MEMORY CONTROLLER section.
+
- isp: image signal processor

Required properties:
@@ -139,6 +161,12 @@ of the following host1x client modules:
- reset-names: Must include the following entries:
- isp

+ Optional properties:
+ - interconnects: Must contain entry for the ISP memory clients.
+ - interconnect-names: Must include name of the interconnect path for each
+ interconnect entry. Consult TRM documentation for information about
+ available memory clients, see MEMORY CONTROLLER section.
+
- gr2d: 2D graphics engine

Required properties:
@@ -152,6 +180,12 @@ of the following host1x client modules:
- reset-names: Must include the following entries:
- 2d

+ Optional properties:
+ - interconnects: Must contain entry for the GR2D memory clients.
+ - interconnect-names: Must include name of the interconnect path for each
+ interconnect entry. Consult TRM documentation for information about
+ available memory clients, see MEMORY CONTROLLER section.
+
- gr3d: 3D graphics engine

Required properties:
@@ -170,6 +204,12 @@ of the following host1x client modules:
- 3d
- 3d2 (Only required on SoCs with two 3D clocks)

+ Optional properties:
+ - interconnects: Must contain entry for the GR3D memory clients.
+ - interconnect-names: Must include name of the interconnect path for each
+ interconnect entry. Consult TRM documentation for information about
+ available memory clients, see MEMORY CONTROLLER section.
+
- dc: display controller

Required properties:
@@ -197,6 +237,10 @@ of the following host1x client modules:
- nvidia,hpd-gpio: specifies a GPIO used for hotplug detection
- nvidia,edid: supplies a binary EDID blob
- nvidia,panel: phandle of a display panel
+ - interconnects: Must contain entry for the DC memory clients.
+ - interconnect-names: Must include name of the interconnect path for each
+ interconnect entry. Consult TRM documentation for information about
+ available memory clients, see MEMORY CONTROLLER section.

- hdmi: High Definition Multimedia Interface

@@ -345,6 +389,12 @@ of the following host1x client modules:
- reset-names: Must include the following entries:
- vic

+ Optional properties:
+ - interconnects: Must contain entry for the VIC memory clients.
+ - interconnect-names: Must include name of the interconnect path for each
+ interconnect entry. Consult TRM documentation for information about
+ available memory clients, see MEMORY CONTROLLER section.
+
Example:

/ {
@@ -498,6 +548,15 @@ Example:
resets = <&tegra_car 27>;
reset-names = "dc";

+ interconnects = <&mc TEGRA20_MC_DISPLAY0A &emc>,
+ <&mc TEGRA20_MC_DISPLAY0B &emc>,
+ <&mc TEGRA20_MC_DISPLAY0C &emc>,
+ <&mc TEGRA20_MC_DISPLAYHC &emc>;
+ interconnect-names = "wina",
+ "winb",
+ "winc",
+ "cursor";
+
rgb {
status = "disabled";
};
@@ -513,6 +572,15 @@ Example:
resets = <&tegra_car 26>;
reset-names = "dc";

+ interconnects = <&mc TEGRA20_MC_DISPLAY0AB &emc>,
+ <&mc TEGRA20_MC_DISPLAY0BB &emc>,
+ <&mc TEGRA20_MC_DISPLAY0CB &emc>,
+ <&mc TEGRA20_MC_DISPLAYHCB &emc>;
+ interconnect-names = "wina",
+ "winb",
+ "winc",
+ "cursor";
+
rgb {
status = "disabled";
};
--
2.27.0

2020-11-04 20:13:08

by Dmitry Osipenko

[permalink] [raw]
Subject: [PATCH v7 29/47] memory: tegra: Use devm_platform_ioremap_resource()

Use devm_platform_ioremap_resource() helper which makes code a bit
cleaner.

Acked-by: Thierry Reding <[email protected]>
Signed-off-by: Dmitry Osipenko <[email protected]>
---
drivers/memory/tegra/tegra124-emc.c | 4 +---
drivers/memory/tegra/tegra20-emc.c | 4 +---
2 files changed, 2 insertions(+), 6 deletions(-)

diff --git a/drivers/memory/tegra/tegra124-emc.c b/drivers/memory/tegra/tegra124-emc.c
index 35dbceb7f841..ee8ee39e98ed 100644
--- a/drivers/memory/tegra/tegra124-emc.c
+++ b/drivers/memory/tegra/tegra124-emc.c
@@ -1179,7 +1179,6 @@ static int tegra_emc_probe(struct platform_device *pdev)
{
struct device_node *np;
struct tegra_emc *emc;
- struct resource *res;
u32 ram_code;
int err;

@@ -1189,8 +1188,7 @@ static int tegra_emc_probe(struct platform_device *pdev)

emc->dev = &pdev->dev;

- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- emc->regs = devm_ioremap_resource(&pdev->dev, res);
+ emc->regs = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(emc->regs))
return PTR_ERR(emc->regs);

diff --git a/drivers/memory/tegra/tegra20-emc.c b/drivers/memory/tegra/tegra20-emc.c
index 027f46287dbf..5ba4e495bfc3 100644
--- a/drivers/memory/tegra/tegra20-emc.c
+++ b/drivers/memory/tegra/tegra20-emc.c
@@ -654,7 +654,6 @@ static int tegra_emc_probe(struct platform_device *pdev)
{
struct device_node *np;
struct tegra_emc *emc;
- struct resource *res;
int irq, err;

/* driver has nothing to do in a case of memory timing absence */
@@ -689,8 +688,7 @@ static int tegra_emc_probe(struct platform_device *pdev)
if (err)
return err;

- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- emc->regs = devm_ioremap_resource(&pdev->dev, res);
+ emc->regs = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(emc->regs))
return PTR_ERR(emc->regs);

--
2.27.0

2020-11-04 20:13:08

by Dmitry Osipenko

[permalink] [raw]
Subject: [PATCH v7 10/47] dt-bindings: memory: tegra30: emc: Document new interconnect property

External memory controller is interconnected with memory controller and
with external memory. Document new interconnect property which turns
External Memory Controller into interconnect provider.

Acked-by: Rob Herring <[email protected]>
Signed-off-by: Dmitry Osipenko <[email protected]>
---
.../bindings/memory-controllers/nvidia,tegra30-emc.yaml | 6 ++++++
1 file changed, 6 insertions(+)

diff --git a/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra30-emc.yaml b/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra30-emc.yaml
index 112bae2fcbbd..c243986db420 100644
--- a/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra30-emc.yaml
+++ b/Documentation/devicetree/bindings/memory-controllers/nvidia,tegra30-emc.yaml
@@ -31,6 +31,9 @@ properties:
interrupts:
maxItems: 1

+ "#interconnect-cells":
+ const: 0
+
nvidia,memory-controller:
$ref: /schemas/types.yaml#/definitions/phandle
description:
@@ -214,6 +217,7 @@ required:
- interrupts
- clocks
- nvidia,memory-controller
+ - "#interconnect-cells"

additionalProperties: false

@@ -227,6 +231,8 @@ examples:

nvidia,memory-controller = <&mc>;

+ #interconnect-cells = <0>;
+
emc-timings-1 {
nvidia,ram-code = <1>;

--
2.27.0

2020-11-04 20:13:09

by Dmitry Osipenko

[permalink] [raw]
Subject: [PATCH v7 27/47] ARM: tegra: Add DVFS properties to Tegra124 EMC and ACTMON device-tree nodes

Add EMC OPP DVFS/DFS tables and interconnect paths that will be used for
dynamic memory bandwidth scaling based on memory utilization statistics.
Remove unsupported EMC OPPs from board device-trees.

Note that ACTMON watches all memory interconnect paths, but we use a
single CPU-READ interconnect path for driving memory bandwidth, for
simplicity.

Signed-off-by: Dmitry Osipenko <[email protected]>
---
arch/arm/boot/dts/tegra124-apalis-emc.dtsi | 8 +
.../arm/boot/dts/tegra124-jetson-tk1-emc.dtsi | 8 +
arch/arm/boot/dts/tegra124-nyan-big-emc.dtsi | 10 +
.../arm/boot/dts/tegra124-nyan-blaze-emc.dtsi | 10 +
.../boot/dts/tegra124-peripherals-opp.dtsi | 419 ++++++++++++++++++
arch/arm/boot/dts/tegra124.dtsi | 6 +
6 files changed, 461 insertions(+)
create mode 100644 arch/arm/boot/dts/tegra124-peripherals-opp.dtsi

diff --git a/arch/arm/boot/dts/tegra124-apalis-emc.dtsi b/arch/arm/boot/dts/tegra124-apalis-emc.dtsi
index 32401457ae71..a7ac805eeed5 100644
--- a/arch/arm/boot/dts/tegra124-apalis-emc.dtsi
+++ b/arch/arm/boot/dts/tegra124-apalis-emc.dtsi
@@ -1465,3 +1465,11 @@ timing-924000000 {
};
};
};
+
+&emc_icc_dvfs_opp_table {
+ /delete-node/ opp@1200000000,1100;
+};
+
+&emc_bw_dfs_opp_table {
+ /delete-node/ opp@1200000000;
+};
diff --git a/arch/arm/boot/dts/tegra124-jetson-tk1-emc.dtsi b/arch/arm/boot/dts/tegra124-jetson-tk1-emc.dtsi
index 861d3f22116b..df4e463afbd1 100644
--- a/arch/arm/boot/dts/tegra124-jetson-tk1-emc.dtsi
+++ b/arch/arm/boot/dts/tegra124-jetson-tk1-emc.dtsi
@@ -2420,3 +2420,11 @@ timing-924000000 {
};
};
};
+
+&emc_icc_dvfs_opp_table {
+ /delete-node/ opp@1200000000,1100;
+};
+
+&emc_bw_dfs_opp_table {
+ /delete-node/ opp@1200000000;
+};
diff --git a/arch/arm/boot/dts/tegra124-nyan-big-emc.dtsi b/arch/arm/boot/dts/tegra124-nyan-big-emc.dtsi
index c91647d13a50..a0f56cc9da5c 100644
--- a/arch/arm/boot/dts/tegra124-nyan-big-emc.dtsi
+++ b/arch/arm/boot/dts/tegra124-nyan-big-emc.dtsi
@@ -6649,3 +6649,13 @@ timing-792000000 {
};
};
};
+
+&emc_icc_dvfs_opp_table {
+ /delete-node/ opp@924000000,1100;
+ /delete-node/ opp@1200000000,1100;
+};
+
+&emc_bw_dfs_opp_table {
+ /delete-node/ opp@924000000;
+ /delete-node/ opp@1200000000;
+};
diff --git a/arch/arm/boot/dts/tegra124-nyan-blaze-emc.dtsi b/arch/arm/boot/dts/tegra124-nyan-blaze-emc.dtsi
index d2beea0bd15f..35c98734d35f 100644
--- a/arch/arm/boot/dts/tegra124-nyan-blaze-emc.dtsi
+++ b/arch/arm/boot/dts/tegra124-nyan-blaze-emc.dtsi
@@ -2048,3 +2048,13 @@ timing-792000000 {
};
};
};
+
+&emc_icc_dvfs_opp_table {
+ /delete-node/ opp@924000000,1100;
+ /delete-node/ opp@1200000000,1100;
+};
+
+&emc_bw_dfs_opp_table {
+ /delete-node/ opp@924000000;
+ /delete-node/ opp@1200000000;
+};
diff --git a/arch/arm/boot/dts/tegra124-peripherals-opp.dtsi b/arch/arm/boot/dts/tegra124-peripherals-opp.dtsi
new file mode 100644
index 000000000000..49d9420a3289
--- /dev/null
+++ b/arch/arm/boot/dts/tegra124-peripherals-opp.dtsi
@@ -0,0 +1,419 @@
+// SPDX-License-Identifier: GPL-2.0
+
+/ {
+ emc_icc_dvfs_opp_table: emc-dvfs-opp-table {
+ compatible = "operating-points-v2";
+
+ opp@12750000,800 {
+ opp-microvolt = <800000 800000 1150000>;
+ opp-hz = /bits/ 64 <12750000>;
+ opp-supported-hw = <0x0003>;
+ };
+
+ opp@12750000,950 {
+ opp-microvolt = <950000 950000 1150000>;
+ opp-hz = /bits/ 64 <12750000>;
+ opp-supported-hw = <0x0008>;
+ };
+
+ opp@12750000,1050 {
+ opp-microvolt = <1050000 1050000 1150000>;
+ opp-hz = /bits/ 64 <12750000>;
+ opp-supported-hw = <0x0010>;
+ };
+
+ opp@12750000,1110 {
+ opp-microvolt = <1110000 1110000 1150000>;
+ opp-hz = /bits/ 64 <12750000>;
+ opp-supported-hw = <0x0004>;
+ };
+
+ opp@20400000,800 {
+ opp-microvolt = <800000 800000 1150000>;
+ opp-hz = /bits/ 64 <20400000>;
+ opp-supported-hw = <0x0003>;
+ };
+
+ opp@20400000,950 {
+ opp-microvolt = <950000 950000 1150000>;
+ opp-hz = /bits/ 64 <20400000>;
+ opp-supported-hw = <0x0008>;
+ };
+
+ opp@20400000,1050 {
+ opp-microvolt = <1050000 1050000 1150000>;
+ opp-hz = /bits/ 64 <20400000>;
+ opp-supported-hw = <0x0010>;
+ };
+
+ opp@20400000,1110 {
+ opp-microvolt = <1110000 1110000 1150000>;
+ opp-hz = /bits/ 64 <20400000>;
+ opp-supported-hw = <0x0004>;
+ };
+
+ opp@40800000,800 {
+ opp-microvolt = <800000 800000 1150000>;
+ opp-hz = /bits/ 64 <40800000>;
+ opp-supported-hw = <0x0003>;
+ };
+
+ opp@40800000,950 {
+ opp-microvolt = <950000 950000 1150000>;
+ opp-hz = /bits/ 64 <40800000>;
+ opp-supported-hw = <0x0008>;
+ };
+
+ opp@40800000,1050 {
+ opp-microvolt = <1050000 1050000 1150000>;
+ opp-hz = /bits/ 64 <40800000>;
+ opp-supported-hw = <0x0010>;
+ };
+
+ opp@40800000,1110 {
+ opp-microvolt = <1110000 1110000 1150000>;
+ opp-hz = /bits/ 64 <40800000>;
+ opp-supported-hw = <0x0004>;
+ };
+
+ opp@68000000,800 {
+ opp-microvolt = <800000 800000 1150000>;
+ opp-hz = /bits/ 64 <68000000>;
+ opp-supported-hw = <0x0003>;
+ };
+
+ opp@68000000,950 {
+ opp-microvolt = <950000 950000 1150000>;
+ opp-hz = /bits/ 64 <68000000>;
+ opp-supported-hw = <0x0008>;
+ };
+
+ opp@68000000,1050 {
+ opp-microvolt = <1050000 1050000 1150000>;
+ opp-hz = /bits/ 64 <68000000>;
+ opp-supported-hw = <0x0010>;
+ };
+
+ opp@68000000,1110 {
+ opp-microvolt = <1110000 1110000 1150000>;
+ opp-hz = /bits/ 64 <68000000>;
+ opp-supported-hw = <0x0004>;
+ };
+
+ opp@102000000,800 {
+ opp-microvolt = <800000 800000 1150000>;
+ opp-hz = /bits/ 64 <102000000>;
+ opp-supported-hw = <0x0003>;
+ };
+
+ opp@102000000,950 {
+ opp-microvolt = <950000 950000 1150000>;
+ opp-hz = /bits/ 64 <102000000>;
+ opp-supported-hw = <0x0008>;
+ };
+
+ opp@102000000,1050 {
+ opp-microvolt = <1050000 1050000 1150000>;
+ opp-hz = /bits/ 64 <102000000>;
+ opp-supported-hw = <0x0010>;
+ };
+
+ opp@102000000,1110 {
+ opp-microvolt = <1110000 1110000 1150000>;
+ opp-hz = /bits/ 64 <102000000>;
+ opp-supported-hw = <0x0004>;
+ };
+
+ opp@204000000,800 {
+ opp-microvolt = <800000 800000 1150000>;
+ opp-hz = /bits/ 64 <204000000>;
+ opp-supported-hw = <0x0003>;
+ };
+
+ opp@204000000,950 {
+ opp-microvolt = <950000 950000 1150000>;
+ opp-hz = /bits/ 64 <204000000>;
+ opp-supported-hw = <0x0008>;
+ };
+
+ opp@204000000,1050 {
+ opp-microvolt = <1050000 1050000 1150000>;
+ opp-hz = /bits/ 64 <204000000>;
+ opp-supported-hw = <0x0010>;
+ };
+
+ opp@204000000,1110 {
+ opp-microvolt = <1110000 1110000 1150000>;
+ opp-hz = /bits/ 64 <204000000>;
+ opp-supported-hw = <0x0004>;
+ };
+
+ opp@264000000,800 {
+ opp-microvolt = <800000 800000 1150000>;
+ opp-hz = /bits/ 64 <264000000>;
+ opp-supported-hw = <0x0003>;
+ };
+
+ opp@264000000,950 {
+ opp-microvolt = <950000 950000 1150000>;
+ opp-hz = /bits/ 64 <264000000>;
+ opp-supported-hw = <0x0008>;
+ };
+
+ opp@264000000,1050 {
+ opp-microvolt = <1050000 1050000 1150000>;
+ opp-hz = /bits/ 64 <264000000>;
+ opp-supported-hw = <0x0010>;
+ };
+
+ opp@264000000,1110 {
+ opp-microvolt = <1110000 1110000 1150000>;
+ opp-hz = /bits/ 64 <264000000>;
+ opp-supported-hw = <0x0004>;
+ };
+
+ opp@300000000,850 {
+ opp-microvolt = <850000 850000 1150000>;
+ opp-hz = /bits/ 64 <300000000>;
+ opp-supported-hw = <0x0003>;
+ };
+
+ opp@300000000,950 {
+ opp-microvolt = <950000 950000 1150000>;
+ opp-hz = /bits/ 64 <300000000>;
+ opp-supported-hw = <0x0008>;
+ };
+
+ opp@300000000,1050 {
+ opp-microvolt = <1050000 1050000 1150000>;
+ opp-hz = /bits/ 64 <300000000>;
+ opp-supported-hw = <0x0010>;
+ };
+
+ opp@300000000,1110 {
+ opp-microvolt = <1110000 1110000 1150000>;
+ opp-hz = /bits/ 64 <300000000>;
+ opp-supported-hw = <0x0004>;
+ };
+
+ opp@348000000,850 {
+ opp-microvolt = <850000 850000 1150000>;
+ opp-hz = /bits/ 64 <348000000>;
+ opp-supported-hw = <0x0003>;
+ };
+
+ opp@348000000,950 {
+ opp-microvolt = <950000 950000 1150000>;
+ opp-hz = /bits/ 64 <348000000>;
+ opp-supported-hw = <0x0008>;
+ };
+
+ opp@348000000,1050 {
+ opp-microvolt = <1050000 1050000 1150000>;
+ opp-hz = /bits/ 64 <348000000>;
+ opp-supported-hw = <0x0010>;
+ };
+
+ opp@348000000,1110 {
+ opp-microvolt = <1110000 1110000 1150000>;
+ opp-hz = /bits/ 64 <348000000>;
+ opp-supported-hw = <0x0004>;
+ };
+
+ opp@396000000,950 {
+ opp-microvolt = <950000 950000 1150000>;
+ opp-hz = /bits/ 64 <396000000>;
+ opp-supported-hw = <0x0008>;
+ };
+
+ opp@396000000,1000 {
+ opp-microvolt = <1000000 1000000 1150000>;
+ opp-hz = /bits/ 64 <396000000>;
+ opp-supported-hw = <0x0003>;
+ };
+
+ opp@396000000,1050 {
+ opp-microvolt = <1050000 1050000 1150000>;
+ opp-hz = /bits/ 64 <396000000>;
+ opp-supported-hw = <0x0010>;
+ };
+
+ opp@396000000,1110 {
+ opp-microvolt = <1110000 1110000 1150000>;
+ opp-hz = /bits/ 64 <396000000>;
+ opp-supported-hw = <0x0004>;
+ };
+
+ opp@528000000,950 {
+ opp-microvolt = <950000 950000 1150000>;
+ opp-hz = /bits/ 64 <528000000>;
+ opp-supported-hw = <0x0008>;
+ };
+
+ opp@528000000,1000 {
+ opp-microvolt = <1000000 1000000 1150000>;
+ opp-hz = /bits/ 64 <528000000>;
+ opp-supported-hw = <0x0003>;
+ };
+
+ opp@528000000,1050 {
+ opp-microvolt = <1050000 1050000 1150000>;
+ opp-hz = /bits/ 64 <528000000>;
+ opp-supported-hw = <0x0010>;
+ };
+
+ opp@528000000,1110 {
+ opp-microvolt = <1110000 1110000 1150000>;
+ opp-hz = /bits/ 64 <528000000>;
+ opp-supported-hw = <0x0004>;
+ };
+
+ opp@600000000,950 {
+ opp-microvolt = <950000 950000 1150000>;
+ opp-hz = /bits/ 64 <600000000>;
+ opp-supported-hw = <0x0008>;
+ };
+
+ opp@600000000,1000 {
+ opp-microvolt = <1000000 1000000 1150000>;
+ opp-hz = /bits/ 64 <600000000>;
+ opp-supported-hw = <0x0003>;
+ };
+
+ opp@600000000,1050 {
+ opp-microvolt = <1050000 1050000 1150000>;
+ opp-hz = /bits/ 64 <600000000>;
+ opp-supported-hw = <0x0010>;
+ };
+
+ opp@600000000,1110 {
+ opp-microvolt = <1110000 1110000 1150000>;
+ opp-hz = /bits/ 64 <600000000>;
+ opp-supported-hw = <0x0004>;
+ };
+
+ opp@792000000,1000 {
+ opp-microvolt = <1000000 1000000 1150000>;
+ opp-hz = /bits/ 64 <792000000>;
+ opp-supported-hw = <0x000B>;
+ };
+
+ opp@792000000,1050 {
+ opp-microvolt = <1050000 1050000 1150000>;
+ opp-hz = /bits/ 64 <792000000>;
+ opp-supported-hw = <0x0010>;
+ };
+
+ opp@792000000,1110 {
+ opp-microvolt = <1110000 1110000 1150000>;
+ opp-hz = /bits/ 64 <792000000>;
+ opp-supported-hw = <0x0004>;
+ };
+
+ opp@924000000,1100 {
+ opp-microvolt = <1100000 1100000 1150000>;
+ opp-hz = /bits/ 64 <924000000>;
+ opp-supported-hw = <0x0013>;
+ };
+
+ opp@1200000000,1100 {
+ opp-microvolt = <1100000 1100000 1150000>;
+ opp-hz = /bits/ 64 <1200000000>;
+ opp-supported-hw = <0x0003>;
+ };
+ };
+
+ emc_bw_dfs_opp_table: emc-bandwidth-opp-table {
+ compatible = "operating-points-v2";
+
+ opp@12750000 {
+ opp-hz = /bits/ 64 <12750000>;
+ opp-supported-hw = <0x001F>;
+ opp-peak-kBps = <204000>;
+ };
+
+ opp@20400000 {
+ opp-hz = /bits/ 64 <20400000>;
+ opp-supported-hw = <0x001F>;
+ opp-peak-kBps = <326400>;
+ };
+
+ opp@40800000 {
+ opp-hz = /bits/ 64 <40800000>;
+ opp-supported-hw = <0x001F>;
+ opp-peak-kBps = <652800>;
+ };
+
+ opp@68000000 {
+ opp-hz = /bits/ 64 <68000000>;
+ opp-supported-hw = <0x001F>;
+ opp-peak-kBps = <1088000>;
+ };
+
+ opp@102000000 {
+ opp-hz = /bits/ 64 <102000000>;
+ opp-supported-hw = <0x001F>;
+ opp-peak-kBps = <1632000>;
+ };
+
+ opp@204000000 {
+ opp-hz = /bits/ 64 <204000000>;
+ opp-supported-hw = <0x001F>;
+ opp-peak-kBps = <3264000>;
+ };
+
+ opp@264000000 {
+ opp-hz = /bits/ 64 <264000000>;
+ opp-supported-hw = <0x001F>;
+ opp-peak-kBps = <4224000>;
+ };
+
+ opp@300000000 {
+ opp-hz = /bits/ 64 <300000000>;
+ opp-supported-hw = <0x001F>;
+ opp-peak-kBps = <4800000>;
+ };
+
+ opp@348000000 {
+ opp-hz = /bits/ 64 <348000000>;
+ opp-supported-hw = <0x001F>;
+ opp-peak-kBps = <5568000>;
+ };
+
+ opp@396000000 {
+ opp-hz = /bits/ 64 <396000000>;
+ opp-supported-hw = <0x001F>;
+ opp-peak-kBps = <6336000>;
+ };
+
+ opp@528000000 {
+ opp-hz = /bits/ 64 <528000000>;
+ opp-supported-hw = <0x001F>;
+ opp-peak-kBps = <8448000>;
+ };
+
+ opp@600000000 {
+ opp-hz = /bits/ 64 <600000000>;
+ opp-supported-hw = <0x001F>;
+ opp-peak-kBps = <9600000>;
+ };
+
+ opp@792000000 {
+ opp-hz = /bits/ 64 <792000000>;
+ opp-supported-hw = <0x001F>;
+ opp-peak-kBps = <12672000>;
+ };
+
+ opp@924000000 {
+ opp-hz = /bits/ 64 <924000000>;
+ opp-supported-hw = <0x0013>;
+ opp-peak-kBps = <14784000>;
+ };
+
+ opp@1200000000 {
+ opp-hz = /bits/ 64 <1200000000>;
+ opp-supported-hw = <0x0003>;
+ opp-peak-kBps = <19200000>;
+ };
+ };
+};
diff --git a/arch/arm/boot/dts/tegra124.dtsi b/arch/arm/boot/dts/tegra124.dtsi
index 1801e30b1d3a..46441d10a3fc 100644
--- a/arch/arm/boot/dts/tegra124.dtsi
+++ b/arch/arm/boot/dts/tegra124.dtsi
@@ -8,6 +8,8 @@
#include <dt-bindings/thermal/tegra124-soctherm.h>
#include <dt-bindings/soc/tegra-pmc.h>

+#include "tegra124-peripherals-opp.dtsi"
+
/ {
compatible = "nvidia,tegra124";
interrupt-parent = <&lic>;
@@ -290,6 +292,9 @@ actmon@6000c800 {
clock-names = "actmon", "emc";
resets = <&tegra_car 119>;
reset-names = "actmon";
+ operating-points-v2 = <&emc_bw_dfs_opp_table>;
+ interconnects = <&mc TEGRA124_MC_MPCORER &emc>;
+ interconnect-names = "cpu-read";
};

gpio: gpio@6000d000 {
@@ -660,6 +665,7 @@ emc: external-memory-controller@7001b000 {
clock-names = "emc";

nvidia,memory-controller = <&mc>;
+ operating-points-v2 = <&emc_icc_dvfs_opp_table>;

#interconnect-cells = <0>;
};
--
2.27.0

2020-11-05 04:47:15

by Chanwoo Choi

[permalink] [raw]
Subject: Re: [PATCH v7 36/47] memory: tegra20-emc: Add devfreq support

On 11/5/20 1:49 AM, Dmitry Osipenko wrote:
> Add devfreq support to the Tegra20 EMC driver. Memory utilization
> statistics will be periodically polled from the memory controller and
> appropriate minimum clock rate will be selected by the devfreq governor.
>
> Signed-off-by: Dmitry Osipenko <[email protected]>
> ---
> drivers/memory/tegra/Kconfig | 2 +
> drivers/memory/tegra/tegra20-emc.c | 92 ++++++++++++++++++++++++++++++
> 2 files changed, 94 insertions(+)
>
> diff --git a/drivers/memory/tegra/Kconfig b/drivers/memory/tegra/Kconfig
> index ac3dfe155505..76e9a3b10839 100644
> --- a/drivers/memory/tegra/Kconfig
> +++ b/drivers/memory/tegra/Kconfig
> @@ -12,6 +12,8 @@ config TEGRA20_EMC
> tristate "NVIDIA Tegra20 External Memory Controller driver"
> default y
> depends on TEGRA_MC && ARCH_TEGRA_2x_SOC
> + select DEVFREQ_GOV_SIMPLE_ONDEMAND
> + select PM_DEVFREQ
> select PM_OPP

nitpick. If you select PM_DEVFREQ, don't need to select 'PM_OPP'
bacause PM_DEVFREQ use OPP as mandatory with 'select PM_OPP' in Kconfig.


> help
> This driver is for the External Memory Controller (EMC) found on
> diff --git a/drivers/memory/tegra/tegra20-emc.c b/drivers/memory/tegra/tegra20-emc.c
> index 5e10aa97809f..9946b957bb01 100644
> --- a/drivers/memory/tegra/tegra20-emc.c
> +++ b/drivers/memory/tegra/tegra20-emc.c
> @@ -8,6 +8,7 @@
> #include <linux/clk.h>
> #include <linux/clk/tegra.h>
> #include <linux/debugfs.h>
> +#include <linux/devfreq.h>
> #include <linux/err.h>
> #include <linux/interconnect-provider.h>
> #include <linux/interrupt.h>
> @@ -102,6 +103,10 @@
>
> #define EMC_FBIO_CFG5_DRAM_WIDTH_X16 BIT(4)
>
> +#define EMC_PWR_GATHER_CLEAR (1 << 8)
> +#define EMC_PWR_GATHER_DISABLE (2 << 8)
> +#define EMC_PWR_GATHER_ENABLE (3 << 8)
> +
> static const u16 emc_timing_registers[] = {
> EMC_RC,
> EMC_RFC,
> @@ -157,6 +162,7 @@ struct emc_timing {
> };
>
> enum emc_rate_request_type {
> + EMC_RATE_DEVFREQ,
> EMC_RATE_DEBUG,
> EMC_RATE_ICC,
> EMC_RATE_TYPE_MAX,
> @@ -193,6 +199,9 @@ struct tegra_emc {
>
> /* protect shared rate-change code path */
> struct mutex rate_lock;
> +
> + struct devfreq_simple_ondemand_data ondemand_data;
> + struct devfreq *devfreq;
> };
>
> static irqreturn_t tegra_emc_isr(int irq, void *data)
> @@ -952,6 +961,88 @@ static int tegra_emc_opp_table_init(struct tegra_emc *emc)
> return err;
> }
>
> +static int tegra_emc_devfreq_target(struct device *dev, unsigned long *freq,
> + u32 flags)
> +{
> + struct tegra_emc *emc = dev_get_drvdata(dev);
> + struct dev_pm_opp *opp;
> + unsigned long rate;
> +
> + opp = devfreq_recommended_opp(dev, freq, flags);
> + if (IS_ERR(opp)) {
> + dev_err(dev, "failed to find opp for %lu Hz\n", *freq);
> + return PTR_ERR(opp);
> + }
> +
> + rate = dev_pm_opp_get_freq(opp);
> + dev_pm_opp_put(opp);
> +
> + return emc_set_min_rate(emc, rate, EMC_RATE_DEVFREQ);
> +}
> +
> +static int tegra_emc_devfreq_get_dev_status(struct device *dev,
> + struct devfreq_dev_status *stat)
> +{
> + struct tegra_emc *emc = dev_get_drvdata(dev);
> +
> + /* freeze counters */
> + writel_relaxed(EMC_PWR_GATHER_DISABLE, emc->regs + EMC_STAT_CONTROL);
> +
> + /*
> + * busy_time: number of clocks EMC request was accepted
> + * total_time: number of clocks PWR_GATHER control was set to ENABLE
> + */
> + stat->busy_time = readl_relaxed(emc->regs + EMC_STAT_PWR_COUNT);
> + stat->total_time = readl_relaxed(emc->regs + EMC_STAT_PWR_CLOCKS);
> + stat->current_frequency = clk_get_rate(emc->clk);
> +
> + /* clear counters and restart */
> + writel_relaxed(EMC_PWR_GATHER_CLEAR, emc->regs + EMC_STAT_CONTROL);
> + writel_relaxed(EMC_PWR_GATHER_ENABLE, emc->regs + EMC_STAT_CONTROL);
> +
> + return 0;
> +}
> +
> +static struct devfreq_dev_profile tegra_emc_devfreq_profile = {
> + .polling_ms = 30,
> + .target = tegra_emc_devfreq_target,
> + .get_dev_status = tegra_emc_devfreq_get_dev_status,
> +};
> +
> +static int tegra_emc_devfreq_init(struct tegra_emc *emc)
> +{
> + int err;
> +
> + /*
> + * PWR_COUNT is 1/2 of PWR_CLOCKS at max, and thus, the up-threshold
> + * should be less than 50. Secondly, multiple active memory clients
> + * may cause over 20% of lost clock cycles due to stalls caused by
> + * competing memory accesses. This means that threshold should be
> + * set to a less than 30 in order to have a properly working governor.
> + */
> + emc->ondemand_data.upthreshold = 20;
> +
> + /*
> + * Reset statistic gathers state, select global bandwidth for the
> + * statistics collection mode and set clocks counter saturation
> + * limit to maximum.
> + */
> + writel_relaxed(0x00000000, emc->regs + EMC_STAT_CONTROL);
> + writel_relaxed(0x00000000, emc->regs + EMC_STAT_LLMC_CONTROL);
> + writel_relaxed(0xffffffff, emc->regs + EMC_STAT_PWR_CLOCK_LIMIT);
> +
> + emc->devfreq = devfreq_add_device(emc->dev, &tegra_emc_devfreq_profile,
> + DEVFREQ_GOV_SIMPLE_ONDEMAND,
> + &emc->ondemand_data);

Do you want to use 'devfreq_add_device' instead of
'devm_devfreq_add_device()'? If you have to use 'devfreq_add_device'
due to some reason, you need to call 'devfreq_remove_device' on exit.

> + if (IS_ERR(emc->devfreq)) {
> + err = PTR_ERR(emc->devfreq);
> + dev_err(emc->dev, "failed to initialize devfreq: %d", err);
> + return err;
> + }
> +
> + return 0;
> +}
> +
> static int tegra_emc_probe(struct platform_device *pdev)
> {
> struct device_node *np;
> @@ -1019,6 +1110,7 @@ static int tegra_emc_probe(struct platform_device *pdev)
> tegra_emc_rate_requests_init(emc);
> tegra_emc_debugfs_init(emc);
> tegra_emc_interconnect_init(emc);
> + tegra_emc_devfreq_init(emc);
>
> /*
> * Don't allow the kernel module to be unloaded. Unloading adds some
>


--
Best Regards,
Chanwoo Choi
Samsung Electronics

2020-11-05 13:55:09

by Dmitry Osipenko

[permalink] [raw]
Subject: Re: [PATCH v7 36/47] memory: tegra20-emc: Add devfreq support

05.11.2020 05:30, Chanwoo Choi пишет:
> On 11/5/20 1:49 AM, Dmitry Osipenko wrote:
>> Add devfreq support to the Tegra20 EMC driver. Memory utilization
>> statistics will be periodically polled from the memory controller and
>> appropriate minimum clock rate will be selected by the devfreq governor.
>>
>> Signed-off-by: Dmitry Osipenko <[email protected]>
>> ---
>> drivers/memory/tegra/Kconfig | 2 +
>> drivers/memory/tegra/tegra20-emc.c | 92 ++++++++++++++++++++++++++++++
>> 2 files changed, 94 insertions(+)
>>
>> diff --git a/drivers/memory/tegra/Kconfig b/drivers/memory/tegra/Kconfig
>> index ac3dfe155505..76e9a3b10839 100644
>> --- a/drivers/memory/tegra/Kconfig
>> +++ b/drivers/memory/tegra/Kconfig
>> @@ -12,6 +12,8 @@ config TEGRA20_EMC
>> tristate "NVIDIA Tegra20 External Memory Controller driver"
>> default y
>> depends on TEGRA_MC && ARCH_TEGRA_2x_SOC
>> + select DEVFREQ_GOV_SIMPLE_ONDEMAND
>> + select PM_DEVFREQ
>> select PM_OPP
>
> nitpick. If you select PM_DEVFREQ, don't need to select 'PM_OPP'
> bacause PM_DEVFREQ use OPP as mandatory with 'select PM_OPP' in Kconfig.

Ok

...
>> +static int tegra_emc_devfreq_init(struct tegra_emc *emc)
>> +{
>> + int err;
>> +
>> + /*
>> + * PWR_COUNT is 1/2 of PWR_CLOCKS at max, and thus, the up-threshold
>> + * should be less than 50. Secondly, multiple active memory clients
>> + * may cause over 20% of lost clock cycles due to stalls caused by
>> + * competing memory accesses. This means that threshold should be
>> + * set to a less than 30 in order to have a properly working governor.
>> + */
>> + emc->ondemand_data.upthreshold = 20;
>> +
>> + /*
>> + * Reset statistic gathers state, select global bandwidth for the
>> + * statistics collection mode and set clocks counter saturation
>> + * limit to maximum.
>> + */
>> + writel_relaxed(0x00000000, emc->regs + EMC_STAT_CONTROL);
>> + writel_relaxed(0x00000000, emc->regs + EMC_STAT_LLMC_CONTROL);
>> + writel_relaxed(0xffffffff, emc->regs + EMC_STAT_PWR_CLOCK_LIMIT);
>> +
>> + emc->devfreq = devfreq_add_device(emc->dev, &tegra_emc_devfreq_profile,
>> + DEVFREQ_GOV_SIMPLE_ONDEMAND,
>> + &emc->ondemand_data);
>
> Do you want to use 'devfreq_add_device' instead of
> 'devm_devfreq_add_device()'? If you have to use 'devfreq_add_device'
> due to some reason, you need to call 'devfreq_remove_device' on exit.

The reason I didn't use the devm here is because the EMC-clk callback
should be unregistered *after* devfreq is removed.

Thinking a bit more about it, I guess the best variant will be to add
devm support to the clk callback registration and then it should be
possible to use devm for the devfreq. I'll try to implement it in v8,
thanks.

2020-11-06 18:35:59

by Krzysztof Kozlowski

[permalink] [raw]
Subject: Re: [PATCH v7 10/47] dt-bindings: memory: tegra30: emc: Document new interconnect property

On Wed, Nov 04, 2020 at 07:48:46PM +0300, Dmitry Osipenko wrote:
> External memory controller is interconnected with memory controller and
> with external memory. Document new interconnect property which turns
> External Memory Controller into interconnect provider.
>
> Acked-by: Rob Herring <[email protected]>
> Signed-off-by: Dmitry Osipenko <[email protected]>
> ---
> .../bindings/memory-controllers/nvidia,tegra30-emc.yaml | 6 ++++++
> 1 file changed, 6 insertions(+)

Thanks, applied.

Best regards,
Krzysztof

2020-11-06 18:38:05

by Krzysztof Kozlowski

[permalink] [raw]
Subject: Re: [PATCH v7 16/47] dt-bindings: host1x: Document new interconnect properties

On Wed, Nov 04, 2020 at 07:48:52PM +0300, Dmitry Osipenko wrote:
> Most of Host1x devices have at least one memory client. These clients
> are directly connected to the memory controller. The new interconnect
> properties represent the memory client's connection to the memory
> controller.
>
> Reviewed-by: Rob Herring <[email protected]>
> Signed-off-by: Dmitry Osipenko <[email protected]>
> ---
> .../display/tegra/nvidia,tegra20-host1x.txt | 68 +++++++++++++++++++
> 1 file changed, 68 insertions(+)

Does not look like patch for memory controller drivers but I guess
better to keep it with others. Let me know if it should go via different
tree.

Thanks, applied.

Best regards,
Krzysztof

2020-11-06 19:05:46

by Krzysztof Kozlowski

[permalink] [raw]
Subject: Re: [PATCH v7 29/47] memory: tegra: Use devm_platform_ioremap_resource()

On Wed, Nov 04, 2020 at 07:49:05PM +0300, Dmitry Osipenko wrote:
> Use devm_platform_ioremap_resource() helper which makes code a bit
> cleaner.
>
> Acked-by: Thierry Reding <[email protected]>
> Signed-off-by: Dmitry Osipenko <[email protected]>
> ---
> drivers/memory/tegra/tegra124-emc.c | 4 +---
> drivers/memory/tegra/tegra20-emc.c | 4 +---
> 2 files changed, 2 insertions(+), 6 deletions(-)

Thanks, applied.

Best regards,
Krzysztof

2020-11-06 19:15:32

by Krzysztof Kozlowski

[permalink] [raw]
Subject: Re: [PATCH v7 36/47] memory: tegra20-emc: Add devfreq support

On Wed, Nov 04, 2020 at 07:49:12PM +0300, Dmitry Osipenko wrote:
> Add devfreq support to the Tegra20 EMC driver. Memory utilization
> statistics will be periodically polled from the memory controller and
> appropriate minimum clock rate will be selected by the devfreq governor.
>
> Signed-off-by: Dmitry Osipenko <[email protected]>
> ---
> drivers/memory/tegra/Kconfig | 2 +
> drivers/memory/tegra/tegra20-emc.c | 92 ++++++++++++++++++++++++++++++
> 2 files changed, 94 insertions(+)
>

I see this one still received comments. I skipped the DTS patches and
applied everything till patch #35. I understand you will send v8, so in
such case please skip the applied ones (you can rebase on my for-next or
on Monday's linux-next).

Best regards,
Krzysztof

2020-11-06 21:57:52

by Dmitry Osipenko

[permalink] [raw]
Subject: Re: [PATCH v7 36/47] memory: tegra20-emc: Add devfreq support

06.11.2020 22:13, Krzysztof Kozlowski пишет:
> On Wed, Nov 04, 2020 at 07:49:12PM +0300, Dmitry Osipenko wrote:
>> Add devfreq support to the Tegra20 EMC driver. Memory utilization
>> statistics will be periodically polled from the memory controller and
>> appropriate minimum clock rate will be selected by the devfreq governor.
>>
>> Signed-off-by: Dmitry Osipenko <[email protected]>
>> ---
>> drivers/memory/tegra/Kconfig | 2 +
>> drivers/memory/tegra/tegra20-emc.c | 92 ++++++++++++++++++++++++++++++
>> 2 files changed, 94 insertions(+)
>>
>
> I see this one still received comments. I skipped the DTS patches and
> applied everything till patch #35. I understand you will send v8, so in
> such case please skip the applied ones (you can rebase on my for-next or
> on Monday's linux-next).

Thank you! I'll also need to wait for a reply from Viresh Kumar in other
thread regarding dev_pm_opp_get_opp_table() usage and then will probably
need to correct patch #35+ as well now, since turned out it may be wrong
for drivers to use dev_pm_opp_get_opp_table().