Tightly-Coupled Memories(TCMs) are low-latency memory that provides
predictable instruction execution and predictable data load/store
timing. Each Cortex-R5F processor contains exclusive two 64 KB memory
banks on the ATCM and BTCM ports, for a total of 128 KB of memory.
In lockstep mode, both 128KB memory is accessible to the cluster.
As per ZynqMP Ultrascale+ Technical Reference Manual UG1085, following
is address space of TCM memory. The bindings in this patch series
introduces properties to accommodate following address space with
address translation between Linux and Cortex-R5 views.
| | | |
| --- | --- | --- |
| *Mode* | *R5 View* | *Linux view* | Notes |
| *Split Mode* | *start addr*| *start addr* | |
| R5_0 ATCM (64 KB) | 0x0000_0000 | 0xFFE0_0000 | |
| R5_0 BTCM (64 KB) | 0x0002_0000 | 0xFFE2_0000 | |
| R5_1 ATCM (64 KB) | 0x0000_0000 | 0xFFE9_0000 | alias of 0xFFE1_0000 |
| R5_1 BTCM (64 KB) | 0x0002_0000 | 0xFFEB_0000 | alias of 0xFFE3_0000 |
| ___ | ___ | ___ | |
| *Lockstep Mode* | | | |
| R5_0 ATCM (128 KB) | 0x0000_0000 | 0xFFE0_0000 | |
| R5_0 BTCM (128 KB) | 0x0002_0000 | 0xFFE2_0000 | |
References:
UG1085 TCM address space:
https://docs.xilinx.com/r/en-US/ug1085-zynq-ultrascale-trm/Tightly-Coupled-Memory-Address-Map
Changes in v11:
- Fix yamllint warning and reduce indentation as needed
- Remove redundant initialization of the variable
- Return correct error code if memory allocation failed
Changs in v10:
- Add new patch (1/4) to series that changes hardcode TCM addresses in
lockstep mode and removes separate handling of TCM in lockstep and
split mode
- modify number of "reg", "reg-names" and "power-domains" entries
based on cluster mode
- Add extra optional atcm and btcm in "reg" property for lockstep mode
- Add "reg-names" for extra optional atcm and btcm for lockstep mode
- Drop previous Ack as bindings has new change
- Add individual tcm regions via "reg" and "reg-names" for lockstep mode
- Add each tcm's power-domains in lockstep mode
- Drop previous Ack as new change in dts patchset
- Remove redundant changes in driver to handle TCM in lockstep mode
Changes in v9:
- Fix rproc lockstep dts
- Introduce new API to request and release core1 TCM power-domains in
lockstep mode. This will be used during prepare -> add_tcm_banks
callback to enable TCM in lockstep mode.
- Parse TCM from device-tree in lockstep mode and split mode in
uniform way.
- Fix TCM representation in device-tree in lockstep mode.
- Fix comments as suggested
Changes in v8:
- Remove use of pm_domains framework
- Remove checking of pm_domain_id validation to power on/off tcm
- Remove spurious change
- parse power-domains property from device-tree and use EEMI calls
to power on/off TCM instead of using pm domains framework
Changes in v7:
- %s/pm_dev1/pm_dev_core0/r
- %s/pm_dev_link1/pm_dev_core0_link/r
- %s/pm_dev2/pm_dev_core1/r
- %s/pm_dev_link2/pm_dev_core1_link/r
- remove pm_domain_id check to move next patch
- add comment about how 1st entry in pm domain list is used
- fix loop when jump to fail_add_pm_domains loop
- move checking of pm_domain_id from previous patch
- fix mem_bank_data memory allocation
Changes in v6:
- Introduce new node entry for r5f cluster split mode dts and
keep it disabled by default.
- Keep remoteproc lockstep mode enabled by default to maintian
back compatibility.
- Enable split mode only for zcu102 board to demo split mode use
- Remove spurious change
- Handle errors in add_pm_domains function
- Remove redundant code to handle errors from remove_pm_domains
- Missing . at the end of the commit message
- remove redundant initialization of variables
- remove fail_tcm label and relevant code to free memory
acquired using devm_* API. As this will be freed when device free it
- add extra check to see if "reg" property is supported or not
Changes in v5:
- maintain Rob's Ack on bindings patch as no changes in bindings
- split previous patch into multiple patches
- Use pm domain framework to turn on/off TCM
- Add support of parsing TCM information from device-tree
- maintain backward compatibility with previous bindings without
TCM information available in device-tree
This patch series continues previous effort to upstream ZynqMP
TCM bindings:
Previous v4 version link:
https://lore.kernel.org/all/[email protected]/
Previous v3 version link:
https://lore.kernel.org/all/[email protected]/
Radhey Shyam Pandey (1):
dt-bindings: remoteproc: add Tightly Coupled Memory (TCM) bindings
Radhey Shyam Pandey (1):
dt-bindings: remoteproc: add Tightly Coupled Memory (TCM) bindings
Tanmay Shah (3):
remoteproc: zynqmp: fix lockstep mode memory region
dts: zynqmp: add properties for TCM in remoteproc
remoteproc: zynqmp: parse TCM from device tree
.../remoteproc/xlnx,zynqmp-r5fss.yaml | 192 +++++++++++--
.../boot/dts/xilinx/zynqmp-zcu102-rev1.0.dts | 8 +
arch/arm64/boot/dts/xilinx/zynqmp.dtsi | 65 ++++-
drivers/remoteproc/xlnx_r5_remoteproc.c | 257 ++++++++----------
4 files changed, 357 insertions(+), 165 deletions(-)
base-commit: 929654e8f1add50b01d5a56171a31c311b0a739a
--
2.25.1
In lockstep mode, r5 core0 uses TCM of R5 core1. Following is lockstep
mode memory region as per hardware reference manual.
| *TCM* | *R5 View* | *Linux view* |
| R5_0 ATCM (128 KB) | 0x0000_0000 | 0xFFE0_0000 |
| R5_0 BTCM (128 KB) | 0x0002_0000 | 0xFFE2_0000 |
However, driver shouldn't model it as above because R5 core0 TCM and core1
TCM has different power-domains mapped to it.
Hence, TCM address space in lockstep mode should be modeled as 64KB
regions only where each region has its own power-domain as following:
| *TCM* | *R5 View* | *Linux view* |
| R5_0 ATCM0 (64 KB) | 0x0000_0000 | 0xFFE0_0000 |
| R5_0 BTCM0 (64 KB) | 0x0002_0000 | 0xFFE2_0000 |
| R5_0 ATCM1 (64 KB) | 0x0001_0000 | 0xFFE1_0000 |
| R5_0 BTCM1 (64 KB) | 0x0003_0000 | 0xFFE3_0000 |
This makes driver maintanance easy and makes design robust for future
platorms as well.
Signed-off-by: Tanmay Shah <[email protected]>
---
drivers/remoteproc/xlnx_r5_remoteproc.c | 145 ++----------------------
1 file changed, 12 insertions(+), 133 deletions(-)
diff --git a/drivers/remoteproc/xlnx_r5_remoteproc.c b/drivers/remoteproc/xlnx_r5_remoteproc.c
index 4395edea9a64..42b0384d34f2 100644
--- a/drivers/remoteproc/xlnx_r5_remoteproc.c
+++ b/drivers/remoteproc/xlnx_r5_remoteproc.c
@@ -84,12 +84,12 @@ static const struct mem_bank_data zynqmp_tcm_banks_split[] = {
{0xffeb0000UL, 0x20000, 0x10000UL, PD_R5_1_BTCM, "btcm1"},
};
-/* In lockstep mode cluster combines each 64KB TCM and makes 128KB TCM */
+/* In lockstep mode cluster uses each 64KB TCM from second core as well */
static const struct mem_bank_data zynqmp_tcm_banks_lockstep[] = {
- {0xffe00000UL, 0x0, 0x20000UL, PD_R5_0_ATCM, "atcm0"}, /* TCM 128KB each */
- {0xffe20000UL, 0x20000, 0x20000UL, PD_R5_0_BTCM, "btcm0"},
- {0, 0, 0, PD_R5_1_ATCM, ""},
- {0, 0, 0, PD_R5_1_BTCM, ""},
+ {0xffe00000UL, 0x0, 0x10000UL, PD_R5_0_ATCM, "atcm0"}, /* TCM 64KB each */
+ {0xffe20000UL, 0x20000, 0x10000UL, PD_R5_0_BTCM, "btcm0"},
+ {0xffe10000UL, 0x10000, 0x10000UL, PD_R5_1_ATCM, "atcm1"},
+ {0xffe30000UL, 0x30000, 0x10000UL, PD_R5_1_BTCM, "btcm1"},
};
/**
@@ -540,14 +540,14 @@ static int tcm_mem_map(struct rproc *rproc,
}
/*
- * add_tcm_carveout_split_mode()
+ * add_tcm_banks()
* @rproc: single R5 core's corresponding rproc instance
*
- * allocate and add remoteproc carveout for TCM memory in split mode
+ * allocate and add remoteproc carveout for TCM memory
*
* return 0 on success, otherwise non-zero value on failure
*/
-static int add_tcm_carveout_split_mode(struct rproc *rproc)
+static int add_tcm_banks(struct rproc *rproc)
{
struct rproc_mem_entry *rproc_mem;
struct zynqmp_r5_core *r5_core;
@@ -580,10 +580,10 @@ static int add_tcm_carveout_split_mode(struct rproc *rproc)
ZYNQMP_PM_REQUEST_ACK_BLOCKING);
if (ret < 0) {
dev_err(dev, "failed to turn on TCM 0x%x", pm_domain_id);
- goto release_tcm_split;
+ goto release_tcm;
}
- dev_dbg(dev, "TCM carveout split mode %s addr=%llx, da=0x%x, size=0x%lx",
+ dev_dbg(dev, "TCM carveout %s addr=%llx, da=0x%x, size=0x%lx",
bank_name, bank_addr, da, bank_size);
rproc_mem = rproc_mem_entry_init(dev, NULL, bank_addr,
@@ -593,7 +593,7 @@ static int add_tcm_carveout_split_mode(struct rproc *rproc)
if (!rproc_mem) {
ret = -ENOMEM;
zynqmp_pm_release_node(pm_domain_id);
- goto release_tcm_split;
+ goto release_tcm;
}
rproc_add_carveout(rproc, rproc_mem);
@@ -601,7 +601,7 @@ static int add_tcm_carveout_split_mode(struct rproc *rproc)
return 0;
-release_tcm_split:
+release_tcm:
/* If failed, Turn off all TCM banks turned on before */
for (i--; i >= 0; i--) {
pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
@@ -610,127 +610,6 @@ static int add_tcm_carveout_split_mode(struct rproc *rproc)
return ret;
}
-/*
- * add_tcm_carveout_lockstep_mode()
- * @rproc: single R5 core's corresponding rproc instance
- *
- * allocate and add remoteproc carveout for TCM memory in lockstep mode
- *
- * return 0 on success, otherwise non-zero value on failure
- */
-static int add_tcm_carveout_lockstep_mode(struct rproc *rproc)
-{
- struct rproc_mem_entry *rproc_mem;
- struct zynqmp_r5_core *r5_core;
- int i, num_banks, ret;
- phys_addr_t bank_addr;
- size_t bank_size = 0;
- struct device *dev;
- u32 pm_domain_id;
- char *bank_name;
- u32 da;
-
- r5_core = rproc->priv;
- dev = r5_core->dev;
-
- /* Go through zynqmp banks for r5 node */
- num_banks = r5_core->tcm_bank_count;
-
- /*
- * In lockstep mode, TCM is contiguous memory block
- * However, each TCM block still needs to be enabled individually.
- * So, Enable each TCM block individually.
- * Although ATCM and BTCM is contiguous memory block, add two separate
- * carveouts for both.
- */
- for (i = 0; i < num_banks; i++) {
- pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
-
- /* Turn on each TCM bank individually */
- ret = zynqmp_pm_request_node(pm_domain_id,
- ZYNQMP_PM_CAPABILITY_ACCESS, 0,
- ZYNQMP_PM_REQUEST_ACK_BLOCKING);
- if (ret < 0) {
- dev_err(dev, "failed to turn on TCM 0x%x", pm_domain_id);
- goto release_tcm_lockstep;
- }
-
- bank_size = r5_core->tcm_banks[i]->size;
- if (bank_size == 0)
- continue;
-
- bank_addr = r5_core->tcm_banks[i]->addr;
- da = r5_core->tcm_banks[i]->da;
- bank_name = r5_core->tcm_banks[i]->bank_name;
-
- /* Register TCM address range, TCM map and unmap functions */
- rproc_mem = rproc_mem_entry_init(dev, NULL, bank_addr,
- bank_size, da,
- tcm_mem_map, tcm_mem_unmap,
- bank_name);
- if (!rproc_mem) {
- ret = -ENOMEM;
- zynqmp_pm_release_node(pm_domain_id);
- goto release_tcm_lockstep;
- }
-
- /* If registration is success, add carveouts */
- rproc_add_carveout(rproc, rproc_mem);
-
- dev_dbg(dev, "TCM carveout lockstep mode %s addr=0x%llx, da=0x%x, size=0x%lx",
- bank_name, bank_addr, da, bank_size);
- }
-
- return 0;
-
-release_tcm_lockstep:
- /* If failed, Turn off all TCM banks turned on before */
- for (i--; i >= 0; i--) {
- pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
- zynqmp_pm_release_node(pm_domain_id);
- }
- return ret;
-}
-
-/*
- * add_tcm_banks()
- * @rproc: single R5 core's corresponding rproc instance
- *
- * allocate and add remoteproc carveouts for TCM memory based on cluster mode
- *
- * return 0 on success, otherwise non-zero value on failure
- */
-static int add_tcm_banks(struct rproc *rproc)
-{
- struct zynqmp_r5_cluster *cluster;
- struct zynqmp_r5_core *r5_core;
- struct device *dev;
-
- r5_core = rproc->priv;
- if (!r5_core)
- return -EINVAL;
-
- dev = r5_core->dev;
-
- cluster = dev_get_drvdata(dev->parent);
- if (!cluster) {
- dev_err(dev->parent, "Invalid driver data\n");
- return -EINVAL;
- }
-
- /*
- * In lockstep mode TCM banks are one contiguous memory region of 256Kb
- * In split mode, each TCM bank is 64Kb and not contiguous.
- * We add memory carveouts accordingly.
- */
- if (cluster->mode == SPLIT_MODE)
- return add_tcm_carveout_split_mode(rproc);
- else if (cluster->mode == LOCKSTEP_MODE)
- return add_tcm_carveout_lockstep_mode(rproc);
-
- return -EINVAL;
-}
-
/*
* zynqmp_r5_parse_fw()
* @rproc: single R5 core's corresponding rproc instance
--
2.25.1
Add properties as per new bindings in zynqmp remoteproc node
to represent TCM address and size.
This patch also adds alternative remoteproc node to represent
remoteproc cluster in split mode. By default lockstep mode is
enabled and users should disable it before using split mode
dts. Both device-tree nodes can't be used simultaneously one
of them must be disabled. For zcu102-1.0 and zcu102-1.1 board
remoteproc split mode dts node is enabled and lockstep mode
dts is disabled.
Signed-off-by: Tanmay Shah <[email protected]>
---
.../boot/dts/xilinx/zynqmp-zcu102-rev1.0.dts | 8 +++
arch/arm64/boot/dts/xilinx/zynqmp.dtsi | 65 +++++++++++++++++--
2 files changed, 68 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/boot/dts/xilinx/zynqmp-zcu102-rev1.0.dts b/arch/arm64/boot/dts/xilinx/zynqmp-zcu102-rev1.0.dts
index c8f71a1aec89..495ca94b45db 100644
--- a/arch/arm64/boot/dts/xilinx/zynqmp-zcu102-rev1.0.dts
+++ b/arch/arm64/boot/dts/xilinx/zynqmp-zcu102-rev1.0.dts
@@ -14,6 +14,14 @@ / {
compatible = "xlnx,zynqmp-zcu102-rev1.0", "xlnx,zynqmp-zcu102", "xlnx,zynqmp";
};
+&rproc_split {
+ status = "okay";
+};
+
+&rproc_lockstep {
+ status = "disabled";
+};
+
&eeprom {
#address-cells = <1>;
#size-cells = <1>;
diff --git a/arch/arm64/boot/dts/xilinx/zynqmp.dtsi b/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
index eaba466804bc..c8a7fd0f3a1e 100644
--- a/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
+++ b/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
@@ -248,19 +248,74 @@ fpga_full: fpga-full {
ranges;
};
- remoteproc {
+ rproc_lockstep: remoteproc@ffe00000 {
compatible = "xlnx,zynqmp-r5fss";
xlnx,cluster-mode = <1>;
- r5f-0 {
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ ranges = <0x0 0x0 0x0 0xffe00000 0x0 0x10000>,
+ <0x0 0x20000 0x0 0xffe20000 0x0 0x10000>,
+ <0x0 0x10000 0x0 0xffe10000 0x0 0x10000>,
+ <0x0 0x30000 0x0 0xffe30000 0x0 0x10000>;
+
+ r5f@0 {
+ compatible = "xlnx,zynqmp-r5f";
+ reg = <0x0 0x0 0x0 0x10000>,
+ <0x0 0x20000 0x0 0x10000>,
+ <0x0 0x10000 0x0 0x10000>,
+ <0x0 0x30000 0x0 0x10000>;
+ reg-names = "atcm0", "btcm0", "atcm1", "btcm1";
+ power-domains = <&zynqmp_firmware PD_RPU_0>,
+ <&zynqmp_firmware PD_R5_0_ATCM>,
+ <&zynqmp_firmware PD_R5_0_BTCM>,
+ <&zynqmp_firmware PD_R5_1_ATCM>,
+ <&zynqmp_firmware PD_R5_1_BTCM>;
+ memory-region = <&rproc_0_fw_image>;
+ };
+
+ r5f@1 {
+ compatible = "xlnx,zynqmp-r5f";
+ reg = <0x1 0x0 0x0 0x10000>, <0x1 0x20000 0x0 0x10000>;
+ reg-names = "atcm0", "btcm0";
+ power-domains = <&zynqmp_firmware PD_RPU_1>,
+ <&zynqmp_firmware PD_R5_1_ATCM>,
+ <&zynqmp_firmware PD_R5_1_BTCM>;
+ memory-region = <&rproc_1_fw_image>;
+ };
+ };
+
+ rproc_split: remoteproc-split@ffe00000 {
+ status = "disabled";
+ compatible = "xlnx,zynqmp-r5fss";
+ xlnx,cluster-mode = <0>;
+
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ ranges = <0x0 0x0 0x0 0xffe00000 0x0 0x10000>,
+ <0x0 0x20000 0x0 0xffe20000 0x0 0x10000>,
+ <0x1 0x0 0x0 0xffe90000 0x0 0x10000>,
+ <0x1 0x20000 0x0 0xffeb0000 0x0 0x10000>;
+
+ r5f@0 {
compatible = "xlnx,zynqmp-r5f";
- power-domains = <&zynqmp_firmware PD_RPU_0>;
+ reg = <0x0 0x0 0x0 0x10000>, <0x0 0x20000 0x0 0x10000>;
+ reg-names = "atcm0", "btcm0";
+ power-domains = <&zynqmp_firmware PD_RPU_0>,
+ <&zynqmp_firmware PD_R5_0_ATCM>,
+ <&zynqmp_firmware PD_R5_0_BTCM>;
memory-region = <&rproc_0_fw_image>;
};
- r5f-1 {
+ r5f@1 {
compatible = "xlnx,zynqmp-r5f";
- power-domains = <&zynqmp_firmware PD_RPU_1>;
+ reg = <0x1 0x0 0x0 0x10000>, <0x1 0x20000 0x0 0x10000>;
+ reg-names = "atcm0", "btcm0";
+ power-domains = <&zynqmp_firmware PD_RPU_1>,
+ <&zynqmp_firmware PD_R5_1_ATCM>,
+ <&zynqmp_firmware PD_R5_1_BTCM>;
memory-region = <&rproc_1_fw_image>;
};
};
--
2.25.1
ZynqMP TCM information was fixed in driver. Now ZynqMP TCM information
is available in device-tree. Parse TCM information in driver
as per new bindings.
Signed-off-by: Tanmay Shah <[email protected]>
---
Changes in v11:
- Remove redundant initialization of the variable
- return correct error code if memory allocation failed
drivers/remoteproc/xlnx_r5_remoteproc.c | 112 ++++++++++++++++++++++--
1 file changed, 107 insertions(+), 5 deletions(-)
diff --git a/drivers/remoteproc/xlnx_r5_remoteproc.c b/drivers/remoteproc/xlnx_r5_remoteproc.c
index 42b0384d34f2..d4a22caebaad 100644
--- a/drivers/remoteproc/xlnx_r5_remoteproc.c
+++ b/drivers/remoteproc/xlnx_r5_remoteproc.c
@@ -74,8 +74,8 @@ struct mbox_info {
};
/*
- * Hardcoded TCM bank values. This will be removed once TCM bindings are
- * accepted for system-dt specifications and upstreamed in linux kernel
+ * Hardcoded TCM bank values. This will stay in driver to maintain backward
+ * compatibility with device-tree that does not have TCM information.
*/
static const struct mem_bank_data zynqmp_tcm_banks_split[] = {
{0xffe00000UL, 0x0, 0x10000UL, PD_R5_0_ATCM, "atcm0"}, /* TCM 64KB each */
@@ -757,6 +757,103 @@ static struct zynqmp_r5_core *zynqmp_r5_add_rproc_core(struct device *cdev)
return ERR_PTR(ret);
}
+static int zynqmp_r5_get_tcm_node_from_dt(struct zynqmp_r5_cluster *cluster)
+{
+ int i, j, tcm_bank_count, ret, tcm_pd_idx, pd_count;
+ struct of_phandle_args out_args;
+ struct zynqmp_r5_core *r5_core;
+ struct platform_device *cpdev;
+ struct mem_bank_data *tcm;
+ struct device_node *np;
+ struct resource *res;
+ u64 abs_addr, size;
+ struct device *dev;
+
+ for (i = 0; i < cluster->core_count; i++) {
+ r5_core = cluster->r5_cores[i];
+ dev = r5_core->dev;
+ np = r5_core->np;
+
+ pd_count = of_count_phandle_with_args(np, "power-domains",
+ "#power-domain-cells");
+
+ if (pd_count <= 0) {
+ dev_err(dev, "invalid power-domains property, %d\n", pd_count);
+ return -EINVAL;
+ }
+
+ /* First entry in power-domains list is for r5 core, rest for TCM. */
+ tcm_bank_count = pd_count - 1;
+
+ if (tcm_bank_count <= 0) {
+ dev_err(dev, "invalid TCM count %d\n", tcm_bank_count);
+ return -EINVAL;
+ }
+
+ r5_core->tcm_banks = devm_kcalloc(dev, tcm_bank_count,
+ sizeof(struct mem_bank_data *),
+ GFP_KERNEL);
+ if (!r5_core->tcm_banks)
+ return -ENOMEM;
+
+ r5_core->tcm_bank_count = tcm_bank_count;
+ for (j = 0, tcm_pd_idx = 1; j < tcm_bank_count; j++, tcm_pd_idx++) {
+ tcm = devm_kzalloc(dev, sizeof(struct mem_bank_data),
+ GFP_KERNEL);
+ if (!tcm)
+ return -ENOMEM;
+
+ r5_core->tcm_banks[j] = tcm;
+
+ /* Get power-domains id of TCM. */
+ ret = of_parse_phandle_with_args(np, "power-domains",
+ "#power-domain-cells",
+ tcm_pd_idx, &out_args);
+ if (ret) {
+ dev_err(r5_core->dev,
+ "failed to get tcm %d pm domain, ret %d\n",
+ tcm_pd_idx, ret);
+ return ret;
+ }
+ tcm->pm_domain_id = out_args.args[0];
+ of_node_put(out_args.np);
+
+ /* Get TCM address without translation. */
+ ret = of_property_read_reg(np, j, &abs_addr, &size);
+ if (ret) {
+ dev_err(dev, "failed to get reg property\n");
+ return ret;
+ }
+
+ /*
+ * Remote processor can address only 32 bits
+ * so convert 64-bits into 32-bits. This will discard
+ * any unwanted upper 32-bits.
+ */
+ tcm->da = (u32)abs_addr;
+ tcm->size = (u32)size;
+
+ cpdev = to_platform_device(dev);
+ res = platform_get_resource(cpdev, IORESOURCE_MEM, j);
+ if (!res) {
+ dev_err(dev, "failed to get tcm resource\n");
+ return -EINVAL;
+ }
+
+ tcm->addr = (u32)res->start;
+ tcm->bank_name = (char *)res->name;
+ res = devm_request_mem_region(dev, tcm->addr, tcm->size,
+ tcm->bank_name);
+ if (!res) {
+ dev_err(dev, "failed to request tcm resource\n");
+ return -EINVAL;
+ }
+ }
+ }
+
+ return 0;
+}
+
/**
* zynqmp_r5_get_tcm_node()
* Ideally this function should parse tcm node and store information
@@ -835,9 +932,14 @@ static int zynqmp_r5_core_init(struct zynqmp_r5_cluster *cluster,
struct zynqmp_r5_core *r5_core;
int ret, i;
- ret = zynqmp_r5_get_tcm_node(cluster);
- if (ret < 0) {
- dev_err(dev, "can't get tcm node, err %d\n", ret);
+ r5_core = cluster->r5_cores[0];
+ if (of_find_property(r5_core->np, "reg", NULL))
+ ret = zynqmp_r5_get_tcm_node_from_dt(cluster);
+ else
+ ret = zynqmp_r5_get_tcm_node(cluster);
+
+ if (ret) {
+ dev_err(dev, "can't get tcm, err %d\n", ret);
return ret;
}
--
2.25.1
In lockstep mode, r5 core0 uses TCM of R5 core1. Following is lockstep
mode memory region as per hardware reference manual.
| *TCM* | *R5 View* | *Linux view* |
| R5_0 ATCM (128 KB) | 0x0000_0000 | 0xFFE0_0000 |
| R5_0 BTCM (128 KB) | 0x0002_0000 | 0xFFE2_0000 |
However, driver shouldn't model it as above because R5 core0 TCM and core1
TCM has different power-domains mapped to it.
Hence, TCM address space in lockstep mode should be modeled as 64KB
regions only where each region has its own power-domain as following:
| *TCM* | *R5 View* | *Linux view* |
| R5_0 ATCM0 (64 KB) | 0x0000_0000 | 0xFFE0_0000 |
| R5_0 BTCM0 (64 KB) | 0x0002_0000 | 0xFFE2_0000 |
| R5_0 ATCM1 (64 KB) | 0x0001_0000 | 0xFFE1_0000 |
| R5_0 BTCM1 (64 KB) | 0x0003_0000 | 0xFFE3_0000 |
This makes driver maintanance easy and makes design robust for future
platorms as well.
Signed-off-by: Tanmay Shah <[email protected]>
---
drivers/remoteproc/xlnx_r5_remoteproc.c | 145 ++----------------------
1 file changed, 12 insertions(+), 133 deletions(-)
diff --git a/drivers/remoteproc/xlnx_r5_remoteproc.c b/drivers/remoteproc/xlnx_r5_remoteproc.c
index 4395edea9a64..42b0384d34f2 100644
--- a/drivers/remoteproc/xlnx_r5_remoteproc.c
+++ b/drivers/remoteproc/xlnx_r5_remoteproc.c
@@ -84,12 +84,12 @@ static const struct mem_bank_data zynqmp_tcm_banks_split[] = {
{0xffeb0000UL, 0x20000, 0x10000UL, PD_R5_1_BTCM, "btcm1"},
};
-/* In lockstep mode cluster combines each 64KB TCM and makes 128KB TCM */
+/* In lockstep mode cluster uses each 64KB TCM from second core as well */
static const struct mem_bank_data zynqmp_tcm_banks_lockstep[] = {
- {0xffe00000UL, 0x0, 0x20000UL, PD_R5_0_ATCM, "atcm0"}, /* TCM 128KB each */
- {0xffe20000UL, 0x20000, 0x20000UL, PD_R5_0_BTCM, "btcm0"},
- {0, 0, 0, PD_R5_1_ATCM, ""},
- {0, 0, 0, PD_R5_1_BTCM, ""},
+ {0xffe00000UL, 0x0, 0x10000UL, PD_R5_0_ATCM, "atcm0"}, /* TCM 64KB each */
+ {0xffe20000UL, 0x20000, 0x10000UL, PD_R5_0_BTCM, "btcm0"},
+ {0xffe10000UL, 0x10000, 0x10000UL, PD_R5_1_ATCM, "atcm1"},
+ {0xffe30000UL, 0x30000, 0x10000UL, PD_R5_1_BTCM, "btcm1"},
};
/**
@@ -540,14 +540,14 @@ static int tcm_mem_map(struct rproc *rproc,
}
/*
- * add_tcm_carveout_split_mode()
+ * add_tcm_banks()
* @rproc: single R5 core's corresponding rproc instance
*
- * allocate and add remoteproc carveout for TCM memory in split mode
+ * allocate and add remoteproc carveout for TCM memory
*
* return 0 on success, otherwise non-zero value on failure
*/
-static int add_tcm_carveout_split_mode(struct rproc *rproc)
+static int add_tcm_banks(struct rproc *rproc)
{
struct rproc_mem_entry *rproc_mem;
struct zynqmp_r5_core *r5_core;
@@ -580,10 +580,10 @@ static int add_tcm_carveout_split_mode(struct rproc *rproc)
ZYNQMP_PM_REQUEST_ACK_BLOCKING);
if (ret < 0) {
dev_err(dev, "failed to turn on TCM 0x%x", pm_domain_id);
- goto release_tcm_split;
+ goto release_tcm;
}
- dev_dbg(dev, "TCM carveout split mode %s addr=%llx, da=0x%x, size=0x%lx",
+ dev_dbg(dev, "TCM carveout %s addr=%llx, da=0x%x, size=0x%lx",
bank_name, bank_addr, da, bank_size);
rproc_mem = rproc_mem_entry_init(dev, NULL, bank_addr,
@@ -593,7 +593,7 @@ static int add_tcm_carveout_split_mode(struct rproc *rproc)
if (!rproc_mem) {
ret = -ENOMEM;
zynqmp_pm_release_node(pm_domain_id);
- goto release_tcm_split;
+ goto release_tcm;
}
rproc_add_carveout(rproc, rproc_mem);
@@ -601,7 +601,7 @@ static int add_tcm_carveout_split_mode(struct rproc *rproc)
return 0;
-release_tcm_split:
+release_tcm:
/* If failed, Turn off all TCM banks turned on before */
for (i--; i >= 0; i--) {
pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
@@ -610,127 +610,6 @@ static int add_tcm_carveout_split_mode(struct rproc *rproc)
return ret;
}
-/*
- * add_tcm_carveout_lockstep_mode()
- * @rproc: single R5 core's corresponding rproc instance
- *
- * allocate and add remoteproc carveout for TCM memory in lockstep mode
- *
- * return 0 on success, otherwise non-zero value on failure
- */
-static int add_tcm_carveout_lockstep_mode(struct rproc *rproc)
-{
- struct rproc_mem_entry *rproc_mem;
- struct zynqmp_r5_core *r5_core;
- int i, num_banks, ret;
- phys_addr_t bank_addr;
- size_t bank_size = 0;
- struct device *dev;
- u32 pm_domain_id;
- char *bank_name;
- u32 da;
-
- r5_core = rproc->priv;
- dev = r5_core->dev;
-
- /* Go through zynqmp banks for r5 node */
- num_banks = r5_core->tcm_bank_count;
-
- /*
- * In lockstep mode, TCM is contiguous memory block
- * However, each TCM block still needs to be enabled individually.
- * So, Enable each TCM block individually.
- * Although ATCM and BTCM is contiguous memory block, add two separate
- * carveouts for both.
- */
- for (i = 0; i < num_banks; i++) {
- pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
-
- /* Turn on each TCM bank individually */
- ret = zynqmp_pm_request_node(pm_domain_id,
- ZYNQMP_PM_CAPABILITY_ACCESS, 0,
- ZYNQMP_PM_REQUEST_ACK_BLOCKING);
- if (ret < 0) {
- dev_err(dev, "failed to turn on TCM 0x%x", pm_domain_id);
- goto release_tcm_lockstep;
- }
-
- bank_size = r5_core->tcm_banks[i]->size;
- if (bank_size == 0)
- continue;
-
- bank_addr = r5_core->tcm_banks[i]->addr;
- da = r5_core->tcm_banks[i]->da;
- bank_name = r5_core->tcm_banks[i]->bank_name;
-
- /* Register TCM address range, TCM map and unmap functions */
- rproc_mem = rproc_mem_entry_init(dev, NULL, bank_addr,
- bank_size, da,
- tcm_mem_map, tcm_mem_unmap,
- bank_name);
- if (!rproc_mem) {
- ret = -ENOMEM;
- zynqmp_pm_release_node(pm_domain_id);
- goto release_tcm_lockstep;
- }
-
- /* If registration is success, add carveouts */
- rproc_add_carveout(rproc, rproc_mem);
-
- dev_dbg(dev, "TCM carveout lockstep mode %s addr=0x%llx, da=0x%x, size=0x%lx",
- bank_name, bank_addr, da, bank_size);
- }
-
- return 0;
-
-release_tcm_lockstep:
- /* If failed, Turn off all TCM banks turned on before */
- for (i--; i >= 0; i--) {
- pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
- zynqmp_pm_release_node(pm_domain_id);
- }
- return ret;
-}
-
-/*
- * add_tcm_banks()
- * @rproc: single R5 core's corresponding rproc instance
- *
- * allocate and add remoteproc carveouts for TCM memory based on cluster mode
- *
- * return 0 on success, otherwise non-zero value on failure
- */
-static int add_tcm_banks(struct rproc *rproc)
-{
- struct zynqmp_r5_cluster *cluster;
- struct zynqmp_r5_core *r5_core;
- struct device *dev;
-
- r5_core = rproc->priv;
- if (!r5_core)
- return -EINVAL;
-
- dev = r5_core->dev;
-
- cluster = dev_get_drvdata(dev->parent);
- if (!cluster) {
- dev_err(dev->parent, "Invalid driver data\n");
- return -EINVAL;
- }
-
- /*
- * In lockstep mode TCM banks are one contiguous memory region of 256Kb
- * In split mode, each TCM bank is 64Kb and not contiguous.
- * We add memory carveouts accordingly.
- */
- if (cluster->mode == SPLIT_MODE)
- return add_tcm_carveout_split_mode(rproc);
- else if (cluster->mode == LOCKSTEP_MODE)
- return add_tcm_carveout_lockstep_mode(rproc);
-
- return -EINVAL;
-}
-
/*
* zynqmp_r5_parse_fw()
* @rproc: single R5 core's corresponding rproc instance
--
2.25.1
Tightly-Coupled Memories(TCMs) are low-latency memory that provides
predictable instruction execution and predictable data load/store
timing. Each Cortex-R5F processor contains exclusive two 64 KB memory
banks on the ATCM and BTCM ports, for a total of 128 KB of memory.
In lockstep mode, both 128KB memory is accessible to the cluster.
As per ZynqMP Ultrascale+ Technical Reference Manual UG1085, following
is address space of TCM memory. The bindings in this patch series
introduces properties to accommodate following address space with
address translation between Linux and Cortex-R5 views.
| | | |
| --- | --- | --- |
| *Mode* | *R5 View* | *Linux view* | Notes |
| *Split Mode* | *start addr*| *start addr* | |
| R5_0 ATCM (64 KB) | 0x0000_0000 | 0xFFE0_0000 | |
| R5_0 BTCM (64 KB) | 0x0002_0000 | 0xFFE2_0000 | |
| R5_1 ATCM (64 KB) | 0x0000_0000 | 0xFFE9_0000 | alias of 0xFFE1_0000 |
| R5_1 BTCM (64 KB) | 0x0002_0000 | 0xFFEB_0000 | alias of 0xFFE3_0000 |
| ___ | ___ | ___ | |
| *Lockstep Mode* | | | |
| R5_0 ATCM (128 KB) | 0x0000_0000 | 0xFFE0_0000 | |
| R5_0 BTCM (128 KB) | 0x0002_0000 | 0xFFE2_0000 | |
References:
UG1085 TCM address space:
https://docs.xilinx.com/r/en-US/ug1085-zynq-ultrascale-trm/Tightly-Coupled-Memory-Address-Map
Changes in v11:
- Fix yamllint warning and reduce indentation as needed
- Remove redundant initialization of the variable
- Return correct error code if memory allocation failed
Changs in v10:
- Add new patch (1/4) to series that changes hardcode TCM addresses in
lockstep mode and removes separate handling of TCM in lockstep and
split mode
- modify number of "reg", "reg-names" and "power-domains" entries
based on cluster mode
- Add extra optional atcm and btcm in "reg" property for lockstep mode
- Add "reg-names" for extra optional atcm and btcm for lockstep mode
- Drop previous Ack as bindings has new change
- Add individual tcm regions via "reg" and "reg-names" for lockstep mode
- Add each tcm's power-domains in lockstep mode
- Drop previous Ack as new change in dts patchset
- Remove redundant changes in driver to handle TCM in lockstep mode
Changes in v9:
- Fix rproc lockstep dts
- Introduce new API to request and release core1 TCM power-domains in
lockstep mode. This will be used during prepare -> add_tcm_banks
callback to enable TCM in lockstep mode.
- Parse TCM from device-tree in lockstep mode and split mode in
uniform way.
- Fix TCM representation in device-tree in lockstep mode.
- Fix comments as suggested
Changes in v8:
- Remove use of pm_domains framework
- Remove checking of pm_domain_id validation to power on/off tcm
- Remove spurious change
- parse power-domains property from device-tree and use EEMI calls
to power on/off TCM instead of using pm domains framework
Changes in v7:
- %s/pm_dev1/pm_dev_core0/r
- %s/pm_dev_link1/pm_dev_core0_link/r
- %s/pm_dev2/pm_dev_core1/r
- %s/pm_dev_link2/pm_dev_core1_link/r
- remove pm_domain_id check to move next patch
- add comment about how 1st entry in pm domain list is used
- fix loop when jump to fail_add_pm_domains loop
- move checking of pm_domain_id from previous patch
- fix mem_bank_data memory allocation
Changes in v6:
- Introduce new node entry for r5f cluster split mode dts and
keep it disabled by default.
- Keep remoteproc lockstep mode enabled by default to maintian
back compatibility.
- Enable split mode only for zcu102 board to demo split mode use
- Remove spurious change
- Handle errors in add_pm_domains function
- Remove redundant code to handle errors from remove_pm_domains
- Missing . at the end of the commit message
- remove redundant initialization of variables
- remove fail_tcm label and relevant code to free memory
acquired using devm_* API. As this will be freed when device free it
- add extra check to see if "reg" property is supported or not
Changes in v5:
- maintain Rob's Ack on bindings patch as no changes in bindings
- split previous patch into multiple patches
- Use pm domain framework to turn on/off TCM
- Add support of parsing TCM information from device-tree
- maintain backward compatibility with previous bindings without
TCM information available in device-tree
This patch series continues previous effort to upstream ZynqMP
TCM bindings:
Previous v4 version link:
https://lore.kernel.org/all/[email protected]/
Previous v3 version link:
https://lore.kernel.org/all/[email protected]/
Radhey Shyam Pandey (1):
dt-bindings: remoteproc: add Tightly Coupled Memory (TCM) bindings
Radhey Shyam Pandey (1):
dt-bindings: remoteproc: add Tightly Coupled Memory (TCM) bindings
Tanmay Shah (3):
remoteproc: zynqmp: fix lockstep mode memory region
dts: zynqmp: add properties for TCM in remoteproc
remoteproc: zynqmp: parse TCM from device tree
.../remoteproc/xlnx,zynqmp-r5fss.yaml | 192 +++++++++++--
.../boot/dts/xilinx/zynqmp-zcu102-rev1.0.dts | 8 +
arch/arm64/boot/dts/xilinx/zynqmp.dtsi | 65 ++++-
drivers/remoteproc/xlnx_r5_remoteproc.c | 257 ++++++++----------
4 files changed, 357 insertions(+), 165 deletions(-)
base-commit: 929654e8f1add50b01d5a56171a31c311b0a739a
--
2.25.1
From: Radhey Shyam Pandey <[email protected]>
Introduce bindings for TCM memory address space on AMD-xilinx Zynq
UltraScale+ platform. It will help in defining TCM in device-tree
and make it's access platform agnostic and data-driven.
Tightly-coupled memories(TCMs) are low-latency memory that provides
predictable instruction execution and predictable data load/store
timing. Each Cortex-R5F processor contains two 64-bit wide 64 KB memory
banks on the ATCM and BTCM ports, for a total of 128 KB of memory.
The TCM resources(reg, reg-names and power-domain) are documented for
each TCM in the R5 node. The reg and reg-names are made as required
properties as we don't want to hardcode TCM addresses for future
platforms and for zu+ legacy implementation will ensure that the
old dts w/o reg/reg-names works and stable ABI is maintained.
It also extends the examples for TCM split and lockstep modes.
Signed-off-by: Radhey Shyam Pandey <[email protected]>
Signed-off-by: Tanmay Shah <[email protected]>
---
Changes in v11:
- Fix yamllint warning and reduce indentation as needed
.../remoteproc/xlnx,zynqmp-r5fss.yaml | 192 ++++++++++++++++--
1 file changed, 170 insertions(+), 22 deletions(-)
diff --git a/Documentation/devicetree/bindings/remoteproc/xlnx,zynqmp-r5fss.yaml b/Documentation/devicetree/bindings/remoteproc/xlnx,zynqmp-r5fss.yaml
index 78aac69f1060..77030edf41fa 100644
--- a/Documentation/devicetree/bindings/remoteproc/xlnx,zynqmp-r5fss.yaml
+++ b/Documentation/devicetree/bindings/remoteproc/xlnx,zynqmp-r5fss.yaml
@@ -20,9 +20,21 @@ properties:
compatible:
const: xlnx,zynqmp-r5fss
+ "#address-cells":
+ const: 2
+
+ "#size-cells":
+ const: 2
+
+ ranges:
+ description: |
+ Standard ranges definition providing address translations for
+ local R5F TCM address spaces to bus addresses.
+
xlnx,cluster-mode:
$ref: /schemas/types.yaml#/definitions/uint32
enum: [0, 1, 2]
+ default: 1
description: |
The RPU MPCore can operate in split mode (Dual-processor performance), Safety
lock-step mode(Both RPU cores execute the same code in lock-step,
@@ -37,7 +49,7 @@ properties:
2: single cpu mode
patternProperties:
- "^r5f-[a-f0-9]+$":
+ "^r5f@[0-9a-f]+$":
type: object
description: |
The RPU is located in the Low Power Domain of the Processor Subsystem.
@@ -54,9 +66,6 @@ patternProperties:
compatible:
const: xlnx,zynqmp-r5f
- power-domains:
- maxItems: 1
-
mboxes:
minItems: 1
items:
@@ -101,35 +110,174 @@ patternProperties:
required:
- compatible
- - power-domains
- unevaluatedProperties: false
+allOf:
+ - if:
+ properties:
+ xlnx,cluster-mode:
+ enum:
+ - 1
+ then:
+ patternProperties:
+ "^r5f@[0-9a-f]+$":
+ type: object
+
+ properties:
+ reg:
+ minItems: 1
+ items:
+ - description: ATCM internal memory
+ - description: BTCM internal memory
+ - description: extra ATCM memory in lockstep mode
+ - description: extra BTCM memory in lockstep mode
+
+ reg-names:
+ minItems: 1
+ items:
+ - const: atcm0
+ - const: btcm0
+ - const: atcm1
+ - const: btcm1
+
+ power-domains:
+ minItems: 2
+ maxItems: 5
+
+ required:
+ - reg
+ - reg-names
+ - power-domains
+
+ else:
+ patternProperties:
+ "^r5f@[0-9a-f]+$":
+ type: object
+
+ properties:
+ reg:
+ minItems: 1
+ items:
+ - description: ATCM internal memory
+ - description: BTCM internal memory
+
+ reg-names:
+ minItems: 1
+ items:
+ - const: atcm0
+ - const: btcm0
+
+ power-domains:
+ minItems: 2
+ maxItems: 3
+
+ required:
+ - reg
+ - reg-names
+ - power-domains
required:
- compatible
+ - "#address-cells"
+ - "#size-cells"
+ - ranges
additionalProperties: false
examples:
- |
- remoteproc {
- compatible = "xlnx,zynqmp-r5fss";
- xlnx,cluster-mode = <1>;
-
- r5f-0 {
- compatible = "xlnx,zynqmp-r5f";
- power-domains = <&zynqmp_firmware 0x7>;
- memory-region = <&rproc_0_fw_image>, <&rpu0vdev0buffer>, <&rpu0vdev0vring0>, <&rpu0vdev0vring1>;
- mboxes = <&ipi_mailbox_rpu0 0>, <&ipi_mailbox_rpu0 1>;
- mbox-names = "tx", "rx";
+ #include <dt-bindings/power/xlnx-zynqmp-power.h>
+
+ // Split mode configuration
+ soc {
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ remoteproc@ffe00000 {
+ compatible = "xlnx,zynqmp-r5fss";
+ xlnx,cluster-mode = <0>;
+
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges = <0x0 0x0 0x0 0xffe00000 0x0 0x10000>,
+ <0x0 0x20000 0x0 0xffe20000 0x0 0x10000>,
+ <0x1 0x0 0x0 0xffe90000 0x0 0x10000>,
+ <0x1 0x20000 0x0 0xffeb0000 0x0 0x10000>;
+
+ r5f@0 {
+ compatible = "xlnx,zynqmp-r5f";
+ reg = <0x0 0x0 0x0 0x10000>, <0x0 0x20000 0x0 0x10000>;
+ reg-names = "atcm0", "btcm0";
+ power-domains = <&zynqmp_firmware PD_RPU_0>,
+ <&zynqmp_firmware PD_R5_0_ATCM>,
+ <&zynqmp_firmware PD_R5_0_BTCM>;
+ memory-region = <&rproc_0_fw_image>, <&rpu0vdev0buffer>,
+ <&rpu0vdev0vring0>, <&rpu0vdev0vring1>;
+ mboxes = <&ipi_mailbox_rpu0 0>, <&ipi_mailbox_rpu0 1>;
+ mbox-names = "tx", "rx";
+ };
+
+ r5f@1 {
+ compatible = "xlnx,zynqmp-r5f";
+ reg = <0x1 0x0 0x0 0x10000>, <0x1 0x20000 0x0 0x10000>;
+ reg-names = "atcm0", "btcm0";
+ power-domains = <&zynqmp_firmware PD_RPU_1>,
+ <&zynqmp_firmware PD_R5_1_ATCM>,
+ <&zynqmp_firmware PD_R5_1_BTCM>;
+ memory-region = <&rproc_1_fw_image>, <&rpu1vdev0buffer>,
+ <&rpu1vdev0vring0>, <&rpu1vdev0vring1>;
+ mboxes = <&ipi_mailbox_rpu1 0>, <&ipi_mailbox_rpu1 1>;
+ mbox-names = "tx", "rx";
+ };
};
+ };
+
+ - |
+ //Lockstep configuration
+ soc {
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ remoteproc@ffe00000 {
+ compatible = "xlnx,zynqmp-r5fss";
+ xlnx,cluster-mode = <1>;
+
+ #address-cells = <2>;
+ #size-cells = <2>;
+ ranges = <0x0 0x0 0x0 0xffe00000 0x0 0x10000>,
+ <0x0 0x20000 0x0 0xffe20000 0x0 0x10000>,
+ <0x0 0x10000 0x0 0xffe10000 0x0 0x10000>,
+ <0x0 0x30000 0x0 0xffe30000 0x0 0x10000>;
+
+ r5f@0 {
+ compatible = "xlnx,zynqmp-r5f";
+ reg = <0x0 0x0 0x0 0x10000>,
+ <0x0 0x20000 0x0 0x10000>,
+ <0x0 0x10000 0x0 0x10000>,
+ <0x0 0x30000 0x0 0x10000>;
+ reg-names = "atcm0", "btcm0", "atcm1", "btcm1";
+ power-domains = <&zynqmp_firmware PD_RPU_0>,
+ <&zynqmp_firmware PD_R5_0_ATCM>,
+ <&zynqmp_firmware PD_R5_0_BTCM>,
+ <&zynqmp_firmware PD_R5_1_ATCM>,
+ <&zynqmp_firmware PD_R5_1_BTCM>;
+ memory-region = <&rproc_0_fw_image>, <&rpu0vdev0buffer>,
+ <&rpu0vdev0vring0>, <&rpu0vdev0vring1>;
+ mboxes = <&ipi_mailbox_rpu0 0>, <&ipi_mailbox_rpu0 1>;
+ mbox-names = "tx", "rx";
+ };
- r5f-1 {
- compatible = "xlnx,zynqmp-r5f";
- power-domains = <&zynqmp_firmware 0x8>;
- memory-region = <&rproc_1_fw_image>, <&rpu1vdev0buffer>, <&rpu1vdev0vring0>, <&rpu1vdev0vring1>;
- mboxes = <&ipi_mailbox_rpu1 0>, <&ipi_mailbox_rpu1 1>;
- mbox-names = "tx", "rx";
+ r5f@1 {
+ compatible = "xlnx,zynqmp-r5f";
+ reg = <0x1 0x0 0x0 0x10000>, <0x1 0x20000 0x0 0x10000>;
+ reg-names = "atcm0", "btcm0";
+ power-domains = <&zynqmp_firmware PD_RPU_1>,
+ <&zynqmp_firmware PD_R5_1_ATCM>,
+ <&zynqmp_firmware PD_R5_1_BTCM>;
+ memory-region = <&rproc_1_fw_image>, <&rpu1vdev0buffer>,
+ <&rpu1vdev0vring0>, <&rpu1vdev0vring1>;
+ mboxes = <&ipi_mailbox_rpu1 0>, <&ipi_mailbox_rpu1 1>;
+ mbox-names = "tx", "rx";
+ };
};
};
...
--
2.25.1
Add properties as per new bindings in zynqmp remoteproc node
to represent TCM address and size.
This patch also adds alternative remoteproc node to represent
remoteproc cluster in split mode. By default lockstep mode is
enabled and users should disable it before using split mode
dts. Both device-tree nodes can't be used simultaneously one
of them must be disabled. For zcu102-1.0 and zcu102-1.1 board
remoteproc split mode dts node is enabled and lockstep mode
dts is disabled.
Signed-off-by: Tanmay Shah <[email protected]>
---
.../boot/dts/xilinx/zynqmp-zcu102-rev1.0.dts | 8 +++
arch/arm64/boot/dts/xilinx/zynqmp.dtsi | 65 +++++++++++++++++--
2 files changed, 68 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/boot/dts/xilinx/zynqmp-zcu102-rev1.0.dts b/arch/arm64/boot/dts/xilinx/zynqmp-zcu102-rev1.0.dts
index c8f71a1aec89..495ca94b45db 100644
--- a/arch/arm64/boot/dts/xilinx/zynqmp-zcu102-rev1.0.dts
+++ b/arch/arm64/boot/dts/xilinx/zynqmp-zcu102-rev1.0.dts
@@ -14,6 +14,14 @@ / {
compatible = "xlnx,zynqmp-zcu102-rev1.0", "xlnx,zynqmp-zcu102", "xlnx,zynqmp";
};
+&rproc_split {
+ status = "okay";
+};
+
+&rproc_lockstep {
+ status = "disabled";
+};
+
&eeprom {
#address-cells = <1>;
#size-cells = <1>;
diff --git a/arch/arm64/boot/dts/xilinx/zynqmp.dtsi b/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
index eaba466804bc..c8a7fd0f3a1e 100644
--- a/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
+++ b/arch/arm64/boot/dts/xilinx/zynqmp.dtsi
@@ -248,19 +248,74 @@ fpga_full: fpga-full {
ranges;
};
- remoteproc {
+ rproc_lockstep: remoteproc@ffe00000 {
compatible = "xlnx,zynqmp-r5fss";
xlnx,cluster-mode = <1>;
- r5f-0 {
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ ranges = <0x0 0x0 0x0 0xffe00000 0x0 0x10000>,
+ <0x0 0x20000 0x0 0xffe20000 0x0 0x10000>,
+ <0x0 0x10000 0x0 0xffe10000 0x0 0x10000>,
+ <0x0 0x30000 0x0 0xffe30000 0x0 0x10000>;
+
+ r5f@0 {
+ compatible = "xlnx,zynqmp-r5f";
+ reg = <0x0 0x0 0x0 0x10000>,
+ <0x0 0x20000 0x0 0x10000>,
+ <0x0 0x10000 0x0 0x10000>,
+ <0x0 0x30000 0x0 0x10000>;
+ reg-names = "atcm0", "btcm0", "atcm1", "btcm1";
+ power-domains = <&zynqmp_firmware PD_RPU_0>,
+ <&zynqmp_firmware PD_R5_0_ATCM>,
+ <&zynqmp_firmware PD_R5_0_BTCM>,
+ <&zynqmp_firmware PD_R5_1_ATCM>,
+ <&zynqmp_firmware PD_R5_1_BTCM>;
+ memory-region = <&rproc_0_fw_image>;
+ };
+
+ r5f@1 {
+ compatible = "xlnx,zynqmp-r5f";
+ reg = <0x1 0x0 0x0 0x10000>, <0x1 0x20000 0x0 0x10000>;
+ reg-names = "atcm0", "btcm0";
+ power-domains = <&zynqmp_firmware PD_RPU_1>,
+ <&zynqmp_firmware PD_R5_1_ATCM>,
+ <&zynqmp_firmware PD_R5_1_BTCM>;
+ memory-region = <&rproc_1_fw_image>;
+ };
+ };
+
+ rproc_split: remoteproc-split@ffe00000 {
+ status = "disabled";
+ compatible = "xlnx,zynqmp-r5fss";
+ xlnx,cluster-mode = <0>;
+
+ #address-cells = <2>;
+ #size-cells = <2>;
+
+ ranges = <0x0 0x0 0x0 0xffe00000 0x0 0x10000>,
+ <0x0 0x20000 0x0 0xffe20000 0x0 0x10000>,
+ <0x1 0x0 0x0 0xffe90000 0x0 0x10000>,
+ <0x1 0x20000 0x0 0xffeb0000 0x0 0x10000>;
+
+ r5f@0 {
compatible = "xlnx,zynqmp-r5f";
- power-domains = <&zynqmp_firmware PD_RPU_0>;
+ reg = <0x0 0x0 0x0 0x10000>, <0x0 0x20000 0x0 0x10000>;
+ reg-names = "atcm0", "btcm0";
+ power-domains = <&zynqmp_firmware PD_RPU_0>,
+ <&zynqmp_firmware PD_R5_0_ATCM>,
+ <&zynqmp_firmware PD_R5_0_BTCM>;
memory-region = <&rproc_0_fw_image>;
};
- r5f-1 {
+ r5f@1 {
compatible = "xlnx,zynqmp-r5f";
- power-domains = <&zynqmp_firmware PD_RPU_1>;
+ reg = <0x1 0x0 0x0 0x10000>, <0x1 0x20000 0x0 0x10000>;
+ reg-names = "atcm0", "btcm0";
+ power-domains = <&zynqmp_firmware PD_RPU_1>,
+ <&zynqmp_firmware PD_R5_1_ATCM>,
+ <&zynqmp_firmware PD_R5_1_BTCM>;
memory-region = <&rproc_1_fw_image>;
};
};
--
2.25.1
ZynqMP TCM information was fixed in driver. Now ZynqMP TCM information
is available in device-tree. Parse TCM information in driver
as per new bindings.
Signed-off-by: Tanmay Shah <[email protected]>
---
Changes in v11:
- Remove redundant initialization of the variable
- return correct error code if memory allocation failed
drivers/remoteproc/xlnx_r5_remoteproc.c | 112 ++++++++++++++++++++++--
1 file changed, 107 insertions(+), 5 deletions(-)
diff --git a/drivers/remoteproc/xlnx_r5_remoteproc.c b/drivers/remoteproc/xlnx_r5_remoteproc.c
index 42b0384d34f2..d4a22caebaad 100644
--- a/drivers/remoteproc/xlnx_r5_remoteproc.c
+++ b/drivers/remoteproc/xlnx_r5_remoteproc.c
@@ -74,8 +74,8 @@ struct mbox_info {
};
/*
- * Hardcoded TCM bank values. This will be removed once TCM bindings are
- * accepted for system-dt specifications and upstreamed in linux kernel
+ * Hardcoded TCM bank values. This will stay in driver to maintain backward
+ * compatibility with device-tree that does not have TCM information.
*/
static const struct mem_bank_data zynqmp_tcm_banks_split[] = {
{0xffe00000UL, 0x0, 0x10000UL, PD_R5_0_ATCM, "atcm0"}, /* TCM 64KB each */
@@ -757,6 +757,103 @@ static struct zynqmp_r5_core *zynqmp_r5_add_rproc_core(struct device *cdev)
return ERR_PTR(ret);
}
+static int zynqmp_r5_get_tcm_node_from_dt(struct zynqmp_r5_cluster *cluster)
+{
+ int i, j, tcm_bank_count, ret, tcm_pd_idx, pd_count;
+ struct of_phandle_args out_args;
+ struct zynqmp_r5_core *r5_core;
+ struct platform_device *cpdev;
+ struct mem_bank_data *tcm;
+ struct device_node *np;
+ struct resource *res;
+ u64 abs_addr, size;
+ struct device *dev;
+
+ for (i = 0; i < cluster->core_count; i++) {
+ r5_core = cluster->r5_cores[i];
+ dev = r5_core->dev;
+ np = r5_core->np;
+
+ pd_count = of_count_phandle_with_args(np, "power-domains",
+ "#power-domain-cells");
+
+ if (pd_count <= 0) {
+ dev_err(dev, "invalid power-domains property, %d\n", pd_count);
+ return -EINVAL;
+ }
+
+ /* First entry in power-domains list is for r5 core, rest for TCM. */
+ tcm_bank_count = pd_count - 1;
+
+ if (tcm_bank_count <= 0) {
+ dev_err(dev, "invalid TCM count %d\n", tcm_bank_count);
+ return -EINVAL;
+ }
+
+ r5_core->tcm_banks = devm_kcalloc(dev, tcm_bank_count,
+ sizeof(struct mem_bank_data *),
+ GFP_KERNEL);
+ if (!r5_core->tcm_banks)
+ return -ENOMEM;
+
+ r5_core->tcm_bank_count = tcm_bank_count;
+ for (j = 0, tcm_pd_idx = 1; j < tcm_bank_count; j++, tcm_pd_idx++) {
+ tcm = devm_kzalloc(dev, sizeof(struct mem_bank_data),
+ GFP_KERNEL);
+ if (!tcm)
+ return -ENOMEM;
+
+ r5_core->tcm_banks[j] = tcm;
+
+ /* Get power-domains id of TCM. */
+ ret = of_parse_phandle_with_args(np, "power-domains",
+ "#power-domain-cells",
+ tcm_pd_idx, &out_args);
+ if (ret) {
+ dev_err(r5_core->dev,
+ "failed to get tcm %d pm domain, ret %d\n",
+ tcm_pd_idx, ret);
+ return ret;
+ }
+ tcm->pm_domain_id = out_args.args[0];
+ of_node_put(out_args.np);
+
+ /* Get TCM address without translation. */
+ ret = of_property_read_reg(np, j, &abs_addr, &size);
+ if (ret) {
+ dev_err(dev, "failed to get reg property\n");
+ return ret;
+ }
+
+ /*
+ * Remote processor can address only 32 bits
+ * so convert 64-bits into 32-bits. This will discard
+ * any unwanted upper 32-bits.
+ */
+ tcm->da = (u32)abs_addr;
+ tcm->size = (u32)size;
+
+ cpdev = to_platform_device(dev);
+ res = platform_get_resource(cpdev, IORESOURCE_MEM, j);
+ if (!res) {
+ dev_err(dev, "failed to get tcm resource\n");
+ return -EINVAL;
+ }
+
+ tcm->addr = (u32)res->start;
+ tcm->bank_name = (char *)res->name;
+ res = devm_request_mem_region(dev, tcm->addr, tcm->size,
+ tcm->bank_name);
+ if (!res) {
+ dev_err(dev, "failed to request tcm resource\n");
+ return -EINVAL;
+ }
+ }
+ }
+
+ return 0;
+}
+
/**
* zynqmp_r5_get_tcm_node()
* Ideally this function should parse tcm node and store information
@@ -835,9 +932,14 @@ static int zynqmp_r5_core_init(struct zynqmp_r5_cluster *cluster,
struct zynqmp_r5_core *r5_core;
int ret, i;
- ret = zynqmp_r5_get_tcm_node(cluster);
- if (ret < 0) {
- dev_err(dev, "can't get tcm node, err %d\n", ret);
+ r5_core = cluster->r5_cores[0];
+ if (of_find_property(r5_core->np, "reg", NULL))
+ ret = zynqmp_r5_get_tcm_node_from_dt(cluster);
+ else
+ ret = zynqmp_r5_get_tcm_node(cluster);
+
+ if (ret) {
+ dev_err(dev, "can't get tcm, err %d\n", ret);
return ret;
}
--
2.25.1
Hello,
By mistake same set of patches were sent twice in same git send-email command.
Anyone can be reviewed. Please let me know if I need to take any action to fix it.
Thanks.
On 2/19/24 11:44 AM, Tanmay Shah wrote:
> Tightly-Coupled Memories(TCMs) are low-latency memory that provides
> predictable instruction execution and predictable data load/store
> timing. Each Cortex-R5F processor contains exclusive two 64 KB memory
> banks on the ATCM and BTCM ports, for a total of 128 KB of memory.
> In lockstep mode, both 128KB memory is accessible to the cluster.
>
> As per ZynqMP Ultrascale+ Technical Reference Manual UG1085, following
> is address space of TCM memory. The bindings in this patch series
> introduces properties to accommodate following address space with
> address translation between Linux and Cortex-R5 views.
>
> | | | |
> | --- | --- | --- |
> | *Mode* | *R5 View* | *Linux view* | Notes |
> | *Split Mode* | *start addr*| *start addr* | |
> | R5_0 ATCM (64 KB) | 0x0000_0000 | 0xFFE0_0000 | |
> | R5_0 BTCM (64 KB) | 0x0002_0000 | 0xFFE2_0000 | |
> | R5_1 ATCM (64 KB) | 0x0000_0000 | 0xFFE9_0000 | alias of 0xFFE1_0000 |
> | R5_1 BTCM (64 KB) | 0x0002_0000 | 0xFFEB_0000 | alias of 0xFFE3_0000 |
> | ___ | ___ | ___ | |
> | *Lockstep Mode* | | | |
> | R5_0 ATCM (128 KB) | 0x0000_0000 | 0xFFE0_0000 | |
> | R5_0 BTCM (128 KB) | 0x0002_0000 | 0xFFE2_0000 | |
>
> References:
> UG1085 TCM address space:
> https://docs.xilinx.com/r/en-US/ug1085-zynq-ultrascale-trm/Tightly-Coupled-Memory-Address-Map
>
> Changes in v11:
> - Fix yamllint warning and reduce indentation as needed
> - Remove redundant initialization of the variable
> - Return correct error code if memory allocation failed
>
> Changs in v10:
> - Add new patch (1/4) to series that changes hardcode TCM addresses in
> lockstep mode and removes separate handling of TCM in lockstep and
> split mode
> - modify number of "reg", "reg-names" and "power-domains" entries
> based on cluster mode
> - Add extra optional atcm and btcm in "reg" property for lockstep mode
> - Add "reg-names" for extra optional atcm and btcm for lockstep mode
> - Drop previous Ack as bindings has new change
> - Add individual tcm regions via "reg" and "reg-names" for lockstep mode
> - Add each tcm's power-domains in lockstep mode
> - Drop previous Ack as new change in dts patchset
> - Remove redundant changes in driver to handle TCM in lockstep mode
>
> Changes in v9:
> - Fix rproc lockstep dts
> - Introduce new API to request and release core1 TCM power-domains in
> lockstep mode. This will be used during prepare -> add_tcm_banks
> callback to enable TCM in lockstep mode.
> - Parse TCM from device-tree in lockstep mode and split mode in
> uniform way.
> - Fix TCM representation in device-tree in lockstep mode.
> - Fix comments as suggested
>
> Changes in v8:
> - Remove use of pm_domains framework
> - Remove checking of pm_domain_id validation to power on/off tcm
> - Remove spurious change
> - parse power-domains property from device-tree and use EEMI calls
> to power on/off TCM instead of using pm domains framework
>
> Changes in v7:
> - %s/pm_dev1/pm_dev_core0/r
> - %s/pm_dev_link1/pm_dev_core0_link/r
> - %s/pm_dev2/pm_dev_core1/r
> - %s/pm_dev_link2/pm_dev_core1_link/r
> - remove pm_domain_id check to move next patch
> - add comment about how 1st entry in pm domain list is used
> - fix loop when jump to fail_add_pm_domains loop
> - move checking of pm_domain_id from previous patch
> - fix mem_bank_data memory allocation
>
> Changes in v6:
> - Introduce new node entry for r5f cluster split mode dts and
> keep it disabled by default.
> - Keep remoteproc lockstep mode enabled by default to maintian
> back compatibility.
> - Enable split mode only for zcu102 board to demo split mode use
> - Remove spurious change
> - Handle errors in add_pm_domains function
> - Remove redundant code to handle errors from remove_pm_domains
> - Missing . at the end of the commit message
> - remove redundant initialization of variables
> - remove fail_tcm label and relevant code to free memory
> acquired using devm_* API. As this will be freed when device free it
> - add extra check to see if "reg" property is supported or not
>
> Changes in v5:
> - maintain Rob's Ack on bindings patch as no changes in bindings
> - split previous patch into multiple patches
> - Use pm domain framework to turn on/off TCM
> - Add support of parsing TCM information from device-tree
> - maintain backward compatibility with previous bindings without
> TCM information available in device-tree
>
> This patch series continues previous effort to upstream ZynqMP
> TCM bindings:
> Previous v4 version link:
> https://lore.kernel.org/all/[email protected]/
>
> Previous v3 version link:
> https://lore.kernel.org/all/[email protected]/
> Radhey Shyam Pandey (1):
> dt-bindings: remoteproc: add Tightly Coupled Memory (TCM) bindings
>
>
>
> Radhey Shyam Pandey (1):
> dt-bindings: remoteproc: add Tightly Coupled Memory (TCM) bindings
>
> Tanmay Shah (3):
> remoteproc: zynqmp: fix lockstep mode memory region
> dts: zynqmp: add properties for TCM in remoteproc
> remoteproc: zynqmp: parse TCM from device tree
>
> .../remoteproc/xlnx,zynqmp-r5fss.yaml | 192 +++++++++++--
> .../boot/dts/xilinx/zynqmp-zcu102-rev1.0.dts | 8 +
> arch/arm64/boot/dts/xilinx/zynqmp.dtsi | 65 ++++-
> drivers/remoteproc/xlnx_r5_remoteproc.c | 257 ++++++++----------
> 4 files changed, 357 insertions(+), 165 deletions(-)
>
>
> base-commit: 929654e8f1add50b01d5a56171a31c311b0a739a
On Mon, Feb 19, 2024 at 09:44:37AM -0800, Tanmay Shah wrote:
> ZynqMP TCM information was fixed in driver. Now ZynqMP TCM information
> is available in device-tree. Parse TCM information in driver
> as per new bindings.
>
> Signed-off-by: Tanmay Shah <[email protected]>
> ---
>
> Changes in v11:
> - Remove redundant initialization of the variable
> - return correct error code if memory allocation failed
Where is that? I looked really hard but couldn't find it.
>
> drivers/remoteproc/xlnx_r5_remoteproc.c | 112 ++++++++++++++++++++++--
> 1 file changed, 107 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/remoteproc/xlnx_r5_remoteproc.c b/drivers/remoteproc/xlnx_r5_remoteproc.c
> index 42b0384d34f2..d4a22caebaad 100644
> --- a/drivers/remoteproc/xlnx_r5_remoteproc.c
> +++ b/drivers/remoteproc/xlnx_r5_remoteproc.c
> @@ -74,8 +74,8 @@ struct mbox_info {
> };
>
> /*
> - * Hardcoded TCM bank values. This will be removed once TCM bindings are
> - * accepted for system-dt specifications and upstreamed in linux kernel
> + * Hardcoded TCM bank values. This will stay in driver to maintain backward
> + * compatibility with device-tree that does not have TCM information.
> */
> static const struct mem_bank_data zynqmp_tcm_banks_split[] = {
> {0xffe00000UL, 0x0, 0x10000UL, PD_R5_0_ATCM, "atcm0"}, /* TCM 64KB each */
> @@ -757,6 +757,103 @@ static struct zynqmp_r5_core *zynqmp_r5_add_rproc_core(struct device *cdev)
> return ERR_PTR(ret);
> }
>
> +static int zynqmp_r5_get_tcm_node_from_dt(struct zynqmp_r5_cluster *cluster)
> +{
> + int i, j, tcm_bank_count, ret, tcm_pd_idx, pd_count;
> + struct of_phandle_args out_args;
> + struct zynqmp_r5_core *r5_core;
> + struct platform_device *cpdev;
> + struct mem_bank_data *tcm;
> + struct device_node *np;
> + struct resource *res;
> + u64 abs_addr, size;
> + struct device *dev;
> +
> + for (i = 0; i < cluster->core_count; i++) {
> + r5_core = cluster->r5_cores[i];
> + dev = r5_core->dev;
> + np = r5_core->np;
> +
> + pd_count = of_count_phandle_with_args(np, "power-domains",
> + "#power-domain-cells");
> +
> + if (pd_count <= 0) {
> + dev_err(dev, "invalid power-domains property, %d\n", pd_count);
> + return -EINVAL;
> + }
> +
> + /* First entry in power-domains list is for r5 core, rest for TCM. */
> + tcm_bank_count = pd_count - 1;
> +
> + if (tcm_bank_count <= 0) {
> + dev_err(dev, "invalid TCM count %d\n", tcm_bank_count);
> + return -EINVAL;
> + }
> +
> + r5_core->tcm_banks = devm_kcalloc(dev, tcm_bank_count,
> + sizeof(struct mem_bank_data *),
> + GFP_KERNEL);
> + if (!r5_core->tcm_banks)
> + return -ENOMEM;
> +
> + r5_core->tcm_bank_count = tcm_bank_count;
> + for (j = 0, tcm_pd_idx = 1; j < tcm_bank_count; j++, tcm_pd_idx++) {
> + tcm = devm_kzalloc(dev, sizeof(struct mem_bank_data),
> + GFP_KERNEL);
> + if (!tcm)
> + return -ENOMEM;
> +
> + r5_core->tcm_banks[j] = tcm;
> +
> + /* Get power-domains id of TCM. */
> + ret = of_parse_phandle_with_args(np, "power-domains",
> + "#power-domain-cells",
> + tcm_pd_idx, &out_args);
> + if (ret) {
> + dev_err(r5_core->dev,
> + "failed to get tcm %d pm domain, ret %d\n",
> + tcm_pd_idx, ret);
> + return ret;
> + }
> + tcm->pm_domain_id = out_args.args[0];
> + of_node_put(out_args.np);
> +
> + /* Get TCM address without translation. */
> + ret = of_property_read_reg(np, j, &abs_addr, &size);
> + if (ret) {
> + dev_err(dev, "failed to get reg property\n");
> + return ret;
> + }
> +
> + /*
> + * Remote processor can address only 32 bits
> + * so convert 64-bits into 32-bits. This will discard
> + * any unwanted upper 32-bits.
> + */
> + tcm->da = (u32)abs_addr;
> + tcm->size = (u32)size;
> +
> + cpdev = to_platform_device(dev);
> + res = platform_get_resource(cpdev, IORESOURCE_MEM, j);
> + if (!res) {
> + dev_err(dev, "failed to get tcm resource\n");
> + return -EINVAL;
> + }
> +
> + tcm->addr = (u32)res->start;
> + tcm->bank_name = (char *)res->name;
> + res = devm_request_mem_region(dev, tcm->addr, tcm->size,
> + tcm->bank_name);
> + if (!res) {
> + dev_err(dev, "failed to request tcm resource\n");
> + return -EINVAL;
> + }
> + }
> + }
> +
> + return 0;
> +}
> +
> /**
> * zynqmp_r5_get_tcm_node()
> * Ideally this function should parse tcm node and store information
> @@ -835,9 +932,14 @@ static int zynqmp_r5_core_init(struct zynqmp_r5_cluster *cluster,
> struct zynqmp_r5_core *r5_core;
> int ret, i;
>
> - ret = zynqmp_r5_get_tcm_node(cluster);
> - if (ret < 0) {
> - dev_err(dev, "can't get tcm node, err %d\n", ret);
> + r5_core = cluster->r5_cores[0];
> + if (of_find_property(r5_core->np, "reg", NULL))
> + ret = zynqmp_r5_get_tcm_node_from_dt(cluster);
> + else
> + ret = zynqmp_r5_get_tcm_node(cluster);
> +
> + if (ret) {
> + dev_err(dev, "can't get tcm, err %d\n", ret);
> return ret;
> }
>
> --
> 2.25.1
>
On Mon, Feb 19, 2024 at 09:44:34AM -0800, Tanmay Shah wrote:
> In lockstep mode, r5 core0 uses TCM of R5 core1. Following is lockstep
> mode memory region as per hardware reference manual.
>
> | *TCM* | *R5 View* | *Linux view* |
> | R5_0 ATCM (128 KB) | 0x0000_0000 | 0xFFE0_0000 |
> | R5_0 BTCM (128 KB) | 0x0002_0000 | 0xFFE2_0000 |
>
> However, driver shouldn't model it as above because R5 core0 TCM and core1
> TCM has different power-domains mapped to it.
> Hence, TCM address space in lockstep mode should be modeled as 64KB
> regions only where each region has its own power-domain as following:
>
> | *TCM* | *R5 View* | *Linux view* |
> | R5_0 ATCM0 (64 KB) | 0x0000_0000 | 0xFFE0_0000 |
> | R5_0 BTCM0 (64 KB) | 0x0002_0000 | 0xFFE2_0000 |
> | R5_0 ATCM1 (64 KB) | 0x0001_0000 | 0xFFE1_0000 |
> | R5_0 BTCM1 (64 KB) | 0x0003_0000 | 0xFFE3_0000 |
>
> This makes driver maintanance easy and makes design robust for future
> platorms as well.
>
> Signed-off-by: Tanmay Shah <[email protected]>
Now that I have a clearer picture of where things are going, I am adding this
patch to rproc-next.
I'll wait for the DT crew for the rest of this set.
Thanks,
Mathieu
> ---
> drivers/remoteproc/xlnx_r5_remoteproc.c | 145 ++----------------------
> 1 file changed, 12 insertions(+), 133 deletions(-)
>
> diff --git a/drivers/remoteproc/xlnx_r5_remoteproc.c b/drivers/remoteproc/xlnx_r5_remoteproc.c
> index 4395edea9a64..42b0384d34f2 100644
> --- a/drivers/remoteproc/xlnx_r5_remoteproc.c
> +++ b/drivers/remoteproc/xlnx_r5_remoteproc.c
> @@ -84,12 +84,12 @@ static const struct mem_bank_data zynqmp_tcm_banks_split[] = {
> {0xffeb0000UL, 0x20000, 0x10000UL, PD_R5_1_BTCM, "btcm1"},
> };
>
> -/* In lockstep mode cluster combines each 64KB TCM and makes 128KB TCM */
> +/* In lockstep mode cluster uses each 64KB TCM from second core as well */
> static const struct mem_bank_data zynqmp_tcm_banks_lockstep[] = {
> - {0xffe00000UL, 0x0, 0x20000UL, PD_R5_0_ATCM, "atcm0"}, /* TCM 128KB each */
> - {0xffe20000UL, 0x20000, 0x20000UL, PD_R5_0_BTCM, "btcm0"},
> - {0, 0, 0, PD_R5_1_ATCM, ""},
> - {0, 0, 0, PD_R5_1_BTCM, ""},
> + {0xffe00000UL, 0x0, 0x10000UL, PD_R5_0_ATCM, "atcm0"}, /* TCM 64KB each */
> + {0xffe20000UL, 0x20000, 0x10000UL, PD_R5_0_BTCM, "btcm0"},
> + {0xffe10000UL, 0x10000, 0x10000UL, PD_R5_1_ATCM, "atcm1"},
> + {0xffe30000UL, 0x30000, 0x10000UL, PD_R5_1_BTCM, "btcm1"},
> };
>
> /**
> @@ -540,14 +540,14 @@ static int tcm_mem_map(struct rproc *rproc,
> }
>
> /*
> - * add_tcm_carveout_split_mode()
> + * add_tcm_banks()
> * @rproc: single R5 core's corresponding rproc instance
> *
> - * allocate and add remoteproc carveout for TCM memory in split mode
> + * allocate and add remoteproc carveout for TCM memory
> *
> * return 0 on success, otherwise non-zero value on failure
> */
> -static int add_tcm_carveout_split_mode(struct rproc *rproc)
> +static int add_tcm_banks(struct rproc *rproc)
> {
> struct rproc_mem_entry *rproc_mem;
> struct zynqmp_r5_core *r5_core;
> @@ -580,10 +580,10 @@ static int add_tcm_carveout_split_mode(struct rproc *rproc)
> ZYNQMP_PM_REQUEST_ACK_BLOCKING);
> if (ret < 0) {
> dev_err(dev, "failed to turn on TCM 0x%x", pm_domain_id);
> - goto release_tcm_split;
> + goto release_tcm;
> }
>
> - dev_dbg(dev, "TCM carveout split mode %s addr=%llx, da=0x%x, size=0x%lx",
> + dev_dbg(dev, "TCM carveout %s addr=%llx, da=0x%x, size=0x%lx",
> bank_name, bank_addr, da, bank_size);
>
> rproc_mem = rproc_mem_entry_init(dev, NULL, bank_addr,
> @@ -593,7 +593,7 @@ static int add_tcm_carveout_split_mode(struct rproc *rproc)
> if (!rproc_mem) {
> ret = -ENOMEM;
> zynqmp_pm_release_node(pm_domain_id);
> - goto release_tcm_split;
> + goto release_tcm;
> }
>
> rproc_add_carveout(rproc, rproc_mem);
> @@ -601,7 +601,7 @@ static int add_tcm_carveout_split_mode(struct rproc *rproc)
>
> return 0;
>
> -release_tcm_split:
> +release_tcm:
> /* If failed, Turn off all TCM banks turned on before */
> for (i--; i >= 0; i--) {
> pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
> @@ -610,127 +610,6 @@ static int add_tcm_carveout_split_mode(struct rproc *rproc)
> return ret;
> }
>
> -/*
> - * add_tcm_carveout_lockstep_mode()
> - * @rproc: single R5 core's corresponding rproc instance
> - *
> - * allocate and add remoteproc carveout for TCM memory in lockstep mode
> - *
> - * return 0 on success, otherwise non-zero value on failure
> - */
> -static int add_tcm_carveout_lockstep_mode(struct rproc *rproc)
> -{
> - struct rproc_mem_entry *rproc_mem;
> - struct zynqmp_r5_core *r5_core;
> - int i, num_banks, ret;
> - phys_addr_t bank_addr;
> - size_t bank_size = 0;
> - struct device *dev;
> - u32 pm_domain_id;
> - char *bank_name;
> - u32 da;
> -
> - r5_core = rproc->priv;
> - dev = r5_core->dev;
> -
> - /* Go through zynqmp banks for r5 node */
> - num_banks = r5_core->tcm_bank_count;
> -
> - /*
> - * In lockstep mode, TCM is contiguous memory block
> - * However, each TCM block still needs to be enabled individually.
> - * So, Enable each TCM block individually.
> - * Although ATCM and BTCM is contiguous memory block, add two separate
> - * carveouts for both.
> - */
> - for (i = 0; i < num_banks; i++) {
> - pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
> -
> - /* Turn on each TCM bank individually */
> - ret = zynqmp_pm_request_node(pm_domain_id,
> - ZYNQMP_PM_CAPABILITY_ACCESS, 0,
> - ZYNQMP_PM_REQUEST_ACK_BLOCKING);
> - if (ret < 0) {
> - dev_err(dev, "failed to turn on TCM 0x%x", pm_domain_id);
> - goto release_tcm_lockstep;
> - }
> -
> - bank_size = r5_core->tcm_banks[i]->size;
> - if (bank_size == 0)
> - continue;
> -
> - bank_addr = r5_core->tcm_banks[i]->addr;
> - da = r5_core->tcm_banks[i]->da;
> - bank_name = r5_core->tcm_banks[i]->bank_name;
> -
> - /* Register TCM address range, TCM map and unmap functions */
> - rproc_mem = rproc_mem_entry_init(dev, NULL, bank_addr,
> - bank_size, da,
> - tcm_mem_map, tcm_mem_unmap,
> - bank_name);
> - if (!rproc_mem) {
> - ret = -ENOMEM;
> - zynqmp_pm_release_node(pm_domain_id);
> - goto release_tcm_lockstep;
> - }
> -
> - /* If registration is success, add carveouts */
> - rproc_add_carveout(rproc, rproc_mem);
> -
> - dev_dbg(dev, "TCM carveout lockstep mode %s addr=0x%llx, da=0x%x, size=0x%lx",
> - bank_name, bank_addr, da, bank_size);
> - }
> -
> - return 0;
> -
> -release_tcm_lockstep:
> - /* If failed, Turn off all TCM banks turned on before */
> - for (i--; i >= 0; i--) {
> - pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
> - zynqmp_pm_release_node(pm_domain_id);
> - }
> - return ret;
> -}
> -
> -/*
> - * add_tcm_banks()
> - * @rproc: single R5 core's corresponding rproc instance
> - *
> - * allocate and add remoteproc carveouts for TCM memory based on cluster mode
> - *
> - * return 0 on success, otherwise non-zero value on failure
> - */
> -static int add_tcm_banks(struct rproc *rproc)
> -{
> - struct zynqmp_r5_cluster *cluster;
> - struct zynqmp_r5_core *r5_core;
> - struct device *dev;
> -
> - r5_core = rproc->priv;
> - if (!r5_core)
> - return -EINVAL;
> -
> - dev = r5_core->dev;
> -
> - cluster = dev_get_drvdata(dev->parent);
> - if (!cluster) {
> - dev_err(dev->parent, "Invalid driver data\n");
> - return -EINVAL;
> - }
> -
> - /*
> - * In lockstep mode TCM banks are one contiguous memory region of 256Kb
> - * In split mode, each TCM bank is 64Kb and not contiguous.
> - * We add memory carveouts accordingly.
> - */
> - if (cluster->mode == SPLIT_MODE)
> - return add_tcm_carveout_split_mode(rproc);
> - else if (cluster->mode == LOCKSTEP_MODE)
> - return add_tcm_carveout_lockstep_mode(rproc);
> -
> - return -EINVAL;
> -}
> -
> /*
> * zynqmp_r5_parse_fw()
> * @rproc: single R5 core's corresponding rproc instance
> --
> 2.25.1
>
On 2/28/24 11:06 AM, Mathieu Poirier wrote:
> On Mon, Feb 19, 2024 at 09:44:37AM -0800, Tanmay Shah wrote:
> > ZynqMP TCM information was fixed in driver. Now ZynqMP TCM information
> > is available in device-tree. Parse TCM information in driver
> > as per new bindings.
> >
> > Signed-off-by: Tanmay Shah <[email protected]>
> > ---
> >
> > Changes in v11:
> > - Remove redundant initialization of the variable
> > - return correct error code if memory allocation failed
>
> Where is that? I looked really hard but couldn't find it.
Hi Mathieu,
Thanks for reviews. Please find my comment below.
>
> >
> > drivers/remoteproc/xlnx_r5_remoteproc.c | 112 ++++++++++++++++++++++--
> > 1 file changed, 107 insertions(+), 5 deletions(-)
> >
> > diff --git a/drivers/remoteproc/xlnx_r5_remoteproc.c b/drivers/remoteproc/xlnx_r5_remoteproc.c
> > index 42b0384d34f2..d4a22caebaad 100644
> > --- a/drivers/remoteproc/xlnx_r5_remoteproc.c
> > +++ b/drivers/remoteproc/xlnx_r5_remoteproc.c
> > @@ -74,8 +74,8 @@ struct mbox_info {
> > };
> >
> > /*
> > - * Hardcoded TCM bank values. This will be removed once TCM bindings are
> > - * accepted for system-dt specifications and upstreamed in linux kernel
> > + * Hardcoded TCM bank values. This will stay in driver to maintain backward
> > + * compatibility with device-tree that does not have TCM information.
> > */
> > static const struct mem_bank_data zynqmp_tcm_banks_split[] = {
> > {0xffe00000UL, 0x0, 0x10000UL, PD_R5_0_ATCM, "atcm0"}, /* TCM 64KB each */
> > @@ -757,6 +757,103 @@ static struct zynqmp_r5_core *zynqmp_r5_add_rproc_core(struct device *cdev)
> > return ERR_PTR(ret);
> > }
> >
> > +static int zynqmp_r5_get_tcm_node_from_dt(struct zynqmp_r5_cluster *cluster)
> > +{
> > + int i, j, tcm_bank_count, ret, tcm_pd_idx, pd_count;
> > + struct of_phandle_args out_args;
> > + struct zynqmp_r5_core *r5_core;
> > + struct platform_device *cpdev;
> > + struct mem_bank_data *tcm;
> > + struct device_node *np;
> > + struct resource *res;
> > + u64 abs_addr, size;
> > + struct device *dev;
> > +
> > + for (i = 0; i < cluster->core_count; i++) {
> > + r5_core = cluster->r5_cores[i];
> > + dev = r5_core->dev;
> > + np = r5_core->np;
> > +
> > + pd_count = of_count_phandle_with_args(np, "power-domains",
> > + "#power-domain-cells");
> > +
> > + if (pd_count <= 0) {
> > + dev_err(dev, "invalid power-domains property, %d\n", pd_count);
> > + return -EINVAL;
> > + }
> > +
> > + /* First entry in power-domains list is for r5 core, rest for TCM. */
> > + tcm_bank_count = pd_count - 1;
> > +
> > + if (tcm_bank_count <= 0) {
> > + dev_err(dev, "invalid TCM count %d\n", tcm_bank_count);
> > + return -EINVAL;
> > + }
> > +
> > + r5_core->tcm_banks = devm_kcalloc(dev, tcm_bank_count,
> > + sizeof(struct mem_bank_data *),
> > + GFP_KERNEL);
> > + if (!r5_core->tcm_banks)
> > + return -ENOMEM;
Hi Mathiue,
Here: in v10 it was following:
+ if (!r5_core->tcm_banks)
+ ret = -ENOMEM;
Which is not correct. Somehow I missed to return the error code instead it was just storing the error in variable.
> > +
> > + r5_core->tcm_bank_count = tcm_bank_count;
> > + for (j = 0, tcm_pd_idx = 1; j < tcm_bank_count; j++, tcm_pd_idx++) {
> > + tcm = devm_kzalloc(dev, sizeof(struct mem_bank_data),
> > + GFP_KERNEL);
> > + if (!tcm)
> > + return -ENOMEM;
> > +
> > + r5_core->tcm_banks[j] = tcm;
> > +
> > + /* Get power-domains id of TCM. */
> > + ret = of_parse_phandle_with_args(np, "power-domains",
> > + "#power-domain-cells",
> > + tcm_pd_idx, &out_args);
> > + if (ret) {
> > + dev_err(r5_core->dev,
> > + "failed to get tcm %d pm domain, ret %d\n",
> > + tcm_pd_idx, ret);
> > + return ret;
> > + }
> > + tcm->pm_domain_id = out_args.args[0];
> > + of_node_put(out_args.np);
> > +
> > + /* Get TCM address without translation. */
> > + ret = of_property_read_reg(np, j, &abs_addr, &size);
> > + if (ret) {
> > + dev_err(dev, "failed to get reg property\n");
> > + return ret;
> > + }
> > +
> > + /*
> > + * Remote processor can address only 32 bits
> > + * so convert 64-bits into 32-bits. This will discard
> > + * any unwanted upper 32-bits.
> > + */
> > + tcm->da = (u32)abs_addr;
> > + tcm->size = (u32)size;
> > +
> > + cpdev = to_platform_device(dev);
> > + res = platform_get_resource(cpdev, IORESOURCE_MEM, j);
> > + if (!res) {
> > + dev_err(dev, "failed to get tcm resource\n");
> > + return -EINVAL;
> > + }
> > +
> > + tcm->addr = (u32)res->start;
> > + tcm->bank_name = (char *)res->name;
> > + res = devm_request_mem_region(dev, tcm->addr, tcm->size,
> > + tcm->bank_name);
> > + if (!res) {
> > + dev_err(dev, "failed to request tcm resource\n");
> > + return -EINVAL;
> > + }
> > + }
> > + }
> > +
> > + return 0;
> > +}
> > +
> > /**
> > * zynqmp_r5_get_tcm_node()
> > * Ideally this function should parse tcm node and store information
> > @@ -835,9 +932,14 @@ static int zynqmp_r5_core_init(struct zynqmp_r5_cluster *cluster,
> > struct zynqmp_r5_core *r5_core;
> > int ret, i;
> >
> > - ret = zynqmp_r5_get_tcm_node(cluster);
> > - if (ret < 0) {
> > - dev_err(dev, "can't get tcm node, err %d\n", ret);
> > + r5_core = cluster->r5_cores[0];
> > + if (of_find_property(r5_core->np, "reg", NULL))
> > + ret = zynqmp_r5_get_tcm_node_from_dt(cluster);
> > + else
> > + ret = zynqmp_r5_get_tcm_node(cluster);
> > +
> > + if (ret) {
> > + dev_err(dev, "can't get tcm, err %d\n", ret);
> > return ret;
> > }
> >
> > --
> > 2.25.1
> >
On 2/28/24 11:08 AM, Mathieu Poirier wrote:
> On Mon, Feb 19, 2024 at 09:44:34AM -0800, Tanmay Shah wrote:
> > In lockstep mode, r5 core0 uses TCM of R5 core1. Following is lockstep
> > mode memory region as per hardware reference manual.
> >
> > | *TCM* | *R5 View* | *Linux view* |
> > | R5_0 ATCM (128 KB) | 0x0000_0000 | 0xFFE0_0000 |
> > | R5_0 BTCM (128 KB) | 0x0002_0000 | 0xFFE2_0000 |
> >
> > However, driver shouldn't model it as above because R5 core0 TCM and core1
> > TCM has different power-domains mapped to it.
> > Hence, TCM address space in lockstep mode should be modeled as 64KB
> > regions only where each region has its own power-domain as following:
> >
> > | *TCM* | *R5 View* | *Linux view* |
> > | R5_0 ATCM0 (64 KB) | 0x0000_0000 | 0xFFE0_0000 |
> > | R5_0 BTCM0 (64 KB) | 0x0002_0000 | 0xFFE2_0000 |
> > | R5_0 ATCM1 (64 KB) | 0x0001_0000 | 0xFFE1_0000 |
> > | R5_0 BTCM1 (64 KB) | 0x0003_0000 | 0xFFE3_0000 |
> >
> > This makes driver maintanance easy and makes design robust for future
> > platorms as well.
> >
> > Signed-off-by: Tanmay Shah <[email protected]>
>
> Now that I have a clearer picture of where things are going, I am adding this
> patch to rproc-next.
>
> I'll wait for the DT crew for the rest of this set.
Hi Mathieu,
Is it okay if we wait for DT crew to clear new bindings as well before taking this one to rproc-next ?
Just in case any modifications needed further?
Tanmay
>
> Thanks,
> Mathieu
>
> > ---
> > drivers/remoteproc/xlnx_r5_remoteproc.c | 145 ++----------------------
> > 1 file changed, 12 insertions(+), 133 deletions(-)
> >
> > diff --git a/drivers/remoteproc/xlnx_r5_remoteproc.c b/drivers/remoteproc/xlnx_r5_remoteproc.c
> > index 4395edea9a64..42b0384d34f2 100644
> > --- a/drivers/remoteproc/xlnx_r5_remoteproc.c
> > +++ b/drivers/remoteproc/xlnx_r5_remoteproc.c
> > @@ -84,12 +84,12 @@ static const struct mem_bank_data zynqmp_tcm_banks_split[] = {
> > {0xffeb0000UL, 0x20000, 0x10000UL, PD_R5_1_BTCM, "btcm1"},
> > };
> >
> > -/* In lockstep mode cluster combines each 64KB TCM and makes 128KB TCM */
> > +/* In lockstep mode cluster uses each 64KB TCM from second core as well */
> > static const struct mem_bank_data zynqmp_tcm_banks_lockstep[] = {
> > - {0xffe00000UL, 0x0, 0x20000UL, PD_R5_0_ATCM, "atcm0"}, /* TCM 128KB each */
> > - {0xffe20000UL, 0x20000, 0x20000UL, PD_R5_0_BTCM, "btcm0"},
> > - {0, 0, 0, PD_R5_1_ATCM, ""},
> > - {0, 0, 0, PD_R5_1_BTCM, ""},
> > + {0xffe00000UL, 0x0, 0x10000UL, PD_R5_0_ATCM, "atcm0"}, /* TCM 64KB each */
> > + {0xffe20000UL, 0x20000, 0x10000UL, PD_R5_0_BTCM, "btcm0"},
> > + {0xffe10000UL, 0x10000, 0x10000UL, PD_R5_1_ATCM, "atcm1"},
> > + {0xffe30000UL, 0x30000, 0x10000UL, PD_R5_1_BTCM, "btcm1"},
> > };
> >
> > /**
> > @@ -540,14 +540,14 @@ static int tcm_mem_map(struct rproc *rproc,
> > }
> >
> > /*
> > - * add_tcm_carveout_split_mode()
> > + * add_tcm_banks()
> > * @rproc: single R5 core's corresponding rproc instance
> > *
> > - * allocate and add remoteproc carveout for TCM memory in split mode
> > + * allocate and add remoteproc carveout for TCM memory
> > *
> > * return 0 on success, otherwise non-zero value on failure
> > */
> > -static int add_tcm_carveout_split_mode(struct rproc *rproc)
> > +static int add_tcm_banks(struct rproc *rproc)
> > {
> > struct rproc_mem_entry *rproc_mem;
> > struct zynqmp_r5_core *r5_core;
> > @@ -580,10 +580,10 @@ static int add_tcm_carveout_split_mode(struct rproc *rproc)
> > ZYNQMP_PM_REQUEST_ACK_BLOCKING);
> > if (ret < 0) {
> > dev_err(dev, "failed to turn on TCM 0x%x", pm_domain_id);
> > - goto release_tcm_split;
> > + goto release_tcm;
> > }
> >
> > - dev_dbg(dev, "TCM carveout split mode %s addr=%llx, da=0x%x, size=0x%lx",
> > + dev_dbg(dev, "TCM carveout %s addr=%llx, da=0x%x, size=0x%lx",
> > bank_name, bank_addr, da, bank_size);
> >
> > rproc_mem = rproc_mem_entry_init(dev, NULL, bank_addr,
> > @@ -593,7 +593,7 @@ static int add_tcm_carveout_split_mode(struct rproc *rproc)
> > if (!rproc_mem) {
> > ret = -ENOMEM;
> > zynqmp_pm_release_node(pm_domain_id);
> > - goto release_tcm_split;
> > + goto release_tcm;
> > }
> >
> > rproc_add_carveout(rproc, rproc_mem);
> > @@ -601,7 +601,7 @@ static int add_tcm_carveout_split_mode(struct rproc *rproc)
> >
> > return 0;
> >
> > -release_tcm_split:
> > +release_tcm:
> > /* If failed, Turn off all TCM banks turned on before */
> > for (i--; i >= 0; i--) {
> > pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
> > @@ -610,127 +610,6 @@ static int add_tcm_carveout_split_mode(struct rproc *rproc)
> > return ret;
> > }
> >
> > -/*
> > - * add_tcm_carveout_lockstep_mode()
> > - * @rproc: single R5 core's corresponding rproc instance
> > - *
> > - * allocate and add remoteproc carveout for TCM memory in lockstep mode
> > - *
> > - * return 0 on success, otherwise non-zero value on failure
> > - */
> > -static int add_tcm_carveout_lockstep_mode(struct rproc *rproc)
> > -{
> > - struct rproc_mem_entry *rproc_mem;
> > - struct zynqmp_r5_core *r5_core;
> > - int i, num_banks, ret;
> > - phys_addr_t bank_addr;
> > - size_t bank_size = 0;
> > - struct device *dev;
> > - u32 pm_domain_id;
> > - char *bank_name;
> > - u32 da;
> > -
> > - r5_core = rproc->priv;
> > - dev = r5_core->dev;
> > -
> > - /* Go through zynqmp banks for r5 node */
> > - num_banks = r5_core->tcm_bank_count;
> > -
> > - /*
> > - * In lockstep mode, TCM is contiguous memory block
> > - * However, each TCM block still needs to be enabled individually.
> > - * So, Enable each TCM block individually.
> > - * Although ATCM and BTCM is contiguous memory block, add two separate
> > - * carveouts for both.
> > - */
> > - for (i = 0; i < num_banks; i++) {
> > - pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
> > -
> > - /* Turn on each TCM bank individually */
> > - ret = zynqmp_pm_request_node(pm_domain_id,
> > - ZYNQMP_PM_CAPABILITY_ACCESS, 0,
> > - ZYNQMP_PM_REQUEST_ACK_BLOCKING);
> > - if (ret < 0) {
> > - dev_err(dev, "failed to turn on TCM 0x%x", pm_domain_id);
> > - goto release_tcm_lockstep;
> > - }
> > -
> > - bank_size = r5_core->tcm_banks[i]->size;
> > - if (bank_size == 0)
> > - continue;
> > -
> > - bank_addr = r5_core->tcm_banks[i]->addr;
> > - da = r5_core->tcm_banks[i]->da;
> > - bank_name = r5_core->tcm_banks[i]->bank_name;
> > -
> > - /* Register TCM address range, TCM map and unmap functions */
> > - rproc_mem = rproc_mem_entry_init(dev, NULL, bank_addr,
> > - bank_size, da,
> > - tcm_mem_map, tcm_mem_unmap,
> > - bank_name);
> > - if (!rproc_mem) {
> > - ret = -ENOMEM;
> > - zynqmp_pm_release_node(pm_domain_id);
> > - goto release_tcm_lockstep;
> > - }
> > -
> > - /* If registration is success, add carveouts */
> > - rproc_add_carveout(rproc, rproc_mem);
> > -
> > - dev_dbg(dev, "TCM carveout lockstep mode %s addr=0x%llx, da=0x%x, size=0x%lx",
> > - bank_name, bank_addr, da, bank_size);
> > - }
> > -
> > - return 0;
> > -
> > -release_tcm_lockstep:
> > - /* If failed, Turn off all TCM banks turned on before */
> > - for (i--; i >= 0; i--) {
> > - pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
> > - zynqmp_pm_release_node(pm_domain_id);
> > - }
> > - return ret;
> > -}
> > -
> > -/*
> > - * add_tcm_banks()
> > - * @rproc: single R5 core's corresponding rproc instance
> > - *
> > - * allocate and add remoteproc carveouts for TCM memory based on cluster mode
> > - *
> > - * return 0 on success, otherwise non-zero value on failure
> > - */
> > -static int add_tcm_banks(struct rproc *rproc)
> > -{
> > - struct zynqmp_r5_cluster *cluster;
> > - struct zynqmp_r5_core *r5_core;
> > - struct device *dev;
> > -
> > - r5_core = rproc->priv;
> > - if (!r5_core)
> > - return -EINVAL;
> > -
> > - dev = r5_core->dev;
> > -
> > - cluster = dev_get_drvdata(dev->parent);
> > - if (!cluster) {
> > - dev_err(dev->parent, "Invalid driver data\n");
> > - return -EINVAL;
> > - }
> > -
> > - /*
> > - * In lockstep mode TCM banks are one contiguous memory region of 256Kb
> > - * In split mode, each TCM bank is 64Kb and not contiguous.
> > - * We add memory carveouts accordingly.
> > - */
> > - if (cluster->mode == SPLIT_MODE)
> > - return add_tcm_carveout_split_mode(rproc);
> > - else if (cluster->mode == LOCKSTEP_MODE)
> > - return add_tcm_carveout_lockstep_mode(rproc);
> > -
> > - return -EINVAL;
> > -}
> > -
> > /*
> > * zynqmp_r5_parse_fw()
> > * @rproc: single R5 core's corresponding rproc instance
> > --
> > 2.25.1
> >
On Wed, 28 Feb 2024 at 12:24, Tanmay Shah <[email protected]> wrote:
>
>
> On 2/28/24 11:08 AM, Mathieu Poirier wrote:
> > On Mon, Feb 19, 2024 at 09:44:34AM -0800, Tanmay Shah wrote:
> > > In lockstep mode, r5 core0 uses TCM of R5 core1. Following is lockstep
> > > mode memory region as per hardware reference manual.
> > >
> > > | *TCM* | *R5 View* | *Linux view* |
> > > | R5_0 ATCM (128 KB) | 0x0000_0000 | 0xFFE0_0000 |
> > > | R5_0 BTCM (128 KB) | 0x0002_0000 | 0xFFE2_0000 |
> > >
> > > However, driver shouldn't model it as above because R5 core0 TCM and core1
> > > TCM has different power-domains mapped to it.
> > > Hence, TCM address space in lockstep mode should be modeled as 64KB
> > > regions only where each region has its own power-domain as following:
> > >
> > > | *TCM* | *R5 View* | *Linux view* |
> > > | R5_0 ATCM0 (64 KB) | 0x0000_0000 | 0xFFE0_0000 |
> > > | R5_0 BTCM0 (64 KB) | 0x0002_0000 | 0xFFE2_0000 |
> > > | R5_0 ATCM1 (64 KB) | 0x0001_0000 | 0xFFE1_0000 |
> > > | R5_0 BTCM1 (64 KB) | 0x0003_0000 | 0xFFE3_0000 |
> > >
> > > This makes driver maintanance easy and makes design robust for future
> > > platorms as well.
> > >
> > > Signed-off-by: Tanmay Shah <[email protected]>
> >
> > Now that I have a clearer picture of where things are going, I am adding this
> > patch to rproc-next.
> >
> > I'll wait for the DT crew for the rest of this set.
>
> Hi Mathieu,
>
> Is it okay if we wait for DT crew to clear new bindings as well before taking this one to rproc-next ?
>
> Just in case any modifications needed further?
>
Sure, we can do that too.
>
> Tanmay
>
>
> >
> > Thanks,
> > Mathieu
> >
> > > ---
> > > drivers/remoteproc/xlnx_r5_remoteproc.c | 145 ++----------------------
> > > 1 file changed, 12 insertions(+), 133 deletions(-)
> > >
> > > diff --git a/drivers/remoteproc/xlnx_r5_remoteproc.c b/drivers/remoteproc/xlnx_r5_remoteproc.c
> > > index 4395edea9a64..42b0384d34f2 100644
> > > --- a/drivers/remoteproc/xlnx_r5_remoteproc.c
> > > +++ b/drivers/remoteproc/xlnx_r5_remoteproc.c
> > > @@ -84,12 +84,12 @@ static const struct mem_bank_data zynqmp_tcm_banks_split[] = {
> > > {0xffeb0000UL, 0x20000, 0x10000UL, PD_R5_1_BTCM, "btcm1"},
> > > };
> > >
> > > -/* In lockstep mode cluster combines each 64KB TCM and makes 128KB TCM */
> > > +/* In lockstep mode cluster uses each 64KB TCM from second core as well */
> > > static const struct mem_bank_data zynqmp_tcm_banks_lockstep[] = {
> > > - {0xffe00000UL, 0x0, 0x20000UL, PD_R5_0_ATCM, "atcm0"}, /* TCM 128KB each */
> > > - {0xffe20000UL, 0x20000, 0x20000UL, PD_R5_0_BTCM, "btcm0"},
> > > - {0, 0, 0, PD_R5_1_ATCM, ""},
> > > - {0, 0, 0, PD_R5_1_BTCM, ""},
> > > + {0xffe00000UL, 0x0, 0x10000UL, PD_R5_0_ATCM, "atcm0"}, /* TCM 64KB each */
> > > + {0xffe20000UL, 0x20000, 0x10000UL, PD_R5_0_BTCM, "btcm0"},
> > > + {0xffe10000UL, 0x10000, 0x10000UL, PD_R5_1_ATCM, "atcm1"},
> > > + {0xffe30000UL, 0x30000, 0x10000UL, PD_R5_1_BTCM, "btcm1"},
> > > };
> > >
> > > /**
> > > @@ -540,14 +540,14 @@ static int tcm_mem_map(struct rproc *rproc,
> > > }
> > >
> > > /*
> > > - * add_tcm_carveout_split_mode()
> > > + * add_tcm_banks()
> > > * @rproc: single R5 core's corresponding rproc instance
> > > *
> > > - * allocate and add remoteproc carveout for TCM memory in split mode
> > > + * allocate and add remoteproc carveout for TCM memory
> > > *
> > > * return 0 on success, otherwise non-zero value on failure
> > > */
> > > -static int add_tcm_carveout_split_mode(struct rproc *rproc)
> > > +static int add_tcm_banks(struct rproc *rproc)
> > > {
> > > struct rproc_mem_entry *rproc_mem;
> > > struct zynqmp_r5_core *r5_core;
> > > @@ -580,10 +580,10 @@ static int add_tcm_carveout_split_mode(struct rproc *rproc)
> > > ZYNQMP_PM_REQUEST_ACK_BLOCKING);
> > > if (ret < 0) {
> > > dev_err(dev, "failed to turn on TCM 0x%x", pm_domain_id);
> > > - goto release_tcm_split;
> > > + goto release_tcm;
> > > }
> > >
> > > - dev_dbg(dev, "TCM carveout split mode %s addr=%llx, da=0x%x, size=0x%lx",
> > > + dev_dbg(dev, "TCM carveout %s addr=%llx, da=0x%x, size=0x%lx",
> > > bank_name, bank_addr, da, bank_size);
> > >
> > > rproc_mem = rproc_mem_entry_init(dev, NULL, bank_addr,
> > > @@ -593,7 +593,7 @@ static int add_tcm_carveout_split_mode(struct rproc *rproc)
> > > if (!rproc_mem) {
> > > ret = -ENOMEM;
> > > zynqmp_pm_release_node(pm_domain_id);
> > > - goto release_tcm_split;
> > > + goto release_tcm;
> > > }
> > >
> > > rproc_add_carveout(rproc, rproc_mem);
> > > @@ -601,7 +601,7 @@ static int add_tcm_carveout_split_mode(struct rproc *rproc)
> > >
> > > return 0;
> > >
> > > -release_tcm_split:
> > > +release_tcm:
> > > /* If failed, Turn off all TCM banks turned on before */
> > > for (i--; i >= 0; i--) {
> > > pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
> > > @@ -610,127 +610,6 @@ static int add_tcm_carveout_split_mode(struct rproc *rproc)
> > > return ret;
> > > }
> > >
> > > -/*
> > > - * add_tcm_carveout_lockstep_mode()
> > > - * @rproc: single R5 core's corresponding rproc instance
> > > - *
> > > - * allocate and add remoteproc carveout for TCM memory in lockstep mode
> > > - *
> > > - * return 0 on success, otherwise non-zero value on failure
> > > - */
> > > -static int add_tcm_carveout_lockstep_mode(struct rproc *rproc)
> > > -{
> > > - struct rproc_mem_entry *rproc_mem;
> > > - struct zynqmp_r5_core *r5_core;
> > > - int i, num_banks, ret;
> > > - phys_addr_t bank_addr;
> > > - size_t bank_size = 0;
> > > - struct device *dev;
> > > - u32 pm_domain_id;
> > > - char *bank_name;
> > > - u32 da;
> > > -
> > > - r5_core = rproc->priv;
> > > - dev = r5_core->dev;
> > > -
> > > - /* Go through zynqmp banks for r5 node */
> > > - num_banks = r5_core->tcm_bank_count;
> > > -
> > > - /*
> > > - * In lockstep mode, TCM is contiguous memory block
> > > - * However, each TCM block still needs to be enabled individually.
> > > - * So, Enable each TCM block individually.
> > > - * Although ATCM and BTCM is contiguous memory block, add two separate
> > > - * carveouts for both.
> > > - */
> > > - for (i = 0; i < num_banks; i++) {
> > > - pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
> > > -
> > > - /* Turn on each TCM bank individually */
> > > - ret = zynqmp_pm_request_node(pm_domain_id,
> > > - ZYNQMP_PM_CAPABILITY_ACCESS, 0,
> > > - ZYNQMP_PM_REQUEST_ACK_BLOCKING);
> > > - if (ret < 0) {
> > > - dev_err(dev, "failed to turn on TCM 0x%x", pm_domain_id);
> > > - goto release_tcm_lockstep;
> > > - }
> > > -
> > > - bank_size = r5_core->tcm_banks[i]->size;
> > > - if (bank_size == 0)
> > > - continue;
> > > -
> > > - bank_addr = r5_core->tcm_banks[i]->addr;
> > > - da = r5_core->tcm_banks[i]->da;
> > > - bank_name = r5_core->tcm_banks[i]->bank_name;
> > > -
> > > - /* Register TCM address range, TCM map and unmap functions */
> > > - rproc_mem = rproc_mem_entry_init(dev, NULL, bank_addr,
> > > - bank_size, da,
> > > - tcm_mem_map, tcm_mem_unmap,
> > > - bank_name);
> > > - if (!rproc_mem) {
> > > - ret = -ENOMEM;
> > > - zynqmp_pm_release_node(pm_domain_id);
> > > - goto release_tcm_lockstep;
> > > - }
> > > -
> > > - /* If registration is success, add carveouts */
> > > - rproc_add_carveout(rproc, rproc_mem);
> > > -
> > > - dev_dbg(dev, "TCM carveout lockstep mode %s addr=0x%llx, da=0x%x, size=0x%lx",
> > > - bank_name, bank_addr, da, bank_size);
> > > - }
> > > -
> > > - return 0;
> > > -
> > > -release_tcm_lockstep:
> > > - /* If failed, Turn off all TCM banks turned on before */
> > > - for (i--; i >= 0; i--) {
> > > - pm_domain_id = r5_core->tcm_banks[i]->pm_domain_id;
> > > - zynqmp_pm_release_node(pm_domain_id);
> > > - }
> > > - return ret;
> > > -}
> > > -
> > > -/*
> > > - * add_tcm_banks()
> > > - * @rproc: single R5 core's corresponding rproc instance
> > > - *
> > > - * allocate and add remoteproc carveouts for TCM memory based on cluster mode
> > > - *
> > > - * return 0 on success, otherwise non-zero value on failure
> > > - */
> > > -static int add_tcm_banks(struct rproc *rproc)
> > > -{
> > > - struct zynqmp_r5_cluster *cluster;
> > > - struct zynqmp_r5_core *r5_core;
> > > - struct device *dev;
> > > -
> > > - r5_core = rproc->priv;
> > > - if (!r5_core)
> > > - return -EINVAL;
> > > -
> > > - dev = r5_core->dev;
> > > -
> > > - cluster = dev_get_drvdata(dev->parent);
> > > - if (!cluster) {
> > > - dev_err(dev->parent, "Invalid driver data\n");
> > > - return -EINVAL;
> > > - }
> > > -
> > > - /*
> > > - * In lockstep mode TCM banks are one contiguous memory region of 256Kb
> > > - * In split mode, each TCM bank is 64Kb and not contiguous.
> > > - * We add memory carveouts accordingly.
> > > - */
> > > - if (cluster->mode == SPLIT_MODE)
> > > - return add_tcm_carveout_split_mode(rproc);
> > > - else if (cluster->mode == LOCKSTEP_MODE)
> > > - return add_tcm_carveout_lockstep_mode(rproc);
> > > -
> > > - return -EINVAL;
> > > -}
> > > -
> > > /*
> > > * zynqmp_r5_parse_fw()
> > > * @rproc: single R5 core's corresponding rproc instance
> > > --
> > > 2.25.1
> > >