This series contains Arm's NPU Ethos-U driver for NXP i.MX93 platform.
Ethos-U Linux driver is to provide an example of how a rich operating
system like Linux can dispatch inferences to an Arm Cortex-M subsystem,
consisting of an Arm Cortex-M and an Arm Ethos-U NPU.
----------------------------------------------------------------
Alison Wang (8):
ethosu: Add Arm Ethos-U driver
ethosu: Use RPMsg messaging protocol based on i.MX Rpmsg implementation
ethosu: Add inference type option for model and op
ethosu: Add suspend/resume power management
ethosu: Use ids for identifying messages sent to Ethos-U firmware
ethosu: Add core message about network info
ethosu: Add core message about inference cancellation
ethosu: Add rwlock when alloc and remove msg id
drivers/firmware/Kconfig | 1 +
drivers/firmware/Makefile | 1 +
drivers/firmware/ethosu/Kconfig | 24 ++++
drivers/firmware/ethosu/Makefile | 31 +++++
drivers/firmware/ethosu/ethosu_buffer.c | 319 +++++++++++++++++++++++++++++++++++++++++++
drivers/firmware/ethosu/ethosu_buffer.h | 106 +++++++++++++++
drivers/firmware/ethosu/ethosu_cancel_inference.c | 185 +++++++++++++++++++++++++
drivers/firmware/ethosu/ethosu_cancel_inference.h | 55 ++++++++
drivers/firmware/ethosu/ethosu_capabilities.c | 157 ++++++++++++++++++++++
drivers/firmware/ethosu/ethosu_capabilities.h | 47 +++++++
drivers/firmware/ethosu/ethosu_core_interface.h | 276 ++++++++++++++++++++++++++++++++++++++
drivers/firmware/ethosu/ethosu_device.c | 404 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
drivers/firmware/ethosu/ethosu_device.h | 81 +++++++++++
drivers/firmware/ethosu/ethosu_driver.c | 201 +++++++++++++++++++++++++++
drivers/firmware/ethosu/ethosu_inference.c | 529 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
drivers/firmware/ethosu/ethosu_inference.h | 125 +++++++++++++++++
drivers/firmware/ethosu/ethosu_network.c | 229 +++++++++++++++++++++++++++++++
drivers/firmware/ethosu/ethosu_network.h | 84 ++++++++++++
drivers/firmware/ethosu/ethosu_network_info.c | 184 +++++++++++++++++++++++++
drivers/firmware/ethosu/ethosu_network_info.h | 56 ++++++++
drivers/firmware/ethosu/ethosu_rpmsg.c | 414 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
drivers/firmware/ethosu/ethosu_rpmsg.h | 155 +++++++++++++++++++++
drivers/firmware/ethosu/uapi/ethosu.h | 285 +++++++++++++++++++++++++++++++++++++++
23 files changed, 3949 insertions(+)
create mode 100644 drivers/firmware/ethosu/Kconfig
create mode 100644 drivers/firmware/ethosu/Makefile
create mode 100644 drivers/firmware/ethosu/ethosu_buffer.c
create mode 100644 drivers/firmware/ethosu/ethosu_buffer.h
create mode 100644 drivers/firmware/ethosu/ethosu_cancel_inference.c
create mode 100644 drivers/firmware/ethosu/ethosu_cancel_inference.h
create mode 100644 drivers/firmware/ethosu/ethosu_capabilities.c
create mode 100644 drivers/firmware/ethosu/ethosu_capabilities.h
create mode 100644 drivers/firmware/ethosu/ethosu_core_interface.h
create mode 100644 drivers/firmware/ethosu/ethosu_device.c
create mode 100644 drivers/firmware/ethosu/ethosu_device.h
create mode 100644 drivers/firmware/ethosu/ethosu_driver.c
create mode 100644 drivers/firmware/ethosu/ethosu_inference.c
create mode 100644 drivers/firmware/ethosu/ethosu_inference.h
create mode 100644 drivers/firmware/ethosu/ethosu_network.c
create mode 100644 drivers/firmware/ethosu/ethosu_network.h
create mode 100644 drivers/firmware/ethosu/ethosu_network_info.c
create mode 100644 drivers/firmware/ethosu/ethosu_network_info.h
create mode 100644 drivers/firmware/ethosu/ethosu_rpmsg.c
create mode 100644 drivers/firmware/ethosu/ethosu_rpmsg.h
create mode 100644 drivers/firmware/ethosu/uapi/ethosu.h
This patch adds inference type option for model and op.
Signed-off-by: Feng Guo <[email protected]>
Reviewed-by: Peng Fan <[email protected]>
---
drivers/firmware/ethosu/ethosu_core_interface.h | 6 +++++-
drivers/firmware/ethosu/ethosu_inference.c | 14 +++++++++++++-
drivers/firmware/ethosu/ethosu_inference.h | 1 +
drivers/firmware/ethosu/ethosu_rpmsg.c | 4 +++-
drivers/firmware/ethosu/ethosu_rpmsg.h | 3 ++-
drivers/firmware/ethosu/uapi/ethosu.h | 11 ++++++++++-
6 files changed, 34 insertions(+), 5 deletions(-)
diff --git a/drivers/firmware/ethosu/ethosu_core_interface.h b/drivers/firmware/ethosu/ethosu_core_interface.h
index ef63c3b55352..b60508e4792f 100644
--- a/drivers/firmware/ethosu/ethosu_core_interface.h
+++ b/drivers/firmware/ethosu/ethosu_core_interface.h
@@ -35,13 +35,16 @@ namespace EthosU {
#define ETHOSU_CORE_BUFFER_MAX 16
/** Maximum number of PMU counters to be returned for inference */
-#define ETHOSU_CORE_PMU_MAX 8
+#define ETHOSU_CORE_PMU_MAX 4
#define ETHOSU_CORE_MSG_MAGIC 0x41457631
#define ETHOSU_CORE_MSG_VERSION_MAJOR 0
#define ETHOSU_CORE_MSG_VERSION_MINOR 2
#define ETHOSU_CORE_MSG_VERSION_PATCH 0
+#define ETHOSU_CORE_INFERENCE_MODEL 0
+#define ETHOSU_CORE_INFERENCE_OP 1
+
/**
* enum ethosu_core_msg_type - Message types
*
@@ -107,6 +110,7 @@ struct ethosu_core_inference_req {
struct ethosu_core_buffer network;
uint8_t pmu_event_config[ETHOSU_CORE_PMU_MAX];
uint32_t pmu_cycle_counter_enable;
+ uint32_t inference_type;
};
struct ethosu_core_inference_rsp {
diff --git a/drivers/firmware/ethosu/ethosu_inference.c b/drivers/firmware/ethosu/ethosu_inference.c
index 853947a2334e..b5e94e19bce0 100644
--- a/drivers/firmware/ethosu/ethosu_inference.c
+++ b/drivers/firmware/ethosu/ethosu_inference.c
@@ -92,7 +92,8 @@ static int ethosu_inference_send(struct ethosu_inference *inf)
inf->net->buf,
inf->pmu_event_config,
ETHOSU_PMU_EVENT_MAX,
- inf->pmu_cycle_counter_enable);
+ inf->pmu_cycle_counter_enable,
+ inf->inference_type);
if (ret)
return ret;
@@ -233,6 +234,17 @@ int ethosu_inference_create(struct ethosu_device *edev,
inf = devm_kzalloc(edev->dev, sizeof(*inf), GFP_KERNEL);
if (!inf)
return -ENOMEM;
+ switch (uapi->inference_type) {
+ case ETHOSU_UAPI_INFERENCE_MODEL:
+ inf->inference_type = ETHOSU_CORE_INFERENCE_MODEL;
+ break;
+ case ETHOSU_UAPI_INFERENCE_OP:
+ inf->inference_type = ETHOSU_CORE_INFERENCE_OP;
+ break;
+ default:
+ inf->inference_type = ETHOSU_CORE_INFERENCE_MODEL;
+ break;
+ }
inf->edev = edev;
inf->net = net;
diff --git a/drivers/firmware/ethosu/ethosu_inference.h b/drivers/firmware/ethosu/ethosu_inference.h
index 07370ca01f22..8414eadbda92 100644
--- a/drivers/firmware/ethosu/ethosu_inference.h
+++ b/drivers/firmware/ethosu/ethosu_inference.h
@@ -74,6 +74,7 @@ struct ethosu_inference {
uint32_t pmu_event_count[ETHOSU_PMU_EVENT_MAX];
uint32_t pmu_cycle_counter_enable;
uint64_t pmu_cycle_counter_count;
+ uint32_t inference_type;
struct list_head list;
};
diff --git a/drivers/firmware/ethosu/ethosu_rpmsg.c b/drivers/firmware/ethosu/ethosu_rpmsg.c
index cb7e4556f635..e4cf398468e4 100644
--- a/drivers/firmware/ethosu/ethosu_rpmsg.c
+++ b/drivers/firmware/ethosu/ethosu_rpmsg.c
@@ -107,7 +107,8 @@ int ethosu_rpmsg_inference(struct ethosu_rpmsg *erp,
struct ethosu_buffer *network,
uint8_t *pmu_event_config,
uint8_t pmu_event_config_count,
- uint8_t pmu_cycle_counter_enable)
+ uint8_t pmu_cycle_counter_enable,
+ uint32_t inference_type)
{
struct ethosu_core_msg msg = {
.magic = ETHOSU_CORE_MSG_MAGIC,
@@ -132,6 +133,7 @@ int ethosu_rpmsg_inference(struct ethosu_rpmsg *erp,
req.ifm_count = ifm_count;
req.ofm_count = ofm_count;
req.pmu_cycle_counter_enable = pmu_cycle_counter_enable;
+ req.inference_type = inference_type;
for (i = 0; i < ifm_count; i++)
ethosu_core_set_size(ifm[i], &req.ifm[i]);
diff --git a/drivers/firmware/ethosu/ethosu_rpmsg.h b/drivers/firmware/ethosu/ethosu_rpmsg.h
index dd08e0d7945b..a4a639997a26 100644
--- a/drivers/firmware/ethosu/ethosu_rpmsg.h
+++ b/drivers/firmware/ethosu/ethosu_rpmsg.h
@@ -63,7 +63,8 @@ int ethosu_rpmsg_inference(struct ethosu_rpmsg *erp,
struct ethosu_buffer *network,
uint8_t *pmu_event_config,
uint8_t pmu_event_config_count,
- uint8_t pmu_cycle_counter_enable
+ uint8_t pmu_cycle_counter_enable,
+ uint32_t inference_type
);
int ethosu_rpmsg_init(struct ethosu_rpmsg *erp,
diff --git a/drivers/firmware/ethosu/uapi/ethosu.h b/drivers/firmware/ethosu/uapi/ethosu.h
index 903316dff7b3..c4a0df67336c 100644
--- a/drivers/firmware/ethosu/uapi/ethosu.h
+++ b/drivers/firmware/ethosu/uapi/ethosu.h
@@ -62,7 +62,7 @@ namespace EthosU {
#define ETHOSU_FD_MAX 16
/* Maximum number of PMUs available */
-#define ETHOSU_PMU_EVENT_MAX 8
+#define ETHOSU_PMU_EVENT_MAX 4
/****************************************************************************
* Types
@@ -173,6 +173,14 @@ struct ethosu_uapi_device_capabilities {
__u32 driver_major_rev;
};
+/**
+ * enum ethosu_uapi_inference_type - Inference type
+ */
+enum ethosu_uapi_inference_type {
+ ETHOSU_UAPI_INFERENCE_MODEL = 0,
+ ETHOSU_UAPI_INFERENCE_OP
+};
+
/**
* struct ethosu_uapi_inference_create - Create network request
* @ifm_count: Number of IFM file descriptors
@@ -185,6 +193,7 @@ struct ethosu_uapi_inference_create {
__u32 ifm_fd[ETHOSU_FD_MAX];
__u32 ofm_count;
__u32 ofm_fd[ETHOSU_FD_MAX];
+ enum ethosu_uapi_inference_type inference_type;
struct ethosu_uapi_pmu_config pmu_config;
};
--
2.17.1
Avoid to use pointers to kernel memory as identify messages, prior this
change, if memory is reused that can lead to unexpected conflicts.
Remove ref counting from capabilities since memory is freed in only one
place.
Finally, extract the capabilities code in its own files.
Signed-off-by: Davide Grohmann <[email protected]>
Signed-off-by: Kristofer Jonsson <[email protected]>
Signed-off-by: Alison Wang <[email protected]>
---
drivers/firmware/ethosu/Kconfig | 2 +-
drivers/firmware/ethosu/Makefile | 5 +-
drivers/firmware/ethosu/ethosu_buffer.c | 34 +--
drivers/firmware/ethosu/ethosu_buffer.h | 2 +-
drivers/firmware/ethosu/ethosu_capabilities.c | 157 ++++++++++++
drivers/firmware/ethosu/ethosu_capabilities.h | 47 ++++
.../firmware/ethosu/ethosu_core_interface.h | 20 +-
drivers/firmware/ethosu/ethosu_device.c | 229 +++++-------------
drivers/firmware/ethosu/ethosu_device.h | 17 +-
drivers/firmware/ethosu/ethosu_driver.c | 4 +-
drivers/firmware/ethosu/ethosu_inference.c | 229 ++++++++++++------
drivers/firmware/ethosu/ethosu_inference.h | 14 +-
drivers/firmware/ethosu/ethosu_network.c | 22 +-
drivers/firmware/ethosu/ethosu_network.h | 2 +-
drivers/firmware/ethosu/ethosu_rpmsg.c | 72 +++++-
drivers/firmware/ethosu/ethosu_rpmsg.h | 53 +++-
drivers/firmware/ethosu/uapi/ethosu.h | 8 +-
17 files changed, 606 insertions(+), 311 deletions(-)
create mode 100644 drivers/firmware/ethosu/ethosu_capabilities.c
create mode 100644 drivers/firmware/ethosu/ethosu_capabilities.h
diff --git a/drivers/firmware/ethosu/Kconfig b/drivers/firmware/ethosu/Kconfig
index ce837f45f8e5..4d3b1b86aabe 100644
--- a/drivers/firmware/ethosu/Kconfig
+++ b/drivers/firmware/ethosu/Kconfig
@@ -1,5 +1,5 @@
#
-# (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
+# (C) COPYRIGHT 2020,2022 ARM Limited.
#
# This program is free software and is provided to you under the terms of the
# GNU General Public License version 2 as published by the Free Software
diff --git a/drivers/firmware/ethosu/Makefile b/drivers/firmware/ethosu/Makefile
index 0b3fe72c9c4f..fa0731adad35 100644
--- a/drivers/firmware/ethosu/Makefile
+++ b/drivers/firmware/ethosu/Makefile
@@ -1,5 +1,5 @@
#
-# (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
+# Copyright (c) 2020,2022 Arm Limited.
#
# This program is free software and is provided to you under the terms of the
# GNU General Public License version 2 as published by the Free Software
@@ -25,4 +25,5 @@ ethosu-objs := ethosu_driver.o \
ethosu_device.o \
ethosu_inference.o \
ethosu_rpmsg.o \
- ethosu_network.o
+ ethosu_network.o \
+ ethosu_capabilities.o
diff --git a/drivers/firmware/ethosu/ethosu_buffer.c b/drivers/firmware/ethosu/ethosu_buffer.c
index c8aa8031a8de..d71db348a240 100644
--- a/drivers/firmware/ethosu/ethosu_buffer.c
+++ b/drivers/firmware/ethosu/ethosu_buffer.c
@@ -1,5 +1,5 @@
/*
- * (C) COPYRIGHT 2020-2021 Arm Limited. All rights reserved.
+ * Copyright (c) 2020-2022 Arm Limited.
*
* This program is free software and is provided to you under the terms of the
* GNU General Public License version 2 as published by the Free Software
@@ -126,7 +126,7 @@ static void ethosu_buffer_destroy(struct kref *kref)
struct ethosu_buffer *buf =
container_of(kref, struct ethosu_buffer, kref);
- dev_info(buf->edev->dev, "Buffer destroy. handle=0x%pK\n", buf);
+ dev_dbg(buf->edev->dev, "Buffer destroy. buf=0x%pK\n", buf);
dma_free_coherent(buf->edev->dev, buf->capacity, buf->cpu_addr,
buf->dma_addr_orig);
@@ -138,7 +138,8 @@ static int ethosu_buffer_release(struct inode *inode,
{
struct ethosu_buffer *buf = file->private_data;
- dev_info(buf->edev->dev, "Buffer release. handle=0x%pK\n", buf);
+ dev_dbg(buf->edev->dev, "Buffer release. file=0x%pK, buf=0x%pK\n",
+ file, buf);
ethosu_buffer_put(buf);
@@ -151,7 +152,8 @@ static int ethosu_buffer_mmap(struct file *file,
struct ethosu_buffer *buf = file->private_data;
int ret;
- dev_info(buf->edev->dev, "Buffer mmap. handle=0x%pK\n", buf);
+ dev_dbg(buf->edev->dev, "Buffer mmap. file=0x%pK, buf=0x%pK\n",
+ file, buf);
ret = dma_mmap_coherent(buf->edev->dev, vma, buf->cpu_addr,
buf->dma_addr_orig,
@@ -172,7 +174,9 @@ static long ethosu_buffer_ioctl(struct file *file,
if (ret)
return ret;
- dev_info(buf->edev->dev, "Ioctl. cmd=%u, arg=%lu\n", cmd, arg);
+ dev_dbg(buf->edev->dev,
+ "Buffer ioctl. file=0x%pK, buf=0x%pK, cmd=0x%x, arg=%lu\n",
+ file, buf, cmd, arg);
switch (cmd) {
case ETHOSU_IOCTL_BUFFER_SET: {
@@ -181,9 +185,9 @@ static long ethosu_buffer_ioctl(struct file *file,
if (copy_from_user(&uapi, udata, sizeof(uapi)))
break;
- dev_info(buf->edev->dev,
- "Ioctl: Buffer set. size=%u, offset=%u\n",
- uapi.size, uapi.offset);
+ dev_dbg(buf->edev->dev,
+ "Buffer ioctl: Buffer set. size=%u, offset=%u\n",
+ uapi.size, uapi.offset);
ret = ethosu_buffer_resize(buf, uapi.size, uapi.offset);
break;
@@ -194,9 +198,9 @@ static long ethosu_buffer_ioctl(struct file *file,
uapi.size = buf->size;
uapi.offset = buf->offset;
- dev_info(buf->edev->dev,
- "Ioctl: Buffer get. size=%u, offset=%u\n",
- uapi.size, uapi.offset);
+ dev_dbg(buf->edev->dev,
+ "Buffer ioctl: Buffer get. size=%u, offset=%u\n",
+ uapi.size, uapi.offset);
if (copy_to_user(udata, &uapi, sizeof(uapi)))
break;
@@ -252,10 +256,10 @@ int ethosu_buffer_create(struct ethosu_device *edev,
buf->file = fget(ret);
fput(buf->file);
- dev_info(buf->edev->dev,
- "Buffer create. handle=0x%pK, capacity=%zu, cpu_addr=0x%pK, dma_addr=0x%llx, dma_addr_orig=0x%llx, phys_addr=0x%llx\n",
- buf, capacity, buf->cpu_addr, buf->dma_addr,
- buf->dma_addr_orig, virt_to_phys(buf->cpu_addr));
+ dev_dbg(buf->edev->dev,
+ "Buffer create. file=0x%pK, fd=%d, buf=0x%pK, capacity=%zu, cpu_addr=0x%pK, dma_addr=0x%llx, dma_addr_orig=0x%llx, phys_addr=0x%llx\n",
+ buf->file, ret, buf, capacity, buf->cpu_addr, buf->dma_addr,
+ buf->dma_addr_orig, virt_to_phys(buf->cpu_addr));
return ret;
diff --git a/drivers/firmware/ethosu/ethosu_buffer.h b/drivers/firmware/ethosu/ethosu_buffer.h
index 14f26c2b0a9d..7a21fcb36b2d 100644
--- a/drivers/firmware/ethosu/ethosu_buffer.h
+++ b/drivers/firmware/ethosu/ethosu_buffer.h
@@ -1,5 +1,5 @@
/*
- * (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
+ * Copyright (c) 2020,2022 ARM Limited.
*
* This program is free software and is provided to you under the terms of the
* GNU General Public License version 2 as published by the Free Software
diff --git a/drivers/firmware/ethosu/ethosu_capabilities.c b/drivers/firmware/ethosu/ethosu_capabilities.c
new file mode 100644
index 000000000000..5d7818feafe1
--- /dev/null
+++ b/drivers/firmware/ethosu/ethosu_capabilities.c
@@ -0,0 +1,157 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Copyright (c) 2022 Arm Limited.
+ */
+
+/****************************************************************************
+ * Includes
+ ****************************************************************************/
+
+#include "ethosu_capabilities.h"
+
+#include "ethosu_device.h"
+
+#include <linux/delay.h>
+#include <linux/dma-mapping.h>
+#include <linux/errno.h>
+
+/****************************************************************************
+ * Defines
+ ****************************************************************************/
+
+#define CAPABILITIES_RESP_TIMEOUT_MS 2000
+
+/****************************************************************************
+ * Functions
+ ****************************************************************************/
+
+static inline int ethosu_capabilities_send(struct ethosu_capabilities *cap)
+{
+ return ethosu_rpmsg_capabilities_request(&cap->edev->erp,
+ &cap->msg);
+}
+
+static void ethosu_capabilities_fail(struct ethosu_rpmsg_msg *msg)
+{
+ struct ethosu_capabilities *cap =
+ container_of(msg, typeof(*cap), msg);
+
+ if (completion_done(&cap->done))
+ return;
+
+ cap->errno = -EFAULT;
+ complete(&cap->done);
+}
+
+static int ethosu_capabilities_resend(struct ethosu_rpmsg_msg *msg)
+{
+ struct ethosu_capabilities *cap =
+ container_of(msg, typeof(*cap), msg);
+
+ /* Don't resend request if response has already been received */
+ if (completion_done(&cap->done))
+ return 0;
+
+ /* Resend request */
+ return ethosu_capabilities_send(cap);
+}
+
+void ethosu_capability_rsp(struct ethosu_device *edev,
+ struct ethosu_core_msg_capabilities_rsp *rsp)
+{
+ int id = (int)rsp->user_arg;
+ struct ethosu_rpmsg_msg *msg;
+ struct ethosu_capabilities *cap;
+
+ msg = ethosu_rpmsg_find(&edev->erp, id);
+ if (IS_ERR(msg)) {
+ dev_warn(edev->dev,
+ "Id for capabilities msg not found. id=%d\n",
+ id);
+
+ return;
+ }
+
+ cap = container_of(msg, typeof(*cap), msg);
+
+ if (completion_done(&cap->done))
+ return;
+
+ cap->uapi->hw_id.version_status = rsp->version_status;
+ cap->uapi->hw_id.version_minor = rsp->version_minor;
+ cap->uapi->hw_id.version_major = rsp->version_major;
+ cap->uapi->hw_id.product_major = rsp->product_major;
+ cap->uapi->hw_id.arch_patch_rev = rsp->arch_patch_rev;
+ cap->uapi->hw_id.arch_minor_rev = rsp->arch_minor_rev;
+ cap->uapi->hw_id.arch_major_rev = rsp->arch_major_rev;
+ cap->uapi->driver_patch_rev = rsp->driver_patch_rev;
+ cap->uapi->driver_minor_rev = rsp->driver_minor_rev;
+ cap->uapi->driver_major_rev = rsp->driver_major_rev;
+ cap->uapi->hw_cfg.macs_per_cc = rsp->macs_per_cc;
+ cap->uapi->hw_cfg.cmd_stream_version = rsp->cmd_stream_version;
+ cap->uapi->hw_cfg.custom_dma = rsp->custom_dma;
+
+ cap->errno = 0;
+ complete(&cap->done);
+}
+
+int ethosu_capabilities_request(struct ethosu_device *edev,
+ struct ethosu_uapi_device_capabilities *uapi)
+{
+ struct ethosu_capabilities *cap;
+ int ret;
+ int timeout;
+
+ cap = devm_kzalloc(edev->dev, sizeof(struct ethosu_capabilities),
+ GFP_KERNEL);
+ if (!cap)
+ return -ENOMEM;
+
+ cap->edev = edev;
+ cap->uapi = uapi;
+ init_completion(&cap->done);
+ cap->msg.fail = ethosu_capabilities_fail;
+ cap->msg.resend = ethosu_capabilities_resend;
+
+ ret = ethosu_rpmsg_register(&cap->edev->erp, &cap->msg);
+ if (ret < 0)
+ goto kfree;
+
+ dev_dbg(edev->dev, "Capabilities create. Id=%d, handle=0x%p\n",
+ cap->msg.id, cap);
+
+ ret = ethosu_capabilities_send(cap);
+ if (ret != 0)
+ goto deregister;
+
+ /* Unlock the mutex before going to block on the condition */
+ mutex_unlock(&edev->mutex);
+
+ /* wait for response to arrive back */
+ timeout = wait_for_completion_timeout(&cap->done,
+ msecs_to_jiffies(CAPABILITIES_RESP_TIMEOUT_MS));
+
+ /* take back the mutex before resuming to do anything */
+ mutex_lock(&edev->mutex);
+
+ if (timeout == 0) {
+ dev_warn(edev->dev, "Capabilities response timeout");
+ ret = -ETIME;
+ goto deregister;
+ }
+
+ if (cap->errno) {
+ ret = cap->errno;
+ goto deregister;
+ }
+
+deregister:
+ ethosu_rpmsg_deregister(&cap->edev->erp, &cap->msg);
+
+kfree:
+ dev_dbg(cap->edev->dev, "Capabilities destroy. Id=%d, handle=0x%p\n",
+ cap->msg.id, cap);
+ devm_kfree(cap->edev->dev, cap);
+
+ return ret;
+}
diff --git a/drivers/firmware/ethosu/ethosu_capabilities.h b/drivers/firmware/ethosu/ethosu_capabilities.h
new file mode 100644
index 000000000000..87ddac3f635c
--- /dev/null
+++ b/drivers/firmware/ethosu/ethosu_capabilities.h
@@ -0,0 +1,47 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Copyright (c) 2022 Arm Limited.
+ */
+
+#ifndef ETHOSU_CAPABILITIES_H
+#define ETHOSU_CAPABILITIES_H
+
+/****************************************************************************
+ * Includes
+ ****************************************************************************/
+
+#include "ethosu_core_interface.h"
+#include "ethosu_rpmsg.h"
+
+#include <linux/types.h>
+#include <linux/completion.h>
+
+/****************************************************************************
+ * Types
+ ****************************************************************************/
+
+struct ethosu_device;
+struct ethosu_uapi_device_capabilities;
+
+/**
+ * struct ethosu_capabilities - Capabilities internal struct
+ */
+struct ethosu_capabilities {
+ struct ethosu_device *edev;
+ struct completion done;
+ struct ethosu_uapi_device_capabilities *uapi;
+ struct ethosu_rpmsg_msg msg;
+ int errno;
+};
+
+/****************************************************************************
+ * Functions
+ ****************************************************************************/
+
+int ethosu_capabilities_request(struct ethosu_device *edev,
+ struct ethosu_uapi_device_capabilities *uapi);
+
+void ethosu_capability_rsp(struct ethosu_device *edev,
+ struct ethosu_core_msg_capabilities_rsp *rsp);
+
+#endif
diff --git a/drivers/firmware/ethosu/ethosu_core_interface.h b/drivers/firmware/ethosu/ethosu_core_interface.h
index 9267e96aec9b..11b695a5219d 100644
--- a/drivers/firmware/ethosu/ethosu_core_interface.h
+++ b/drivers/firmware/ethosu/ethosu_core_interface.h
@@ -1,5 +1,5 @@
/*
- * (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
+ * Copyright (c) 2020-2022 Arm Limited.
*
* This program is free software and is provided to you under the terms of the
* GNU General Public License version 2 as published by the Free Software
@@ -93,11 +93,24 @@ struct ethosu_core_queue {
uint8_t data[];
};
+/**
+ * enum ethosu_core_status - Status
+ */
enum ethosu_core_status {
ETHOSU_CORE_STATUS_OK,
- ETHOSU_CORE_STATUS_ERROR
+ ETHOSU_CORE_STATUS_ERROR,
+ ETHOSU_CORE_STATUS_RUNNING,
+ ETHOSU_CORE_STATUS_REJECTED,
+ ETHOSU_CORE_STATUS_ABORTED,
+ ETHOSU_CORE_STATUS_ABORTING,
};
+/**
+ * struct ethosu_core_buffer - Buffer descriptor
+ *
+ * Pointer and size to a buffer within the Ethos-U
+ * address space.
+ */
struct ethosu_core_buffer {
uint32_t ptr;
uint32_t size;
@@ -115,6 +128,9 @@ struct ethosu_core_inference_req {
uint32_t inference_type;
};
+/**
+ * struct ethosu_core_inference_rsp - Inference response
+ */
struct ethosu_core_inference_rsp {
uint64_t user_arg;
uint32_t ofm_count;
diff --git a/drivers/firmware/ethosu/ethosu_device.c b/drivers/firmware/ethosu/ethosu_device.c
index 1627f25c8a95..6643a7aaad5b 100644
--- a/drivers/firmware/ethosu/ethosu_device.c
+++ b/drivers/firmware/ethosu/ethosu_device.c
@@ -1,5 +1,5 @@
/*
- * (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
+ * Copyright (c) 2020-2022 Arm Limited.
* Copyright 2020-2022 NXP
*
* This program is free software and is provided to you under the terms of the
@@ -27,11 +27,11 @@
#include "ethosu_buffer.h"
#include "ethosu_core_interface.h"
+#include "ethosu_capabilities.h"
#include "ethosu_inference.h"
#include "ethosu_network.h"
#include "uapi/ethosu.h"
-#include <linux/delay.h>
#include <linux/dma-mapping.h>
#include <linux/errno.h>
#include <linux/fs.h>
@@ -48,8 +48,6 @@
#define DMA_ADDR_BITS 32 /* Number of address bits */
-#define CAPABILITIES_RESP_TIMEOUT_MS 2000
-
#define ETHOSU_FIRMWARE_NAME "ethosu_firmware"
/****************************************************************************
@@ -60,70 +58,6 @@
* Functions
****************************************************************************/
-static void ethosu_capabilities_destroy(struct kref *kref)
-{
- struct ethosu_capabilities *cap =
- container_of(kref, struct ethosu_capabilities, refcount);
-
- list_del(&cap->list);
-
- devm_kfree(cap->edev->dev, cap);
-}
-
-static int ethosu_capabilities_find(struct ethosu_capabilities *cap,
- struct list_head *capabilties_list)
-{
- struct ethosu_capabilities *cur;
-
- list_for_each_entry(cur, capabilties_list, list) {
- if (cur == cap)
- return 0;
- }
-
- return -EINVAL;
-}
-
-static int ethosu_capability_rsp(struct ethosu_device *edev,
- struct ethosu_core_msg_capabilities_rsp *msg)
-{
- struct ethosu_capabilities *cap;
- struct ethosu_uapi_device_capabilities *capabilities;
- int ret;
-
- cap = (struct ethosu_capabilities *)msg->user_arg;
- ret = ethosu_capabilities_find(cap, &edev->capabilities_list);
- if (0 != ret) {
- dev_warn(edev->dev,
- "Handle not found in capabilities list. handle=0x%p\n",
- cap);
-
- /* NOTE: do not call complete or kref_put on invalid data! */
- return ret;
- }
-
- capabilities = cap->capabilities;
-
- capabilities->hw_id.version_status = msg->version_status;
- capabilities->hw_id.version_minor = msg->version_minor;
- capabilities->hw_id.version_major = msg->version_major;
- capabilities->hw_id.product_major = msg->product_major;
- capabilities->hw_id.arch_patch_rev = msg->arch_patch_rev;
- capabilities->hw_id.arch_minor_rev = msg->arch_minor_rev;
- capabilities->hw_id.arch_major_rev = msg->arch_major_rev;
- capabilities->driver_patch_rev = msg->driver_patch_rev;
- capabilities->driver_minor_rev = msg->driver_minor_rev;
- capabilities->driver_major_rev = msg->driver_major_rev;
- capabilities->hw_cfg.macs_per_cc = msg->macs_per_cc;
- capabilities->hw_cfg.cmd_stream_version = msg->cmd_stream_version;
- capabilities->hw_cfg.custom_dma = msg->custom_dma;
-
- complete(&cap->done);
-
- kref_put(&cap->refcount, ethosu_capabilities_destroy);
-
- return 0;
-}
-
/* Incoming messages */
static int ethosu_handle_msg(struct ethosu_device *edev, void *data)
{
@@ -158,14 +92,14 @@ static int ethosu_handle_msg(struct ethosu_device *edev, void *data)
ret = -EBADMSG;
break;
case ETHOSU_CORE_MSG_PING:
- dev_info(edev->dev, "Msg: Ping\n");
+ dev_dbg(edev->dev, "Msg: Ping\n");
ret = ethosu_rpmsg_pong(&edev->erp);
break;
case ETHOSU_CORE_MSG_PONG:
- dev_info(edev->dev, "Msg: Pong\n");
+ dev_dbg(edev->dev, "Msg: Pong\n");
break;
case ETHOSU_CORE_MSG_POWER_RSP:
- dev_info(edev->dev, "Msg: Power response\n");
+ dev_dbg(edev->dev, "Msg: Power response\n");
break;
case ETHOSU_CORE_MSG_INFERENCE_RSP:
if (header->length != sizeof(struct ethosu_core_inference_rsp)) {
@@ -176,10 +110,10 @@ static int ethosu_handle_msg(struct ethosu_device *edev, void *data)
break;
}
- dev_info(edev->dev,
- "Msg: Inference response. user_arg=0x%llx, ofm_count=%u, status=%u\n",
- rsp->user_arg, rsp->ofm_count,
- rsp->status);
+ dev_dbg(edev->dev,
+ "Msg: Inference response. user_arg=0x%llx, ofm_count=%u, status=%u\n",
+ rsp->user_arg, rsp->ofm_count,
+ rsp->status);
ethosu_inference_rsp(edev, rsp);
break;
case ETHOSU_CORE_MSG_VERSION_RSP:
@@ -191,9 +125,9 @@ static int ethosu_handle_msg(struct ethosu_device *edev, void *data)
break;
}
- dev_info(edev->dev, "Msg: Version response v%u.%u.%u\n",
- version->major, version->minor,
- version->patch);
+ dev_dbg(edev->dev, "Msg: Version response v%u.%u.%u\n",
+ version->major, version->minor,
+ version->patch);
/* Check major and minor version match, else return error */
if (version->major != ETHOSU_CORE_MSG_VERSION_MAJOR ||
@@ -215,24 +149,24 @@ static int ethosu_handle_msg(struct ethosu_device *edev, void *data)
break;
}
- dev_info(edev->dev,
- "Msg: Capabilities response ua%llx vs%hhu v%hhu.%hhu p%hhu av%hhu.%hhu.%hhu dv%hhu.%hhu.%hhu mcc%hhu csv%hhu cd%hhu\n",
- capabilities->user_arg,
- capabilities->version_status,
- capabilities->version_major,
- capabilities->version_minor,
- capabilities->product_major,
- capabilities->arch_major_rev,
- capabilities->arch_minor_rev,
- capabilities->arch_patch_rev,
- capabilities->driver_major_rev,
- capabilities->driver_minor_rev,
- capabilities->driver_patch_rev,
- capabilities->macs_per_cc,
- capabilities->cmd_stream_version,
- capabilities->custom_dma);
-
- ret = ethosu_capability_rsp(edev, capabilities);
+ dev_dbg(edev->dev,
+ "Msg: Capabilities response ua%llx vs%hhu v%hhu.%hhu p%hhu av%hhu.%hhu.%hhu dv%hhu.%hhu.%hhu mcc%hhu csv%hhu cd%hhu\n",
+ capabilities->user_arg,
+ capabilities->version_status,
+ capabilities->version_major,
+ capabilities->version_minor,
+ capabilities->product_major,
+ capabilities->arch_major_rev,
+ capabilities->arch_minor_rev,
+ capabilities->arch_patch_rev,
+ capabilities->driver_major_rev,
+ capabilities->driver_minor_rev,
+ capabilities->driver_patch_rev,
+ capabilities->macs_per_cc,
+ capabilities->cmd_stream_version,
+ capabilities->custom_dma);
+
+ ethosu_capability_rsp(edev, capabilities);
break;
default:
/* This should not happen due to version checks */
@@ -255,7 +189,7 @@ static int ethosu_open(struct inode *inode,
file->private_data = edev;
- dev_info(edev->dev, "Opening device node.\n");
+ dev_dbg(edev->dev, "Device open. file=0x%pK\n", file);
if (of_property_read_u32(edev->dev->of_node, "fsl,cm33-proc",
&rproc_phandle)) {
@@ -287,64 +221,6 @@ static int ethosu_open(struct inode *inode,
return nonseekable_open(inode, file);
}
-static int ethosu_send_capabilities_request(struct ethosu_device *edev,
- void __user *udata)
-{
- struct ethosu_uapi_device_capabilities uapi;
- struct ethosu_capabilities *cap;
- int ret;
- int timeout;
-
- cap = devm_kzalloc(edev->dev, sizeof(struct ethosu_capabilities),
- GFP_KERNEL);
- if (!cap)
- return -ENOMEM;
-
- cap->edev = edev;
- cap->capabilities = &uapi;
- kref_init(&cap->refcount);
- init_completion(&cap->done);
- list_add(&cap->list, &edev->capabilities_list);
-
- ret = ethosu_rpmsg_capabilities_request(&edev->erp, cap);
- if (0 != ret)
- goto put_kref;
-
- /*
- * Increase ref counter since we sent the pointer out to
- * response handler thread. That thread is responsible to
- * decrease the ref counter before exiting. So the memory
- * can be freed.
- *
- * NOTE: if no response is received back, the memory is leaked.
- */
- kref_get(&cap->refcount);
- /* Unlock the mutex before going to block on the condition */
- mutex_unlock(&edev->mutex);
- /* wait for response to arrive back */
- timeout = wait_for_completion_timeout(&cap->done,
- msecs_to_jiffies(
- CAPABILITIES_RESP_TIMEOUT_MS));
- /* take back the mutex before resuming to do anything */
- ret = mutex_lock_interruptible(&edev->mutex);
- if (0 != ret)
- goto put_kref;
-
- if (0 == timeout /* timed out*/) {
- dev_warn(edev->dev,
- "Msg: Capabilities response lost - timeout\n");
- ret = -EIO;
- goto put_kref;
- }
-
- ret = copy_to_user(udata, &uapi, sizeof(uapi)) ? -EFAULT : 0;
-
-put_kref:
- kref_put(&cap->refcount, ethosu_capabilities_destroy);
-
- return ret;
-}
-
static long ethosu_ioctl(struct file *file,
unsigned int cmd,
unsigned long arg)
@@ -357,32 +233,41 @@ static long ethosu_ioctl(struct file *file,
if (ret)
return ret;
- dev_info(edev->dev, "Ioctl. cmd=%u, arg=%lu\n", cmd, arg);
+ dev_dbg(edev->dev, "Device ioctl. file=0x%pK, cmd=0x%x, arg=0x%lx\n",
+ file, cmd, arg);
switch (cmd) {
case ETHOSU_IOCTL_VERSION_REQ:
- dev_info(edev->dev, "Ioctl: Send version request\n");
+ dev_dbg(edev->dev, "Device ioctl: Send version request\n");
ret = ethosu_rpmsg_version_request(&edev->erp);
break;
- case ETHOSU_IOCTL_CAPABILITIES_REQ:
- dev_info(edev->dev, "Ioctl: Send capabilities request\n");
- ret = ethosu_send_capabilities_request(edev, udata);
+ case ETHOSU_IOCTL_CAPABILITIES_REQ: {
+ struct ethosu_uapi_device_capabilities uapi;
+
+ dev_dbg(edev->dev,
+ "Device ioctl: Send capabilities request\n");
+
+ ret = ethosu_capabilities_request(edev, &uapi);
+ if (ret)
+ break;
+
+ ret = copy_to_user(udata, &uapi, sizeof(uapi)) ? -EFAULT : 0;
break;
+ }
case ETHOSU_IOCTL_PING: {
- dev_info(edev->dev, "Ioctl: Send ping\n");
+ dev_dbg(edev->dev, "Device ioctl: Send ping\n");
ret = ethosu_rpmsg_ping(&edev->erp);
break;
}
case ETHOSU_IOCTL_BUFFER_CREATE: {
struct ethosu_uapi_buffer_create uapi;
- dev_info(edev->dev, "Ioctl: Buffer create\n");
-
if (copy_from_user(&uapi, udata, sizeof(uapi)))
break;
- dev_info(edev->dev, "Ioctl: Buffer. capacity=%u\n",
- uapi.capacity);
+ dev_dbg(edev->dev,
+ "Device ioctl: Buffer create. capacity=%u\n",
+ uapi.capacity);
ret = ethosu_buffer_create(edev, uapi.capacity);
break;
@@ -393,7 +278,9 @@ static long ethosu_ioctl(struct file *file,
if (copy_from_user(&uapi, udata, sizeof(uapi)))
break;
- dev_info(edev->dev, "Ioctl: Network. fd=%u\n", uapi.fd);
+ dev_dbg(edev->dev,
+ "Device ioctl: Network create. fd=%u\n",
+ uapi.fd);
ret = ethosu_network_create(edev, &uapi);
break;
@@ -444,8 +331,6 @@ int ethosu_dev_init(struct ethosu_device *edev,
edev->class = class;
edev->devt = devt;
mutex_init(&edev->mutex);
- INIT_LIST_HEAD(&edev->capabilities_list);
- INIT_LIST_HEAD(&edev->inference_list);
ret = of_reserved_mem_device_init(edev->dev);
if (ret)
@@ -474,9 +359,9 @@ int ethosu_dev_init(struct ethosu_device *edev,
goto del_cdev;
}
- dev_info(edev->dev,
- "Created Arm Ethos-U device. name=%s, major=%d, minor=%d\n",
- dev_name(sysdev), MAJOR(edev->devt), MINOR(edev->devt));
+ dev_dbg(edev->dev,
+ "Created Arm Ethos-U device. name=%s, major=%d, minor=%d\n",
+ dev_name(sysdev), MAJOR(edev->devt), MINOR(edev->devt));
return 0;
@@ -498,6 +383,4 @@ void ethosu_dev_deinit(struct ethosu_device *edev)
device_destroy(edev->class, edev->cdev.dev);
cdev_del(&edev->cdev);
of_reserved_mem_device_release(edev->dev);
-
- dev_info(edev->dev, "%s\n", __FUNCTION__);
}
diff --git a/drivers/firmware/ethosu/ethosu_device.h b/drivers/firmware/ethosu/ethosu_device.h
index da470241b6a1..d027d8d9b11d 100644
--- a/drivers/firmware/ethosu/ethosu_device.h
+++ b/drivers/firmware/ethosu/ethosu_device.h
@@ -1,5 +1,5 @@
/*
- * (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
+ * Copyright (c) 2020-2022 Arm Limited.
*
* This program is free software and is provided to you under the terms of the
* GNU General Public License version 2 as published by the Free Software
@@ -32,8 +32,6 @@
#include <linux/cdev.h>
#include <linux/io.h>
#include <linux/mutex.h>
-#include <linux/workqueue.h>
-#include <linux/completion.h>
/****************************************************************************
* Types
@@ -49,22 +47,9 @@ struct ethosu_device {
dev_t devt;
struct mutex mutex;
struct ethosu_rpmsg erp;
- struct list_head capabilities_list;
- struct list_head inference_list;
bool open;
};
-/**
- * struct ethosu_capabilities - Capabilities internal struct
- */
-struct ethosu_capabilities {
- struct ethosu_device *edev;
- struct completion done;
- struct kref refcount;
- struct ethosu_uapi_device_capabilities *capabilities;
- struct list_head list;
-};
-
/****************************************************************************
* Functions
****************************************************************************/
diff --git a/drivers/firmware/ethosu/ethosu_driver.c b/drivers/firmware/ethosu/ethosu_driver.c
index d3306ef625ae..59ae79256564 100644
--- a/drivers/firmware/ethosu/ethosu_driver.c
+++ b/drivers/firmware/ethosu/ethosu_driver.c
@@ -1,5 +1,5 @@
/*
- * (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
+ * Copyright (c) 2020-2022 Arm Limited.
* Copyright 2020-2022 NXP
*
* This program is free software and is provided to you under the terms of the
@@ -60,7 +60,7 @@ static int ethosu_pdev_probe(struct platform_device *pdev)
int minor;
int ret;
- dev_info(&pdev->dev, "Probe\n");
+ dev_dbg(&pdev->dev, "Probe\n");
minor = find_first_zero_bit(minors, MINOR_COUNT);
if (minor >= MINOR_COUNT) {
diff --git a/drivers/firmware/ethosu/ethosu_inference.c b/drivers/firmware/ethosu/ethosu_inference.c
index b5e94e19bce0..947988507792 100644
--- a/drivers/firmware/ethosu/ethosu_inference.c
+++ b/drivers/firmware/ethosu/ethosu_inference.c
@@ -1,5 +1,5 @@
/*
- * (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
+ * Copyright (c) 2020,2022 Arm Limited.
*
* This program is free software and is provided to you under the terms of the
* GNU General Public License version 2 as published by the Free Software
@@ -71,6 +71,18 @@ static const char *status_to_string(const enum ethosu_uapi_status status)
case ETHOSU_UAPI_STATUS_ERROR: {
return "Error";
}
+ case ETHOSU_UAPI_STATUS_RUNNING: {
+ return "Running";
+ }
+ case ETHOSU_UAPI_STATUS_REJECTED: {
+ return "Rejected";
+ }
+ case ETHOSU_UAPI_STATUS_ABORTED: {
+ return "Aborted";
+ }
+ case ETHOSU_UAPI_STATUS_ABORTING: {
+ return "Aborting";
+ }
default: {
return "Unknown";
}
@@ -81,40 +93,82 @@ static int ethosu_inference_send(struct ethosu_inference *inf)
{
int ret;
- if (inf->pending)
- return -EINVAL;
-
inf->status = ETHOSU_UAPI_STATUS_ERROR;
- ret = ethosu_rpmsg_inference(&inf->edev->erp, inf,
- inf->ifm_count, inf->ifm,
- inf->ofm_count, inf->ofm,
- inf->net->buf,
- inf->pmu_event_config,
- ETHOSU_PMU_EVENT_MAX,
- inf->pmu_cycle_counter_enable,
- inf->inference_type);
- if (ret)
+ ret = ethosu_rpmsg_inference(&inf->edev->erp, &inf->msg,
+ inf->ifm_count, inf->ifm,
+ inf->ofm_count, inf->ofm,
+ inf->net->buf,
+ inf->pmu_event_config,
+ ETHOSU_PMU_EVENT_MAX,
+ inf->pmu_cycle_counter_enable,
+ inf->inference_type);
+ if (ret) {
+ dev_warn(inf->edev->dev,
+ "Failed to send inference request. inf=0x%pK, ret=%d",
+ inf, ret);
+
return ret;
+ }
- inf->pending = true;
+ inf->status = ETHOSU_UAPI_STATUS_RUNNING;
ethosu_inference_get(inf);
return 0;
}
-static int ethosu_inference_find(struct ethosu_inference *inf,
- struct list_head *inference_list)
+static void ethosu_inference_fail(struct ethosu_rpmsg_msg *msg)
+{
+ struct ethosu_inference *inf =
+ container_of(msg, typeof(*inf), msg);
+ int ret;
+
+ if (inf->done)
+ return;
+
+ /* Decrement reference count if inference was pending response */
+ ret = ethosu_inference_put(inf);
+ if (ret)
+ return;
+
+ /* Set status accordingly to the inference state */
+ inf->status = inf->status == ETHOSU_UAPI_STATUS_ABORTING ?
+ ETHOSU_UAPI_STATUS_ABORTED :
+ ETHOSU_UAPI_STATUS_ERROR;
+ /* Mark it done and wake up the waiting process */
+ inf->done = true;
+ wake_up_interruptible(&inf->waitq);
+}
+
+static int ethosu_inference_resend(struct ethosu_rpmsg_msg *msg)
{
- struct ethosu_inference *cur;
+ struct ethosu_inference *inf =
+ container_of(msg, typeof(*inf), msg);
+ int ret;
- list_for_each_entry(cur, inference_list, list) {
- if (cur == inf)
- return 0;
+ /* Don't resend request if response has already been received */
+ if (inf->done)
+ return 0;
+
+ /* If marked as ABORTING simply fail it and return */
+ if (inf->status == ETHOSU_UAPI_STATUS_ABORTING) {
+ ethosu_inference_fail(msg);
+
+ return 0;
}
- return -EINVAL;
+ /* Decrement reference count for pending request */
+ ret = ethosu_inference_put(inf);
+ if (ret)
+ return 0;
+
+ /* Resend request */
+ ret = ethosu_inference_send(inf);
+ if (ret)
+ return ret;
+
+ return 0;
}
static bool ethosu_inference_verify(struct file *file)
@@ -127,11 +181,11 @@ static void ethosu_inference_kref_destroy(struct kref *kref)
struct ethosu_inference *inf =
container_of(kref, struct ethosu_inference, kref);
- dev_info(inf->edev->dev,
- "Inference destroy. handle=0x%pK, status=%d\n",
- inf, inf->status);
+ dev_dbg(inf->edev->dev,
+ "Inference destroy. inf=0x%pK, status=%d, ifm_count=%u, ofm_count=%u",
+ inf, inf->status, inf->ifm_count, inf->ofm_count);
- list_del(&inf->list);
+ ethosu_rpmsg_deregister(&inf->edev->erp, &inf->msg);
while (inf->ifm_count-- > 0)
ethosu_buffer_put(inf->ifm[inf->ifm_count]);
@@ -148,9 +202,9 @@ static int ethosu_inference_release(struct inode *inode,
{
struct ethosu_inference *inf = file->private_data;
- dev_info(inf->edev->dev,
- "Inference release. handle=0x%pK, status=%d\n",
- inf, inf->status);
+ dev_dbg(inf->edev->dev,
+ "Inference release. file=0x%pK, inf=0x%pK",
+ file, inf);
ethosu_inference_put(inf);
@@ -165,7 +219,7 @@ static unsigned int ethosu_inference_poll(struct file *file,
poll_wait(file, &inf->waitq, wait);
- if (!inf->pending)
+ if (inf->done)
ret |= POLLIN;
return ret;
@@ -183,7 +237,9 @@ static long ethosu_inference_ioctl(struct file *file,
if (ret)
return ret;
- dev_info(inf->edev->dev, "Ioctl: cmd=%u, arg=%lu\n", cmd, arg);
+ dev_dbg(inf->edev->dev,
+ "Inference ioctl: file=0x%pK, inf=0x%pK, cmd=0x%x, arg=%lu",
+ file, inf, cmd, arg);
switch (cmd) {
case ETHOSU_IOCTL_INFERENCE_STATUS: {
@@ -202,16 +258,16 @@ static long ethosu_inference_ioctl(struct file *file,
uapi.pmu_config.cycle_count = inf->pmu_cycle_counter_enable;
uapi.pmu_count.cycle_count = inf->pmu_cycle_counter_count;
- dev_info(inf->edev->dev,
- "Ioctl: Inference status. status=%s (%d)\n",
- status_to_string(uapi.status), uapi.status);
+ dev_dbg(inf->edev->dev,
+ "Inference ioctl: Inference status. status=%s (%d)\n",
+ status_to_string(uapi.status), uapi.status);
ret = copy_to_user(udata, &uapi, sizeof(uapi)) ? -EFAULT : 0;
break;
}
default: {
- dev_err(inf->edev->dev, "Invalid ioctl. cmd=%u, arg=%lu",
+ dev_err(inf->edev->dev, "Invalid ioctl. cmd=%u, arg=%lu\n",
cmd, arg);
break;
}
@@ -231,9 +287,19 @@ int ethosu_inference_create(struct ethosu_device *edev,
int fd;
int ret = -ENOMEM;
+ if (uapi->ifm_count > ETHOSU_FD_MAX ||
+ uapi->ofm_count > ETHOSU_FD_MAX) {
+ dev_warn(edev->dev,
+ "Too many IFM and/or OFM buffers for inference. ifm_count=%u, ofm_count=%u",
+ uapi->ifm_count, uapi->ofm_count);
+
+ return -EFAULT;
+ }
+
inf = devm_kzalloc(edev->dev, sizeof(*inf), GFP_KERNEL);
if (!inf)
return -ENOMEM;
+
switch (uapi->inference_type) {
case ETHOSU_UAPI_INFERENCE_MODEL:
inf->inference_type = ETHOSU_CORE_INFERENCE_MODEL;
@@ -248,10 +314,17 @@ int ethosu_inference_create(struct ethosu_device *edev,
inf->edev = edev;
inf->net = net;
- inf->pending = false;
+ inf->done = false;
inf->status = ETHOSU_UAPI_STATUS_ERROR;
kref_init(&inf->kref);
init_waitqueue_head(&inf->waitq);
+ inf->msg.fail = ethosu_inference_fail;
+ inf->msg.resend = ethosu_inference_resend;
+
+ /* Add inference to pending list */
+ ret = ethosu_rpmsg_register(&edev->erp, &inf->msg);
+ if (ret < 0)
+ goto kfree;
/* Get pointer to IFM buffers */
for (i = 0; i < uapi->ifm_count; i++) {
@@ -276,10 +349,10 @@ int ethosu_inference_create(struct ethosu_device *edev,
}
/* Configure PMU and cycle counter */
- dev_info(inf->edev->dev,
- "Configuring events for PMU. events=[%u, %u, %u, %u]\n",
- uapi->pmu_config.events[0], uapi->pmu_config.events[1],
- uapi->pmu_config.events[2], uapi->pmu_config.events[3]);
+ dev_dbg(inf->edev->dev,
+ "Configuring events for PMU. events=[%u, %u, %u, %u]\n",
+ uapi->pmu_config.events[0], uapi->pmu_config.events[1],
+ uapi->pmu_config.events[2], uapi->pmu_config.events[3]);
/* Configure events and reset count for all events */
for (i = 0; i < ETHOSU_PMU_EVENT_MAX; i++) {
@@ -288,7 +361,7 @@ int ethosu_inference_create(struct ethosu_device *edev,
}
if (uapi->pmu_config.cycle_count)
- dev_info(inf->edev->dev, "Enabling cycle counter\n");
+ dev_dbg(inf->edev->dev, "Enabling cycle counter\n");
/* Configure cycle counter and reset any previous count */
inf->pmu_cycle_counter_enable = uapi->pmu_config.cycle_count;
@@ -297,6 +370,11 @@ int ethosu_inference_create(struct ethosu_device *edev,
/* Increment network reference count */
ethosu_network_get(net);
+ /* Send inference request to Arm Ethos-U subsystem */
+ ret = ethosu_inference_send(inf);
+ if (ret)
+ goto put_net;
+
/* Create file descriptor */
ret = fd = anon_inode_getfd("ethosu-inference", ðosu_inference_fops,
inf, O_RDWR | O_CLOEXEC);
@@ -307,14 +385,9 @@ int ethosu_inference_create(struct ethosu_device *edev,
inf->file = fget(ret);
fput(inf->file);
- /* Add inference to inference list */
- list_add(&inf->list, &edev->inference_list);
-
- /* Send inference request to Arm Ethos-U subsystem */
- (void)ethosu_inference_send(inf);
-
- dev_info(edev->dev, "Inference create. handle=0x%pK, fd=%d",
- inf, fd);
+ dev_dbg(edev->dev,
+ "Inference create. file=0x%pK, fd=%d, inf=0x%p, net=0x%pK, msg.id=0x%x",
+ inf->file, fd, inf, inf->net, inf->msg.id);
return fd;
@@ -329,6 +402,7 @@ int ethosu_inference_create(struct ethosu_device *edev,
while (inf->ifm_count-- > 0)
ethosu_buffer_put(inf->ifm[inf->ifm_count]);
+kfree:
devm_kfree(edev->dev, inf);
return ret;
@@ -361,29 +435,30 @@ void ethosu_inference_get(struct ethosu_inference *inf)
kref_get(&inf->kref);
}
-void ethosu_inference_put(struct ethosu_inference *inf)
+int ethosu_inference_put(struct ethosu_inference *inf)
{
- kref_put(&inf->kref, ðosu_inference_kref_destroy);
+ return kref_put(&inf->kref, ðosu_inference_kref_destroy);
}
void ethosu_inference_rsp(struct ethosu_device *edev,
struct ethosu_core_inference_rsp *rsp)
{
- struct ethosu_inference *inf =
- (struct ethosu_inference *)rsp->user_arg;
+ int id = (int)rsp->user_arg;
+ struct ethosu_rpmsg_msg *msg;
+ struct ethosu_inference *inf;
int ret;
int i;
- ret = ethosu_inference_find(inf, &edev->inference_list);
- if (ret) {
+ msg = ethosu_rpmsg_find(&edev->erp, id);
+ if (IS_ERR(msg)) {
dev_warn(edev->dev,
- "Handle not found in inference list. handle=0x%p\n",
- rsp);
+ "Id for inference msg not found. Id=%d\n",
+ id);
return;
}
- inf->pending = false;
+ inf = container_of(msg, typeof(*inf), msg);
if (rsp->status == ETHOSU_CORE_STATUS_OK &&
inf->ofm_count <= ETHOSU_CORE_BUFFER_MAX) {
@@ -400,29 +475,37 @@ void ethosu_inference_rsp(struct ethosu_device *edev,
if (ret)
inf->status = ETHOSU_UAPI_STATUS_ERROR;
}
+ } else if (rsp->status == ETHOSU_CORE_STATUS_REJECTED) {
+ inf->status = ETHOSU_UAPI_STATUS_REJECTED;
+ } else if (rsp->status == ETHOSU_CORE_STATUS_ABORTED) {
+ inf->status = ETHOSU_UAPI_STATUS_ABORTED;
} else {
inf->status = ETHOSU_UAPI_STATUS_ERROR;
}
- for (i = 0; i < ETHOSU_CORE_PMU_MAX; i++) {
- inf->pmu_event_config[i] = rsp->pmu_event_config[i];
- inf->pmu_event_count[i] = rsp->pmu_event_count[i];
- }
+ if (inf->status == ETHOSU_UAPI_STATUS_OK) {
+ for (i = 0; i < ETHOSU_CORE_PMU_MAX; i++) {
+ inf->pmu_event_config[i] = rsp->pmu_event_config[i];
+ inf->pmu_event_count[i] = rsp->pmu_event_count[i];
+ }
+
+ inf->pmu_cycle_counter_enable = rsp->pmu_cycle_counter_enable;
+ inf->pmu_cycle_counter_count = rsp->pmu_cycle_counter_count;
- inf->pmu_cycle_counter_enable = rsp->pmu_cycle_counter_enable;
- inf->pmu_cycle_counter_count = rsp->pmu_cycle_counter_count;
+ dev_dbg(edev->dev,
+ "PMU events. config=[%u, %u, %u, %u], count=[%u, %u, %u, %u]\n",
+ inf->pmu_event_config[0], inf->pmu_event_config[1],
+ inf->pmu_event_config[2], inf->pmu_event_config[3],
+ inf->pmu_event_count[0], inf->pmu_event_count[1],
+ inf->pmu_event_count[2], inf->pmu_event_count[3]);
- dev_info(edev->dev,
- "PMU events. config=[%u, %u, %u, %u], count=[%u, %u, %u, %u]\n",
- inf->pmu_event_config[0], inf->pmu_event_config[1],
- inf->pmu_event_config[2], inf->pmu_event_config[3],
- inf->pmu_event_count[0], inf->pmu_event_count[1],
- inf->pmu_event_count[2], inf->pmu_event_count[3]);
+ dev_dbg(edev->dev,
+ "PMU cycle counter. enable=%u, count=%llu\n",
+ inf->pmu_cycle_counter_enable,
+ inf->pmu_cycle_counter_count);
+ }
- dev_info(edev->dev,
- "PMU cycle counter. enable=%u, count=%llu\n",
- inf->pmu_cycle_counter_enable,
- inf->pmu_cycle_counter_count);
+ inf->done = true;
wake_up_interruptible(&inf->waitq);
ethosu_inference_put(inf);
diff --git a/drivers/firmware/ethosu/ethosu_inference.h b/drivers/firmware/ethosu/ethosu_inference.h
index 8414eadbda92..311c727b2897 100644
--- a/drivers/firmware/ethosu/ethosu_inference.h
+++ b/drivers/firmware/ethosu/ethosu_inference.h
@@ -1,5 +1,5 @@
/*
- * (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
+ * Copyright (c) 2020,2022 ARM Limited.
*
* This program is free software and is provided to you under the terms of the
* GNU General Public License version 2 as published by the Free Software
@@ -25,6 +25,7 @@
* Includes
****************************************************************************/
+#include "ethosu_rpmsg.h"
#include "uapi/ethosu.h"
#include <linux/kref.h>
@@ -48,34 +49,35 @@ struct file;
* @file: File handle
* @kref: Reference counter
* @waitq: Wait queue
+ * @done: Wait condition is done
* @ifm: Pointer to IFM buffer
* @ofm: Pointer to OFM buffer
* @net: Pointer to network
- * @pending: Pending response from the firmware
* @status: Inference status
* @pmu_event_config: PMU event configuration
* @pmu_event_count: PMU event count after inference
* @pmu_cycle_counter_enable: PMU cycle counter config
* @pmu_cycle_counter_count: PMU cycle counter count after inference
+ * @msg: Rpmsg message
*/
struct ethosu_inference {
struct ethosu_device *edev;
struct file *file;
struct kref kref;
wait_queue_head_t waitq;
+ bool done;
uint32_t ifm_count;
struct ethosu_buffer *ifm[ETHOSU_FD_MAX];
uint32_t ofm_count;
struct ethosu_buffer *ofm[ETHOSU_FD_MAX];
struct ethosu_network *net;
- bool pending;
enum ethosu_uapi_status status;
uint8_t pmu_event_config[ETHOSU_PMU_EVENT_MAX];
uint32_t pmu_event_count[ETHOSU_PMU_EVENT_MAX];
uint32_t pmu_cycle_counter_enable;
uint64_t pmu_cycle_counter_count;
uint32_t inference_type;
- struct list_head list;
+ struct ethosu_rpmsg_msg msg;
};
/****************************************************************************
@@ -109,8 +111,10 @@ void ethosu_inference_get(struct ethosu_inference *inf);
/**
* ethosu_inference_put() - Put inference
+ *
+ * Return: 1 if object was removed, else 0.
*/
-void ethosu_inference_put(struct ethosu_inference *inf);
+int ethosu_inference_put(struct ethosu_inference *inf);
/**
* ethosu_inference_rsp() - Handle inference response
diff --git a/drivers/firmware/ethosu/ethosu_network.c b/drivers/firmware/ethosu/ethosu_network.c
index 4d68f0537e38..a7249619291b 100644
--- a/drivers/firmware/ethosu/ethosu_network.c
+++ b/drivers/firmware/ethosu/ethosu_network.c
@@ -1,5 +1,5 @@
/*
- * (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
+ * Copyright (c) 2020,2022 Arm Limited.
*
* This program is free software and is provided to you under the terms of the
* GNU General Public License version 2 as published by the Free Software
@@ -67,7 +67,7 @@ static void ethosu_network_destroy(struct kref *kref)
struct ethosu_network *net =
container_of(kref, struct ethosu_network, kref);
- dev_info(net->edev->dev, "Network destroy. handle=0x%pK\n", net);
+ dev_dbg(net->edev->dev, "Network destroy. net=0x%pK\n", net);
ethosu_buffer_put(net->buf);
devm_kfree(net->edev->dev, net);
@@ -78,7 +78,8 @@ static int ethosu_network_release(struct inode *inode,
{
struct ethosu_network *net = file->private_data;
- dev_info(net->edev->dev, "Network release. handle=0x%pK\n", net);
+ dev_dbg(net->edev->dev, "Network release. file=0x%pK, net=0x%pK\n",
+ file, net);
ethosu_network_put(net);
@@ -97,7 +98,9 @@ static long ethosu_network_ioctl(struct file *file,
if (ret)
return ret;
- dev_info(net->edev->dev, "Ioctl: cmd=%u, arg=%lu\n", cmd, arg);
+ dev_dbg(net->edev->dev,
+ "Network ioctl: file=0x%pK, net=0x%pK, cmd=0x%x, arg=0x%lx\n",
+ file, net, cmd, arg);
switch (cmd) {
case ETHOSU_IOCTL_INFERENCE_CREATE: {
@@ -106,9 +109,9 @@ static long ethosu_network_ioctl(struct file *file,
if (copy_from_user(&uapi, udata, sizeof(uapi)))
break;
- dev_info(net->edev->dev,
- "Ioctl: Inference. ifm_fd=%u, ofm_fd=%u\n",
- uapi.ifm_fd[0], uapi.ofm_fd[0]);
+ dev_dbg(net->edev->dev,
+ "Network ioctl: Inference. ifm_fd=%u, ofm_fd=%u\n",
+ uapi.ifm_fd[0], uapi.ofm_fd[0]);
ret = ethosu_inference_create(net->edev, net, &uapi);
break;
@@ -154,8 +157,9 @@ int ethosu_network_create(struct ethosu_device *edev,
net->file = fget(ret);
fput(net->file);
- dev_info(edev->dev, "Network create. handle=0x%pK",
- net);
+ dev_dbg(edev->dev,
+ "Network create. file=0x%pK, fd=%d, net=0x%pK, buf=0x%pK",
+ net->file, ret, net, net->buf);
return ret;
diff --git a/drivers/firmware/ethosu/ethosu_network.h b/drivers/firmware/ethosu/ethosu_network.h
index bb70afcb2572..4236e1586efb 100644
--- a/drivers/firmware/ethosu/ethosu_network.h
+++ b/drivers/firmware/ethosu/ethosu_network.h
@@ -1,5 +1,5 @@
/*
- * (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
+ * Copyright (c) 2020,2022 Arm Limited.
*
* This program is free software and is provided to you under the terms of the
* GNU General Public License version 2 as published by the Free Software
diff --git a/drivers/firmware/ethosu/ethosu_rpmsg.c b/drivers/firmware/ethosu/ethosu_rpmsg.c
index a0320b8407d1..a92439b81c96 100644
--- a/drivers/firmware/ethosu/ethosu_rpmsg.c
+++ b/drivers/firmware/ethosu/ethosu_rpmsg.c
@@ -32,6 +32,61 @@ static void ethosu_core_set_capacity(struct ethosu_buffer *buf,
cbuf->size = (uint32_t)buf->capacity - buf->offset - buf->size;
}
+int ethosu_rpmsg_register(struct ethosu_rpmsg *erp,
+ struct ethosu_rpmsg_msg *msg)
+{
+ msg->id = idr_alloc_cyclic(&erp->msg_idr, msg, 0, INT_MAX, GFP_KERNEL);
+ if (msg->id < 0)
+ return msg->id;
+
+ return 0;
+}
+
+void ethosu_rpmsg_deregister(struct ethosu_rpmsg *erp,
+ struct ethosu_rpmsg_msg *msg)
+{
+ idr_remove(&erp->msg_idr, msg->id);
+}
+
+struct ethosu_rpmsg_msg *ethosu_rpmsg_find(struct ethosu_rpmsg *erp,
+ int msg_id)
+{
+ struct ethosu_rpmsg_msg *ptr =
+ (struct ethosu_rpmsg_msg *)idr_find(&erp->msg_idr, msg_id);
+
+ if (!ptr)
+ return ERR_PTR(-EINVAL);
+
+ return ptr;
+}
+
+void ethosu_rpmsg_fail(struct ethosu_rpmsg *erp)
+{
+ struct ethosu_rpmsg_msg *cur;
+ int id;
+
+ idr_for_each_entry(&erp->msg_idr, cur, id) {
+ cur->fail(cur);
+ }
+}
+
+void ethosu_rpmsg_resend(struct ethosu_rpmsg *erp)
+{
+ struct ethosu_rpmsg_msg *cur;
+ struct rpmsg_device *rpdev = erp->rpdev;
+ int id;
+ int ret;
+
+ idr_for_each_entry(&erp->msg_idr, cur, id) {
+ ret = cur->resend(cur);
+ if (ret) {
+ dev_warn(&rpdev->dev, "Failed to resend msg. ret=%d",
+ ret);
+ cur->fail(cur);
+ }
+ }
+}
+
static int ethosu_rpmsg_send(struct ethosu_rpmsg *erp, uint32_t type)
{
struct ethosu_core_msg msg;
@@ -69,7 +124,8 @@ int ethosu_rpmsg_version_request(struct ethosu_rpmsg *erp)
return ethosu_rpmsg_send(erp, ETHOSU_CORE_MSG_VERSION_REQ);
}
-int ethosu_rpmsg_capabilities_request(struct ethosu_rpmsg *erp, void *user_arg)
+int ethosu_rpmsg_capabilities_request(struct ethosu_rpmsg *erp,
+ struct ethosu_rpmsg_msg *rpmsg)
{
struct ethosu_core_msg msg = {
.magic = ETHOSU_CORE_MSG_MAGIC,
@@ -77,7 +133,7 @@ int ethosu_rpmsg_capabilities_request(struct ethosu_rpmsg *erp, void *user_arg)
.length = sizeof(struct ethosu_core_capabilities_req)
};
struct ethosu_core_capabilities_req req = {
- .user_arg = (uint64_t)user_arg
+ .user_arg = rpmsg->id
};
struct rpmsg_device *rpdev = erp->rpdev;
uint8_t data[sizeof(struct ethosu_core_msg) +
@@ -128,7 +184,7 @@ int ethosu_rpmsg_power_request(struct ethosu_rpmsg *erp,
}
int ethosu_rpmsg_inference(struct ethosu_rpmsg *erp,
- void *user_arg,
+ struct ethosu_rpmsg_msg *rpmsg,
uint32_t ifm_count,
struct ethosu_buffer **ifm,
uint32_t ofm_count,
@@ -158,7 +214,7 @@ int ethosu_rpmsg_inference(struct ethosu_rpmsg *erp,
return -EINVAL;
}
- req.user_arg = (uint64_t)user_arg;
+ req.user_arg = rpmsg->id;
req.ifm_count = ifm_count;
req.ofm_count = ofm_count;
req.pmu_cycle_counter_enable = pmu_cycle_counter_enable;
@@ -215,8 +271,8 @@ static int ethosu_rpmsg_probe(struct rpmsg_device *rpdev)
grp->rpdev = rpdev;
dev_set_drvdata(&rpdev->dev, grp);
- dev_info(&rpdev->dev, "new channel: 0x%x -> 0x%x!\n",
- rpdev->src, rpdev->dst);
+ dev_dbg(&rpdev->dev, "new channel: 0x%x -> 0x%x!\n",
+ rpdev->src, rpdev->dst);
ret = rpmsg_send(rpdev->ept, MSG, strlen(MSG));
if (ret) {
@@ -229,7 +285,7 @@ static int ethosu_rpmsg_probe(struct rpmsg_device *rpdev)
static void ethosu_rpmsg_remove(struct rpmsg_device *rpdev)
{
- dev_info(&rpdev->dev, "rpmsg ethosu driver is removed\n");
+ dev_dbg(&rpdev->dev, "rpmsg ethosu driver is removed\n");
}
static struct rpmsg_device_id rpmsg_driver_ethosu_id_table[] = {
@@ -252,6 +308,8 @@ int ethosu_rpmsg_init(struct ethosu_rpmsg *erp,
grp = erp;
erp->callback = callback;
erp->user_arg = user_arg;
+ erp->ping_count = 0;
+ idr_init(&erp->msg_idr);
return register_rpmsg_driver(ðosu_rpmsg_driver);
}
diff --git a/drivers/firmware/ethosu/ethosu_rpmsg.h b/drivers/firmware/ethosu/ethosu_rpmsg.h
index cee3d96c7895..74ee2fdea5cb 100644
--- a/drivers/firmware/ethosu/ethosu_rpmsg.h
+++ b/drivers/firmware/ethosu/ethosu_rpmsg.h
@@ -6,8 +6,11 @@
#ifndef ETHOSU_RPMSG_H
#define ETHOSU_RPMSG_H
+#include <linux/idr.h>
#include <linux/types.h>
#include <linux/completion.h>
+#include <linux/workqueue.h>
+
#include "ethosu_core_interface.h"
struct device;
@@ -22,7 +25,16 @@ struct ethosu_rpmsg {
ethosu_rpmsg_cb callback;
void *user_arg;
struct completion rpmsg_ready;
+ struct idr msg_idr;
+ unsigned int ping_count;
+};
+
+struct ethosu_rpmsg_msg {
+ int id;
+ void (*fail)(struct ethosu_rpmsg_msg *msg);
+ int (*resend)(struct ethosu_rpmsg_msg *msg);
};
+
/**
* ethosu_rpmsg_ping() - Send ping message
*
@@ -50,7 +62,7 @@ int ethosu_rpmsg_version_request(struct ethosu_rpmsg *erp);
* Return: 0 on success, else error code
*/
int ethosu_rpmsg_capabilities_request(struct ethosu_rpmsg *erp,
- void *user_arg);
+ struct ethosu_rpmsg_msg *rpmsg);
/**
* ethosu_rpmsg_power_request - Send power request
@@ -66,7 +78,7 @@ int ethosu_rpmsg_power_request(struct ethosu_rpmsg *erp,
* Return: 0 on success, else error code.
*/
int ethosu_rpmsg_inference(struct ethosu_rpmsg *erp,
- void *user_arg,
+ struct ethosu_rpmsg_msg *rpmsg,
uint32_t ifm_count,
struct ethosu_buffer **ifm,
uint32_t ofm_count,
@@ -82,4 +94,41 @@ int ethosu_rpmsg_init(struct ethosu_rpmsg *erp,
ethosu_rpmsg_cb callback, void *user_arg);
int ethosu_rpmsg_deinit(struct ethosu_rpmsg *erp);
+
+/**
+ * ethosu_rpmsg_register() - Register the ethosu_rpmsg_msg in ethosu_rpmsg
+ *
+ * Return: 0 on success, else error code.
+ */
+int ethosu_rpmsg_register(struct ethosu_rpmsg *erp,
+ struct ethosu_rpmsg_msg *msg);
+
+/**
+ * ethosu_rpmsg_free_id() - Free the id of the ethosu_rpmsg_msg
+ */
+void ethosu_rpmsg_deregister(struct ethosu_rpmsg *erp,
+ struct ethosu_rpmsg_msg *msg);
+
+/**
+ * ethosu_rpmsg_find() - Find rpmsg message
+ *
+ * Return: a valid pointer on success, otherwise an error ptr.
+ */
+struct ethosu_rpmsg_msg *ethosu_rpmsg_find(struct ethosu_rpmsg *erq,
+ int msg_id);
+
+/**
+ * ethosu_rpmsg_fail() - Fail rpmsg messages
+ *
+ * Call fail() callback on all messages in pending list.
+ */
+void ethosu_rpmsg_fail(struct ethosu_rpmsg *erp);
+
+/**
+ * ethosu_rpmsg_resend() - Resend rpmsg messages
+ *
+ * Call resend() callback on all messages in pending list.
+ */
+void ethosu_rpmsg_resend(struct ethosu_rpmsg *erp);
+
#endif /* ETHOSU_RPMSG_H */
diff --git a/drivers/firmware/ethosu/uapi/ethosu.h b/drivers/firmware/ethosu/uapi/ethosu.h
index c4a0df67336c..648cec40473b 100644
--- a/drivers/firmware/ethosu/uapi/ethosu.h
+++ b/drivers/firmware/ethosu/uapi/ethosu.h
@@ -1,5 +1,5 @@
/*
- * (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
+ * Copyright (c) 2020-2022 Arm Limited.
*
* This program is free software and is provided to you under the terms of the
* GNU General Public License version 2 as published by the Free Software
@@ -73,7 +73,11 @@ namespace EthosU {
*/
enum ethosu_uapi_status {
ETHOSU_UAPI_STATUS_OK,
- ETHOSU_UAPI_STATUS_ERROR
+ ETHOSU_UAPI_STATUS_ERROR,
+ ETHOSU_UAPI_STATUS_RUNNING,
+ ETHOSU_UAPI_STATUS_REJECTED,
+ ETHOSU_UAPI_STATUS_ABORTED,
+ ETHOSU_UAPI_STATUS_ABORTING,
};
/**
--
2.17.1
This patch adds suspend and resume power management support. The
corresponding message will be sent to Cortex-M core via RPMsg protocol.
Signed-off-by: Alison Wang <[email protected]>
Signed-off-by: Feng Guo <[email protected]>
Reviewed-by: Peng Fan <[email protected]>
---
drivers/firmware/ethosu/ethosu_buffer.c | 3 ++
.../firmware/ethosu/ethosu_core_interface.h | 14 ++++++++
drivers/firmware/ethosu/ethosu_device.c | 36 +++++++++++++++++++
drivers/firmware/ethosu/ethosu_device.h | 10 ++++++
drivers/firmware/ethosu/ethosu_driver.c | 29 +++++++++++++++
drivers/firmware/ethosu/ethosu_rpmsg.c | 30 ++++++++++++++++
drivers/firmware/ethosu/ethosu_rpmsg.h | 11 ++++++
7 files changed, 133 insertions(+)
diff --git a/drivers/firmware/ethosu/ethosu_buffer.c b/drivers/firmware/ethosu/ethosu_buffer.c
index 43a433355f30..c8aa8031a8de 100644
--- a/drivers/firmware/ethosu/ethosu_buffer.c
+++ b/drivers/firmware/ethosu/ethosu_buffer.c
@@ -222,6 +222,9 @@ int ethosu_buffer_create(struct ethosu_device *edev,
struct ethosu_buffer *buf;
int ret = -ENOMEM;
+ if (!capacity)
+ return -EINVAL;
+
buf = devm_kzalloc(edev->dev, sizeof(*buf), GFP_KERNEL);
if (!buf)
return -ENOMEM;
diff --git a/drivers/firmware/ethosu/ethosu_core_interface.h b/drivers/firmware/ethosu/ethosu_core_interface.h
index b60508e4792f..9267e96aec9b 100644
--- a/drivers/firmware/ethosu/ethosu_core_interface.h
+++ b/drivers/firmware/ethosu/ethosu_core_interface.h
@@ -60,6 +60,8 @@ enum ethosu_core_msg_type {
ETHOSU_CORE_MSG_VERSION_RSP,
ETHOSU_CORE_MSG_CAPABILITIES_REQ,
ETHOSU_CORE_MSG_CAPABILITIES_RSP,
+ ETHOSU_CORE_MSG_POWER_REQ,
+ ETHOSU_CORE_MSG_POWER_RSP,
ETHOSU_CORE_MSG_MAX
};
@@ -161,6 +163,18 @@ struct ethosu_core_msg_capabilities_rsp {
uint32_t custom_dma;
};
+enum ethosu_core_power_req_type {
+ ETHOSU_CORE_POWER_REQ_SUSPEND = 0,
+ ETHOSU_CORE_POWER_REQ_RESUME
+};
+
+/**
+ * struct ethosu_core_power_req - Message power request
+ */
+struct ethosu_core_power_req {
+ enum ethosu_core_power_req_type type;
+};
+
/**
* enum ethosu_core_msg_err_type - Error types
*/
diff --git a/drivers/firmware/ethosu/ethosu_device.c b/drivers/firmware/ethosu/ethosu_device.c
index b2d4737080ac..1627f25c8a95 100644
--- a/drivers/firmware/ethosu/ethosu_device.c
+++ b/drivers/firmware/ethosu/ethosu_device.c
@@ -38,6 +38,7 @@
#include <linux/interrupt.h>
#include <linux/io.h>
#include <linux/of_reserved_mem.h>
+#include <linux/remoteproc.h>
#include <linux/uaccess.h>
#include <linux/slab.h>
@@ -49,6 +50,8 @@
#define CAPABILITIES_RESP_TIMEOUT_MS 2000
+#define ETHOSU_FIRMWARE_NAME "ethosu_firmware"
+
/****************************************************************************
* Types
****************************************************************************/
@@ -161,6 +164,9 @@ static int ethosu_handle_msg(struct ethosu_device *edev, void *data)
case ETHOSU_CORE_MSG_PONG:
dev_info(edev->dev, "Msg: Pong\n");
break;
+ case ETHOSU_CORE_MSG_POWER_RSP:
+ dev_info(edev->dev, "Msg: Power response\n");
+ break;
case ETHOSU_CORE_MSG_INFERENCE_RSP:
if (header->length != sizeof(struct ethosu_core_inference_rsp)) {
dev_warn(edev->dev,
@@ -243,11 +249,41 @@ static int ethosu_open(struct inode *inode,
{
struct ethosu_device *edev =
container_of(inode->i_cdev, struct ethosu_device, cdev);
+ phandle rproc_phandle;
+ struct rproc *rproc;
+ int ret = 0;
file->private_data = edev;
dev_info(edev->dev, "Opening device node.\n");
+ if (of_property_read_u32(edev->dev->of_node, "fsl,cm33-proc",
+ &rproc_phandle)) {
+ dev_err(edev->dev, "could not get rproc phandle\n");
+ return -ENODEV;
+ }
+
+ rproc = rproc_get_by_phandle(rproc_phandle);
+ if (!rproc) {
+ dev_err(edev->dev, "could not get rproc handle\n");
+ return -EINVAL;
+ }
+
+ ret = rproc_set_firmware(rproc, ETHOSU_FIRMWARE_NAME);
+
+ if (!ret && atomic_read(&rproc->power) == 0) {
+ init_completion(&edev->erp.rpmsg_ready);
+ ret = rproc_boot(rproc);
+ if (ret)
+ dev_err(edev->dev, "could not boot a remote processor\n");
+ else
+ wait_for_completion_interruptible(&edev->erp.rpmsg_ready);
+ } else {
+ dev_err(edev->dev, "can't change firmware or remote processor is running\n");
+ }
+
+ edev->open = true;
+
return nonseekable_open(inode, file);
}
diff --git a/drivers/firmware/ethosu/ethosu_device.h b/drivers/firmware/ethosu/ethosu_device.h
index 3943c8df8f28..da470241b6a1 100644
--- a/drivers/firmware/ethosu/ethosu_device.h
+++ b/drivers/firmware/ethosu/ethosu_device.h
@@ -51,6 +51,7 @@ struct ethosu_device {
struct ethosu_rpmsg erp;
struct list_head capabilities_list;
struct list_head inference_list;
+ bool open;
};
/**
@@ -83,4 +84,13 @@ int ethosu_dev_init(struct ethosu_device *edev,
*/
void ethosu_dev_deinit(struct ethosu_device *edev);
+/**
+ * ethosu_suspend() - Suspend the device
+ */
+int ethosu_suspend(struct device *dev);
+/**
+ * ethosu_resume() - Resume the device
+ */
+int ethosu_resume(struct device *dev);
+
#endif /* ETHOSU_DEVICE_H */
diff --git a/drivers/firmware/ethosu/ethosu_driver.c b/drivers/firmware/ethosu/ethosu_driver.c
index 27931e9aa97c..d3306ef625ae 100644
--- a/drivers/firmware/ethosu/ethosu_driver.c
+++ b/drivers/firmware/ethosu/ethosu_driver.c
@@ -27,6 +27,7 @@
#include <linux/of_address.h>
#include <linux/platform_device.h>
+#include "ethosu_core_interface.h"
#include "ethosu_device.h"
/****************************************************************************
@@ -101,6 +102,33 @@ static int ethosu_pdev_remove(struct platform_device *pdev)
return 0;
}
+int ethosu_suspend(struct device *dev)
+{
+ struct ethosu_device *edev = dev->driver_data;
+ int ret = 0;
+
+ if (edev->open)
+ ret = ethosu_rpmsg_power_request(&edev->erp, ETHOSU_CORE_POWER_REQ_SUSPEND);
+
+ return ret;
+}
+
+int ethosu_resume(struct device *dev)
+{
+ struct ethosu_device *edev = dev->driver_data;
+ int ret = 0;
+
+ if (edev->open)
+ ret = ethosu_rpmsg_power_request(&edev->erp, ETHOSU_CORE_POWER_REQ_RESUME);
+
+ return ret;
+}
+
+struct dev_pm_ops ethosu_pm_ops = {
+ .suspend = ethosu_suspend,
+ .resume = ethosu_resume,
+};
+
static const struct of_device_id ethosu_pdev_match[] = {
{ .compatible = "arm,ethosu" },
{ /* Sentinel */ },
@@ -115,6 +143,7 @@ static struct platform_driver ethosu_pdev_driver = {
.name = ETHOSU_DRIVER_NAME,
.owner = THIS_MODULE,
.of_match_table = of_match_ptr(ethosu_pdev_match),
+ .pm = ðosu_pm_ops,
},
};
diff --git a/drivers/firmware/ethosu/ethosu_rpmsg.c b/drivers/firmware/ethosu/ethosu_rpmsg.c
index e4cf398468e4..a0320b8407d1 100644
--- a/drivers/firmware/ethosu/ethosu_rpmsg.c
+++ b/drivers/firmware/ethosu/ethosu_rpmsg.c
@@ -98,6 +98,35 @@ int ethosu_rpmsg_capabilities_request(struct ethosu_rpmsg *erp, void *user_arg)
return 0;
}
+int ethosu_rpmsg_power_request(struct ethosu_rpmsg *erp,
+ enum ethosu_core_power_req_type power_type)
+{
+ struct ethosu_core_msg msg = {
+ .magic = ETHOSU_CORE_MSG_MAGIC,
+ .type = ETHOSU_CORE_MSG_POWER_REQ,
+ .length = sizeof(struct ethosu_core_power_req)
+ };
+ struct ethosu_core_power_req req;
+ struct rpmsg_device *rpdev = erp->rpdev;
+ uint8_t data[sizeof(struct ethosu_core_msg) +
+ sizeof(struct ethosu_core_power_req)];
+ int ret;
+
+ req.type = power_type;
+ memcpy(data, &msg, sizeof(struct ethosu_core_msg));
+ memcpy(data + sizeof(struct ethosu_core_msg), &req,
+ sizeof(struct ethosu_core_power_req));
+
+ ret = rpmsg_send(rpdev->ept, (void *)&data,
+ sizeof(struct ethosu_core_msg) +
+ sizeof(struct ethosu_core_power_req));
+ if (ret) {
+ dev_err(&rpdev->dev, "rpmsg_send failed: %d\n", ret);
+ return ret;
+ }
+ return 0;
+}
+
int ethosu_rpmsg_inference(struct ethosu_rpmsg *erp,
void *user_arg,
uint32_t ifm_count,
@@ -193,6 +222,7 @@ static int ethosu_rpmsg_probe(struct rpmsg_device *rpdev)
if (ret) {
dev_err(&rpdev->dev, "rpmsg_send failed: %d\n", ret);
}
+ complete(&grp->rpmsg_ready);
return 0;
}
diff --git a/drivers/firmware/ethosu/ethosu_rpmsg.h b/drivers/firmware/ethosu/ethosu_rpmsg.h
index a4a639997a26..cee3d96c7895 100644
--- a/drivers/firmware/ethosu/ethosu_rpmsg.h
+++ b/drivers/firmware/ethosu/ethosu_rpmsg.h
@@ -7,6 +7,8 @@
#define ETHOSU_RPMSG_H
#include <linux/types.h>
+#include <linux/completion.h>
+#include "ethosu_core_interface.h"
struct device;
struct ethosu_buffer;
@@ -19,6 +21,7 @@ struct ethosu_rpmsg {
struct rpmsg_device *rpdev;
ethosu_rpmsg_cb callback;
void *user_arg;
+ struct completion rpmsg_ready;
};
/**
* ethosu_rpmsg_ping() - Send ping message
@@ -49,6 +52,14 @@ int ethosu_rpmsg_version_request(struct ethosu_rpmsg *erp);
int ethosu_rpmsg_capabilities_request(struct ethosu_rpmsg *erp,
void *user_arg);
+/**
+ * ethosu_rpmsg_power_request - Send power request
+ *
+ * Return: 0 on success, else error code
+ */
+int ethosu_rpmsg_power_request(struct ethosu_rpmsg *erp,
+ enum ethosu_core_power_req_type power_type);
+
/**
* ethosu_rpmsg_inference() - Send inference
*
--
2.17.1
RPMsg is used for the communication mechanism between Cortex-A and
Cortex-M for Ethos-U driver.
Signed-off-by: Lei Xu <[email protected]>
Signed-off-by: Alison Wang <[email protected]>
Reviewed-by: Peng Fan <[email protected]>
---
drivers/firmware/ethosu/Makefile | 2 +-
drivers/firmware/ethosu/ethosu_device.c | 143 ++++----
drivers/firmware/ethosu/ethosu_device.h | 8 +-
drivers/firmware/ethosu/ethosu_driver.c | 23 +-
drivers/firmware/ethosu/ethosu_inference.c | 2 +-
drivers/firmware/ethosu/ethosu_mailbox.c | 377 ---------------------
drivers/firmware/ethosu/ethosu_mailbox.h | 140 --------
drivers/firmware/ethosu/ethosu_rpmsg.c | 235 +++++++++++++
drivers/firmware/ethosu/ethosu_rpmsg.h | 73 ++++
9 files changed, 383 insertions(+), 620 deletions(-)
delete mode 100644 drivers/firmware/ethosu/ethosu_mailbox.c
delete mode 100644 drivers/firmware/ethosu/ethosu_mailbox.h
create mode 100644 drivers/firmware/ethosu/ethosu_rpmsg.c
create mode 100644 drivers/firmware/ethosu/ethosu_rpmsg.h
diff --git a/drivers/firmware/ethosu/Makefile b/drivers/firmware/ethosu/Makefile
index 933efee1b22f..0b3fe72c9c4f 100644
--- a/drivers/firmware/ethosu/Makefile
+++ b/drivers/firmware/ethosu/Makefile
@@ -24,5 +24,5 @@ ethosu-objs := ethosu_driver.o \
ethosu_buffer.o \
ethosu_device.o \
ethosu_inference.o \
- ethosu_mailbox.o \
+ ethosu_rpmsg.o \
ethosu_network.o
diff --git a/drivers/firmware/ethosu/ethosu_device.c b/drivers/firmware/ethosu/ethosu_device.c
index e6f1e8012b06..b2d4737080ac 100644
--- a/drivers/firmware/ethosu/ethosu_device.c
+++ b/drivers/firmware/ethosu/ethosu_device.c
@@ -1,5 +1,6 @@
/*
* (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
+ * Copyright 2020-2022 NXP
*
* This program is free software and is provided to you under the terms of the
* GNU General Public License version 2 as published by the Free Software
@@ -121,76 +122,76 @@ static int ethosu_capability_rsp(struct ethosu_device *edev,
}
/* Incoming messages */
-static int ethosu_handle_msg(struct ethosu_device *edev)
+static int ethosu_handle_msg(struct ethosu_device *edev, void *data)
{
- int ret;
- struct ethosu_core_msg header;
-
- union {
- struct ethosu_core_msg_err error;
- struct ethosu_core_inference_rsp inf;
- struct ethosu_core_msg_version version;
- struct ethosu_core_msg_capabilities_rsp capabilities;
- } data;
-
- /* Read message */
- ret = ethosu_mailbox_read(&edev->mailbox, &header, &data, sizeof(data));
- if (ret)
- return ret;
-
- switch (header.type) {
+ int ret = 0;
+ struct ethosu_core_msg *header = (struct ethosu_core_msg *)data;
+ struct ethosu_core_msg_err *error =
+ (struct ethosu_core_msg_err *)
+ ((char *)data + sizeof(struct ethosu_core_msg));
+ struct ethosu_core_inference_rsp *rsp =
+ (struct ethosu_core_inference_rsp *)
+ ((char *)data + sizeof(struct ethosu_core_msg));
+ struct ethosu_core_msg_version *version =
+ (struct ethosu_core_msg_version *)
+ ((char *)data + sizeof(struct ethosu_core_msg));
+ struct ethosu_core_msg_capabilities_rsp *capabilities =
+ (struct ethosu_core_msg_capabilities_rsp *)
+ ((char *)data + sizeof(struct ethosu_core_msg));
+
+ switch (header->type) {
case ETHOSU_CORE_MSG_ERR:
- if (header.length != sizeof(data.error)) {
+ if (header->length != sizeof(struct ethosu_core_msg_err)) {
dev_warn(edev->dev,
- "Msg: Error message of incorrect size. size=%u, expected=%zu\n", header.length,
- sizeof(data.error));
+ "Msg: Error message of incorrect size. size=%u, expected=%zu\n", header->length,
+ sizeof(struct ethosu_core_msg_err));
ret = -EBADMSG;
break;
}
- data.error.msg[sizeof(data.error.msg) - 1] = '\0';
+ error->msg[sizeof(error->msg) - 1] = '\0';
dev_warn(edev->dev, "Msg: Error. type=%u, msg=\"%s\"\n",
- data.error.type, data.error.msg);
+ error->type, error->msg);
ret = -EBADMSG;
break;
case ETHOSU_CORE_MSG_PING:
dev_info(edev->dev, "Msg: Ping\n");
- ret = ethosu_mailbox_pong(&edev->mailbox);
+ ret = ethosu_rpmsg_pong(&edev->erp);
break;
case ETHOSU_CORE_MSG_PONG:
dev_info(edev->dev, "Msg: Pong\n");
break;
case ETHOSU_CORE_MSG_INFERENCE_RSP:
- if (header.length != sizeof(data.inf)) {
+ if (header->length != sizeof(struct ethosu_core_inference_rsp)) {
dev_warn(edev->dev,
- "Msg: Inference response of incorrect size. size=%u, expected=%zu\n", header.length,
- sizeof(data.inf));
+ "Msg: Inference response of incorrect size. size=%u, expected=%zu\n", header->length,
+ sizeof(struct ethosu_core_inference_rsp));
ret = -EBADMSG;
break;
}
dev_info(edev->dev,
"Msg: Inference response. user_arg=0x%llx, ofm_count=%u, status=%u\n",
- data.inf.user_arg, data.inf.ofm_count,
- data.inf.status);
- ethosu_inference_rsp(edev, &data.inf);
+ rsp->user_arg, rsp->ofm_count,
+ rsp->status);
+ ethosu_inference_rsp(edev, rsp);
break;
case ETHOSU_CORE_MSG_VERSION_RSP:
- if (header.length != sizeof(data.version)) {
+ if (header->length != sizeof(struct ethosu_core_msg_version)) {
dev_warn(edev->dev,
- "Msg: Version response of incorrect size. size=%u, expected=%zu\n", header.length,
- sizeof(data.version));
+ "Msg: Version response of incorrect size. size=%u, expected=%zu\n", header->length,
+ sizeof(struct ethosu_core_msg_version));
ret = -EBADMSG;
break;
}
dev_info(edev->dev, "Msg: Version response v%u.%u.%u\n",
- data.version.major, data.version.minor,
- data.version.patch);
+ version->major, version->minor,
+ version->patch);
/* Check major and minor version match, else return error */
- if (data.version.major != ETHOSU_CORE_MSG_VERSION_MAJOR ||
- data.version.minor != ETHOSU_CORE_MSG_VERSION_MINOR) {
+ if (version->major != ETHOSU_CORE_MSG_VERSION_MAJOR ||
+ version->minor != ETHOSU_CORE_MSG_VERSION_MINOR) {
dev_warn(edev->dev, "Msg: Version mismatch detected! ");
dev_warn(edev->dev, "Local version: v%u.%u.%u\n",
ETHOSU_CORE_MSG_VERSION_MAJOR,
@@ -200,32 +201,32 @@ static int ethosu_handle_msg(struct ethosu_device *edev)
break;
case ETHOSU_CORE_MSG_CAPABILITIES_RSP:
- if (header.length != sizeof(data.capabilities)) {
+ if (header->length != sizeof(struct ethosu_core_msg_capabilities_rsp)) {
dev_warn(edev->dev,
- "Msg: Capabilities response of incorrect size. size=%u, expected=%zu\n", header.length,
- sizeof(data.capabilities));
+ "Msg: Capabilities response of incorrect size. size=%u, expected=%zu\n", header->length,
+ sizeof(struct ethosu_core_msg_capabilities_rsp));
ret = -EBADMSG;
break;
}
dev_info(edev->dev,
"Msg: Capabilities response ua%llx vs%hhu v%hhu.%hhu p%hhu av%hhu.%hhu.%hhu dv%hhu.%hhu.%hhu mcc%hhu csv%hhu cd%hhu\n",
- data.capabilities.user_arg,
- data.capabilities.version_status,
- data.capabilities.version_major,
- data.capabilities.version_minor,
- data.capabilities.product_major,
- data.capabilities.arch_major_rev,
- data.capabilities.arch_minor_rev,
- data.capabilities.arch_patch_rev,
- data.capabilities.driver_major_rev,
- data.capabilities.driver_minor_rev,
- data.capabilities.driver_patch_rev,
- data.capabilities.macs_per_cc,
- data.capabilities.cmd_stream_version,
- data.capabilities.custom_dma);
-
- ret = ethosu_capability_rsp(edev, &data.capabilities);
+ capabilities->user_arg,
+ capabilities->version_status,
+ capabilities->version_major,
+ capabilities->version_minor,
+ capabilities->product_major,
+ capabilities->arch_major_rev,
+ capabilities->arch_minor_rev,
+ capabilities->arch_patch_rev,
+ capabilities->driver_major_rev,
+ capabilities->driver_minor_rev,
+ capabilities->driver_patch_rev,
+ capabilities->macs_per_cc,
+ capabilities->cmd_stream_version,
+ capabilities->custom_dma);
+
+ ret = ethosu_capability_rsp(edev, capabilities);
break;
default:
/* This should not happen due to version checks */
@@ -269,7 +270,7 @@ static int ethosu_send_capabilities_request(struct ethosu_device *edev,
init_completion(&cap->done);
list_add(&cap->list, &edev->capabilities_list);
- ret = ethosu_mailbox_capabilities_request(&edev->mailbox, cap);
+ ret = ethosu_rpmsg_capabilities_request(&edev->erp, cap);
if (0 != ret)
goto put_kref;
@@ -325,7 +326,7 @@ static long ethosu_ioctl(struct file *file,
switch (cmd) {
case ETHOSU_IOCTL_VERSION_REQ:
dev_info(edev->dev, "Ioctl: Send version request\n");
- ret = ethosu_mailbox_version_request(&edev->mailbox);
+ ret = ethosu_rpmsg_version_request(&edev->erp);
break;
case ETHOSU_IOCTL_CAPABILITIES_REQ:
dev_info(edev->dev, "Ioctl: Send capabilities request\n");
@@ -333,7 +334,7 @@ static long ethosu_ioctl(struct file *file,
break;
case ETHOSU_IOCTL_PING: {
dev_info(edev->dev, "Ioctl: Send ping\n");
- ret = ethosu_mailbox_ping(&edev->mailbox);
+ ret = ethosu_rpmsg_ping(&edev->erp);
break;
}
case ETHOSU_IOCTL_BUFFER_CREATE: {
@@ -373,7 +374,7 @@ static long ethosu_ioctl(struct file *file,
return ret;
}
-static void ethosu_mbox_rx(void *user_arg)
+static void ethosu_rpmsg_rx(void *user_arg, void *data)
{
struct ethosu_device *edev = user_arg;
int ret;
@@ -381,13 +382,8 @@ static void ethosu_mbox_rx(void *user_arg)
mutex_lock(&edev->mutex);
do {
- ret = ethosu_handle_msg(edev);
- if (ret && ret != -ENOMSG)
- /* Need to start over in case of error, empty the queue
- * by fast-forwarding read position to write position.
- * */
- ethosu_mailbox_reset(&edev->mailbox);
- } while (ret == 0);
+ ret = ethosu_handle_msg(edev, data);
+ } while (ret != 0);
mutex_unlock(&edev->mutex);
}
@@ -395,9 +391,7 @@ static void ethosu_mbox_rx(void *user_arg)
int ethosu_dev_init(struct ethosu_device *edev,
struct device *dev,
struct class *class,
- dev_t devt,
- struct resource *in_queue,
- struct resource *out_queue)
+ dev_t devt)
{
static const struct file_operations fops = {
.owner = THIS_MODULE,
@@ -423,8 +417,7 @@ int ethosu_dev_init(struct ethosu_device *edev,
dma_set_mask_and_coherent(edev->dev, DMA_BIT_MASK(DMA_ADDR_BITS));
- ret = ethosu_mailbox_init(&edev->mailbox, dev, in_queue, out_queue,
- ethosu_mbox_rx, edev);
+ ret = ethosu_rpmsg_init(&edev->erp, ethosu_rpmsg_rx, edev);
if (ret)
goto release_reserved_mem;
@@ -434,7 +427,7 @@ int ethosu_dev_init(struct ethosu_device *edev,
ret = cdev_add(&edev->cdev, edev->devt, 1);
if (ret) {
dev_err(edev->dev, "Failed to add character device.\n");
- goto deinit_mailbox;
+ goto deinit_rpmsg;
}
sysdev = device_create(edev->class, NULL, edev->devt, edev,
@@ -454,8 +447,8 @@ int ethosu_dev_init(struct ethosu_device *edev,
del_cdev:
cdev_del(&edev->cdev);
-deinit_mailbox:
- ethosu_mailbox_deinit(&edev->mailbox);
+deinit_rpmsg:
+ ethosu_rpmsg_deinit(&edev->erp);
release_reserved_mem:
of_reserved_mem_device_release(edev->dev);
@@ -465,7 +458,7 @@ int ethosu_dev_init(struct ethosu_device *edev,
void ethosu_dev_deinit(struct ethosu_device *edev)
{
- ethosu_mailbox_deinit(&edev->mailbox);
+ ethosu_rpmsg_deinit(&edev->erp);
device_destroy(edev->class, edev->cdev.dev);
cdev_del(&edev->cdev);
of_reserved_mem_device_release(edev->dev);
diff --git a/drivers/firmware/ethosu/ethosu_device.h b/drivers/firmware/ethosu/ethosu_device.h
index 3afdda84862d..3943c8df8f28 100644
--- a/drivers/firmware/ethosu/ethosu_device.h
+++ b/drivers/firmware/ethosu/ethosu_device.h
@@ -26,7 +26,7 @@
****************************************************************************/
#include "uapi/ethosu.h"
-#include "ethosu_mailbox.h"
+#include "ethosu_rpmsg.h"
#include <linux/device.h>
#include <linux/cdev.h>
@@ -48,7 +48,7 @@ struct ethosu_device {
struct class *class;
dev_t devt;
struct mutex mutex;
- struct ethosu_mailbox mailbox;
+ struct ethosu_rpmsg erp;
struct list_head capabilities_list;
struct list_head inference_list;
};
@@ -76,9 +76,7 @@ struct ethosu_capabilities {
int ethosu_dev_init(struct ethosu_device *edev,
struct device *dev,
struct class *class,
- dev_t devt,
- struct resource *in_queue,
- struct resource *out_queue);
+ dev_t devt);
/**
* ethosu_dev_deinit() - Initialize the device
diff --git a/drivers/firmware/ethosu/ethosu_driver.c b/drivers/firmware/ethosu/ethosu_driver.c
index 9d02431d3194..27931e9aa97c 100644
--- a/drivers/firmware/ethosu/ethosu_driver.c
+++ b/drivers/firmware/ethosu/ethosu_driver.c
@@ -1,5 +1,6 @@
/*
* (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
+ * Copyright 2020-2022 NXP
*
* This program is free software and is provided to you under the terms of the
* GNU General Public License version 2 as published by the Free Software
@@ -55,8 +56,6 @@ static DECLARE_BITMAP(minors, MINOR_COUNT);
static int ethosu_pdev_probe(struct platform_device *pdev)
{
struct ethosu_device *edev;
- struct resource *in_queue_res;
- struct resource *out_queue_res;
int minor;
int ret;
@@ -69,23 +68,6 @@ static int ethosu_pdev_probe(struct platform_device *pdev)
return -ENOMEM;
}
- /* Get path to TCM memory */
- in_queue_res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
- "in_queue");
- if (IS_ERR(in_queue_res)) {
- dev_err(&pdev->dev, "Failed to get in_queue resource.\n");
-
- return PTR_ERR(in_queue_res);
- }
-
- out_queue_res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
- "out_queue");
- if (IS_ERR(out_queue_res)) {
- dev_err(&pdev->dev, "Failed to get out_queue resource.\n");
-
- return PTR_ERR(out_queue_res);
- }
-
/* Allocate memory for Arm Ethos-U device */
edev = devm_kzalloc(&pdev->dev, sizeof(*edev), GFP_KERNEL);
if (!edev)
@@ -95,8 +77,7 @@ static int ethosu_pdev_probe(struct platform_device *pdev)
/* Initialize device */
ret = ethosu_dev_init(edev, &pdev->dev, ethosu_class,
- MKDEV(MAJOR(devt), minor), in_queue_res,
- out_queue_res);
+ MKDEV(MAJOR(devt), minor));
if (ret)
goto free_dev;
diff --git a/drivers/firmware/ethosu/ethosu_inference.c b/drivers/firmware/ethosu/ethosu_inference.c
index 6fde92c148a0..853947a2334e 100644
--- a/drivers/firmware/ethosu/ethosu_inference.c
+++ b/drivers/firmware/ethosu/ethosu_inference.c
@@ -86,7 +86,7 @@ static int ethosu_inference_send(struct ethosu_inference *inf)
inf->status = ETHOSU_UAPI_STATUS_ERROR;
- ret = ethosu_mailbox_inference(&inf->edev->mailbox, inf,
+ ret = ethosu_rpmsg_inference(&inf->edev->erp, inf,
inf->ifm_count, inf->ifm,
inf->ofm_count, inf->ofm,
inf->net->buf,
diff --git a/drivers/firmware/ethosu/ethosu_mailbox.c b/drivers/firmware/ethosu/ethosu_mailbox.c
deleted file mode 100644
index 7f159f3b0a60..000000000000
--- a/drivers/firmware/ethosu/ethosu_mailbox.c
+++ /dev/null
@@ -1,377 +0,0 @@
-/*
- * (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
- *
- * This program is free software and is provided to you under the terms of the
- * GNU General Public License version 2 as published by the Free Software
- * Foundation, and any use by you of this program is subject to the terms
- * of such GNU licence.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, you can access it online at
- * http://www.gnu.org/licenses/gpl-2.0.html.
- *
- * SPDX-License-Identifier: GPL-2.0-only
- */
-
-/****************************************************************************
- * Includes
- ****************************************************************************/
-
-#include "ethosu_mailbox.h"
-
-#include "ethosu_buffer.h"
-#include "ethosu_core_interface.h"
-#include "ethosu_device.h"
-
-#include <linux/resource.h>
-#include <linux/uio.h>
-
-/****************************************************************************
- * Functions
- ****************************************************************************/
-
-static void ethosu_core_set_size(struct ethosu_buffer *buf,
- struct ethosu_core_buffer *cbuf)
-{
- cbuf->ptr = (uint32_t)buf->dma_addr + buf->offset;
- cbuf->size = (uint32_t)buf->size;
-}
-
-static void ethosu_core_set_capacity(struct ethosu_buffer *buf,
- struct ethosu_core_buffer *cbuf)
-{
- cbuf->ptr = (uint32_t)buf->dma_addr + buf->offset + buf->size;
- cbuf->size = (uint32_t)buf->capacity - buf->offset - buf->size;
-}
-
-static size_t ethosu_queue_available(struct ethosu_core_queue *queue)
-{
- size_t size = queue->header.write - queue->header.read;
-
- if (queue->header.read > queue->header.write)
- size += queue->header.size;
-
- return size;
-}
-
-static size_t ethosu_queue_capacity(struct ethosu_core_queue *queue)
-{
- return queue->header.size - ethosu_queue_available(queue);
-}
-
-static int ethosu_queue_write(struct ethosu_mailbox *mbox,
- const struct kvec *vec,
- size_t length)
-{
- struct ethosu_core_queue *queue = mbox->in_queue;
- uint8_t *dst = &queue->data[0];
- uint32_t wpos = queue->header.write;
- size_t total_size;
- size_t i;
- int ret;
-
- for (i = 0, total_size = 0; i < length; i++)
- total_size += vec[i].iov_len;
-
- if (total_size > ethosu_queue_capacity(queue))
- return -EINVAL;
-
- for (i = 0; i < length; i++) {
- const uint8_t *src = vec[i].iov_base;
- const uint8_t *end = src + vec[i].iov_len;
-
- while (src < end) {
- dst[wpos] = *src++;
- wpos = (wpos + 1) % queue->header.size;
- }
- }
-
- queue->header.write = wpos;
-
- ret = mbox_send_message(mbox->tx, queue);
- if (ret < 0)
- return ret;
-
- return 0;
-}
-
-static int ethosu_queue_write_msg(struct ethosu_mailbox *mbox,
- uint32_t type,
- void *data,
- size_t length)
-{
- struct ethosu_core_msg msg = {
- .magic = ETHOSU_CORE_MSG_MAGIC,
- .type = type, .length= length
- };
- const struct kvec vec[2] = {
- { &msg, sizeof(msg) },
- { data, length }
- };
-
- return ethosu_queue_write(mbox, vec, 2);
-}
-
-static int ethosu_queue_read(struct ethosu_mailbox *mbox,
- void *data,
- size_t length)
-{
- struct ethosu_core_queue *queue = mbox->out_queue;
- uint8_t *src = &queue->data[0];
- uint8_t *dst = (uint8_t *)data;
- const uint8_t *end = dst + length;
- uint32_t rpos = queue->header.read;
- size_t queue_avail = ethosu_queue_available(queue);
-
- if (length == 0)
- return 0;
- else if (queue_avail == 0)
- return -ENOMSG;
- else if (length > queue_avail)
- return -EBADMSG;
-
- while (dst < end) {
- *dst++ = src[rpos];
- rpos = (rpos + 1) % queue->header.size;
- }
-
- queue->header.read = rpos;
-
- return 0;
-}
-
-void ethosu_mailbox_reset(struct ethosu_mailbox *mbox)
-{
- mbox->out_queue->header.read = mbox->out_queue->header.write;
-}
-
-int ethosu_mailbox_read(struct ethosu_mailbox *mbox,
- struct ethosu_core_msg *header,
- void *data,
- size_t length)
-{
- int ret;
-
- /* Read message header magic */
- ret = ethosu_queue_read(mbox, header, sizeof(*header));
- if (ret) {
- if (ret != -ENOMSG)
- dev_warn(mbox->dev,
- "Msg: Failed to read message header\n");
-
- return ret;
- }
-
- if (header->magic != ETHOSU_CORE_MSG_MAGIC) {
- dev_warn(mbox->dev,
- "Msg: Invalid magic. Got: %08X but expected %08X\n",
- header->magic, ETHOSU_CORE_MSG_MAGIC);
-
- return -EINVAL;
- }
-
- dev_info(mbox->dev,
- "mbox: Read msg header. magic=%08X, type=%u, length=%u",
- header->magic, header->type, header->length);
-
- /* Check that payload is not larger than allocated buffer */
- if (header->length > length) {
- dev_warn(mbox->dev,
- "Msg: Buffer size (%zu) too small for message (%u)\n",
- sizeof(data), header->length);
-
- return -ENOMEM;
- }
-
- /* Read payload data */
- ret = ethosu_queue_read(mbox, data, header->length);
- if (ret) {
- dev_warn(mbox->dev, "Msg: Failed to read payload data\n");
-
- return -EBADMSG;
- }
-
- return 0;
-}
-
-int ethosu_mailbox_ping(struct ethosu_mailbox *mbox)
-{
- return ethosu_queue_write_msg(mbox, ETHOSU_CORE_MSG_PING, NULL, 0);
-}
-
-int ethosu_mailbox_pong(struct ethosu_mailbox *mbox)
-{
- return ethosu_queue_write_msg(mbox, ETHOSU_CORE_MSG_PONG, NULL, 0);
-}
-
-int ethosu_mailbox_version_request(struct ethosu_mailbox *mbox)
-{
- return ethosu_queue_write_msg(mbox, ETHOSU_CORE_MSG_VERSION_REQ, NULL,
- 0);
-}
-
-int ethosu_mailbox_capabilities_request(struct ethosu_mailbox *mbox,
- void *user_arg)
-{
- struct ethosu_core_capabilities_req req = {
- .user_arg = (ptrdiff_t)user_arg
- };
-
- return ethosu_queue_write_msg(mbox, ETHOSU_CORE_MSG_CAPABILITIES_REQ,
- &req,
- sizeof(req));
-}
-
-int ethosu_mailbox_inference(struct ethosu_mailbox *mbox,
- void *user_arg,
- uint32_t ifm_count,
- struct ethosu_buffer **ifm,
- uint32_t ofm_count,
- struct ethosu_buffer **ofm,
- struct ethosu_buffer *network,
- uint8_t *pmu_event_config,
- uint8_t pmu_event_config_count,
- uint8_t pmu_cycle_counter_enable)
-{
- struct ethosu_core_inference_req inf;
- uint32_t i;
-
- /* Verify that the uapi and core has the same number of pmus */
- if (pmu_event_config_count != ETHOSU_CORE_PMU_MAX) {
- dev_err(mbox->dev, "PMU count misconfigured.\n");
-
- return -EINVAL;
- }
-
- inf.user_arg = (ptrdiff_t)user_arg;
- inf.ifm_count = ifm_count;
- inf.ofm_count = ofm_count;
- inf.pmu_cycle_counter_enable = pmu_cycle_counter_enable;
-
- for (i = 0; i < ifm_count; i++)
- ethosu_core_set_size(ifm[i], &inf.ifm[i]);
-
- for (i = 0; i < ofm_count; i++)
- ethosu_core_set_capacity(ofm[i], &inf.ofm[i]);
-
- for (i = 0; i < ETHOSU_CORE_PMU_MAX; i++)
- inf.pmu_event_config[i] = pmu_event_config[i];
-
- ethosu_core_set_size(network, &inf.network);
-
- return ethosu_queue_write_msg(mbox, ETHOSU_CORE_MSG_INFERENCE_REQ,
- &inf, sizeof(inf));
-}
-
-static void ethosu_mailbox_rx_work(struct work_struct *work)
-{
- struct ethosu_mailbox *mbox = container_of(work, typeof(*mbox), work);
-
- mbox->callback(mbox->user_arg);
-}
-
-static void ethosu_mailbox_rx_callback(struct mbox_client *client,
- void *message)
-{
- struct ethosu_mailbox *mbox =
- container_of(client, typeof(*mbox), client);
-
- dev_info(mbox->dev, "mbox: Received message.\n");
-
- queue_work(mbox->wq, &mbox->work);
-}
-
-static void ethosu_mailbox_tx_done(struct mbox_client *client,
- void *message,
- int r)
-{
- if (r)
- dev_warn(client->dev, "mbox: Failed sending message (%d)\n", r);
- else
- dev_info(client->dev, "mbox: Message sent\n");
-}
-
-int ethosu_mailbox_init(struct ethosu_mailbox *mbox,
- struct device *dev,
- struct resource *in_queue,
- struct resource *out_queue,
- ethosu_mailbox_cb callback,
- void *user_arg)
-{
- int ret;
-
- mbox->dev = dev;
- mbox->callback = callback;
- mbox->user_arg = user_arg;
-
- mbox->client.dev = dev;
- mbox->client.rx_callback = ethosu_mailbox_rx_callback;
- mbox->client.tx_prepare = NULL; /* preparation of data is handled
- * through the
- * queue functions */
- mbox->client.tx_done = ethosu_mailbox_tx_done;
- mbox->client.tx_block = true;
- mbox->client.knows_txdone = false;
- mbox->client.tx_tout = 500;
-
- mbox->in_queue = devm_ioremap_resource(mbox->dev, in_queue);
- if (IS_ERR(mbox->in_queue))
- return PTR_ERR(mbox->in_queue);
-
- mbox->out_queue = devm_ioremap_resource(mbox->dev, out_queue);
- if (IS_ERR(mbox->out_queue)) {
- ret = PTR_ERR(mbox->out_queue);
- goto unmap_in_queue;
- }
-
- mbox->wq = create_singlethread_workqueue("ethosu_workqueue");
- if (!mbox->wq) {
- dev_err(mbox->dev, "Failed to create work queue\n");
- ret = -EINVAL;
- goto unmap_out_queue;
- }
-
- INIT_WORK(&mbox->work, ethosu_mailbox_rx_work);
-
- mbox->tx = mbox_request_channel_byname(&mbox->client, "tx");
- if (IS_ERR(mbox->tx)) {
- dev_warn(mbox->dev, "mbox: Failed to request tx channel\n");
- ret = PTR_ERR(mbox->tx);
- goto workqueue_destroy;
- }
-
- mbox->rx = mbox_request_channel_byname(&mbox->client, "rx");
- if (IS_ERR(mbox->rx)) {
- dev_info(dev, "mbox: Using same channel for RX and TX\n");
- mbox->rx = mbox->tx;
- }
-
- return 0;
-
-workqueue_destroy:
- destroy_workqueue(mbox->wq);
-
-unmap_out_queue:
- devm_iounmap(mbox->dev, mbox->out_queue);
-
-unmap_in_queue:
- devm_iounmap(mbox->dev, mbox->in_queue);
-
- return ret;
-}
-
-void ethosu_mailbox_deinit(struct ethosu_mailbox *mbox)
-{
- if (mbox->rx != mbox->tx)
- mbox_free_channel(mbox->rx);
-
- mbox_free_channel(mbox->tx);
- destroy_workqueue(mbox->wq);
- devm_iounmap(mbox->dev, mbox->out_queue);
- devm_iounmap(mbox->dev, mbox->in_queue);
-}
diff --git a/drivers/firmware/ethosu/ethosu_mailbox.h b/drivers/firmware/ethosu/ethosu_mailbox.h
deleted file mode 100644
index 5cd5e62198b8..000000000000
--- a/drivers/firmware/ethosu/ethosu_mailbox.h
+++ /dev/null
@@ -1,140 +0,0 @@
-/*
- * (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
- *
- * This program is free software and is provided to you under the terms of the
- * GNU General Public License version 2 as published by the Free Software
- * Foundation, and any use by you of this program is subject to the terms
- * of such GNU licence.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, you can access it online at
- * http://www.gnu.org/licenses/gpl-2.0.html.
- *
- * SPDX-License-Identifier: GPL-2.0-only
- */
-
-#ifndef ETHOSU_MAILBOX_H
-#define ETHOSU_MAILBOX_H
-
-/****************************************************************************
- * Includes
- ****************************************************************************/
-#include "ethosu_core_interface.h"
-
-#include <linux/types.h>
-#include <linux/mailbox_client.h>
-#include <linux/workqueue.h>
-
-/****************************************************************************
- * Types
- ****************************************************************************/
-
-struct device;
-struct ethosu_buffer;
-struct ethosu_device;
-struct ethosu_core_msg;
-struct ethosu_core_queue;
-struct resource;
-
-typedef void (*ethosu_mailbox_cb)(void *user_arg);
-
-struct ethosu_mailbox {
- struct device *dev;
- struct workqueue_struct *wq;
- struct work_struct work;
- struct ethosu_core_queue __iomem *in_queue;
- struct ethosu_core_queue __iomem *out_queue;
- struct mbox_client client;
- struct mbox_chan *rx;
- struct mbox_chan *tx;
- ethosu_mailbox_cb callback;
- void *user_arg;
-};
-
-/****************************************************************************
- * Functions
- ****************************************************************************/
-
-/**
- * ethosu_mailbox_init() - Initialize mailbox
- *
- * Return: 0 on success, else error code.
- */
-int ethosu_mailbox_init(struct ethosu_mailbox *mbox,
- struct device *dev,
- struct resource *in_queue,
- struct resource *out_queue,
- ethosu_mailbox_cb callback,
- void *user_arg);
-
-/**
- * ethosu_mailbox_deinit() - Deinitialize mailbox
- */
-void ethosu_mailbox_deinit(struct ethosu_mailbox *mbox);
-
-/**
- * ethosu_mailbox_read() - Read message from mailbox
- *
- * Return: 0 message read, else error code.
- */
-int ethosu_mailbox_read(struct ethosu_mailbox *mbox,
- struct ethosu_core_msg *header,
- void *data,
- size_t length);
-
-/**
- * ethosu_mailbox_reset() - Reset to end of queue
- */
-void ethosu_mailbox_reset(struct ethosu_mailbox *mbox);
-
-/**
- * ethosu_mailbox_ping() - Send ping message
- *
- * Return: 0 on success, else error code.
- */
-int ethosu_mailbox_ping(struct ethosu_mailbox *mbox);
-
-/**
- * ethosu_mailbox_pong() - Send pong response
- *
- * Return: 0 on success, else error code.
- */
-int ethosu_mailbox_pong(struct ethosu_mailbox *mbox);
-
-/**
- * ethosu_mailbox_version_response - Send version request
- *
- * Return: 0 on succes, else error code
- */
-int ethosu_mailbox_version_request(struct ethosu_mailbox *mbox);
-
-/**
- * ethosu_mailbox_capabilities_request() - Send capabilities request
- *
- * Return: 0 on success, else error code.
- */
-int ethosu_mailbox_capabilities_request(struct ethosu_mailbox *mbox,
- void *user_arg);
-
-/**
- * ethosu_mailbox_inference() - Send inference
- *
- * Return: 0 on success, else error code.
- */
-int ethosu_mailbox_inference(struct ethosu_mailbox *mbox,
- void *user_arg,
- uint32_t ifm_count,
- struct ethosu_buffer **ifm,
- uint32_t ofm_count,
- struct ethosu_buffer **ofm,
- struct ethosu_buffer *network,
- uint8_t *pmu_event_config,
- uint8_t pmu_event_config_count,
- uint8_t pmu_cycle_counter_enable);
-
-#endif /* ETHOSU_MAILBOX_H */
diff --git a/drivers/firmware/ethosu/ethosu_rpmsg.c b/drivers/firmware/ethosu/ethosu_rpmsg.c
new file mode 100644
index 000000000000..cb7e4556f635
--- /dev/null
+++ b/drivers/firmware/ethosu/ethosu_rpmsg.c
@@ -0,0 +1,235 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright 2020-2022 NXP
+ */
+
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/resource.h>
+#include <linux/rpmsg.h>
+#include <linux/uio.h>
+#include <linux/virtio.h>
+
+#include "ethosu_buffer.h"
+#include "ethosu_core_interface.h"
+#include "ethosu_device.h"
+
+struct ethosu_rpmsg *grp;
+
+#define MSG "ethosu say hello!"
+
+static void ethosu_core_set_size(struct ethosu_buffer *buf,
+ struct ethosu_core_buffer *cbuf)
+{
+ cbuf->ptr = (uint32_t)buf->dma_addr + buf->offset;
+ cbuf->size = (uint32_t)buf->size;
+}
+
+static void ethosu_core_set_capacity(struct ethosu_buffer *buf,
+ struct ethosu_core_buffer *cbuf)
+{
+ cbuf->ptr = (uint32_t)buf->dma_addr + buf->offset + buf->size;
+ cbuf->size = (uint32_t)buf->capacity - buf->offset - buf->size;
+}
+
+static int ethosu_rpmsg_send(struct ethosu_rpmsg *erp, uint32_t type)
+{
+ struct ethosu_core_msg msg;
+ struct rpmsg_device *rpdev = erp->rpdev;
+ int ret;
+
+ msg.magic = ETHOSU_CORE_MSG_MAGIC;
+ msg.type = type;
+ msg.length = 0;
+
+ print_hex_dump(KERN_DEBUG, __func__, DUMP_PREFIX_NONE, 16, 1,
+ (void *)&msg, sizeof(msg), true);
+
+ ret = rpmsg_send(rpdev->ept, (void *)&msg, sizeof(msg));
+ if (ret) {
+ dev_err(&rpdev->dev, "rpmsg_send failed: %d\n", ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+int ethosu_rpmsg_ping(struct ethosu_rpmsg *erp)
+{
+ return ethosu_rpmsg_send(erp, ETHOSU_CORE_MSG_PING);
+}
+
+int ethosu_rpmsg_pong(struct ethosu_rpmsg *erp)
+{
+ return ethosu_rpmsg_send(erp, ETHOSU_CORE_MSG_PONG);
+}
+
+int ethosu_rpmsg_version_request(struct ethosu_rpmsg *erp)
+{
+ return ethosu_rpmsg_send(erp, ETHOSU_CORE_MSG_VERSION_REQ);
+}
+
+int ethosu_rpmsg_capabilities_request(struct ethosu_rpmsg *erp, void *user_arg)
+{
+ struct ethosu_core_msg msg = {
+ .magic = ETHOSU_CORE_MSG_MAGIC,
+ .type = ETHOSU_CORE_MSG_CAPABILITIES_REQ,
+ .length = sizeof(struct ethosu_core_capabilities_req)
+ };
+ struct ethosu_core_capabilities_req req = {
+ .user_arg = (uint64_t)user_arg
+ };
+ struct rpmsg_device *rpdev = erp->rpdev;
+ uint8_t data[sizeof(struct ethosu_core_msg) +
+ sizeof(struct ethosu_core_capabilities_req)];
+ int ret;
+
+ memcpy(data, &msg, sizeof(struct ethosu_core_msg));
+ memcpy(data + sizeof(struct ethosu_core_msg), &req,
+ sizeof(struct ethosu_core_capabilities_req));
+
+ ret = rpmsg_send(rpdev->ept, (void *)&data,
+ sizeof(struct ethosu_core_msg) +
+ sizeof(struct ethosu_core_capabilities_req));
+ if (ret) {
+ dev_err(&rpdev->dev, "rpmsg_send failed: %d\n", ret);
+ return ret;
+ }
+ return 0;
+}
+
+int ethosu_rpmsg_inference(struct ethosu_rpmsg *erp,
+ void *user_arg,
+ uint32_t ifm_count,
+ struct ethosu_buffer **ifm,
+ uint32_t ofm_count,
+ struct ethosu_buffer **ofm,
+ struct ethosu_buffer *network,
+ uint8_t *pmu_event_config,
+ uint8_t pmu_event_config_count,
+ uint8_t pmu_cycle_counter_enable)
+{
+ struct ethosu_core_msg msg = {
+ .magic = ETHOSU_CORE_MSG_MAGIC,
+ .type = ETHOSU_CORE_MSG_INFERENCE_REQ,
+ .length = sizeof(struct ethosu_core_inference_req)
+ };
+ struct ethosu_core_inference_req req;
+ struct rpmsg_device *rpdev = erp->rpdev;
+ uint8_t data[sizeof(struct ethosu_core_msg) +
+ sizeof(struct ethosu_core_inference_req)];
+ int ret;
+ uint32_t i;
+
+ /* Verify that the uapi and core has the same number of pmus */
+ if (pmu_event_config_count != ETHOSU_CORE_PMU_MAX) {
+ dev_err(&rpdev->dev, "PMU count misconfigured.\n");
+
+ return -EINVAL;
+ }
+
+ req.user_arg = (uint64_t)user_arg;
+ req.ifm_count = ifm_count;
+ req.ofm_count = ofm_count;
+ req.pmu_cycle_counter_enable = pmu_cycle_counter_enable;
+
+ for (i = 0; i < ifm_count; i++)
+ ethosu_core_set_size(ifm[i], &req.ifm[i]);
+
+ for (i = 0; i < ofm_count; i++)
+ ethosu_core_set_capacity(ofm[i], &req.ofm[i]);
+
+ for (i = 0; i < ETHOSU_CORE_PMU_MAX; i++)
+ req.pmu_event_config[i] = pmu_event_config[i];
+
+ ethosu_core_set_size(network, &req.network);
+
+ memcpy(data, &msg, sizeof(struct ethosu_core_msg));
+ memcpy(data + sizeof(struct ethosu_core_msg), &req,
+ sizeof(struct ethosu_core_inference_req));
+
+ ret = rpmsg_send(rpdev->ept, (void *)&data,
+ sizeof(struct ethosu_core_msg) +
+ sizeof(struct ethosu_core_inference_req));
+ if (ret) {
+ dev_err(&rpdev->dev, "rpmsg_send failed: %d\n", ret);
+ return ret;
+ }
+
+ return 0;
+}
+
+static int rpmsg_ethosu_cb(struct rpmsg_device *rpdev,
+ void *data, int len, void *priv, u32 src)
+{
+ struct ethosu_rpmsg *rpmsg = dev_get_drvdata(&rpdev->dev);
+
+ if (len == 0)
+ return 0;
+
+ dev_dbg(&rpdev->dev, "msg(<- src 0x%x) len %d\n", src, len);
+
+ print_hex_dump(KERN_DEBUG, __func__, DUMP_PREFIX_NONE, 16, 1,
+ data, len, true);
+
+ rpmsg->callback(rpmsg->user_arg, data);
+
+ return 0;
+}
+
+static int ethosu_rpmsg_probe(struct rpmsg_device *rpdev)
+{
+ int ret;
+
+ grp->rpdev = rpdev;
+ dev_set_drvdata(&rpdev->dev, grp);
+
+ dev_info(&rpdev->dev, "new channel: 0x%x -> 0x%x!\n",
+ rpdev->src, rpdev->dst);
+
+ ret = rpmsg_send(rpdev->ept, MSG, strlen(MSG));
+ if (ret) {
+ dev_err(&rpdev->dev, "rpmsg_send failed: %d\n", ret);
+ }
+
+ return 0;
+}
+
+static void ethosu_rpmsg_remove(struct rpmsg_device *rpdev)
+{
+ dev_info(&rpdev->dev, "rpmsg ethosu driver is removed\n");
+}
+
+static struct rpmsg_device_id rpmsg_driver_ethosu_id_table[] = {
+ { .name = "rpmsg-ethosu-channel" },
+ { },
+};
+
+static struct rpmsg_driver ethosu_rpmsg_driver = {
+ .drv.name = KBUILD_MODNAME,
+ .drv.owner = THIS_MODULE,
+ .id_table = rpmsg_driver_ethosu_id_table,
+ .probe = ethosu_rpmsg_probe,
+ .callback = rpmsg_ethosu_cb,
+ .remove = ethosu_rpmsg_remove,
+};
+
+int ethosu_rpmsg_init(struct ethosu_rpmsg *erp,
+ ethosu_rpmsg_cb callback, void *user_arg)
+{
+ grp = erp;
+ erp->callback = callback;
+ erp->user_arg = user_arg;
+
+ return register_rpmsg_driver(ðosu_rpmsg_driver);
+}
+
+int ethosu_rpmsg_deinit(struct ethosu_rpmsg *erp)
+{
+ erp->callback = NULL;
+ erp->user_arg = NULL;
+ erp->rpdev = NULL;
+
+ unregister_rpmsg_driver(ðosu_rpmsg_driver);
+ return 0;
+}
diff --git a/drivers/firmware/ethosu/ethosu_rpmsg.h b/drivers/firmware/ethosu/ethosu_rpmsg.h
new file mode 100644
index 000000000000..dd08e0d7945b
--- /dev/null
+++ b/drivers/firmware/ethosu/ethosu_rpmsg.h
@@ -0,0 +1,73 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright 2020-2022 NXP
+ */
+
+#ifndef ETHOSU_RPMSG_H
+#define ETHOSU_RPMSG_H
+
+#include <linux/types.h>
+
+struct device;
+struct ethosu_buffer;
+struct ethosu_device;
+struct ethosu_core_msg;
+
+typedef void (*ethosu_rpmsg_cb)(void *user_arg, void *data);
+
+struct ethosu_rpmsg {
+ struct rpmsg_device *rpdev;
+ ethosu_rpmsg_cb callback;
+ void *user_arg;
+};
+/**
+ * ethosu_rpmsg_ping() - Send ping message
+ *
+ * Return: 0 on success, else error code.
+ */
+int ethosu_rpmsg_ping(struct ethosu_rpmsg *erp);
+
+/**
+ * ethosu_rpmsg_pong() - Send pong message
+ *
+ * Return: 0 on success, else error code.
+ */
+int ethosu_rpmsg_pong(struct ethosu_rpmsg *erp);
+
+/**
+ * ethosu_rpmsg_version_response - Send version request
+ *
+ * Return: 0 on success, else error code
+ */
+int ethosu_rpmsg_version_request(struct ethosu_rpmsg *erp);
+
+/**
+ * ethosu_rpmsg_capabilities_request - Send capabilities request
+ *
+ * Return: 0 on success, else error code
+ */
+int ethosu_rpmsg_capabilities_request(struct ethosu_rpmsg *erp,
+ void *user_arg);
+
+/**
+ * ethosu_rpmsg_inference() - Send inference
+ *
+ * Return: 0 on success, else error code.
+ */
+int ethosu_rpmsg_inference(struct ethosu_rpmsg *erp,
+ void *user_arg,
+ uint32_t ifm_count,
+ struct ethosu_buffer **ifm,
+ uint32_t ofm_count,
+ struct ethosu_buffer **ofm,
+ struct ethosu_buffer *network,
+ uint8_t *pmu_event_config,
+ uint8_t pmu_event_config_count,
+ uint8_t pmu_cycle_counter_enable
+ );
+
+int ethosu_rpmsg_init(struct ethosu_rpmsg *erp,
+ ethosu_rpmsg_cb callback, void *user_arg);
+
+int ethosu_rpmsg_deinit(struct ethosu_rpmsg *erp);
+#endif /* ETHOSU_RPMSG_H */
--
2.17.1
Ethos-U Linux driver is to provide an example of how a rich operating
system like Linux can dispatch inferences to an Arm Cortex-M subsystem,
consisting of an Arm Cortex-M and an Arm Ethos-U NPU.
Link: https://git.mlplatform.org/ml/ethos-u/ethos-u-linux-driver-stack.git
Tag: 22.02
Signed-off-by: Kristofer Jonsson <[email protected]>
Signed-off-by: Per Astrand <[email protected]>
Signed-off-by: Jonny Svärd <[email protected]>
Signed-off-by: Lior Dekel <[email protected]>
Signed-off-by: Henrik Hoglind <[email protected]>
Signed-off-by: Davide Grohmann <[email protected]>
Signed-off-by: Alison Wang <[email protected]>
---
drivers/firmware/Kconfig | 1 +
drivers/firmware/Makefile | 1 +
drivers/firmware/ethosu/Kconfig | 24 +
drivers/firmware/ethosu/Makefile | 28 ++
drivers/firmware/ethosu/ethosu_buffer.c | 312 ++++++++++++
drivers/firmware/ethosu/ethosu_buffer.h | 106 ++++
.../firmware/ethosu/ethosu_core_interface.h | 183 +++++++
drivers/firmware/ethosu/ethosu_device.c | 474 ++++++++++++++++++
drivers/firmware/ethosu/ethosu_device.h | 88 ++++
drivers/firmware/ethosu/ethosu_driver.c | 191 +++++++
drivers/firmware/ethosu/ethosu_inference.c | 417 +++++++++++++++
drivers/firmware/ethosu/ethosu_inference.h | 120 +++++
drivers/firmware/ethosu/ethosu_mailbox.c | 377 ++++++++++++++
drivers/firmware/ethosu/ethosu_mailbox.h | 140 ++++++
drivers/firmware/ethosu/ethosu_network.c | 201 ++++++++
drivers/firmware/ethosu/ethosu_network.h | 81 +++
drivers/firmware/ethosu/uapi/ethosu.h | 207 ++++++++
17 files changed, 2951 insertions(+)
create mode 100644 drivers/firmware/ethosu/Kconfig
create mode 100644 drivers/firmware/ethosu/Makefile
create mode 100644 drivers/firmware/ethosu/ethosu_buffer.c
create mode 100644 drivers/firmware/ethosu/ethosu_buffer.h
create mode 100644 drivers/firmware/ethosu/ethosu_core_interface.h
create mode 100644 drivers/firmware/ethosu/ethosu_device.c
create mode 100644 drivers/firmware/ethosu/ethosu_device.h
create mode 100644 drivers/firmware/ethosu/ethosu_driver.c
create mode 100644 drivers/firmware/ethosu/ethosu_inference.c
create mode 100644 drivers/firmware/ethosu/ethosu_inference.h
create mode 100644 drivers/firmware/ethosu/ethosu_mailbox.c
create mode 100644 drivers/firmware/ethosu/ethosu_mailbox.h
create mode 100644 drivers/firmware/ethosu/ethosu_network.c
create mode 100644 drivers/firmware/ethosu/ethosu_network.h
create mode 100644 drivers/firmware/ethosu/uapi/ethosu.h
diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig
index b59e3041fd62..bbad6dc81e7b 100644
--- a/drivers/firmware/Kconfig
+++ b/drivers/firmware/Kconfig
@@ -308,6 +308,7 @@ source "drivers/firmware/broadcom/Kconfig"
source "drivers/firmware/cirrus/Kconfig"
source "drivers/firmware/google/Kconfig"
source "drivers/firmware/efi/Kconfig"
+source "drivers/firmware/ethosu/Kconfig"
source "drivers/firmware/imx/Kconfig"
source "drivers/firmware/meson/Kconfig"
source "drivers/firmware/psci/Kconfig"
diff --git a/drivers/firmware/Makefile b/drivers/firmware/Makefile
index 28fcddcd688f..6c597a78daa7 100644
--- a/drivers/firmware/Makefile
+++ b/drivers/firmware/Makefile
@@ -33,6 +33,7 @@ obj-y += cirrus/
obj-y += meson/
obj-$(CONFIG_GOOGLE_FIRMWARE) += google/
obj-y += efi/
+obj-$(CONFIG_ETHOSU) += ethosu/
obj-y += imx/
obj-y += psci/
obj-y += smccc/
diff --git a/drivers/firmware/ethosu/Kconfig b/drivers/firmware/ethosu/Kconfig
new file mode 100644
index 000000000000..ce837f45f8e5
--- /dev/null
+++ b/drivers/firmware/ethosu/Kconfig
@@ -0,0 +1,24 @@
+#
+# (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
+#
+# This program is free software and is provided to you under the terms of the
+# GNU General Public License version 2 as published by the Free Software
+# Foundation, and any use by you of this program is subject to the terms
+# of such GNU licence.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, you can access it online at
+# http://www.gnu.org/licenses/gpl-2.0.html.
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+
+config ETHOSU
+ tristate "Arm Ethos-U NPU support"
+ help
+ Arm Ethos-U NPU driver.
diff --git a/drivers/firmware/ethosu/Makefile b/drivers/firmware/ethosu/Makefile
new file mode 100644
index 000000000000..933efee1b22f
--- /dev/null
+++ b/drivers/firmware/ethosu/Makefile
@@ -0,0 +1,28 @@
+#
+# (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
+#
+# This program is free software and is provided to you under the terms of the
+# GNU General Public License version 2 as published by the Free Software
+# Foundation, and any use by you of this program is subject to the terms
+# of such GNU licence.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, you can access it online at
+# http://www.gnu.org/licenses/gpl-2.0.html.
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+
+obj-$(CONFIG_ETHOSU) = ethosu.o
+
+ethosu-objs := ethosu_driver.o \
+ ethosu_buffer.o \
+ ethosu_device.o \
+ ethosu_inference.o \
+ ethosu_mailbox.o \
+ ethosu_network.o
diff --git a/drivers/firmware/ethosu/ethosu_buffer.c b/drivers/firmware/ethosu/ethosu_buffer.c
new file mode 100644
index 000000000000..43a433355f30
--- /dev/null
+++ b/drivers/firmware/ethosu/ethosu_buffer.c
@@ -0,0 +1,312 @@
+/*
+ * (C) COPYRIGHT 2020-2021 Arm Limited. All rights reserved.
+ *
+ * This program is free software and is provided to you under the terms of the
+ * GNU General Public License version 2 as published by the Free Software
+ * Foundation, and any use by you of this program is subject to the terms
+ * of such GNU licence.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, you can access it online at
+ * http://www.gnu.org/licenses/gpl-2.0.html.
+ *
+ * SPDX-License-Identifier: GPL-2.0-only
+ */
+
+/****************************************************************************
+ * Includes
+ ****************************************************************************/
+
+#include "ethosu_buffer.h"
+
+#include "ethosu_device.h"
+#include "uapi/ethosu.h"
+
+#include <linux/anon_inodes.h>
+#include <linux/dma-mapping.h>
+#include <linux/of_address.h>
+#include <linux/file.h>
+#include <linux/fs.h>
+#include <linux/uaccess.h>
+
+/****************************************************************************
+ * Variables
+ ****************************************************************************/
+
+static int ethosu_buffer_release(struct inode *inode,
+ struct file *file);
+
+static int ethosu_buffer_mmap(struct file *file,
+ struct vm_area_struct *vma);
+
+static long ethosu_buffer_ioctl(struct file *file,
+ unsigned int cmd,
+ unsigned long arg);
+
+static const struct file_operations ethosu_buffer_fops = {
+ .release = ðosu_buffer_release,
+ .mmap = ðosu_buffer_mmap,
+ .unlocked_ioctl = ðosu_buffer_ioctl,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl = ðosu_buffer_ioctl,
+#endif
+};
+
+/****************************************************************************
+ * Functions
+ ****************************************************************************/
+
+/*
+ * The 'dma-ranges' device tree property for shared dma memory does not seem
+ * to be fully supported for coherent memory. Therefor we apply the DMA range
+ * offset ourselves.
+ */
+static dma_addr_t ethosu_buffer_dma_ranges(struct device *dev,
+ dma_addr_t dma_addr,
+ size_t dma_buf_size)
+{
+ struct device_node *node = dev->of_node;
+ const __be32 *ranges;
+ int len;
+ int naddr;
+ int nsize;
+ int inc;
+ int i;
+
+ if (!node)
+ return dma_addr;
+
+ /* Get the #address-cells and #size-cells properties */
+ naddr = of_n_addr_cells(node);
+ nsize = of_n_size_cells(node);
+
+ /* Read the 'dma-ranges' property */
+ ranges = of_get_property(node, "dma-ranges", &len);
+ if (!ranges || len <= 0)
+ return dma_addr;
+
+ dev_dbg(dev, "ranges=%p, len=%d, naddr=%d, nsize=%d\n",
+ ranges, len, naddr, nsize);
+
+ len /= sizeof(*ranges);
+ inc = naddr + naddr + nsize;
+
+ for (i = 0; (i + inc) <= len; i += inc) {
+ dma_addr_t daddr;
+ dma_addr_t paddr;
+ dma_addr_t size;
+
+ daddr = of_read_number(&ranges[i], naddr);
+ paddr = of_read_number(&ranges[i + naddr], naddr);
+ size = of_read_number(&ranges[i + naddr + naddr], nsize);
+
+ dev_dbg(dev, "daddr=0x%llx, paddr=0x%llx, size=0x%llx\n",
+ daddr, paddr, size);
+
+ if (dma_addr >= paddr &&
+ (dma_addr + dma_buf_size) < (paddr + size))
+ return dma_addr + daddr - paddr;
+ }
+
+ return dma_addr;
+}
+
+static bool ethosu_buffer_verify(struct file *file)
+{
+ return file->f_op == ðosu_buffer_fops;
+}
+
+static void ethosu_buffer_destroy(struct kref *kref)
+{
+ struct ethosu_buffer *buf =
+ container_of(kref, struct ethosu_buffer, kref);
+
+ dev_info(buf->edev->dev, "Buffer destroy. handle=0x%pK\n", buf);
+
+ dma_free_coherent(buf->edev->dev, buf->capacity, buf->cpu_addr,
+ buf->dma_addr_orig);
+ devm_kfree(buf->edev->dev, buf);
+}
+
+static int ethosu_buffer_release(struct inode *inode,
+ struct file *file)
+{
+ struct ethosu_buffer *buf = file->private_data;
+
+ dev_info(buf->edev->dev, "Buffer release. handle=0x%pK\n", buf);
+
+ ethosu_buffer_put(buf);
+
+ return 0;
+}
+
+static int ethosu_buffer_mmap(struct file *file,
+ struct vm_area_struct *vma)
+{
+ struct ethosu_buffer *buf = file->private_data;
+ int ret;
+
+ dev_info(buf->edev->dev, "Buffer mmap. handle=0x%pK\n", buf);
+
+ ret = dma_mmap_coherent(buf->edev->dev, vma, buf->cpu_addr,
+ buf->dma_addr_orig,
+ buf->capacity);
+
+ return ret;
+}
+
+static long ethosu_buffer_ioctl(struct file *file,
+ unsigned int cmd,
+ unsigned long arg)
+{
+ struct ethosu_buffer *buf = file->private_data;
+ void __user *udata = (void __user *)arg;
+ int ret = -EINVAL;
+
+ ret = mutex_lock_interruptible(&buf->edev->mutex);
+ if (ret)
+ return ret;
+
+ dev_info(buf->edev->dev, "Ioctl. cmd=%u, arg=%lu\n", cmd, arg);
+
+ switch (cmd) {
+ case ETHOSU_IOCTL_BUFFER_SET: {
+ struct ethosu_uapi_buffer uapi;
+
+ if (copy_from_user(&uapi, udata, sizeof(uapi)))
+ break;
+
+ dev_info(buf->edev->dev,
+ "Ioctl: Buffer set. size=%u, offset=%u\n",
+ uapi.size, uapi.offset);
+
+ ret = ethosu_buffer_resize(buf, uapi.size, uapi.offset);
+ break;
+ }
+ case ETHOSU_IOCTL_BUFFER_GET: {
+ struct ethosu_uapi_buffer uapi;
+
+ uapi.size = buf->size;
+ uapi.offset = buf->offset;
+
+ dev_info(buf->edev->dev,
+ "Ioctl: Buffer get. size=%u, offset=%u\n",
+ uapi.size, uapi.offset);
+
+ if (copy_to_user(udata, &uapi, sizeof(uapi)))
+ break;
+
+ ret = 0;
+ break;
+ }
+ default: {
+ dev_err(buf->edev->dev, "Invalid ioctl. cmd=%u, arg=%lu",
+ cmd, arg);
+ break;
+ }
+ }
+
+ mutex_unlock(&buf->edev->mutex);
+
+ return ret;
+}
+
+int ethosu_buffer_create(struct ethosu_device *edev,
+ size_t capacity)
+{
+ struct ethosu_buffer *buf;
+ int ret = -ENOMEM;
+
+ buf = devm_kzalloc(edev->dev, sizeof(*buf), GFP_KERNEL);
+ if (!buf)
+ return -ENOMEM;
+
+ buf->edev = edev;
+ buf->capacity = capacity;
+ buf->offset = 0;
+ buf->size = 0;
+ kref_init(&buf->kref);
+
+ buf->cpu_addr = dma_alloc_coherent(buf->edev->dev, capacity,
+ &buf->dma_addr_orig, GFP_KERNEL);
+ if (!buf->cpu_addr)
+ goto free_buf;
+
+ buf->dma_addr = ethosu_buffer_dma_ranges(buf->edev->dev,
+ buf->dma_addr_orig,
+ buf->capacity);
+
+ ret = anon_inode_getfd("ethosu-buffer", ðosu_buffer_fops, buf,
+ O_RDWR | O_CLOEXEC);
+ if (ret < 0)
+ goto free_dma;
+
+ buf->file = fget(ret);
+ fput(buf->file);
+
+ dev_info(buf->edev->dev,
+ "Buffer create. handle=0x%pK, capacity=%zu, cpu_addr=0x%pK, dma_addr=0x%llx, dma_addr_orig=0x%llx, phys_addr=0x%llx\n",
+ buf, capacity, buf->cpu_addr, buf->dma_addr,
+ buf->dma_addr_orig, virt_to_phys(buf->cpu_addr));
+
+ return ret;
+
+free_dma:
+ dma_free_coherent(buf->edev->dev, buf->capacity, buf->cpu_addr,
+ buf->dma_addr_orig);
+
+free_buf:
+ devm_kfree(buf->edev->dev, buf);
+
+ return ret;
+}
+
+struct ethosu_buffer *ethosu_buffer_get_from_fd(int fd)
+{
+ struct ethosu_buffer *buf;
+ struct file *file;
+
+ file = fget(fd);
+ if (!file)
+ return ERR_PTR(-EINVAL);
+
+ if (!ethosu_buffer_verify(file)) {
+ fput(file);
+
+ return ERR_PTR(-EINVAL);
+ }
+
+ buf = file->private_data;
+ ethosu_buffer_get(buf);
+ fput(file);
+
+ return buf;
+}
+
+void ethosu_buffer_get(struct ethosu_buffer *buf)
+{
+ kref_get(&buf->kref);
+}
+
+void ethosu_buffer_put(struct ethosu_buffer *buf)
+{
+ kref_put(&buf->kref, ethosu_buffer_destroy);
+}
+
+int ethosu_buffer_resize(struct ethosu_buffer *buf,
+ size_t size,
+ size_t offset)
+{
+ if ((size + offset) > buf->capacity)
+ return -EINVAL;
+
+ buf->size = size;
+ buf->offset = offset;
+
+ return 0;
+}
diff --git a/drivers/firmware/ethosu/ethosu_buffer.h b/drivers/firmware/ethosu/ethosu_buffer.h
new file mode 100644
index 000000000000..14f26c2b0a9d
--- /dev/null
+++ b/drivers/firmware/ethosu/ethosu_buffer.h
@@ -0,0 +1,106 @@
+/*
+ * (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
+ *
+ * This program is free software and is provided to you under the terms of the
+ * GNU General Public License version 2 as published by the Free Software
+ * Foundation, and any use by you of this program is subject to the terms
+ * of such GNU licence.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, you can access it online at
+ * http://www.gnu.org/licenses/gpl-2.0.html.
+ *
+ * SPDX-License-Identifier: GPL-2.0-only
+ */
+
+#ifndef ETHOSU_BUFFER_H
+#define ETHOSU_BUFFER_H
+
+/****************************************************************************
+ * Includes
+ ****************************************************************************/
+
+#include <linux/kref.h>
+#include <linux/types.h>
+
+/****************************************************************************
+ * Types
+ ****************************************************************************/
+
+struct ethosu_device;
+struct device;
+
+/**
+ * struct ethosu_buffer - Buffer
+ * @dev: Device
+ * @file: File
+ * @kref: Reference counting
+ * @capacity: Maximum capacity of the buffer
+ * @offset: Offset to first byte of buffer
+ * @size: Size of the data in the buffer
+ * @cpu_addr: Kernel mapped address
+ * @dma_addr: DMA address
+ * @dma_addr_orig: Original DMA address before range mapping
+ *
+ * 'offset + size' must not be larger than 'capacity'.
+ */
+struct ethosu_buffer {
+ struct ethosu_device *edev;
+ struct file *file;
+ struct kref kref;
+ size_t capacity;
+ size_t offset;
+ size_t size;
+ void *cpu_addr;
+ dma_addr_t dma_addr;
+ dma_addr_t dma_addr_orig;
+};
+
+/****************************************************************************
+ * Functions
+ ****************************************************************************/
+
+/**
+ * ethosu_buffer_create() - Create buffer
+ *
+ * This function must be called in the context of a user space process.
+ *
+ * Return: fd on success, else error code.
+ */
+int ethosu_buffer_create(struct ethosu_device *edev,
+ size_t capacity);
+
+/**
+ * ethosu_buffer_get_from_fd() - Get buffer handle from fd
+ *
+ * This function must be called from a user space context.
+ *
+ * Return: Pointer on success, else ERR_PTR.
+ */
+struct ethosu_buffer *ethosu_buffer_get_from_fd(int fd);
+
+/**
+ * ethosu_buffer_get() - Put buffer
+ */
+void ethosu_buffer_get(struct ethosu_buffer *buf);
+
+/**
+ * ethosu_buffer_put() - Put buffer
+ */
+void ethosu_buffer_put(struct ethosu_buffer *buf);
+
+/**
+ * ethosu_buffer_resize() - Resize and validate buffer
+ *
+ * Return: 0 on success, else error code.
+ */
+int ethosu_buffer_resize(struct ethosu_buffer *buf,
+ size_t size,
+ size_t offset);
+
+#endif /* ETHOSU_BUFFER_H */
diff --git a/drivers/firmware/ethosu/ethosu_core_interface.h b/drivers/firmware/ethosu/ethosu_core_interface.h
new file mode 100644
index 000000000000..ef63c3b55352
--- /dev/null
+++ b/drivers/firmware/ethosu/ethosu_core_interface.h
@@ -0,0 +1,183 @@
+/*
+ * (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
+ *
+ * This program is free software and is provided to you under the terms of the
+ * GNU General Public License version 2 as published by the Free Software
+ * Foundation, and any use by you of this program is subject to the terms
+ * of such GNU licence.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, you can access it online at
+ * http://www.gnu.org/licenses/gpl-2.0.html.
+ *
+ * SPDX-License-Identifier: GPL-2.0-only
+ */
+
+#ifndef ETHOSU_CORE_INTERFACE_H
+#define ETHOSU_CORE_INTERFACE_H
+
+#ifdef __KERNEL__
+#include <linux/types.h>
+#else
+#include <stdint.h>
+#endif
+
+#ifdef __cplusplus
+namespace EthosU {
+#endif
+
+/** Maximum number of IFM/OFM buffers per inference */
+#define ETHOSU_CORE_BUFFER_MAX 16
+
+/** Maximum number of PMU counters to be returned for inference */
+#define ETHOSU_CORE_PMU_MAX 8
+
+#define ETHOSU_CORE_MSG_MAGIC 0x41457631
+#define ETHOSU_CORE_MSG_VERSION_MAJOR 0
+#define ETHOSU_CORE_MSG_VERSION_MINOR 2
+#define ETHOSU_CORE_MSG_VERSION_PATCH 0
+
+/**
+ * enum ethosu_core_msg_type - Message types
+ *
+ * Types for the messages sent between the host and the core subsystem.
+ */
+enum ethosu_core_msg_type {
+ ETHOSU_CORE_MSG_ERR = 1,
+ ETHOSU_CORE_MSG_PING,
+ ETHOSU_CORE_MSG_PONG,
+ ETHOSU_CORE_MSG_INFERENCE_REQ,
+ ETHOSU_CORE_MSG_INFERENCE_RSP,
+ ETHOSU_CORE_MSG_VERSION_REQ,
+ ETHOSU_CORE_MSG_VERSION_RSP,
+ ETHOSU_CORE_MSG_CAPABILITIES_REQ,
+ ETHOSU_CORE_MSG_CAPABILITIES_RSP,
+ ETHOSU_CORE_MSG_MAX
+};
+
+/**
+ * struct ethosu_core_msg - Message header
+ */
+struct ethosu_core_msg {
+ uint32_t magic;
+ uint32_t type;
+ uint32_t length;
+};
+
+/**
+ * struct ethosu_core_queue_header - Message queue header
+ */
+struct ethosu_core_queue_header {
+ uint32_t size;
+ uint32_t read;
+ uint32_t write;
+};
+
+/**
+ * struct ethosu_core_queue - Message queue
+ *
+ * Dynamically sized message queue.
+ */
+struct ethosu_core_queue {
+ struct ethosu_core_queue_header header;
+ uint8_t data[];
+};
+
+enum ethosu_core_status {
+ ETHOSU_CORE_STATUS_OK,
+ ETHOSU_CORE_STATUS_ERROR
+};
+
+struct ethosu_core_buffer {
+ uint32_t ptr;
+ uint32_t size;
+};
+
+struct ethosu_core_inference_req {
+ uint64_t user_arg;
+ uint32_t ifm_count;
+ struct ethosu_core_buffer ifm[ETHOSU_CORE_BUFFER_MAX];
+ uint32_t ofm_count;
+ struct ethosu_core_buffer ofm[ETHOSU_CORE_BUFFER_MAX];
+ struct ethosu_core_buffer network;
+ uint8_t pmu_event_config[ETHOSU_CORE_PMU_MAX];
+ uint32_t pmu_cycle_counter_enable;
+};
+
+struct ethosu_core_inference_rsp {
+ uint64_t user_arg;
+ uint32_t ofm_count;
+ uint32_t ofm_size[ETHOSU_CORE_BUFFER_MAX];
+ uint32_t status;
+ uint8_t pmu_event_config[ETHOSU_CORE_PMU_MAX];
+ uint32_t pmu_event_count[ETHOSU_CORE_PMU_MAX];
+ uint32_t pmu_cycle_counter_enable;
+ uint64_t pmu_cycle_counter_count;
+};
+
+/**
+ * struct ethosu_core_msg_verson - Message protocol version
+ */
+struct ethosu_core_msg_version {
+ uint8_t major;
+ uint8_t minor;
+ uint8_t patch;
+ uint8_t _reserved;
+};
+
+/**
+ * struct ethosu_core_capabilities_req - Message capabilities request
+ */
+struct ethosu_core_capabilities_req {
+ uint64_t user_arg;
+};
+
+/**
+ * struct ethosu_core_capabilities_rsp - Message capabilities response
+ */
+struct ethosu_core_msg_capabilities_rsp {
+ uint64_t user_arg;
+ uint32_t version_status;
+ uint32_t version_minor;
+ uint32_t version_major;
+ uint32_t product_major;
+ uint32_t arch_patch_rev;
+ uint32_t arch_minor_rev;
+ uint32_t arch_major_rev;
+ uint32_t driver_patch_rev;
+ uint32_t driver_minor_rev;
+ uint32_t driver_major_rev;
+ uint32_t macs_per_cc;
+ uint32_t cmd_stream_version;
+ uint32_t custom_dma;
+};
+
+/**
+ * enum ethosu_core_msg_err_type - Error types
+ */
+enum ethosu_core_msg_err_type {
+ ETHOSU_CORE_MSG_ERR_GENERIC = 0,
+ ETHOSU_CORE_MSG_ERR_UNSUPPORTED_TYPE,
+ ETHOSU_CORE_MSG_ERR_INVALID_PAYLOAD,
+ ETHOSU_CORE_MSG_ERR_INVALID_SIZE,
+ ETHOSU_CORE_MSG_ERR_INVALID_MAGIC,
+ ETHOSU_CORE_MSG_ERR_MAX
+};
+
+/**
+ * struct ethosu_core_msg_err - Error message struct
+ */
+struct ethosu_core_msg_err {
+ uint32_t type; /* optional use of extra error code */
+ char msg[128];
+};
+#ifdef __cplusplus
+} /*namespace EthosU */
+#endif
+
+#endif
diff --git a/drivers/firmware/ethosu/ethosu_device.c b/drivers/firmware/ethosu/ethosu_device.c
new file mode 100644
index 000000000000..e6f1e8012b06
--- /dev/null
+++ b/drivers/firmware/ethosu/ethosu_device.c
@@ -0,0 +1,474 @@
+/*
+ * (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
+ *
+ * This program is free software and is provided to you under the terms of the
+ * GNU General Public License version 2 as published by the Free Software
+ * Foundation, and any use by you of this program is subject to the terms
+ * of such GNU licence.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, you can access it online at
+ * http://www.gnu.org/licenses/gpl-2.0.html.
+ *
+ * SPDX-License-Identifier: GPL-2.0-only
+ */
+
+/****************************************************************************
+ * Includes
+ ****************************************************************************/
+
+#include "ethosu_device.h"
+
+#include "ethosu_buffer.h"
+#include "ethosu_core_interface.h"
+#include "ethosu_inference.h"
+#include "ethosu_network.h"
+#include "uapi/ethosu.h"
+
+#include <linux/delay.h>
+#include <linux/dma-mapping.h>
+#include <linux/errno.h>
+#include <linux/fs.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/uaccess.h>
+#include <linux/slab.h>
+
+/****************************************************************************
+ * Defines
+ ****************************************************************************/
+
+#define DMA_ADDR_BITS 32 /* Number of address bits */
+
+#define CAPABILITIES_RESP_TIMEOUT_MS 2000
+
+/****************************************************************************
+ * Types
+ ****************************************************************************/
+
+/****************************************************************************
+ * Functions
+ ****************************************************************************/
+
+static void ethosu_capabilities_destroy(struct kref *kref)
+{
+ struct ethosu_capabilities *cap =
+ container_of(kref, struct ethosu_capabilities, refcount);
+
+ list_del(&cap->list);
+
+ devm_kfree(cap->edev->dev, cap);
+}
+
+static int ethosu_capabilities_find(struct ethosu_capabilities *cap,
+ struct list_head *capabilties_list)
+{
+ struct ethosu_capabilities *cur;
+
+ list_for_each_entry(cur, capabilties_list, list) {
+ if (cur == cap)
+ return 0;
+ }
+
+ return -EINVAL;
+}
+
+static int ethosu_capability_rsp(struct ethosu_device *edev,
+ struct ethosu_core_msg_capabilities_rsp *msg)
+{
+ struct ethosu_capabilities *cap;
+ struct ethosu_uapi_device_capabilities *capabilities;
+ int ret;
+
+ cap = (struct ethosu_capabilities *)msg->user_arg;
+ ret = ethosu_capabilities_find(cap, &edev->capabilities_list);
+ if (0 != ret) {
+ dev_warn(edev->dev,
+ "Handle not found in capabilities list. handle=0x%p\n",
+ cap);
+
+ /* NOTE: do not call complete or kref_put on invalid data! */
+ return ret;
+ }
+
+ capabilities = cap->capabilities;
+
+ capabilities->hw_id.version_status = msg->version_status;
+ capabilities->hw_id.version_minor = msg->version_minor;
+ capabilities->hw_id.version_major = msg->version_major;
+ capabilities->hw_id.product_major = msg->product_major;
+ capabilities->hw_id.arch_patch_rev = msg->arch_patch_rev;
+ capabilities->hw_id.arch_minor_rev = msg->arch_minor_rev;
+ capabilities->hw_id.arch_major_rev = msg->arch_major_rev;
+ capabilities->driver_patch_rev = msg->driver_patch_rev;
+ capabilities->driver_minor_rev = msg->driver_minor_rev;
+ capabilities->driver_major_rev = msg->driver_major_rev;
+ capabilities->hw_cfg.macs_per_cc = msg->macs_per_cc;
+ capabilities->hw_cfg.cmd_stream_version = msg->cmd_stream_version;
+ capabilities->hw_cfg.custom_dma = msg->custom_dma;
+
+ complete(&cap->done);
+
+ kref_put(&cap->refcount, ethosu_capabilities_destroy);
+
+ return 0;
+}
+
+/* Incoming messages */
+static int ethosu_handle_msg(struct ethosu_device *edev)
+{
+ int ret;
+ struct ethosu_core_msg header;
+
+ union {
+ struct ethosu_core_msg_err error;
+ struct ethosu_core_inference_rsp inf;
+ struct ethosu_core_msg_version version;
+ struct ethosu_core_msg_capabilities_rsp capabilities;
+ } data;
+
+ /* Read message */
+ ret = ethosu_mailbox_read(&edev->mailbox, &header, &data, sizeof(data));
+ if (ret)
+ return ret;
+
+ switch (header.type) {
+ case ETHOSU_CORE_MSG_ERR:
+ if (header.length != sizeof(data.error)) {
+ dev_warn(edev->dev,
+ "Msg: Error message of incorrect size. size=%u, expected=%zu\n", header.length,
+ sizeof(data.error));
+ ret = -EBADMSG;
+ break;
+ }
+
+ data.error.msg[sizeof(data.error.msg) - 1] = '\0';
+ dev_warn(edev->dev, "Msg: Error. type=%u, msg=\"%s\"\n",
+ data.error.type, data.error.msg);
+ ret = -EBADMSG;
+ break;
+ case ETHOSU_CORE_MSG_PING:
+ dev_info(edev->dev, "Msg: Ping\n");
+ ret = ethosu_mailbox_pong(&edev->mailbox);
+ break;
+ case ETHOSU_CORE_MSG_PONG:
+ dev_info(edev->dev, "Msg: Pong\n");
+ break;
+ case ETHOSU_CORE_MSG_INFERENCE_RSP:
+ if (header.length != sizeof(data.inf)) {
+ dev_warn(edev->dev,
+ "Msg: Inference response of incorrect size. size=%u, expected=%zu\n", header.length,
+ sizeof(data.inf));
+ ret = -EBADMSG;
+ break;
+ }
+
+ dev_info(edev->dev,
+ "Msg: Inference response. user_arg=0x%llx, ofm_count=%u, status=%u\n",
+ data.inf.user_arg, data.inf.ofm_count,
+ data.inf.status);
+ ethosu_inference_rsp(edev, &data.inf);
+ break;
+ case ETHOSU_CORE_MSG_VERSION_RSP:
+ if (header.length != sizeof(data.version)) {
+ dev_warn(edev->dev,
+ "Msg: Version response of incorrect size. size=%u, expected=%zu\n", header.length,
+ sizeof(data.version));
+ ret = -EBADMSG;
+ break;
+ }
+
+ dev_info(edev->dev, "Msg: Version response v%u.%u.%u\n",
+ data.version.major, data.version.minor,
+ data.version.patch);
+
+ /* Check major and minor version match, else return error */
+ if (data.version.major != ETHOSU_CORE_MSG_VERSION_MAJOR ||
+ data.version.minor != ETHOSU_CORE_MSG_VERSION_MINOR) {
+ dev_warn(edev->dev, "Msg: Version mismatch detected! ");
+ dev_warn(edev->dev, "Local version: v%u.%u.%u\n",
+ ETHOSU_CORE_MSG_VERSION_MAJOR,
+ ETHOSU_CORE_MSG_VERSION_MINOR,
+ ETHOSU_CORE_MSG_VERSION_PATCH);
+ }
+
+ break;
+ case ETHOSU_CORE_MSG_CAPABILITIES_RSP:
+ if (header.length != sizeof(data.capabilities)) {
+ dev_warn(edev->dev,
+ "Msg: Capabilities response of incorrect size. size=%u, expected=%zu\n", header.length,
+ sizeof(data.capabilities));
+ ret = -EBADMSG;
+ break;
+ }
+
+ dev_info(edev->dev,
+ "Msg: Capabilities response ua%llx vs%hhu v%hhu.%hhu p%hhu av%hhu.%hhu.%hhu dv%hhu.%hhu.%hhu mcc%hhu csv%hhu cd%hhu\n",
+ data.capabilities.user_arg,
+ data.capabilities.version_status,
+ data.capabilities.version_major,
+ data.capabilities.version_minor,
+ data.capabilities.product_major,
+ data.capabilities.arch_major_rev,
+ data.capabilities.arch_minor_rev,
+ data.capabilities.arch_patch_rev,
+ data.capabilities.driver_major_rev,
+ data.capabilities.driver_minor_rev,
+ data.capabilities.driver_patch_rev,
+ data.capabilities.macs_per_cc,
+ data.capabilities.cmd_stream_version,
+ data.capabilities.custom_dma);
+
+ ret = ethosu_capability_rsp(edev, &data.capabilities);
+ break;
+ default:
+ /* This should not happen due to version checks */
+ dev_warn(edev->dev, "Msg: Protocol error\n");
+ ret = -EPROTO;
+ break;
+ }
+
+ return ret;
+}
+
+static int ethosu_open(struct inode *inode,
+ struct file *file)
+{
+ struct ethosu_device *edev =
+ container_of(inode->i_cdev, struct ethosu_device, cdev);
+
+ file->private_data = edev;
+
+ dev_info(edev->dev, "Opening device node.\n");
+
+ return nonseekable_open(inode, file);
+}
+
+static int ethosu_send_capabilities_request(struct ethosu_device *edev,
+ void __user *udata)
+{
+ struct ethosu_uapi_device_capabilities uapi;
+ struct ethosu_capabilities *cap;
+ int ret;
+ int timeout;
+
+ cap = devm_kzalloc(edev->dev, sizeof(struct ethosu_capabilities),
+ GFP_KERNEL);
+ if (!cap)
+ return -ENOMEM;
+
+ cap->edev = edev;
+ cap->capabilities = &uapi;
+ kref_init(&cap->refcount);
+ init_completion(&cap->done);
+ list_add(&cap->list, &edev->capabilities_list);
+
+ ret = ethosu_mailbox_capabilities_request(&edev->mailbox, cap);
+ if (0 != ret)
+ goto put_kref;
+
+ /*
+ * Increase ref counter since we sent the pointer out to
+ * response handler thread. That thread is responsible to
+ * decrease the ref counter before exiting. So the memory
+ * can be freed.
+ *
+ * NOTE: if no response is received back, the memory is leaked.
+ */
+ kref_get(&cap->refcount);
+ /* Unlock the mutex before going to block on the condition */
+ mutex_unlock(&edev->mutex);
+ /* wait for response to arrive back */
+ timeout = wait_for_completion_timeout(&cap->done,
+ msecs_to_jiffies(
+ CAPABILITIES_RESP_TIMEOUT_MS));
+ /* take back the mutex before resuming to do anything */
+ ret = mutex_lock_interruptible(&edev->mutex);
+ if (0 != ret)
+ goto put_kref;
+
+ if (0 == timeout /* timed out*/) {
+ dev_warn(edev->dev,
+ "Msg: Capabilities response lost - timeout\n");
+ ret = -EIO;
+ goto put_kref;
+ }
+
+ ret = copy_to_user(udata, &uapi, sizeof(uapi)) ? -EFAULT : 0;
+
+put_kref:
+ kref_put(&cap->refcount, ethosu_capabilities_destroy);
+
+ return ret;
+}
+
+static long ethosu_ioctl(struct file *file,
+ unsigned int cmd,
+ unsigned long arg)
+{
+ struct ethosu_device *edev = file->private_data;
+ void __user *udata = (void __user *)arg;
+ int ret = -EINVAL;
+
+ ret = mutex_lock_interruptible(&edev->mutex);
+ if (ret)
+ return ret;
+
+ dev_info(edev->dev, "Ioctl. cmd=%u, arg=%lu\n", cmd, arg);
+
+ switch (cmd) {
+ case ETHOSU_IOCTL_VERSION_REQ:
+ dev_info(edev->dev, "Ioctl: Send version request\n");
+ ret = ethosu_mailbox_version_request(&edev->mailbox);
+ break;
+ case ETHOSU_IOCTL_CAPABILITIES_REQ:
+ dev_info(edev->dev, "Ioctl: Send capabilities request\n");
+ ret = ethosu_send_capabilities_request(edev, udata);
+ break;
+ case ETHOSU_IOCTL_PING: {
+ dev_info(edev->dev, "Ioctl: Send ping\n");
+ ret = ethosu_mailbox_ping(&edev->mailbox);
+ break;
+ }
+ case ETHOSU_IOCTL_BUFFER_CREATE: {
+ struct ethosu_uapi_buffer_create uapi;
+
+ dev_info(edev->dev, "Ioctl: Buffer create\n");
+
+ if (copy_from_user(&uapi, udata, sizeof(uapi)))
+ break;
+
+ dev_info(edev->dev, "Ioctl: Buffer. capacity=%u\n",
+ uapi.capacity);
+
+ ret = ethosu_buffer_create(edev, uapi.capacity);
+ break;
+ }
+ case ETHOSU_IOCTL_NETWORK_CREATE: {
+ struct ethosu_uapi_network_create uapi;
+
+ if (copy_from_user(&uapi, udata, sizeof(uapi)))
+ break;
+
+ dev_info(edev->dev, "Ioctl: Network. fd=%u\n", uapi.fd);
+
+ ret = ethosu_network_create(edev, &uapi);
+ break;
+ }
+ default: {
+ dev_err(edev->dev, "Invalid ioctl. cmd=%u, arg=%lu",
+ cmd, arg);
+ break;
+ }
+ }
+
+ mutex_unlock(&edev->mutex);
+
+ return ret;
+}
+
+static void ethosu_mbox_rx(void *user_arg)
+{
+ struct ethosu_device *edev = user_arg;
+ int ret;
+
+ mutex_lock(&edev->mutex);
+
+ do {
+ ret = ethosu_handle_msg(edev);
+ if (ret && ret != -ENOMSG)
+ /* Need to start over in case of error, empty the queue
+ * by fast-forwarding read position to write position.
+ * */
+ ethosu_mailbox_reset(&edev->mailbox);
+ } while (ret == 0);
+
+ mutex_unlock(&edev->mutex);
+}
+
+int ethosu_dev_init(struct ethosu_device *edev,
+ struct device *dev,
+ struct class *class,
+ dev_t devt,
+ struct resource *in_queue,
+ struct resource *out_queue)
+{
+ static const struct file_operations fops = {
+ .owner = THIS_MODULE,
+ .open = ðosu_open,
+ .unlocked_ioctl = ðosu_ioctl,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl = ðosu_ioctl,
+#endif
+ };
+ struct device *sysdev;
+ int ret;
+
+ edev->dev = dev;
+ edev->class = class;
+ edev->devt = devt;
+ mutex_init(&edev->mutex);
+ INIT_LIST_HEAD(&edev->capabilities_list);
+ INIT_LIST_HEAD(&edev->inference_list);
+
+ ret = of_reserved_mem_device_init(edev->dev);
+ if (ret)
+ return ret;
+
+ dma_set_mask_and_coherent(edev->dev, DMA_BIT_MASK(DMA_ADDR_BITS));
+
+ ret = ethosu_mailbox_init(&edev->mailbox, dev, in_queue, out_queue,
+ ethosu_mbox_rx, edev);
+ if (ret)
+ goto release_reserved_mem;
+
+ cdev_init(&edev->cdev, &fops);
+ edev->cdev.owner = THIS_MODULE;
+
+ ret = cdev_add(&edev->cdev, edev->devt, 1);
+ if (ret) {
+ dev_err(edev->dev, "Failed to add character device.\n");
+ goto deinit_mailbox;
+ }
+
+ sysdev = device_create(edev->class, NULL, edev->devt, edev,
+ "ethosu%d", MINOR(edev->devt));
+ if (IS_ERR(sysdev)) {
+ dev_err(edev->dev, "Failed to create device.\n");
+ ret = PTR_ERR(sysdev);
+ goto del_cdev;
+ }
+
+ dev_info(edev->dev,
+ "Created Arm Ethos-U device. name=%s, major=%d, minor=%d\n",
+ dev_name(sysdev), MAJOR(edev->devt), MINOR(edev->devt));
+
+ return 0;
+
+del_cdev:
+ cdev_del(&edev->cdev);
+
+deinit_mailbox:
+ ethosu_mailbox_deinit(&edev->mailbox);
+
+release_reserved_mem:
+ of_reserved_mem_device_release(edev->dev);
+
+ return ret;
+}
+
+void ethosu_dev_deinit(struct ethosu_device *edev)
+{
+ ethosu_mailbox_deinit(&edev->mailbox);
+ device_destroy(edev->class, edev->cdev.dev);
+ cdev_del(&edev->cdev);
+ of_reserved_mem_device_release(edev->dev);
+
+ dev_info(edev->dev, "%s\n", __FUNCTION__);
+}
diff --git a/drivers/firmware/ethosu/ethosu_device.h b/drivers/firmware/ethosu/ethosu_device.h
new file mode 100644
index 000000000000..3afdda84862d
--- /dev/null
+++ b/drivers/firmware/ethosu/ethosu_device.h
@@ -0,0 +1,88 @@
+/*
+ * (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
+ *
+ * This program is free software and is provided to you under the terms of the
+ * GNU General Public License version 2 as published by the Free Software
+ * Foundation, and any use by you of this program is subject to the terms
+ * of such GNU licence.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, you can access it online at
+ * http://www.gnu.org/licenses/gpl-2.0.html.
+ *
+ * SPDX-License-Identifier: GPL-2.0-only
+ */
+
+#ifndef ETHOSU_DEVICE_H
+#define ETHOSU_DEVICE_H
+
+/****************************************************************************
+ * Includes
+ ****************************************************************************/
+
+#include "uapi/ethosu.h"
+#include "ethosu_mailbox.h"
+
+#include <linux/device.h>
+#include <linux/cdev.h>
+#include <linux/io.h>
+#include <linux/mutex.h>
+#include <linux/workqueue.h>
+#include <linux/completion.h>
+
+/****************************************************************************
+ * Types
+ ****************************************************************************/
+
+/**
+ * struct ethosu_device - Device structure
+ */
+struct ethosu_device {
+ struct device *dev;
+ struct cdev cdev;
+ struct class *class;
+ dev_t devt;
+ struct mutex mutex;
+ struct ethosu_mailbox mailbox;
+ struct list_head capabilities_list;
+ struct list_head inference_list;
+};
+
+/**
+ * struct ethosu_capabilities - Capabilities internal struct
+ */
+struct ethosu_capabilities {
+ struct ethosu_device *edev;
+ struct completion done;
+ struct kref refcount;
+ struct ethosu_uapi_device_capabilities *capabilities;
+ struct list_head list;
+};
+
+/****************************************************************************
+ * Functions
+ ****************************************************************************/
+
+/**
+ * ethosu_dev_init() - Initialize the device
+ *
+ * Return: 0 on success, else error code.
+ */
+int ethosu_dev_init(struct ethosu_device *edev,
+ struct device *dev,
+ struct class *class,
+ dev_t devt,
+ struct resource *in_queue,
+ struct resource *out_queue);
+
+/**
+ * ethosu_dev_deinit() - Initialize the device
+ */
+void ethosu_dev_deinit(struct ethosu_device *edev);
+
+#endif /* ETHOSU_DEVICE_H */
diff --git a/drivers/firmware/ethosu/ethosu_driver.c b/drivers/firmware/ethosu/ethosu_driver.c
new file mode 100644
index 000000000000..9d02431d3194
--- /dev/null
+++ b/drivers/firmware/ethosu/ethosu_driver.c
@@ -0,0 +1,191 @@
+/*
+ * (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
+ *
+ * This program is free software and is provided to you under the terms of the
+ * GNU General Public License version 2 as published by the Free Software
+ * Foundation, and any use by you of this program is subject to the terms
+ * of such GNU licence.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, you can access it online at
+ * http://www.gnu.org/licenses/gpl-2.0.html.
+ *
+ * SPDX-License-Identifier: GPL-2.0-only
+ */
+
+#include <linux/bitmap.h>
+#include <linux/fs.h>
+#include <linux/module.h>
+#include <linux/io.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/platform_device.h>
+
+#include "ethosu_device.h"
+
+/****************************************************************************
+ * Defines
+ ****************************************************************************/
+
+#define ETHOSU_DRIVER_VERSION "1.0"
+#define ETHOSU_DRIVER_NAME "ethosu"
+
+#define MINOR_BASE 0 /* Minor version starts at 0 */
+#define MINOR_COUNT 64 /* Allocate minor versions */
+
+/****************************************************************************
+ * Variables
+ ****************************************************************************/
+
+static struct class *ethosu_class;
+
+static dev_t devt;
+
+static DECLARE_BITMAP(minors, MINOR_COUNT);
+
+/****************************************************************************
+ * Arm Ethos-U
+ ****************************************************************************/
+
+static int ethosu_pdev_probe(struct platform_device *pdev)
+{
+ struct ethosu_device *edev;
+ struct resource *in_queue_res;
+ struct resource *out_queue_res;
+ int minor;
+ int ret;
+
+ dev_info(&pdev->dev, "Probe\n");
+
+ minor = find_first_zero_bit(minors, MINOR_COUNT);
+ if (minor >= MINOR_COUNT) {
+ dev_err(&pdev->dev, "No more minor numbers.\n");
+
+ return -ENOMEM;
+ }
+
+ /* Get path to TCM memory */
+ in_queue_res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
+ "in_queue");
+ if (IS_ERR(in_queue_res)) {
+ dev_err(&pdev->dev, "Failed to get in_queue resource.\n");
+
+ return PTR_ERR(in_queue_res);
+ }
+
+ out_queue_res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
+ "out_queue");
+ if (IS_ERR(out_queue_res)) {
+ dev_err(&pdev->dev, "Failed to get out_queue resource.\n");
+
+ return PTR_ERR(out_queue_res);
+ }
+
+ /* Allocate memory for Arm Ethos-U device */
+ edev = devm_kzalloc(&pdev->dev, sizeof(*edev), GFP_KERNEL);
+ if (!edev)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, edev);
+
+ /* Initialize device */
+ ret = ethosu_dev_init(edev, &pdev->dev, ethosu_class,
+ MKDEV(MAJOR(devt), minor), in_queue_res,
+ out_queue_res);
+ if (ret)
+ goto free_dev;
+
+ set_bit(minor, minors);
+
+ return 0;
+
+free_dev:
+ devm_kfree(&pdev->dev, edev);
+
+ return ret;
+}
+
+static int ethosu_pdev_remove(struct platform_device *pdev)
+{
+ struct ethosu_device *edev = platform_get_drvdata(pdev);
+
+ clear_bit(MINOR(edev->devt), minors);
+ ethosu_dev_deinit(edev);
+
+ return 0;
+}
+
+static const struct of_device_id ethosu_pdev_match[] = {
+ { .compatible = "arm,ethosu" },
+ { /* Sentinel */ },
+};
+
+MODULE_DEVICE_TABLE(of, ethosu_pdev_match);
+
+static struct platform_driver ethosu_pdev_driver = {
+ .probe = ðosu_pdev_probe,
+ .remove = ðosu_pdev_remove,
+ .driver = {
+ .name = ETHOSU_DRIVER_NAME,
+ .owner = THIS_MODULE,
+ .of_match_table = of_match_ptr(ethosu_pdev_match),
+ },
+};
+
+/****************************************************************************
+ * Module init and exit
+ ****************************************************************************/
+
+static int __init ethosu_init(void)
+{
+ int ret;
+
+ ethosu_class = class_create(THIS_MODULE, ETHOSU_DRIVER_NAME);
+ if (IS_ERR(ethosu_class)) {
+ printk("Failed to create class '%s'.\n", ETHOSU_DRIVER_NAME);
+
+ return PTR_ERR(ethosu_class);
+ }
+
+ ret = alloc_chrdev_region(&devt, MINOR_BASE, MINOR_COUNT,
+ ETHOSU_DRIVER_NAME);
+ if (ret) {
+ printk("Failed to allocate chrdev region.\n");
+ goto destroy_class;
+ }
+
+ ret = platform_driver_register(ðosu_pdev_driver);
+ if (ret) {
+ printk("Failed to register Arm Ethos-U platform driver.\n");
+ goto region_unregister;
+ }
+
+ return 0;
+
+region_unregister:
+ unregister_chrdev_region(devt, MINOR_COUNT);
+
+destroy_class:
+ class_destroy(ethosu_class);
+
+ return ret;
+}
+
+static void __exit ethosu_exit(void)
+{
+ platform_driver_unregister(ðosu_pdev_driver);
+ unregister_chrdev_region(devt, MINOR_COUNT);
+ class_destroy(ethosu_class);
+}
+
+module_init(ethosu_init)
+module_exit(ethosu_exit)
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR("Arm Ltd");
+MODULE_DESCRIPTION("Arm Ethos-U NPU Driver");
+MODULE_VERSION(ETHOSU_DRIVER_VERSION);
diff --git a/drivers/firmware/ethosu/ethosu_inference.c b/drivers/firmware/ethosu/ethosu_inference.c
new file mode 100644
index 000000000000..6fde92c148a0
--- /dev/null
+++ b/drivers/firmware/ethosu/ethosu_inference.c
@@ -0,0 +1,417 @@
+/*
+ * (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
+ *
+ * This program is free software and is provided to you under the terms of the
+ * GNU General Public License version 2 as published by the Free Software
+ * Foundation, and any use by you of this program is subject to the terms
+ * of such GNU licence.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, you can access it online at
+ * http://www.gnu.org/licenses/gpl-2.0.html.
+ *
+ * SPDX-License-Identifier: GPL-2.0-only
+ */
+
+/****************************************************************************
+ * Includes
+ ****************************************************************************/
+
+#include "ethosu_inference.h"
+
+#include "ethosu_buffer.h"
+#include "ethosu_core_interface.h"
+#include "ethosu_device.h"
+#include "ethosu_network.h"
+#include "uapi/ethosu.h"
+
+#include <linux/anon_inodes.h>
+#include <linux/file.h>
+#include <linux/fs.h>
+#include <linux/poll.h>
+
+/****************************************************************************
+ * Variables
+ ****************************************************************************/
+
+static int ethosu_inference_release(struct inode *inode,
+ struct file *file);
+
+static unsigned int ethosu_inference_poll(struct file *file,
+ poll_table *wait);
+
+static long ethosu_inference_ioctl(struct file *file,
+ unsigned int cmd,
+ unsigned long arg);
+
+static const struct file_operations ethosu_inference_fops = {
+ .release = ðosu_inference_release,
+ .poll = ðosu_inference_poll,
+ .unlocked_ioctl = ðosu_inference_ioctl,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl = ðosu_inference_ioctl,
+#endif
+};
+
+/****************************************************************************
+ * Functions
+ ****************************************************************************/
+
+static const char *status_to_string(const enum ethosu_uapi_status status)
+{
+ switch (status) {
+ case ETHOSU_UAPI_STATUS_OK: {
+ return "Ok";
+ }
+ case ETHOSU_UAPI_STATUS_ERROR: {
+ return "Error";
+ }
+ default: {
+ return "Unknown";
+ }
+ }
+}
+
+static int ethosu_inference_send(struct ethosu_inference *inf)
+{
+ int ret;
+
+ if (inf->pending)
+ return -EINVAL;
+
+ inf->status = ETHOSU_UAPI_STATUS_ERROR;
+
+ ret = ethosu_mailbox_inference(&inf->edev->mailbox, inf,
+ inf->ifm_count, inf->ifm,
+ inf->ofm_count, inf->ofm,
+ inf->net->buf,
+ inf->pmu_event_config,
+ ETHOSU_PMU_EVENT_MAX,
+ inf->pmu_cycle_counter_enable);
+ if (ret)
+ return ret;
+
+ inf->pending = true;
+
+ ethosu_inference_get(inf);
+
+ return 0;
+}
+
+static int ethosu_inference_find(struct ethosu_inference *inf,
+ struct list_head *inference_list)
+{
+ struct ethosu_inference *cur;
+
+ list_for_each_entry(cur, inference_list, list) {
+ if (cur == inf)
+ return 0;
+ }
+
+ return -EINVAL;
+}
+
+static bool ethosu_inference_verify(struct file *file)
+{
+ return file->f_op == ðosu_inference_fops;
+}
+
+static void ethosu_inference_kref_destroy(struct kref *kref)
+{
+ struct ethosu_inference *inf =
+ container_of(kref, struct ethosu_inference, kref);
+
+ dev_info(inf->edev->dev,
+ "Inference destroy. handle=0x%pK, status=%d\n",
+ inf, inf->status);
+
+ list_del(&inf->list);
+
+ while (inf->ifm_count-- > 0)
+ ethosu_buffer_put(inf->ifm[inf->ifm_count]);
+
+ while (inf->ofm_count-- > 0)
+ ethosu_buffer_put(inf->ofm[inf->ofm_count]);
+
+ ethosu_network_put(inf->net);
+ devm_kfree(inf->edev->dev, inf);
+}
+
+static int ethosu_inference_release(struct inode *inode,
+ struct file *file)
+{
+ struct ethosu_inference *inf = file->private_data;
+
+ dev_info(inf->edev->dev,
+ "Inference release. handle=0x%pK, status=%d\n",
+ inf, inf->status);
+
+ ethosu_inference_put(inf);
+
+ return 0;
+}
+
+static unsigned int ethosu_inference_poll(struct file *file,
+ poll_table *wait)
+{
+ struct ethosu_inference *inf = file->private_data;
+ int ret = 0;
+
+ poll_wait(file, &inf->waitq, wait);
+
+ if (!inf->pending)
+ ret |= POLLIN;
+
+ return ret;
+}
+
+static long ethosu_inference_ioctl(struct file *file,
+ unsigned int cmd,
+ unsigned long arg)
+{
+ struct ethosu_inference *inf = file->private_data;
+ void __user *udata = (void __user *)arg;
+ int ret;
+
+ ret = mutex_lock_interruptible(&inf->edev->mutex);
+ if (ret)
+ return ret;
+
+ dev_info(inf->edev->dev, "Ioctl: cmd=%u, arg=%lu\n", cmd, arg);
+
+ switch (cmd) {
+ case ETHOSU_IOCTL_INFERENCE_STATUS: {
+ struct ethosu_uapi_result_status uapi;
+ int i;
+
+ uapi.status = inf->status;
+
+ for (i = 0; i < ETHOSU_PMU_EVENT_MAX; i++) {
+ uapi.pmu_config.events[i] =
+ inf->pmu_event_config[i];
+ uapi.pmu_count.events[i] =
+ inf->pmu_event_count[i];
+ }
+
+ uapi.pmu_config.cycle_count = inf->pmu_cycle_counter_enable;
+ uapi.pmu_count.cycle_count = inf->pmu_cycle_counter_count;
+
+ dev_info(inf->edev->dev,
+ "Ioctl: Inference status. status=%s (%d)\n",
+ status_to_string(uapi.status), uapi.status);
+
+ ret = copy_to_user(udata, &uapi, sizeof(uapi)) ? -EFAULT : 0;
+
+ break;
+ }
+ default: {
+ dev_err(inf->edev->dev, "Invalid ioctl. cmd=%u, arg=%lu",
+ cmd, arg);
+ break;
+ }
+ }
+
+ mutex_unlock(&inf->edev->mutex);
+
+ return ret;
+}
+
+int ethosu_inference_create(struct ethosu_device *edev,
+ struct ethosu_network *net,
+ struct ethosu_uapi_inference_create *uapi)
+{
+ struct ethosu_inference *inf;
+ uint32_t i;
+ int fd;
+ int ret = -ENOMEM;
+
+ inf = devm_kzalloc(edev->dev, sizeof(*inf), GFP_KERNEL);
+ if (!inf)
+ return -ENOMEM;
+
+ inf->edev = edev;
+ inf->net = net;
+ inf->pending = false;
+ inf->status = ETHOSU_UAPI_STATUS_ERROR;
+ kref_init(&inf->kref);
+ init_waitqueue_head(&inf->waitq);
+
+ /* Get pointer to IFM buffers */
+ for (i = 0; i < uapi->ifm_count; i++) {
+ inf->ifm[i] = ethosu_buffer_get_from_fd(uapi->ifm_fd[i]);
+ if (IS_ERR(inf->ifm[i])) {
+ ret = PTR_ERR(inf->ifm[i]);
+ goto put_ifm;
+ }
+
+ inf->ifm_count++;
+ }
+
+ /* Get pointer to OFM buffer */
+ for (i = 0; i < uapi->ofm_count; i++) {
+ inf->ofm[i] = ethosu_buffer_get_from_fd(uapi->ofm_fd[i]);
+ if (IS_ERR(inf->ofm[i])) {
+ ret = PTR_ERR(inf->ofm[i]);
+ goto put_ofm;
+ }
+
+ inf->ofm_count++;
+ }
+
+ /* Configure PMU and cycle counter */
+ dev_info(inf->edev->dev,
+ "Configuring events for PMU. events=[%u, %u, %u, %u]\n",
+ uapi->pmu_config.events[0], uapi->pmu_config.events[1],
+ uapi->pmu_config.events[2], uapi->pmu_config.events[3]);
+
+ /* Configure events and reset count for all events */
+ for (i = 0; i < ETHOSU_PMU_EVENT_MAX; i++) {
+ inf->pmu_event_config[i] = uapi->pmu_config.events[i];
+ inf->pmu_event_count[i] = 0;
+ }
+
+ if (uapi->pmu_config.cycle_count)
+ dev_info(inf->edev->dev, "Enabling cycle counter\n");
+
+ /* Configure cycle counter and reset any previous count */
+ inf->pmu_cycle_counter_enable = uapi->pmu_config.cycle_count;
+ inf->pmu_cycle_counter_count = 0;
+
+ /* Increment network reference count */
+ ethosu_network_get(net);
+
+ /* Create file descriptor */
+ ret = fd = anon_inode_getfd("ethosu-inference", ðosu_inference_fops,
+ inf, O_RDWR | O_CLOEXEC);
+ if (ret < 0)
+ goto put_net;
+
+ /* Store pointer to file structure */
+ inf->file = fget(ret);
+ fput(inf->file);
+
+ /* Add inference to inference list */
+ list_add(&inf->list, &edev->inference_list);
+
+ /* Send inference request to Arm Ethos-U subsystem */
+ (void)ethosu_inference_send(inf);
+
+ dev_info(edev->dev, "Inference create. handle=0x%pK, fd=%d",
+ inf, fd);
+
+ return fd;
+
+put_net:
+ ethosu_network_put(inf->net);
+
+put_ofm:
+ while (inf->ofm_count-- > 0)
+ ethosu_buffer_put(inf->ofm[inf->ofm_count]);
+
+put_ifm:
+ while (inf->ifm_count-- > 0)
+ ethosu_buffer_put(inf->ifm[inf->ifm_count]);
+
+ devm_kfree(edev->dev, inf);
+
+ return ret;
+}
+
+struct ethosu_inference *ethosu_inference_get_from_fd(int fd)
+{
+ struct ethosu_inference *inf;
+ struct file *file;
+
+ file = fget(fd);
+ if (!file)
+ return ERR_PTR(-EINVAL);
+
+ if (!ethosu_inference_verify(file)) {
+ fput(file);
+
+ return ERR_PTR(-EINVAL);
+ }
+
+ inf = file->private_data;
+ ethosu_inference_get(inf);
+ fput(file);
+
+ return inf;
+}
+
+void ethosu_inference_get(struct ethosu_inference *inf)
+{
+ kref_get(&inf->kref);
+}
+
+void ethosu_inference_put(struct ethosu_inference *inf)
+{
+ kref_put(&inf->kref, ðosu_inference_kref_destroy);
+}
+
+void ethosu_inference_rsp(struct ethosu_device *edev,
+ struct ethosu_core_inference_rsp *rsp)
+{
+ struct ethosu_inference *inf =
+ (struct ethosu_inference *)rsp->user_arg;
+ int ret;
+ int i;
+
+ ret = ethosu_inference_find(inf, &edev->inference_list);
+ if (ret) {
+ dev_warn(edev->dev,
+ "Handle not found in inference list. handle=0x%p\n",
+ rsp);
+
+ return;
+ }
+
+ inf->pending = false;
+
+ if (rsp->status == ETHOSU_CORE_STATUS_OK &&
+ inf->ofm_count <= ETHOSU_CORE_BUFFER_MAX) {
+ uint32_t i;
+
+ inf->status = ETHOSU_UAPI_STATUS_OK;
+
+ for (i = 0; i < inf->ofm_count; i++) {
+ struct ethosu_buffer *ofm = inf->ofm[i];
+
+ ret = ethosu_buffer_resize(
+ ofm, ofm->size + rsp->ofm_size[i],
+ ofm->offset);
+ if (ret)
+ inf->status = ETHOSU_UAPI_STATUS_ERROR;
+ }
+ } else {
+ inf->status = ETHOSU_UAPI_STATUS_ERROR;
+ }
+
+ for (i = 0; i < ETHOSU_CORE_PMU_MAX; i++) {
+ inf->pmu_event_config[i] = rsp->pmu_event_config[i];
+ inf->pmu_event_count[i] = rsp->pmu_event_count[i];
+ }
+
+ inf->pmu_cycle_counter_enable = rsp->pmu_cycle_counter_enable;
+ inf->pmu_cycle_counter_count = rsp->pmu_cycle_counter_count;
+
+ dev_info(edev->dev,
+ "PMU events. config=[%u, %u, %u, %u], count=[%u, %u, %u, %u]\n",
+ inf->pmu_event_config[0], inf->pmu_event_config[1],
+ inf->pmu_event_config[2], inf->pmu_event_config[3],
+ inf->pmu_event_count[0], inf->pmu_event_count[1],
+ inf->pmu_event_count[2], inf->pmu_event_count[3]);
+
+ dev_info(edev->dev,
+ "PMU cycle counter. enable=%u, count=%llu\n",
+ inf->pmu_cycle_counter_enable,
+ inf->pmu_cycle_counter_count);
+ wake_up_interruptible(&inf->waitq);
+
+ ethosu_inference_put(inf);
+}
diff --git a/drivers/firmware/ethosu/ethosu_inference.h b/drivers/firmware/ethosu/ethosu_inference.h
new file mode 100644
index 000000000000..07370ca01f22
--- /dev/null
+++ b/drivers/firmware/ethosu/ethosu_inference.h
@@ -0,0 +1,120 @@
+/*
+ * (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
+ *
+ * This program is free software and is provided to you under the terms of the
+ * GNU General Public License version 2 as published by the Free Software
+ * Foundation, and any use by you of this program is subject to the terms
+ * of such GNU licence.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, you can access it online at
+ * http://www.gnu.org/licenses/gpl-2.0.html.
+ *
+ * SPDX-License-Identifier: GPL-2.0-only
+ */
+
+#ifndef ETHOSU_INFERENCE_H
+#define ETHOSU_INFERENCE_H
+
+/****************************************************************************
+ * Includes
+ ****************************************************************************/
+
+#include "uapi/ethosu.h"
+
+#include <linux/kref.h>
+#include <linux/types.h>
+#include <linux/wait.h>
+
+/****************************************************************************
+ * Types
+ ****************************************************************************/
+
+struct ethosu_buffer;
+struct ethosu_core_inference_rsp;
+struct ethosu_device;
+struct ethosu_network;
+struct ethosu_uapi_inference_create;
+struct file;
+
+/**
+ * struct ethosu_inference - Inference struct
+ * @edev: Arm Ethos-U device
+ * @file: File handle
+ * @kref: Reference counter
+ * @waitq: Wait queue
+ * @ifm: Pointer to IFM buffer
+ * @ofm: Pointer to OFM buffer
+ * @net: Pointer to network
+ * @pending: Pending response from the firmware
+ * @status: Inference status
+ * @pmu_event_config: PMU event configuration
+ * @pmu_event_count: PMU event count after inference
+ * @pmu_cycle_counter_enable: PMU cycle counter config
+ * @pmu_cycle_counter_count: PMU cycle counter count after inference
+ */
+struct ethosu_inference {
+ struct ethosu_device *edev;
+ struct file *file;
+ struct kref kref;
+ wait_queue_head_t waitq;
+ uint32_t ifm_count;
+ struct ethosu_buffer *ifm[ETHOSU_FD_MAX];
+ uint32_t ofm_count;
+ struct ethosu_buffer *ofm[ETHOSU_FD_MAX];
+ struct ethosu_network *net;
+ bool pending;
+ enum ethosu_uapi_status status;
+ uint8_t pmu_event_config[ETHOSU_PMU_EVENT_MAX];
+ uint32_t pmu_event_count[ETHOSU_PMU_EVENT_MAX];
+ uint32_t pmu_cycle_counter_enable;
+ uint64_t pmu_cycle_counter_count;
+ struct list_head list;
+};
+
+/****************************************************************************
+ * Functions
+ ****************************************************************************/
+
+/**
+ * ethosu_inference_create() - Create inference
+ *
+ * This function must be called in the context of a user space process.
+ *
+ * Return: fd on success, else error code.
+ */
+int ethosu_inference_create(struct ethosu_device *edev,
+ struct ethosu_network *net,
+ struct ethosu_uapi_inference_create *uapi);
+
+/**
+ * ethosu_inference_get_from_fd() - Get inference handle from fd
+ *
+ * This function must be called from a user space context.
+ *
+ * Return: Pointer on success, else ERR_PTR.
+ */
+struct ethosu_inference *ethosu_inference_get_from_fd(int fd);
+
+/**
+ * ethosu_inference_get() - Get inference
+ */
+void ethosu_inference_get(struct ethosu_inference *inf);
+
+/**
+ * ethosu_inference_put() - Put inference
+ */
+void ethosu_inference_put(struct ethosu_inference *inf);
+
+/**
+ * ethosu_inference_rsp() - Handle inference response
+ */
+void ethosu_inference_rsp(struct ethosu_device *edev,
+ struct ethosu_core_inference_rsp *rsp);
+
+#endif /* ETHOSU_INFERENCE_H */
diff --git a/drivers/firmware/ethosu/ethosu_mailbox.c b/drivers/firmware/ethosu/ethosu_mailbox.c
new file mode 100644
index 000000000000..7f159f3b0a60
--- /dev/null
+++ b/drivers/firmware/ethosu/ethosu_mailbox.c
@@ -0,0 +1,377 @@
+/*
+ * (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
+ *
+ * This program is free software and is provided to you under the terms of the
+ * GNU General Public License version 2 as published by the Free Software
+ * Foundation, and any use by you of this program is subject to the terms
+ * of such GNU licence.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, you can access it online at
+ * http://www.gnu.org/licenses/gpl-2.0.html.
+ *
+ * SPDX-License-Identifier: GPL-2.0-only
+ */
+
+/****************************************************************************
+ * Includes
+ ****************************************************************************/
+
+#include "ethosu_mailbox.h"
+
+#include "ethosu_buffer.h"
+#include "ethosu_core_interface.h"
+#include "ethosu_device.h"
+
+#include <linux/resource.h>
+#include <linux/uio.h>
+
+/****************************************************************************
+ * Functions
+ ****************************************************************************/
+
+static void ethosu_core_set_size(struct ethosu_buffer *buf,
+ struct ethosu_core_buffer *cbuf)
+{
+ cbuf->ptr = (uint32_t)buf->dma_addr + buf->offset;
+ cbuf->size = (uint32_t)buf->size;
+}
+
+static void ethosu_core_set_capacity(struct ethosu_buffer *buf,
+ struct ethosu_core_buffer *cbuf)
+{
+ cbuf->ptr = (uint32_t)buf->dma_addr + buf->offset + buf->size;
+ cbuf->size = (uint32_t)buf->capacity - buf->offset - buf->size;
+}
+
+static size_t ethosu_queue_available(struct ethosu_core_queue *queue)
+{
+ size_t size = queue->header.write - queue->header.read;
+
+ if (queue->header.read > queue->header.write)
+ size += queue->header.size;
+
+ return size;
+}
+
+static size_t ethosu_queue_capacity(struct ethosu_core_queue *queue)
+{
+ return queue->header.size - ethosu_queue_available(queue);
+}
+
+static int ethosu_queue_write(struct ethosu_mailbox *mbox,
+ const struct kvec *vec,
+ size_t length)
+{
+ struct ethosu_core_queue *queue = mbox->in_queue;
+ uint8_t *dst = &queue->data[0];
+ uint32_t wpos = queue->header.write;
+ size_t total_size;
+ size_t i;
+ int ret;
+
+ for (i = 0, total_size = 0; i < length; i++)
+ total_size += vec[i].iov_len;
+
+ if (total_size > ethosu_queue_capacity(queue))
+ return -EINVAL;
+
+ for (i = 0; i < length; i++) {
+ const uint8_t *src = vec[i].iov_base;
+ const uint8_t *end = src + vec[i].iov_len;
+
+ while (src < end) {
+ dst[wpos] = *src++;
+ wpos = (wpos + 1) % queue->header.size;
+ }
+ }
+
+ queue->header.write = wpos;
+
+ ret = mbox_send_message(mbox->tx, queue);
+ if (ret < 0)
+ return ret;
+
+ return 0;
+}
+
+static int ethosu_queue_write_msg(struct ethosu_mailbox *mbox,
+ uint32_t type,
+ void *data,
+ size_t length)
+{
+ struct ethosu_core_msg msg = {
+ .magic = ETHOSU_CORE_MSG_MAGIC,
+ .type = type, .length= length
+ };
+ const struct kvec vec[2] = {
+ { &msg, sizeof(msg) },
+ { data, length }
+ };
+
+ return ethosu_queue_write(mbox, vec, 2);
+}
+
+static int ethosu_queue_read(struct ethosu_mailbox *mbox,
+ void *data,
+ size_t length)
+{
+ struct ethosu_core_queue *queue = mbox->out_queue;
+ uint8_t *src = &queue->data[0];
+ uint8_t *dst = (uint8_t *)data;
+ const uint8_t *end = dst + length;
+ uint32_t rpos = queue->header.read;
+ size_t queue_avail = ethosu_queue_available(queue);
+
+ if (length == 0)
+ return 0;
+ else if (queue_avail == 0)
+ return -ENOMSG;
+ else if (length > queue_avail)
+ return -EBADMSG;
+
+ while (dst < end) {
+ *dst++ = src[rpos];
+ rpos = (rpos + 1) % queue->header.size;
+ }
+
+ queue->header.read = rpos;
+
+ return 0;
+}
+
+void ethosu_mailbox_reset(struct ethosu_mailbox *mbox)
+{
+ mbox->out_queue->header.read = mbox->out_queue->header.write;
+}
+
+int ethosu_mailbox_read(struct ethosu_mailbox *mbox,
+ struct ethosu_core_msg *header,
+ void *data,
+ size_t length)
+{
+ int ret;
+
+ /* Read message header magic */
+ ret = ethosu_queue_read(mbox, header, sizeof(*header));
+ if (ret) {
+ if (ret != -ENOMSG)
+ dev_warn(mbox->dev,
+ "Msg: Failed to read message header\n");
+
+ return ret;
+ }
+
+ if (header->magic != ETHOSU_CORE_MSG_MAGIC) {
+ dev_warn(mbox->dev,
+ "Msg: Invalid magic. Got: %08X but expected %08X\n",
+ header->magic, ETHOSU_CORE_MSG_MAGIC);
+
+ return -EINVAL;
+ }
+
+ dev_info(mbox->dev,
+ "mbox: Read msg header. magic=%08X, type=%u, length=%u",
+ header->magic, header->type, header->length);
+
+ /* Check that payload is not larger than allocated buffer */
+ if (header->length > length) {
+ dev_warn(mbox->dev,
+ "Msg: Buffer size (%zu) too small for message (%u)\n",
+ sizeof(data), header->length);
+
+ return -ENOMEM;
+ }
+
+ /* Read payload data */
+ ret = ethosu_queue_read(mbox, data, header->length);
+ if (ret) {
+ dev_warn(mbox->dev, "Msg: Failed to read payload data\n");
+
+ return -EBADMSG;
+ }
+
+ return 0;
+}
+
+int ethosu_mailbox_ping(struct ethosu_mailbox *mbox)
+{
+ return ethosu_queue_write_msg(mbox, ETHOSU_CORE_MSG_PING, NULL, 0);
+}
+
+int ethosu_mailbox_pong(struct ethosu_mailbox *mbox)
+{
+ return ethosu_queue_write_msg(mbox, ETHOSU_CORE_MSG_PONG, NULL, 0);
+}
+
+int ethosu_mailbox_version_request(struct ethosu_mailbox *mbox)
+{
+ return ethosu_queue_write_msg(mbox, ETHOSU_CORE_MSG_VERSION_REQ, NULL,
+ 0);
+}
+
+int ethosu_mailbox_capabilities_request(struct ethosu_mailbox *mbox,
+ void *user_arg)
+{
+ struct ethosu_core_capabilities_req req = {
+ .user_arg = (ptrdiff_t)user_arg
+ };
+
+ return ethosu_queue_write_msg(mbox, ETHOSU_CORE_MSG_CAPABILITIES_REQ,
+ &req,
+ sizeof(req));
+}
+
+int ethosu_mailbox_inference(struct ethosu_mailbox *mbox,
+ void *user_arg,
+ uint32_t ifm_count,
+ struct ethosu_buffer **ifm,
+ uint32_t ofm_count,
+ struct ethosu_buffer **ofm,
+ struct ethosu_buffer *network,
+ uint8_t *pmu_event_config,
+ uint8_t pmu_event_config_count,
+ uint8_t pmu_cycle_counter_enable)
+{
+ struct ethosu_core_inference_req inf;
+ uint32_t i;
+
+ /* Verify that the uapi and core has the same number of pmus */
+ if (pmu_event_config_count != ETHOSU_CORE_PMU_MAX) {
+ dev_err(mbox->dev, "PMU count misconfigured.\n");
+
+ return -EINVAL;
+ }
+
+ inf.user_arg = (ptrdiff_t)user_arg;
+ inf.ifm_count = ifm_count;
+ inf.ofm_count = ofm_count;
+ inf.pmu_cycle_counter_enable = pmu_cycle_counter_enable;
+
+ for (i = 0; i < ifm_count; i++)
+ ethosu_core_set_size(ifm[i], &inf.ifm[i]);
+
+ for (i = 0; i < ofm_count; i++)
+ ethosu_core_set_capacity(ofm[i], &inf.ofm[i]);
+
+ for (i = 0; i < ETHOSU_CORE_PMU_MAX; i++)
+ inf.pmu_event_config[i] = pmu_event_config[i];
+
+ ethosu_core_set_size(network, &inf.network);
+
+ return ethosu_queue_write_msg(mbox, ETHOSU_CORE_MSG_INFERENCE_REQ,
+ &inf, sizeof(inf));
+}
+
+static void ethosu_mailbox_rx_work(struct work_struct *work)
+{
+ struct ethosu_mailbox *mbox = container_of(work, typeof(*mbox), work);
+
+ mbox->callback(mbox->user_arg);
+}
+
+static void ethosu_mailbox_rx_callback(struct mbox_client *client,
+ void *message)
+{
+ struct ethosu_mailbox *mbox =
+ container_of(client, typeof(*mbox), client);
+
+ dev_info(mbox->dev, "mbox: Received message.\n");
+
+ queue_work(mbox->wq, &mbox->work);
+}
+
+static void ethosu_mailbox_tx_done(struct mbox_client *client,
+ void *message,
+ int r)
+{
+ if (r)
+ dev_warn(client->dev, "mbox: Failed sending message (%d)\n", r);
+ else
+ dev_info(client->dev, "mbox: Message sent\n");
+}
+
+int ethosu_mailbox_init(struct ethosu_mailbox *mbox,
+ struct device *dev,
+ struct resource *in_queue,
+ struct resource *out_queue,
+ ethosu_mailbox_cb callback,
+ void *user_arg)
+{
+ int ret;
+
+ mbox->dev = dev;
+ mbox->callback = callback;
+ mbox->user_arg = user_arg;
+
+ mbox->client.dev = dev;
+ mbox->client.rx_callback = ethosu_mailbox_rx_callback;
+ mbox->client.tx_prepare = NULL; /* preparation of data is handled
+ * through the
+ * queue functions */
+ mbox->client.tx_done = ethosu_mailbox_tx_done;
+ mbox->client.tx_block = true;
+ mbox->client.knows_txdone = false;
+ mbox->client.tx_tout = 500;
+
+ mbox->in_queue = devm_ioremap_resource(mbox->dev, in_queue);
+ if (IS_ERR(mbox->in_queue))
+ return PTR_ERR(mbox->in_queue);
+
+ mbox->out_queue = devm_ioremap_resource(mbox->dev, out_queue);
+ if (IS_ERR(mbox->out_queue)) {
+ ret = PTR_ERR(mbox->out_queue);
+ goto unmap_in_queue;
+ }
+
+ mbox->wq = create_singlethread_workqueue("ethosu_workqueue");
+ if (!mbox->wq) {
+ dev_err(mbox->dev, "Failed to create work queue\n");
+ ret = -EINVAL;
+ goto unmap_out_queue;
+ }
+
+ INIT_WORK(&mbox->work, ethosu_mailbox_rx_work);
+
+ mbox->tx = mbox_request_channel_byname(&mbox->client, "tx");
+ if (IS_ERR(mbox->tx)) {
+ dev_warn(mbox->dev, "mbox: Failed to request tx channel\n");
+ ret = PTR_ERR(mbox->tx);
+ goto workqueue_destroy;
+ }
+
+ mbox->rx = mbox_request_channel_byname(&mbox->client, "rx");
+ if (IS_ERR(mbox->rx)) {
+ dev_info(dev, "mbox: Using same channel for RX and TX\n");
+ mbox->rx = mbox->tx;
+ }
+
+ return 0;
+
+workqueue_destroy:
+ destroy_workqueue(mbox->wq);
+
+unmap_out_queue:
+ devm_iounmap(mbox->dev, mbox->out_queue);
+
+unmap_in_queue:
+ devm_iounmap(mbox->dev, mbox->in_queue);
+
+ return ret;
+}
+
+void ethosu_mailbox_deinit(struct ethosu_mailbox *mbox)
+{
+ if (mbox->rx != mbox->tx)
+ mbox_free_channel(mbox->rx);
+
+ mbox_free_channel(mbox->tx);
+ destroy_workqueue(mbox->wq);
+ devm_iounmap(mbox->dev, mbox->out_queue);
+ devm_iounmap(mbox->dev, mbox->in_queue);
+}
diff --git a/drivers/firmware/ethosu/ethosu_mailbox.h b/drivers/firmware/ethosu/ethosu_mailbox.h
new file mode 100644
index 000000000000..5cd5e62198b8
--- /dev/null
+++ b/drivers/firmware/ethosu/ethosu_mailbox.h
@@ -0,0 +1,140 @@
+/*
+ * (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
+ *
+ * This program is free software and is provided to you under the terms of the
+ * GNU General Public License version 2 as published by the Free Software
+ * Foundation, and any use by you of this program is subject to the terms
+ * of such GNU licence.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, you can access it online at
+ * http://www.gnu.org/licenses/gpl-2.0.html.
+ *
+ * SPDX-License-Identifier: GPL-2.0-only
+ */
+
+#ifndef ETHOSU_MAILBOX_H
+#define ETHOSU_MAILBOX_H
+
+/****************************************************************************
+ * Includes
+ ****************************************************************************/
+#include "ethosu_core_interface.h"
+
+#include <linux/types.h>
+#include <linux/mailbox_client.h>
+#include <linux/workqueue.h>
+
+/****************************************************************************
+ * Types
+ ****************************************************************************/
+
+struct device;
+struct ethosu_buffer;
+struct ethosu_device;
+struct ethosu_core_msg;
+struct ethosu_core_queue;
+struct resource;
+
+typedef void (*ethosu_mailbox_cb)(void *user_arg);
+
+struct ethosu_mailbox {
+ struct device *dev;
+ struct workqueue_struct *wq;
+ struct work_struct work;
+ struct ethosu_core_queue __iomem *in_queue;
+ struct ethosu_core_queue __iomem *out_queue;
+ struct mbox_client client;
+ struct mbox_chan *rx;
+ struct mbox_chan *tx;
+ ethosu_mailbox_cb callback;
+ void *user_arg;
+};
+
+/****************************************************************************
+ * Functions
+ ****************************************************************************/
+
+/**
+ * ethosu_mailbox_init() - Initialize mailbox
+ *
+ * Return: 0 on success, else error code.
+ */
+int ethosu_mailbox_init(struct ethosu_mailbox *mbox,
+ struct device *dev,
+ struct resource *in_queue,
+ struct resource *out_queue,
+ ethosu_mailbox_cb callback,
+ void *user_arg);
+
+/**
+ * ethosu_mailbox_deinit() - Deinitialize mailbox
+ */
+void ethosu_mailbox_deinit(struct ethosu_mailbox *mbox);
+
+/**
+ * ethosu_mailbox_read() - Read message from mailbox
+ *
+ * Return: 0 message read, else error code.
+ */
+int ethosu_mailbox_read(struct ethosu_mailbox *mbox,
+ struct ethosu_core_msg *header,
+ void *data,
+ size_t length);
+
+/**
+ * ethosu_mailbox_reset() - Reset to end of queue
+ */
+void ethosu_mailbox_reset(struct ethosu_mailbox *mbox);
+
+/**
+ * ethosu_mailbox_ping() - Send ping message
+ *
+ * Return: 0 on success, else error code.
+ */
+int ethosu_mailbox_ping(struct ethosu_mailbox *mbox);
+
+/**
+ * ethosu_mailbox_pong() - Send pong response
+ *
+ * Return: 0 on success, else error code.
+ */
+int ethosu_mailbox_pong(struct ethosu_mailbox *mbox);
+
+/**
+ * ethosu_mailbox_version_response - Send version request
+ *
+ * Return: 0 on succes, else error code
+ */
+int ethosu_mailbox_version_request(struct ethosu_mailbox *mbox);
+
+/**
+ * ethosu_mailbox_capabilities_request() - Send capabilities request
+ *
+ * Return: 0 on success, else error code.
+ */
+int ethosu_mailbox_capabilities_request(struct ethosu_mailbox *mbox,
+ void *user_arg);
+
+/**
+ * ethosu_mailbox_inference() - Send inference
+ *
+ * Return: 0 on success, else error code.
+ */
+int ethosu_mailbox_inference(struct ethosu_mailbox *mbox,
+ void *user_arg,
+ uint32_t ifm_count,
+ struct ethosu_buffer **ifm,
+ uint32_t ofm_count,
+ struct ethosu_buffer **ofm,
+ struct ethosu_buffer *network,
+ uint8_t *pmu_event_config,
+ uint8_t pmu_event_config_count,
+ uint8_t pmu_cycle_counter_enable);
+
+#endif /* ETHOSU_MAILBOX_H */
diff --git a/drivers/firmware/ethosu/ethosu_network.c b/drivers/firmware/ethosu/ethosu_network.c
new file mode 100644
index 000000000000..4d68f0537e38
--- /dev/null
+++ b/drivers/firmware/ethosu/ethosu_network.c
@@ -0,0 +1,201 @@
+/*
+ * (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
+ *
+ * This program is free software and is provided to you under the terms of the
+ * GNU General Public License version 2 as published by the Free Software
+ * Foundation, and any use by you of this program is subject to the terms
+ * of such GNU licence.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, you can access it online at
+ * http://www.gnu.org/licenses/gpl-2.0.html.
+ *
+ * SPDX-License-Identifier: GPL-2.0-only
+ */
+
+/****************************************************************************
+ * Includes
+ ****************************************************************************/
+
+#include "ethosu_network.h"
+
+#include "ethosu_buffer.h"
+#include "ethosu_device.h"
+#include "ethosu_inference.h"
+#include "uapi/ethosu.h"
+
+#include <linux/anon_inodes.h>
+#include <linux/file.h>
+#include <linux/fs.h>
+#include <linux/uaccess.h>
+
+/****************************************************************************
+ * Variables
+ ****************************************************************************/
+
+static int ethosu_network_release(struct inode *inode,
+ struct file *file);
+
+static long ethosu_network_ioctl(struct file *file,
+ unsigned int cmd,
+ unsigned long arg);
+
+static const struct file_operations ethosu_network_fops = {
+ .release = ðosu_network_release,
+ .unlocked_ioctl = ðosu_network_ioctl,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl = ðosu_network_ioctl,
+#endif
+};
+
+/****************************************************************************
+ * Functions
+ ****************************************************************************/
+
+static bool ethosu_network_verify(struct file *file)
+{
+ return file->f_op == ðosu_network_fops;
+}
+
+static void ethosu_network_destroy(struct kref *kref)
+{
+ struct ethosu_network *net =
+ container_of(kref, struct ethosu_network, kref);
+
+ dev_info(net->edev->dev, "Network destroy. handle=0x%pK\n", net);
+
+ ethosu_buffer_put(net->buf);
+ devm_kfree(net->edev->dev, net);
+}
+
+static int ethosu_network_release(struct inode *inode,
+ struct file *file)
+{
+ struct ethosu_network *net = file->private_data;
+
+ dev_info(net->edev->dev, "Network release. handle=0x%pK\n", net);
+
+ ethosu_network_put(net);
+
+ return 0;
+}
+
+static long ethosu_network_ioctl(struct file *file,
+ unsigned int cmd,
+ unsigned long arg)
+{
+ struct ethosu_network *net = file->private_data;
+ void __user *udata = (void __user *)arg;
+ int ret = -EINVAL;
+
+ ret = mutex_lock_interruptible(&net->edev->mutex);
+ if (ret)
+ return ret;
+
+ dev_info(net->edev->dev, "Ioctl: cmd=%u, arg=%lu\n", cmd, arg);
+
+ switch (cmd) {
+ case ETHOSU_IOCTL_INFERENCE_CREATE: {
+ struct ethosu_uapi_inference_create uapi;
+
+ if (copy_from_user(&uapi, udata, sizeof(uapi)))
+ break;
+
+ dev_info(net->edev->dev,
+ "Ioctl: Inference. ifm_fd=%u, ofm_fd=%u\n",
+ uapi.ifm_fd[0], uapi.ofm_fd[0]);
+
+ ret = ethosu_inference_create(net->edev, net, &uapi);
+ break;
+ }
+ default: {
+ dev_err(net->edev->dev, "Invalid ioctl. cmd=%u, arg=%lu",
+ cmd, arg);
+ break;
+ }
+ }
+
+ mutex_unlock(&net->edev->mutex);
+
+ return ret;
+}
+
+int ethosu_network_create(struct ethosu_device *edev,
+ struct ethosu_uapi_network_create *uapi)
+{
+ struct ethosu_buffer *buf;
+ struct ethosu_network *net;
+ int ret = -ENOMEM;
+
+ buf = ethosu_buffer_get_from_fd(uapi->fd);
+ if (IS_ERR(buf))
+ return PTR_ERR(buf);
+
+ net = devm_kzalloc(edev->dev, sizeof(*net), GFP_KERNEL);
+ if (!net) {
+ ret = -ENOMEM;
+ goto put_buf;
+ }
+
+ net->edev = edev;
+ net->buf = buf;
+ kref_init(&net->kref);
+
+ ret = anon_inode_getfd("ethosu-network", ðosu_network_fops, net,
+ O_RDWR | O_CLOEXEC);
+ if (ret < 0)
+ goto free_net;
+
+ net->file = fget(ret);
+ fput(net->file);
+
+ dev_info(edev->dev, "Network create. handle=0x%pK",
+ net);
+
+ return ret;
+
+free_net:
+ devm_kfree(edev->dev, net);
+
+put_buf:
+ ethosu_buffer_put(buf);
+
+ return ret;
+}
+
+struct ethosu_network *ethosu_network_get_from_fd(int fd)
+{
+ struct ethosu_network *net;
+ struct file *file;
+
+ file = fget(fd);
+ if (!file)
+ return ERR_PTR(-EINVAL);
+
+ if (!ethosu_network_verify(file)) {
+ fput(file);
+
+ return ERR_PTR(-EINVAL);
+ }
+
+ net = file->private_data;
+ ethosu_network_get(net);
+ fput(file);
+
+ return net;
+}
+
+void ethosu_network_get(struct ethosu_network *net)
+{
+ kref_get(&net->kref);
+}
+
+void ethosu_network_put(struct ethosu_network *net)
+{
+ kref_put(&net->kref, ethosu_network_destroy);
+}
diff --git a/drivers/firmware/ethosu/ethosu_network.h b/drivers/firmware/ethosu/ethosu_network.h
new file mode 100644
index 000000000000..bb70afcb2572
--- /dev/null
+++ b/drivers/firmware/ethosu/ethosu_network.h
@@ -0,0 +1,81 @@
+/*
+ * (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
+ *
+ * This program is free software and is provided to you under the terms of the
+ * GNU General Public License version 2 as published by the Free Software
+ * Foundation, and any use by you of this program is subject to the terms
+ * of such GNU licence.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, you can access it online at
+ * http://www.gnu.org/licenses/gpl-2.0.html.
+ *
+ * SPDX-License-Identifier: GPL-2.0-only
+ */
+
+#ifndef ETHOSU_NETWORK_H
+#define ETHOSU_NETWORK_H
+
+/****************************************************************************
+ * Includes
+ ****************************************************************************/
+
+#include <linux/kref.h>
+#include <linux/types.h>
+
+/****************************************************************************
+ * Types
+ ****************************************************************************/
+
+struct ethosu_buffer;
+struct ethosu_device;
+struct ethosu_uapi_network_create;
+struct device;
+struct file;
+
+struct ethosu_network {
+ struct ethosu_device *edev;
+ struct file *file;
+ struct kref kref;
+ struct ethosu_buffer *buf;
+};
+
+/****************************************************************************
+ * Functions
+ ****************************************************************************/
+
+/**
+ * ethosu_network_create() - Create network
+ *
+ * This function must be called in the context of a user space process.
+ *
+ * Return: fd on success, else error code.
+ */
+int ethosu_network_create(struct ethosu_device *edev,
+ struct ethosu_uapi_network_create *uapi);
+
+/**
+ * ethosu_network_get_from_fd() - Get network handle from fd
+ *
+ * This function must be called from a user space context.
+ *
+ * Return: Pointer on success, else ERR_PTR.
+ */
+struct ethosu_network *ethosu_network_get_from_fd(int fd);
+
+/**
+ * ethosu_network_get() - Get network
+ */
+void ethosu_network_get(struct ethosu_network *net);
+
+/**
+ * ethosu_network_put() - Put network
+ */
+void ethosu_network_put(struct ethosu_network *net);
+
+#endif /* ETHOSU_NETWORK_H */
diff --git a/drivers/firmware/ethosu/uapi/ethosu.h b/drivers/firmware/ethosu/uapi/ethosu.h
new file mode 100644
index 000000000000..903316dff7b3
--- /dev/null
+++ b/drivers/firmware/ethosu/uapi/ethosu.h
@@ -0,0 +1,207 @@
+/*
+ * (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
+ *
+ * This program is free software and is provided to you under the terms of the
+ * GNU General Public License version 2 as published by the Free Software
+ * Foundation, and any use by you of this program is subject to the terms
+ * of such GNU licence.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, you can access it online at
+ * http://www.gnu.org/licenses/gpl-2.0.html.
+ *
+ * SPDX-License-Identifier: GPL-2.0-only
+ */
+
+#ifndef ETHOSU_H
+#define ETHOSU_H
+
+/****************************************************************************
+ * Includes
+ ****************************************************************************/
+
+#include <linux/ioctl.h>
+#include <linux/types.h>
+
+#ifdef __cplusplus
+namespace EthosU {
+#endif
+
+/****************************************************************************
+ * Defines
+ ****************************************************************************/
+
+#define ETHOSU_IOCTL_BASE 0x01
+#define ETHOSU_IO(nr) _IO(ETHOSU_IOCTL_BASE, nr)
+#define ETHOSU_IOR(nr, type) _IOR(ETHOSU_IOCTL_BASE, nr, type)
+#define ETHOSU_IOW(nr, type) _IOW(ETHOSU_IOCTL_BASE, nr, type)
+#define ETHOSU_IOWR(nr, type) _IOWR(ETHOSU_IOCTL_BASE, nr, type)
+
+#define ETHOSU_IOCTL_PING ETHOSU_IO(0x00)
+#define ETHOSU_IOCTL_VERSION_REQ ETHOSU_IO(0x01)
+#define ETHOSU_IOCTL_CAPABILITIES_REQ ETHOSU_IO(0x02)
+#define ETHOSU_IOCTL_BUFFER_CREATE ETHOSU_IOR(0x10, \
+ struct ethosu_uapi_buffer_create)
+#define ETHOSU_IOCTL_BUFFER_SET ETHOSU_IOR(0x11, \
+ struct ethosu_uapi_buffer)
+#define ETHOSU_IOCTL_BUFFER_GET ETHOSU_IOW(0x12, \
+ struct ethosu_uapi_buffer)
+#define ETHOSU_IOCTL_NETWORK_CREATE ETHOSU_IOR(0x20, \
+ struct ethosu_uapi_network_create)
+#define ETHOSU_IOCTL_INFERENCE_CREATE ETHOSU_IOR(0x30, \
+ struct ethosu_uapi_inference_create)
+#define ETHOSU_IOCTL_INFERENCE_STATUS ETHOSU_IOR(0x31, \
+ struct ethosu_uapi_result_status)
+
+/* Maximum number of IFM/OFM file descriptors per network */
+#define ETHOSU_FD_MAX 16
+
+/* Maximum number of PMUs available */
+#define ETHOSU_PMU_EVENT_MAX 8
+
+/****************************************************************************
+ * Types
+ ****************************************************************************/
+
+/**
+ * enum ethosu_uapi_status - Status
+ */
+enum ethosu_uapi_status {
+ ETHOSU_UAPI_STATUS_OK,
+ ETHOSU_UAPI_STATUS_ERROR
+};
+
+/**
+ * struct ethosu_uapi_buffer_create - Create buffer request
+ * @capacity: Maximum capacity of the buffer
+ */
+struct ethosu_uapi_buffer_create {
+ __u32 capacity;
+};
+
+/**
+ * struct ethosu_uapi_buffer - Buffer descriptor
+ * @offset: Offset to where the data starts
+ * @size: Size of the data
+ *
+ * 'offset + size' must not exceed the capacity of the buffer.
+ */
+struct ethosu_uapi_buffer {
+ __u32 offset;
+ __u32 size;
+};
+
+/**
+ * struct ethosu_uapi_network_create - Create network request
+ * @fd: Buffer file descriptor
+ */
+struct ethosu_uapi_network_create {
+ __u32 fd;
+};
+
+/**
+ * struct ethosu_uapi_pmu_config - Configure performance counters
+ * @events: Array of counters to configure, set to non-zero for
+ * each counter to enable corresponding event.
+ * @cycle_count: Set to enable the cycle counter.
+ */
+struct ethosu_uapi_pmu_config {
+ __u32 events[ETHOSU_PMU_EVENT_MAX];
+ __u32 cycle_count;
+};
+
+/**
+ * struct ethosu_uapi_pmu_counts - Status of performance counters
+ * @events: Count for respective configured events.
+ * @cycle_count: Count for cycle counter.
+ */
+struct ethosu_uapi_pmu_counts {
+ __u32 events[ETHOSU_PMU_EVENT_MAX];
+ __u64 cycle_count;
+};
+
+/**
+ * struct ethosu_uapi_device_hw_id - Device hardware identification
+ * @version_status: Version status
+ * @version_minor: Version minor
+ * @version_major: Version major
+ * @product_major: Product major
+ * @arch_patch_rev: Architecture version patch
+ * @arch_minor_rev: Architecture version minor
+ * @arch_major_rev: Architecture version major
+ */
+struct ethosu_uapi_device_hw_id {
+ __u32 version_status;
+ __u32 version_minor;
+ __u32 version_major;
+ __u32 product_major;
+ __u32 arch_patch_rev;
+ __u32 arch_minor_rev;
+ __u32 arch_major_rev;
+};
+
+/**
+ * struct ethosu_uapi_device_hw_cfg - Device hardware configuration
+ * @macs_per_cc: MACs per clock cycle
+ * @cmd_stream_version: NPU command stream version
+ * @custom_dma: Custom DMA enabled
+ */
+struct ethosu_uapi_device_hw_cfg {
+ __u32 macs_per_cc;
+ __u32 cmd_stream_version;
+ __u32 custom_dma;
+};
+
+/**
+ * struct ethosu_uapi_capabilities - Device capabilities
+ * @hw_id: Hardware identification
+ * @hw_cfg: Hardware configuration
+ * @driver_patch_rev: Driver version patch
+ * @driver_minor_rev: Driver version minor
+ * @driver_major_rev: Driver version major
+ */
+struct ethosu_uapi_device_capabilities {
+ struct ethosu_uapi_device_hw_id hw_id;
+ struct ethosu_uapi_device_hw_cfg hw_cfg;
+ __u32 driver_patch_rev;
+ __u32 driver_minor_rev;
+ __u32 driver_major_rev;
+};
+
+/**
+ * struct ethosu_uapi_inference_create - Create network request
+ * @ifm_count: Number of IFM file descriptors
+ * @ifm_fd: IFM buffer file descriptors
+ * @ofm_count: Number of OFM file descriptors
+ * @ofm_fd: OFM buffer file descriptors
+ */
+struct ethosu_uapi_inference_create {
+ __u32 ifm_count;
+ __u32 ifm_fd[ETHOSU_FD_MAX];
+ __u32 ofm_count;
+ __u32 ofm_fd[ETHOSU_FD_MAX];
+ struct ethosu_uapi_pmu_config pmu_config;
+};
+
+/**
+ * struct ethosu_uapi_result_status - Status of inference
+ * @status Status of run inference.
+ * @pmu_config Configured performance counters.
+ * @pmu_count Perfomance counters values, when status is
+ * ETHOSU_UAPI_STATUS_OK.
+ */
+struct ethosu_uapi_result_status {
+ enum ethosu_uapi_status status;
+ struct ethosu_uapi_pmu_config pmu_config;
+ struct ethosu_uapi_pmu_counts pmu_count;
+};
+
+#ifdef __cplusplus
+} /* namespace EthosU */
+#endif
+#endif
--
2.17.1
On Fri, Jun 16, 2023 at 01:59:05PM +0800, Alison Wang wrote:
> This series contains Arm's NPU Ethos-U driver for NXP i.MX93 platform.
>
> Ethos-U Linux driver is to provide an example of how a rich operating
> system like Linux can dispatch inferences to an Arm Cortex-M subsystem,
> consisting of an Arm Cortex-M and an Arm Ethos-U NPU.
What is an "inference"?
Where are the userspace tools (if any) that are needed for this driver
to work properly?
thanks,
greg k-h
On Fri, Jun 16, 2023 at 01:59:06PM +0800, Alison Wang wrote:
> +static int ethosu_buffer_release(struct inode *inode,
> + struct file *file)
> +{
> + struct ethosu_buffer *buf = file->private_data;
> +
> + dev_info(buf->edev->dev, "Buffer release. handle=0x%pK\n", buf);
Oh, and please remove your debugging code that you have here, and all
over the place in the driver. When drivers work properly, they are
quiet, NOT printing out to the information log.
Are you sure that all of those people reviewed this thing?
greg k-h
On Fri, Jun 16, 2023 at 01:59:06PM +0800, Alison Wang wrote:
> Ethos-U Linux driver is to provide an example of how a rich operating
> system like Linux can dispatch inferences to an Arm Cortex-M subsystem,
> consisting of an Arm Cortex-M and an Arm Ethos-U NPU.
>
> Link: https://git.mlplatform.org/ml/ethos-u/ethos-u-linux-driver-stack.git
> Tag: 22.02
What does "Tag:" mean?
>
> Signed-off-by: Kristofer Jonsson <[email protected]>
> Signed-off-by: Per Astrand <[email protected]>
> Signed-off-by: Jonny Sv?rd <[email protected]>
> Signed-off-by: Lior Dekel <[email protected]>
> Signed-off-by: Henrik Hoglind <[email protected]>
> Signed-off-by: Davide Grohmann <[email protected]>
> Signed-off-by: Alison Wang <[email protected]>
If this many people signed off on this, it better be correct :)
> --- /dev/null
> +++ b/drivers/firmware/ethosu/Kconfig
> @@ -0,0 +1,24 @@
> +#
> +# (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
It's not 2020 anymore.
> +#
> +# This program is free software and is provided to you under the terms of the
> +# GNU General Public License version 2 as published by the Free Software
> +# Foundation, and any use by you of this program is subject to the terms
> +# of such GNU licence.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program; if not, you can access it online at
> +# http://www.gnu.org/licenses/gpl-2.0.html.
> +#
> +# SPDX-License-Identifier: GPL-2.0-only
> +#
> +
> +config ETHOSU
> + tristate "Arm Ethos-U NPU support"
> + help
> + Arm Ethos-U NPU driver.
{sigh}
With all of those people, NO ONE ran checkpatch.pl on the thing?
I'm stopping here, please go do that, fix up the obvious issues it tells
you about (hint, no SPDX line and drop the huge license "boiler plate"
texts)
And also provide a REAL configuration help text, 4 words for this is
just not acceptable, what would you do if you were asked to review this?
thanks,
greg k-h
On Fri, Jun 16, 2023 at 01:59:06PM +0800, Alison Wang wrote:
> +static int ethosu_buffer_release(struct inode *inode,
> + struct file *file) {
> + struct ethosu_buffer *buf = file->private_data;
> +
> + dev_info(buf->edev->dev, "Buffer release. handle=0x%pK\n", buf);
Oh, and please remove your debugging code that you have here, and all over the place in the driver. When drivers work properly, they are quiet, NOT printing out to the information log.
Are you sure that all of those people reviewed this thing?
[Alison Wang] These codes are also written by Arm guys. I changed dev_info to dev_dbg in the later patches.
Ok, I will remove these code.
Thanks a lot for your review.
greg k-h
On Fri, Jun 16, 2023 at 01:59:06PM +0800, Alison Wang wrote:
> Ethos-U Linux driver is to provide an example of how a rich operating
> system like Linux can dispatch inferences to an Arm Cortex-M
> subsystem, consisting of an Arm Cortex-M and an Arm Ethos-U NPU.
>
> Link:
> https://git/.
> mlplatform.org%2Fml%2Fethos-u%2Fethos-u-linux-driver-stack.git&data=05
> %7C01%7Calison.wang%40nxp.com%7C4b1681cd19b9450e832208db6e32709b%7C686
> ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C638224935139578328%7CUnknown%7
> CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXV
> CI6Mn0%3D%7C3000%7C%7C%7C&sdata=jwhvw5i1Q2M%2FOLID8TsOOnaI9LqD6y%2BMnn
> jzdvdCXcw%3D&reserved=0
> Tag: 22.02
What does "Tag:" mean?
>
> Signed-off-by: Kristofer Jonsson <[email protected]>
> Signed-off-by: Per Astrand <[email protected]>
> Signed-off-by: Jonny Sv?rd <[email protected]>
> Signed-off-by: Lior Dekel <[email protected]>
> Signed-off-by: Henrik Hoglind <[email protected]>
> Signed-off-by: Davide Grohmann <[email protected]>
> Signed-off-by: Alison Wang <[email protected]>
If this many people signed off on this, it better be correct :)
[Alison Wang] The source codes of this patch come from the kernel folder of https://git.mlplatform.org/ml/ethos-u/ethos-u-linux-driver-stack.git, the detailed tag is 22.02.
So the source codes of this patch are written by Arm guys. The above people are all the authors. I list them all to avoid any missing.
> --- /dev/null
> +++ b/drivers/firmware/ethosu/Kconfig
> @@ -0,0 +1,24 @@
> +#
> +# (C) COPYRIGHT 2020 ARM Limited. All rights reserved.
It's not 2020 anymore.
> +#
> +# This program is free software and is provided to you under the
> +terms of the # GNU General Public License version 2 as published by
> +the Free Software # Foundation, and any use by you of this program is
> +subject to the terms # of such GNU licence.
> +#
> +# This program is distributed in the hope that it will be useful, #
> +but WITHOUT ANY WARRANTY; without even the implied warranty of #
> +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU
> +General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License #
> +along with this program; if not, you can access it online at #
> +http://www.gnu.org/licenses/gpl-2.0.html.
> +#
> +# SPDX-License-Identifier: GPL-2.0-only #
> +
> +config ETHOSU
> + tristate "Arm Ethos-U NPU support"
> + help
> + Arm Ethos-U NPU driver.
{sigh}
With all of those people, NO ONE ran checkpatch.pl on the thing?
I'm stopping here, please go do that, fix up the obvious issues it tells you about (hint, no SPDX line and drop the huge license "boiler plate"
texts)
And also provide a REAL configuration help text, 4 words for this is just not acceptable, what would you do if you were asked to review this?
[Alison Wang] So sorry, I didn't do any change for the issues you mentioned above for this patch.
My previous idea is the first patch comes from Arm guys and the other patches are written by NXP guys. It seems this is not a good idea.
I will change this way in the next version.
Thanks a lot for your review.
thanks,
greg k-h
On Fri, Jun 16, 2023 at 01:59:05PM +0800, Alison Wang wrote:
> This series contains Arm's NPU Ethos-U driver for NXP i.MX93 platform.
>
> Ethos-U Linux driver is to provide an example of how a rich operating
> system like Linux can dispatch inferences to an Arm Cortex-M
> subsystem, consisting of an Arm Cortex-M and an Arm Ethos-U NPU.
What is an "inference"?
[Alison Wang] The inference here means machine learning inference. It is the process of
running data points into a machine learning model to calculate an output such as a single
numerical score. For example, we use this driver, user space application and Cortex-M firmware
to complete an inference about image classification.
Where are the userspace tools (if any) that are needed for this driver to work properly?
[Alison Wang] You could refer to https://github.com/nxp-imx/ethos-u-driver-stack-imx.git .
thanks,
greg k-h
On Fri, Jun 16, 2023 at 08:26:28AM +0000, Alison Wang wrote:
>
> On Fri, Jun 16, 2023 at 01:59:05PM +0800, Alison Wang wrote:
> > This series contains Arm's NPU Ethos-U driver for NXP i.MX93 platform.
> >
> > Ethos-U Linux driver is to provide an example of how a rich operating
> > system like Linux can dispatch inferences to an Arm Cortex-M
> > subsystem, consisting of an Arm Cortex-M and an Arm Ethos-U NPU.
>
> What is an "inference"?
> [Alison Wang] The inference here means machine learning inference. It is the process of
> running data points into a machine learning model to calculate an output such as a single
> numerical score. For example, we use this driver, user space application and Cortex-M firmware
> to complete an inference about image classification.
Nice, so why isn't this under drivers/accel/ then? Does it not need
that framework?
> Where are the userspace tools (if any) that are needed for this driver to work properly?
> [Alison Wang] You could refer to https://github.com/nxp-imx/ethos-u-driver-stack-imx.git .
Please put that in the documentation for the commit.
thanks,
greg k-h
On Fri, Jun 16, 2023 at 08:26:28AM +0000, Alison Wang wrote:
>
> On Fri, Jun 16, 2023 at 01:59:05PM +0800, Alison Wang wrote:
> > This series contains Arm's NPU Ethos-U driver for NXP i.MX93 platform.
> >
> > Ethos-U Linux driver is to provide an example of how a rich
> > operating system like Linux can dispatch inferences to an Arm
> > Cortex-M subsystem, consisting of an Arm Cortex-M and an Arm Ethos-U NPU.
>
> What is an "inference"?
> [Alison Wang] The inference here means machine learning inference. It
> is the process of running data points into a machine learning model to
> calculate an output such as a single numerical score. For example, we
> use this driver, user space application and Cortex-M firmware to complete an inference about image classification.
Nice, so why isn't this under drivers/accel/ then? Does it not need that framework?
[Alison Wang] This driver doesn't need that framework.
> Where are the userspace tools (if any) that are needed for this driver to work properly?
> [Alison Wang] You could refer to https://github.com/nxp-imx/ethos-u-driver-stack-imx.git .
Please put that in the documentation for the commit.
[Alison Wang] Sure, I will.
thanks,
greg k-h
On Fri, Jun 16, 2023 at 10:47:56AM +0000, Alison Wang wrote:
> On Fri, Jun 16, 2023 at 08:26:28AM +0000, Alison Wang wrote:
> >
> > On Fri, Jun 16, 2023 at 01:59:05PM +0800, Alison Wang wrote:
> > > This series contains Arm's NPU Ethos-U driver for NXP i.MX93 platform.
> > >
> > > Ethos-U Linux driver is to provide an example of how a rich
> > > operating system like Linux can dispatch inferences to an Arm
> > > Cortex-M subsystem, consisting of an Arm Cortex-M and an Arm Ethos-U NPU.
> >
> > What is an "inference"?
> > [Alison Wang] The inference here means machine learning inference. It
> > is the process of running data points into a machine learning model to
> > calculate an output such as a single numerical score. For example, we
> > use this driver, user space application and Cortex-M firmware to complete an inference about image classification.
>
> Nice, so why isn't this under drivers/accel/ then? Does it not need that framework?
> [Alison Wang] This driver doesn't need that framework.
Odd quoting style...
Anyway, why not use that famework? That's what that subsystem is for,
so you are going to have to document why this does not fit into that
model at all and you need your own interface instead.
thanks,
greg k-h
Hi Alison,
kernel test robot noticed the following build warnings:
[auto build test WARNING on soc/for-next]
[also build test WARNING on linus/master v6.4-rc6 next-20230616]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Alison-Wang/ethosu-Add-Arm-Ethos-U-driver/20230616-141036
base: https://git.kernel.org/pub/scm/linux/kernel/git/soc/soc.git for-next
patch link: https://lore.kernel.org/r/20230616055913.2360-2-alison.wang%40nxp.com
patch subject: [PATCH 1/8] ethosu: Add Arm Ethos-U driver
config: sparc-allyesconfig (https://download.01.org/0day-ci/archive/20230616/[email protected]/config)
compiler: sparc64-linux-gcc (GCC) 12.3.0
reproduce (this is a W=1 build):
mkdir -p ~/bin
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
git remote add soc https://git.kernel.org/pub/scm/linux/kernel/git/soc/soc.git
git fetch soc for-next
git checkout soc/for-next
b4 shazam https://lore.kernel.org/r/[email protected]
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.3.0 ~/bin/make.cross W=1 O=build_dir ARCH=sparc olddefconfig
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.3.0 ~/bin/make.cross W=1 O=build_dir ARCH=sparc SHELL=/bin/bash drivers/firmware/ethosu/
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
In file included from include/linux/device.h:15,
from include/linux/mailbox_client.h:11,
from drivers/firmware/ethosu/ethosu_mailbox.h:30,
from drivers/firmware/ethosu/ethosu_device.h:29,
from drivers/firmware/ethosu/ethosu_buffer.c:27:
drivers/firmware/ethosu/ethosu_buffer.c: In function 'ethosu_buffer_create':
>> drivers/firmware/ethosu/ethosu_buffer.c:253:18: warning: format '%llx' expects argument of type 'long long unsigned int', but argument 8 has type 'long unsigned int' [-Wformat=]
253 | "Buffer create. handle=0x%pK, capacity=%zu, cpu_addr=0x%pK, dma_addr=0x%llx, dma_addr_orig=0x%llx, phys_addr=0x%llx\n",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/dev_printk.h:110:30: note: in definition of macro 'dev_printk_index_wrap'
110 | _p_func(dev, fmt, ##__VA_ARGS__); \
| ^~~
include/linux/dev_printk.h:150:58: note: in expansion of macro 'dev_fmt'
150 | dev_printk_index_wrap(_dev_info, KERN_INFO, dev, dev_fmt(fmt), ##__VA_ARGS__)
| ^~~~~~~
drivers/firmware/ethosu/ethosu_buffer.c:252:9: note: in expansion of macro 'dev_info'
252 | dev_info(buf->edev->dev,
| ^~~~~~~~
drivers/firmware/ethosu/ethosu_buffer.c:253:132: note: format string is defined here
253 | "Buffer create. handle=0x%pK, capacity=%zu, cpu_addr=0x%pK, dma_addr=0x%llx, dma_addr_orig=0x%llx, phys_addr=0x%llx\n",
| ~~~^
| |
| long long unsigned int
| %lx
vim +253 drivers/firmware/ethosu/ethosu_buffer.c
218
219 int ethosu_buffer_create(struct ethosu_device *edev,
220 size_t capacity)
221 {
222 struct ethosu_buffer *buf;
223 int ret = -ENOMEM;
224
225 buf = devm_kzalloc(edev->dev, sizeof(*buf), GFP_KERNEL);
226 if (!buf)
227 return -ENOMEM;
228
229 buf->edev = edev;
230 buf->capacity = capacity;
231 buf->offset = 0;
232 buf->size = 0;
233 kref_init(&buf->kref);
234
235 buf->cpu_addr = dma_alloc_coherent(buf->edev->dev, capacity,
236 &buf->dma_addr_orig, GFP_KERNEL);
237 if (!buf->cpu_addr)
238 goto free_buf;
239
240 buf->dma_addr = ethosu_buffer_dma_ranges(buf->edev->dev,
241 buf->dma_addr_orig,
242 buf->capacity);
243
244 ret = anon_inode_getfd("ethosu-buffer", ðosu_buffer_fops, buf,
245 O_RDWR | O_CLOEXEC);
246 if (ret < 0)
247 goto free_dma;
248
249 buf->file = fget(ret);
250 fput(buf->file);
251
252 dev_info(buf->edev->dev,
> 253 "Buffer create. handle=0x%pK, capacity=%zu, cpu_addr=0x%pK, dma_addr=0x%llx, dma_addr_orig=0x%llx, phys_addr=0x%llx\n",
254 buf, capacity, buf->cpu_addr, buf->dma_addr,
255 buf->dma_addr_orig, virt_to_phys(buf->cpu_addr));
256
257 return ret;
258
259 free_dma:
260 dma_free_coherent(buf->edev->dev, buf->capacity, buf->cpu_addr,
261 buf->dma_addr_orig);
262
263 free_buf:
264 devm_kfree(buf->edev->dev, buf);
265
266 return ret;
267 }
268
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
On Fri, Jun 16, 2023 at 08:28:05AM +0000, Alison Wang wrote:
> On Fri, Jun 16, 2023 at 01:59:06PM +0800, Alison Wang wrote:
> > Ethos-U Linux driver is to provide an example of how a rich operating
> > system like Linux can dispatch inferences to an Arm Cortex-M
> > subsystem, consisting of an Arm Cortex-M and an Arm Ethos-U NPU.
> >
> > Link:
> > https://git/.
> > mlplatform.org%2Fml%2Fethos-u%2Fethos-u-linux-driver-stack.git&data=05
> > %7C01%7Calison.wang%40nxp.com%7C4b1681cd19b9450e832208db6e32709b%7C686
> > ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C638224935139578328%7CUnknown%7
> > CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXV
> > CI6Mn0%3D%7C3000%7C%7C%7C&sdata=jwhvw5i1Q2M%2FOLID8TsOOnaI9LqD6y%2BMnn
> > jzdvdCXcw%3D&reserved=0
> > Tag: 22.02
>
> What does "Tag:" mean?
>
> >
> > Signed-off-by: Kristofer Jonsson <[email protected]>
> > Signed-off-by: Per Astrand <[email protected]>
> > Signed-off-by: Jonny Sv?rd <[email protected]>
> > Signed-off-by: Lior Dekel <[email protected]>
> > Signed-off-by: Henrik Hoglind <[email protected]>
> > Signed-off-by: Davide Grohmann <[email protected]>
> > Signed-off-by: Alison Wang <[email protected]>
>
> If this many people signed off on this, it better be correct :)
> [Alison Wang] The source codes of this patch come from the kernel folder of https://git.mlplatform.org/ml/ethos-u/ethos-u-linux-driver-stack.git, the detailed tag is 22.02.
> So the source codes of this patch are written by Arm guys. The above people are all the authors. I list them all to avoid any missing.
Hi Alison
Maybe you should spend a few days reading email from others on the
list. Notice that netiquette style is used. Lines are wrapped to
around 80. Quoting is done using > etc. You are expected to follow
these guidelines.
Can i suggest you find somebody in NXP to be your mentor and help you
get up to speed with how mainline works. There are plenty of very good
mainline developers in the Ethernet group for example.
Andrew
On 2023-06-16 06:59, Alison Wang wrote:
[...]
> +/*
> + * The 'dma-ranges' device tree property for shared dma memory does not seem
> + * to be fully supported for coherent memory. Therefor we apply the DMA range
> + * offset ourselves.
> + */
NAK - if there's a bug in the core code, that wants to be fixed, not
bodged around by individual drivers. However from the look of the code
here, the driver appears to be misusing the property in an incorrect
manner anyway. But of course there's no devicetree binding here, so we
don't even really know what it thinks it expects... :/
I'd also agree with Greg that this is definitely not a firmware driver.
IIUC it's not so much a driver for the Ethos-U NPU itself, but one for
this particular subsystem configuration where requests to the NPU are
proxied through a dedicated Cortex-M core. As such, if you don't think
it belongs in drivers/accel, then drivers/remoteproc might be the next
most relevant choice. Also, is there a more specific name for this
particular subsystem, or is there a general expectation that this is the
only way an Ethos-U should ever be exposed to Linux, and it should never
have direct access to the hardware (as it would with an Ethos-N), and
thus there's no chance of ending up with multiple different "Ethos-U"
drivers in future?
Thanks,
Robin.
> +static dma_addr_t ethosu_buffer_dma_ranges(struct device *dev,
> + dma_addr_t dma_addr,
> + size_t dma_buf_size)
> +{
> + struct device_node *node = dev->of_node;
> + const __be32 *ranges;
> + int len;
> + int naddr;
> + int nsize;
> + int inc;
> + int i;
> +
> + if (!node)
> + return dma_addr;
> +
> + /* Get the #address-cells and #size-cells properties */
> + naddr = of_n_addr_cells(node);
> + nsize = of_n_size_cells(node);
> +
> + /* Read the 'dma-ranges' property */
> + ranges = of_get_property(node, "dma-ranges", &len);
> + if (!ranges || len <= 0)
> + return dma_addr;
> +
> + dev_dbg(dev, "ranges=%p, len=%d, naddr=%d, nsize=%d\n",
> + ranges, len, naddr, nsize);
> +
> + len /= sizeof(*ranges);
> + inc = naddr + naddr + nsize;
> +
> + for (i = 0; (i + inc) <= len; i += inc) {
> + dma_addr_t daddr;
> + dma_addr_t paddr;
> + dma_addr_t size;
> +
> + daddr = of_read_number(&ranges[i], naddr);
> + paddr = of_read_number(&ranges[i + naddr], naddr);
> + size = of_read_number(&ranges[i + naddr + naddr], nsize);
> +
> + dev_dbg(dev, "daddr=0x%llx, paddr=0x%llx, size=0x%llx\n",
> + daddr, paddr, size);
> +
> + if (dma_addr >= paddr &&
> + (dma_addr + dma_buf_size) < (paddr + size))
> + return dma_addr + daddr - paddr;
> + }
> +
> + return dma_addr;
> +}
Hi Alison,
kernel test robot noticed the following build warnings:
[auto build test WARNING on soc/for-next]
[also build test WARNING on linus/master v6.4-rc6 next-20230616]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Alison-Wang/ethosu-Add-Arm-Ethos-U-driver/20230616-141036
base: https://git.kernel.org/pub/scm/linux/kernel/git/soc/soc.git for-next
patch link: https://lore.kernel.org/r/20230616055913.2360-2-alison.wang%40nxp.com
patch subject: [PATCH 1/8] ethosu: Add Arm Ethos-U driver
config: i386-allyesconfig (https://download.01.org/0day-ci/archive/20230617/[email protected]/config)
compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
reproduce: (https://download.01.org/0day-ci/archive/20230617/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
In file included from include/linux/printk.h:564,
from include/linux/kernel.h:30,
from arch/x86/include/asm/percpu.h:27,
from arch/x86/include/asm/preempt.h:6,
from include/linux/preempt.h:78,
from include/linux/spinlock.h:56,
from include/linux/kref.h:16,
from drivers/firmware/ethosu/ethosu_buffer.h:28,
from drivers/firmware/ethosu/ethosu_buffer.c:25:
drivers/firmware/ethosu/ethosu_buffer.c: In function 'ethosu_buffer_dma_ranges':
>> drivers/firmware/ethosu/ethosu_buffer.c:108:30: warning: format '%llx' expects argument of type 'long long unsigned int', but argument 4 has type 'dma_addr_t' {aka 'unsigned int'} [-Wformat=]
108 | dev_dbg(dev, "daddr=0x%llx, paddr=0x%llx, size=0x%llx\n",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/dynamic_debug.h:222:29: note: in definition of macro '__dynamic_func_call_cls'
222 | func(&id, ##__VA_ARGS__); \
| ^~~~~~~~~~~
include/linux/dynamic_debug.h:248:9: note: in expansion of macro '_dynamic_func_call_cls'
248 | _dynamic_func_call_cls(_DPRINTK_CLASS_DFLT, fmt, func, ##__VA_ARGS__)
| ^~~~~~~~~~~~~~~~~~~~~~
include/linux/dynamic_debug.h:271:9: note: in expansion of macro '_dynamic_func_call'
271 | _dynamic_func_call(fmt, __dynamic_dev_dbg, \
| ^~~~~~~~~~~~~~~~~~
include/linux/dev_printk.h:155:9: note: in expansion of macro 'dynamic_dev_dbg'
155 | dynamic_dev_dbg(dev, dev_fmt(fmt), ##__VA_ARGS__)
| ^~~~~~~~~~~~~~~
include/linux/dev_printk.h:155:30: note: in expansion of macro 'dev_fmt'
155 | dynamic_dev_dbg(dev, dev_fmt(fmt), ##__VA_ARGS__)
| ^~~~~~~
drivers/firmware/ethosu/ethosu_buffer.c:108:17: note: in expansion of macro 'dev_dbg'
108 | dev_dbg(dev, "daddr=0x%llx, paddr=0x%llx, size=0x%llx\n",
| ^~~~~~~
drivers/firmware/ethosu/ethosu_buffer.c:108:42: note: format string is defined here
108 | dev_dbg(dev, "daddr=0x%llx, paddr=0x%llx, size=0x%llx\n",
| ~~~^
| |
| long long unsigned int
| %x
drivers/firmware/ethosu/ethosu_buffer.c:108:30: warning: format '%llx' expects argument of type 'long long unsigned int', but argument 5 has type 'dma_addr_t' {aka 'unsigned int'} [-Wformat=]
108 | dev_dbg(dev, "daddr=0x%llx, paddr=0x%llx, size=0x%llx\n",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/dynamic_debug.h:222:29: note: in definition of macro '__dynamic_func_call_cls'
222 | func(&id, ##__VA_ARGS__); \
| ^~~~~~~~~~~
include/linux/dynamic_debug.h:248:9: note: in expansion of macro '_dynamic_func_call_cls'
248 | _dynamic_func_call_cls(_DPRINTK_CLASS_DFLT, fmt, func, ##__VA_ARGS__)
| ^~~~~~~~~~~~~~~~~~~~~~
include/linux/dynamic_debug.h:271:9: note: in expansion of macro '_dynamic_func_call'
271 | _dynamic_func_call(fmt, __dynamic_dev_dbg, \
| ^~~~~~~~~~~~~~~~~~
include/linux/dev_printk.h:155:9: note: in expansion of macro 'dynamic_dev_dbg'
155 | dynamic_dev_dbg(dev, dev_fmt(fmt), ##__VA_ARGS__)
| ^~~~~~~~~~~~~~~
include/linux/dev_printk.h:155:30: note: in expansion of macro 'dev_fmt'
155 | dynamic_dev_dbg(dev, dev_fmt(fmt), ##__VA_ARGS__)
| ^~~~~~~
drivers/firmware/ethosu/ethosu_buffer.c:108:17: note: in expansion of macro 'dev_dbg'
108 | dev_dbg(dev, "daddr=0x%llx, paddr=0x%llx, size=0x%llx\n",
| ^~~~~~~
drivers/firmware/ethosu/ethosu_buffer.c:108:56: note: format string is defined here
108 | dev_dbg(dev, "daddr=0x%llx, paddr=0x%llx, size=0x%llx\n",
| ~~~^
| |
| long long unsigned int
| %x
drivers/firmware/ethosu/ethosu_buffer.c:108:30: warning: format '%llx' expects argument of type 'long long unsigned int', but argument 6 has type 'dma_addr_t' {aka 'unsigned int'} [-Wformat=]
108 | dev_dbg(dev, "daddr=0x%llx, paddr=0x%llx, size=0x%llx\n",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/dynamic_debug.h:222:29: note: in definition of macro '__dynamic_func_call_cls'
222 | func(&id, ##__VA_ARGS__); \
| ^~~~~~~~~~~
include/linux/dynamic_debug.h:248:9: note: in expansion of macro '_dynamic_func_call_cls'
248 | _dynamic_func_call_cls(_DPRINTK_CLASS_DFLT, fmt, func, ##__VA_ARGS__)
| ^~~~~~~~~~~~~~~~~~~~~~
include/linux/dynamic_debug.h:271:9: note: in expansion of macro '_dynamic_func_call'
271 | _dynamic_func_call(fmt, __dynamic_dev_dbg, \
| ^~~~~~~~~~~~~~~~~~
include/linux/dev_printk.h:155:9: note: in expansion of macro 'dynamic_dev_dbg'
155 | dynamic_dev_dbg(dev, dev_fmt(fmt), ##__VA_ARGS__)
| ^~~~~~~~~~~~~~~
include/linux/dev_printk.h:155:30: note: in expansion of macro 'dev_fmt'
155 | dynamic_dev_dbg(dev, dev_fmt(fmt), ##__VA_ARGS__)
| ^~~~~~~
drivers/firmware/ethosu/ethosu_buffer.c:108:17: note: in expansion of macro 'dev_dbg'
108 | dev_dbg(dev, "daddr=0x%llx, paddr=0x%llx, size=0x%llx\n",
| ^~~~~~~
drivers/firmware/ethosu/ethosu_buffer.c:108:69: note: format string is defined here
108 | dev_dbg(dev, "daddr=0x%llx, paddr=0x%llx, size=0x%llx\n",
| ~~~^
| |
| long long unsigned int
| %x
In file included from include/linux/device.h:15,
from include/linux/mailbox_client.h:11,
from drivers/firmware/ethosu/ethosu_mailbox.h:30,
from drivers/firmware/ethosu/ethosu_device.h:29,
from drivers/firmware/ethosu/ethosu_buffer.c:27:
drivers/firmware/ethosu/ethosu_buffer.c: In function 'ethosu_buffer_create':
drivers/firmware/ethosu/ethosu_buffer.c:253:18: warning: format '%llx' expects argument of type 'long long unsigned int', but argument 6 has type 'dma_addr_t' {aka 'unsigned int'} [-Wformat=]
253 | "Buffer create. handle=0x%pK, capacity=%zu, cpu_addr=0x%pK, dma_addr=0x%llx, dma_addr_orig=0x%llx, phys_addr=0x%llx\n",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/dev_printk.h:110:30: note: in definition of macro 'dev_printk_index_wrap'
110 | _p_func(dev, fmt, ##__VA_ARGS__); \
| ^~~
include/linux/dev_printk.h:150:58: note: in expansion of macro 'dev_fmt'
150 | dev_printk_index_wrap(_dev_info, KERN_INFO, dev, dev_fmt(fmt), ##__VA_ARGS__)
| ^~~~~~~
drivers/firmware/ethosu/ethosu_buffer.c:252:9: note: in expansion of macro 'dev_info'
252 | dev_info(buf->edev->dev,
| ^~~~~~~~
drivers/firmware/ethosu/ethosu_buffer.c:253:92: note: format string is defined here
253 | "Buffer create. handle=0x%pK, capacity=%zu, cpu_addr=0x%pK, dma_addr=0x%llx, dma_addr_orig=0x%llx, phys_addr=0x%llx\n",
| ~~~^
| |
| long long unsigned int
| %x
drivers/firmware/ethosu/ethosu_buffer.c:253:18: warning: format '%llx' expects argument of type 'long long unsigned int', but argument 7 has type 'dma_addr_t' {aka 'unsigned int'} [-Wformat=]
253 | "Buffer create. handle=0x%pK, capacity=%zu, cpu_addr=0x%pK, dma_addr=0x%llx, dma_addr_orig=0x%llx, phys_addr=0x%llx\n",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/dev_printk.h:110:30: note: in definition of macro 'dev_printk_index_wrap'
110 | _p_func(dev, fmt, ##__VA_ARGS__); \
| ^~~
include/linux/dev_printk.h:150:58: note: in expansion of macro 'dev_fmt'
150 | dev_printk_index_wrap(_dev_info, KERN_INFO, dev, dev_fmt(fmt), ##__VA_ARGS__)
| ^~~~~~~
drivers/firmware/ethosu/ethosu_buffer.c:252:9: note: in expansion of macro 'dev_info'
252 | dev_info(buf->edev->dev,
| ^~~~~~~~
drivers/firmware/ethosu/ethosu_buffer.c:253:114: note: format string is defined here
253 | "Buffer create. handle=0x%pK, capacity=%zu, cpu_addr=0x%pK, dma_addr=0x%llx, dma_addr_orig=0x%llx, phys_addr=0x%llx\n",
| ~~~^
| |
| long long unsigned int
| %x
>> drivers/firmware/ethosu/ethosu_buffer.c:253:18: warning: format '%llx' expects argument of type 'long long unsigned int', but argument 8 has type 'phys_addr_t' {aka 'unsigned int'} [-Wformat=]
253 | "Buffer create. handle=0x%pK, capacity=%zu, cpu_addr=0x%pK, dma_addr=0x%llx, dma_addr_orig=0x%llx, phys_addr=0x%llx\n",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
include/linux/dev_printk.h:110:30: note: in definition of macro 'dev_printk_index_wrap'
110 | _p_func(dev, fmt, ##__VA_ARGS__); \
| ^~~
include/linux/dev_printk.h:150:58: note: in expansion of macro 'dev_fmt'
150 | dev_printk_index_wrap(_dev_info, KERN_INFO, dev, dev_fmt(fmt), ##__VA_ARGS__)
| ^~~~~~~
drivers/firmware/ethosu/ethosu_buffer.c:252:9: note: in expansion of macro 'dev_info'
252 | dev_info(buf->edev->dev,
| ^~~~~~~~
drivers/firmware/ethosu/ethosu_buffer.c:253:132: note: format string is defined here
253 | "Buffer create. handle=0x%pK, capacity=%zu, cpu_addr=0x%pK, dma_addr=0x%llx, dma_addr_orig=0x%llx, phys_addr=0x%llx\n",
| ~~~^
| |
| long long unsigned int
| %x
vim +108 drivers/firmware/ethosu/ethosu_buffer.c
59
60 /****************************************************************************
61 * Functions
62 ****************************************************************************/
63
64 /*
65 * The 'dma-ranges' device tree property for shared dma memory does not seem
66 * to be fully supported for coherent memory. Therefor we apply the DMA range
67 * offset ourselves.
68 */
69 static dma_addr_t ethosu_buffer_dma_ranges(struct device *dev,
70 dma_addr_t dma_addr,
71 size_t dma_buf_size)
72 {
73 struct device_node *node = dev->of_node;
74 const __be32 *ranges;
75 int len;
76 int naddr;
77 int nsize;
78 int inc;
79 int i;
80
81 if (!node)
82 return dma_addr;
83
84 /* Get the #address-cells and #size-cells properties */
85 naddr = of_n_addr_cells(node);
86 nsize = of_n_size_cells(node);
87
88 /* Read the 'dma-ranges' property */
89 ranges = of_get_property(node, "dma-ranges", &len);
90 if (!ranges || len <= 0)
91 return dma_addr;
92
93 dev_dbg(dev, "ranges=%p, len=%d, naddr=%d, nsize=%d\n",
94 ranges, len, naddr, nsize);
95
96 len /= sizeof(*ranges);
97 inc = naddr + naddr + nsize;
98
99 for (i = 0; (i + inc) <= len; i += inc) {
100 dma_addr_t daddr;
101 dma_addr_t paddr;
102 dma_addr_t size;
103
104 daddr = of_read_number(&ranges[i], naddr);
105 paddr = of_read_number(&ranges[i + naddr], naddr);
106 size = of_read_number(&ranges[i + naddr + naddr], nsize);
107
> 108 dev_dbg(dev, "daddr=0x%llx, paddr=0x%llx, size=0x%llx\n",
109 daddr, paddr, size);
110
111 if (dma_addr >= paddr &&
112 (dma_addr + dma_buf_size) < (paddr + size))
113 return dma_addr + daddr - paddr;
114 }
115
116 return dma_addr;
117 }
118
119 static bool ethosu_buffer_verify(struct file *file)
120 {
121 return file->f_op == ðosu_buffer_fops;
122 }
123
124 static void ethosu_buffer_destroy(struct kref *kref)
125 {
126 struct ethosu_buffer *buf =
127 container_of(kref, struct ethosu_buffer, kref);
128
129 dev_info(buf->edev->dev, "Buffer destroy. handle=0x%pK\n", buf);
130
131 dma_free_coherent(buf->edev->dev, buf->capacity, buf->cpu_addr,
132 buf->dma_addr_orig);
133 devm_kfree(buf->edev->dev, buf);
134 }
135
136 static int ethosu_buffer_release(struct inode *inode,
137 struct file *file)
138 {
139 struct ethosu_buffer *buf = file->private_data;
140
141 dev_info(buf->edev->dev, "Buffer release. handle=0x%pK\n", buf);
142
143 ethosu_buffer_put(buf);
144
145 return 0;
146 }
147
148 static int ethosu_buffer_mmap(struct file *file,
149 struct vm_area_struct *vma)
150 {
151 struct ethosu_buffer *buf = file->private_data;
152 int ret;
153
154 dev_info(buf->edev->dev, "Buffer mmap. handle=0x%pK\n", buf);
155
156 ret = dma_mmap_coherent(buf->edev->dev, vma, buf->cpu_addr,
157 buf->dma_addr_orig,
158 buf->capacity);
159
160 return ret;
161 }
162
163 static long ethosu_buffer_ioctl(struct file *file,
164 unsigned int cmd,
165 unsigned long arg)
166 {
167 struct ethosu_buffer *buf = file->private_data;
168 void __user *udata = (void __user *)arg;
169 int ret = -EINVAL;
170
171 ret = mutex_lock_interruptible(&buf->edev->mutex);
172 if (ret)
173 return ret;
174
175 dev_info(buf->edev->dev, "Ioctl. cmd=%u, arg=%lu\n", cmd, arg);
176
177 switch (cmd) {
178 case ETHOSU_IOCTL_BUFFER_SET: {
179 struct ethosu_uapi_buffer uapi;
180
181 if (copy_from_user(&uapi, udata, sizeof(uapi)))
182 break;
183
184 dev_info(buf->edev->dev,
185 "Ioctl: Buffer set. size=%u, offset=%u\n",
186 uapi.size, uapi.offset);
187
188 ret = ethosu_buffer_resize(buf, uapi.size, uapi.offset);
189 break;
190 }
191 case ETHOSU_IOCTL_BUFFER_GET: {
192 struct ethosu_uapi_buffer uapi;
193
194 uapi.size = buf->size;
195 uapi.offset = buf->offset;
196
197 dev_info(buf->edev->dev,
198 "Ioctl: Buffer get. size=%u, offset=%u\n",
199 uapi.size, uapi.offset);
200
201 if (copy_to_user(udata, &uapi, sizeof(uapi)))
202 break;
203
204 ret = 0;
205 break;
206 }
207 default: {
208 dev_err(buf->edev->dev, "Invalid ioctl. cmd=%u, arg=%lu",
209 cmd, arg);
210 break;
211 }
212 }
213
214 mutex_unlock(&buf->edev->mutex);
215
216 return ret;
217 }
218
219 int ethosu_buffer_create(struct ethosu_device *edev,
220 size_t capacity)
221 {
222 struct ethosu_buffer *buf;
223 int ret = -ENOMEM;
224
225 buf = devm_kzalloc(edev->dev, sizeof(*buf), GFP_KERNEL);
226 if (!buf)
227 return -ENOMEM;
228
229 buf->edev = edev;
230 buf->capacity = capacity;
231 buf->offset = 0;
232 buf->size = 0;
233 kref_init(&buf->kref);
234
235 buf->cpu_addr = dma_alloc_coherent(buf->edev->dev, capacity,
236 &buf->dma_addr_orig, GFP_KERNEL);
237 if (!buf->cpu_addr)
238 goto free_buf;
239
240 buf->dma_addr = ethosu_buffer_dma_ranges(buf->edev->dev,
241 buf->dma_addr_orig,
242 buf->capacity);
243
244 ret = anon_inode_getfd("ethosu-buffer", ðosu_buffer_fops, buf,
245 O_RDWR | O_CLOEXEC);
246 if (ret < 0)
247 goto free_dma;
248
249 buf->file = fget(ret);
250 fput(buf->file);
251
252 dev_info(buf->edev->dev,
> 253 "Buffer create. handle=0x%pK, capacity=%zu, cpu_addr=0x%pK, dma_addr=0x%llx, dma_addr_orig=0x%llx, phys_addr=0x%llx\n",
254 buf, capacity, buf->cpu_addr, buf->dma_addr,
255 buf->dma_addr_orig, virt_to_phys(buf->cpu_addr));
256
257 return ret;
258
259 free_dma:
260 dma_free_coherent(buf->edev->dev, buf->capacity, buf->cpu_addr,
261 buf->dma_addr_orig);
262
263 free_buf:
264 devm_kfree(buf->edev->dev, buf);
265
266 return ret;
267 }
268
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
Hi Alison,
kernel test robot noticed the following build warnings:
[auto build test WARNING on soc/for-next]
[also build test WARNING on linus/master v6.4-rc7 next-20230623]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Alison-Wang/ethosu-Add-Arm-Ethos-U-driver/20230616-141036
base: https://git.kernel.org/pub/scm/linux/kernel/git/soc/soc.git for-next
patch link: https://lore.kernel.org/r/20230616055913.2360-2-alison.wang%40nxp.com
patch subject: [PATCH 1/8] ethosu: Add Arm Ethos-U driver
config: mips-randconfig-s032-20230625 (https://download.01.org/0day-ci/archive/20230625/[email protected]/config)
compiler: mipsel-linux-gcc (GCC) 12.3.0
reproduce: (https://download.01.org/0day-ci/archive/20230625/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
sparse warnings: (new ones prefixed by >>)
>> drivers/firmware/ethosu/ethosu_inference.c:54:28: sparse: sparse: incorrect type in initializer (different base types) @@ expected restricted __poll_t ( *poll )( ... ) @@ got unsigned int ( * )( ... ) @@
drivers/firmware/ethosu/ethosu_inference.c:54:28: sparse: expected restricted __poll_t ( *poll )( ... )
drivers/firmware/ethosu/ethosu_inference.c:54:28: sparse: got unsigned int ( * )( ... )
drivers/firmware/ethosu/ethosu_inference.c:361:44: sparse: sparse: non size-preserving integer to pointer cast
vim +54 drivers/firmware/ethosu/ethosu_inference.c
37
38 /****************************************************************************
39 * Variables
40 ****************************************************************************/
41
42 static int ethosu_inference_release(struct inode *inode,
43 struct file *file);
44
45 static unsigned int ethosu_inference_poll(struct file *file,
46 poll_table *wait);
47
48 static long ethosu_inference_ioctl(struct file *file,
49 unsigned int cmd,
50 unsigned long arg);
51
52 static const struct file_operations ethosu_inference_fops = {
53 .release = ðosu_inference_release,
> 54 .poll = ðosu_inference_poll,
55 .unlocked_ioctl = ðosu_inference_ioctl,
56 #ifdef CONFIG_COMPAT
57 .compat_ioctl = ðosu_inference_ioctl,
58 #endif
59 };
60
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
> On Fri, Jun 16, 2023 at 01:59:06PM +0800, Alison Wang wrote:
> > Ethos-U Linux driver is to provide an example of how a rich operating
> > system like Linux can dispatch inferences to an Arm Cortex-M
> > subsystem, consisting of an Arm Cortex-M and an Arm Ethos-U NPU.
> >
> > Link:
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.
> >
> mlplatform.org%2Fml%2Fethos-u%2Fethos-u-linux-driver-stack.git&data=05
> > %7C01%7Calison.wang%40nxp.com%7C4b1681cd19b9450e832208db6e32
> 709b%7C686
> >
> ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C638224935139578328%7C
> Unknown%7
> >
> CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwi
> LCJXV
> >
> CI6Mn0%3D%7C3000%7C%7C%7C&sdata=jwhvw5i1Q2M%2FOLID8TsOOnaI
> 9LqD6y%2BMnn
> > jzdvdCXcw%3D&reserved=0
> > Tag: 22.02
>
> What does "Tag:" mean?
>
> >
> > Signed-off-by: Kristofer Jonsson <[email protected]>
> > Signed-off-by: Per Astrand <[email protected]>
> > Signed-off-by: Jonny Sv?rd <[email protected]>
> > Signed-off-by: Lior Dekel <[email protected]>
> > Signed-off-by: Henrik Hoglind <[email protected]>
> > Signed-off-by: Davide Grohmann <[email protected]>
> > Signed-off-by: Alison Wang <[email protected]>
>
> If this many people signed off on this, it better be correct :)
Our patches are derived from the ethos-u-linux-driver-stack in mlplatform (https://git.mlplatform.org/ml/ethos-u/ethos-u-linux-driver-stack.git).
As a gesture of respect to the original authors, we initially included the ARM authors' names on the signed-off-by line.
However, after discussions with ARM, we would like to clarify Arm has not signed off on this patchset and therefore we will remove their signed-off-by tags on the next update.
Thanks.
Alison