2020-12-01 22:42:02

by mark gross

[permalink] [raw]
Subject: [PATCH 00/22] Intel Vision Processing Unit base enabling part 1

From: mark gross <[email protected]>

The Intel Vision Processing Unit (VPU) is an IP block that is showing up for
the first time as part of the Keem Bay SOC. Keem Bay is a quad core A53 Arm
SOC. It is designed to be used as a stand alone SOC as well as in an PCIe
Vision Processing accelerator add in card.

This part 1 of the patches make up the base or core of the stack needed to
enable both use cases for the VPU.

Part 2 includes 11 more patches that depend on part 1. Those should be ready
in a couple of weeks or less.

I am trying something a bit new with this sequence where I've been working with
the driver developers as a "pre-maintainer" reviewing and enforcing the kernel
expectations as I understand them. Its taken a couple of months to get this
code to the point I feel its ready for public posting. My goal is to make sure
it meets expectations for quality and compliance with kernel expectations and
there will be mostly technical / design issues to talk about.

Thanks for looking at these and providing feedback.

--mark
p.s. I have had a problem my MTA configuration between mutt and git send-email
where I was using msmpt to send from mutt (because 15+ years ago its the first
way I got to work and never changed) while my worstation MTA that git
send-email uses was un-configured resulting in my return-path naming my
workstion withing the firewall. I suck at email administration.

I appologies for the multiple copies.

P.p.s. corrected cc listings.

Daniele Alessandrelli (2):
dt-bindings: Add bindings for Keem Bay IPC driver
keembay-ipc: Add Keem Bay IPC module

Paul Murphy (2):
dt-bindings: Add bindings for Keem Bay VPU IPC driver
keembay-vpu-ipc: Add Keem Bay VPU IPC module

Seamus Kelly (8):
xlink-ipc: Add xlink ipc device tree bindings
xlink-ipc: Add xlink ipc driver
xlink-core: Add xlink core device tree bindings
xlink-core: Add xlink core driver xLink
xlink-core: Enable xlink protocol over pcie
xlink-core: Enable VPU IP management and runtime control
xlink-core: add async channel and events
xlink-core: factorize xlink_ioctl function by creating sub-functions
for each ioctl command

Srikanth Thokala (9):
misc: xlink-pcie: Add documentation for XLink PCIe driver
misc: xlink-pcie: lh: Add PCIe EPF driver for Local Host
misc: xlink-pcie: lh: Add PCIe EP DMA functionality
misc: xlink-pcie: lh: Add core communication logic
misc: xlink-pcie: lh: Prepare changes for adding remote host driver
misc: xlink-pcie: rh: Add PCIe EP driver for Remote Host
misc: xlink-pcie: rh: Add core communication logic
misc: xlink-pcie: Add XLink API interface
misc: xlink-pcie: Add asynchronous event notification support for
XLink

mark gross (1):
Add Vision Processing Unit (VPU) documentation.

.../misc/intel,keembay-xlink-ipc.yaml | 49 +
.../bindings/misc/intel,keembay-xlink.yaml | 27 +
.../bindings/soc/intel/intel,keembay-ipc.yaml | 63 +
.../soc/intel/intel,keembay-vpu-ipc.yaml | 151 ++
Documentation/index.rst | 3 +-
Documentation/vpu/index.rst | 19 +
Documentation/vpu/vpu-stack-overview.rst | 267 +++
Documentation/vpu/xlink-core.rst | 80 +
Documentation/vpu/xlink-ipc.rst | 50 +
Documentation/vpu/xlink-pcie.rst | 91 +
MAINTAINERS | 41 +
drivers/misc/Kconfig | 3 +
drivers/misc/Makefile | 3 +
drivers/misc/xlink-core/Kconfig | 33 +
drivers/misc/xlink-core/Makefile | 5 +
drivers/misc/xlink-core/xlink-core.c | 1335 +++++++++++
drivers/misc/xlink-core/xlink-core.h | 24 +
drivers/misc/xlink-core/xlink-defs.h | 181 ++
drivers/misc/xlink-core/xlink-dispatcher.c | 436 ++++
drivers/misc/xlink-core/xlink-dispatcher.h | 26 +
drivers/misc/xlink-core/xlink-ioctl.c | 584 +++++
drivers/misc/xlink-core/xlink-ioctl.h | 36 +
drivers/misc/xlink-core/xlink-multiplexer.c | 1164 ++++++++++
drivers/misc/xlink-core/xlink-multiplexer.h | 35 +
drivers/misc/xlink-core/xlink-platform.c | 273 +++
drivers/misc/xlink-core/xlink-platform.h | 65 +
drivers/misc/xlink-ipc/Kconfig | 7 +
drivers/misc/xlink-ipc/Makefile | 4 +
drivers/misc/xlink-ipc/xlink-ipc.c | 879 +++++++
drivers/misc/xlink-pcie/Kconfig | 20 +
drivers/misc/xlink-pcie/Makefile | 2 +
drivers/misc/xlink-pcie/common/core.h | 247 ++
drivers/misc/xlink-pcie/common/interface.c | 126 +
drivers/misc/xlink-pcie/common/util.c | 375 +++
drivers/misc/xlink-pcie/common/util.h | 70 +
drivers/misc/xlink-pcie/common/xpcie.h | 120 +
drivers/misc/xlink-pcie/local_host/Makefile | 6 +
drivers/misc/xlink-pcie/local_host/core.c | 905 ++++++++
drivers/misc/xlink-pcie/local_host/dma.c | 577 +++++
drivers/misc/xlink-pcie/local_host/epf.c | 523 +++++
drivers/misc/xlink-pcie/local_host/epf.h | 106 +
drivers/misc/xlink-pcie/remote_host/Makefile | 6 +
drivers/misc/xlink-pcie/remote_host/core.c | 647 ++++++
drivers/misc/xlink-pcie/remote_host/main.c | 96 +
drivers/misc/xlink-pcie/remote_host/pci.c | 525 +++++
drivers/misc/xlink-pcie/remote_host/pci.h | 67 +
drivers/soc/Kconfig | 1 +
drivers/soc/Makefile | 1 +
drivers/soc/intel/Kconfig | 32 +
drivers/soc/intel/Makefile | 5 +
drivers/soc/intel/keembay-ipc.c | 1669 ++++++++++++++
drivers/soc/intel/keembay-vpu-ipc.c | 2036 +++++++++++++++++
include/linux/soc/intel/keembay-ipc.h | 30 +
include/linux/soc/intel/keembay-vpu-ipc.h | 62 +
include/linux/xlink-ipc.h | 48 +
include/linux/xlink.h | 146 ++
include/linux/xlink_drv_inf.h | 72 +
include/uapi/misc/xlink_uapi.h | 145 ++
58 files changed, 14598 insertions(+), 1 deletion(-)
create mode 100644 Documentation/devicetree/bindings/misc/intel,keembay-xlink-ipc.yaml
create mode 100644 Documentation/devicetree/bindings/misc/intel,keembay-xlink.yaml
create mode 100644 Documentation/devicetree/bindings/soc/intel/intel,keembay-ipc.yaml
create mode 100644 Documentation/devicetree/bindings/soc/intel/intel,keembay-vpu-ipc.yaml
create mode 100644 Documentation/vpu/index.rst
create mode 100644 Documentation/vpu/vpu-stack-overview.rst
create mode 100644 Documentation/vpu/xlink-core.rst
create mode 100644 Documentation/vpu/xlink-ipc.rst
create mode 100644 Documentation/vpu/xlink-pcie.rst
create mode 100644 drivers/misc/xlink-core/Kconfig
create mode 100644 drivers/misc/xlink-core/Makefile
create mode 100644 drivers/misc/xlink-core/xlink-core.c
create mode 100644 drivers/misc/xlink-core/xlink-core.h
create mode 100644 drivers/misc/xlink-core/xlink-defs.h
create mode 100644 drivers/misc/xlink-core/xlink-dispatcher.c
create mode 100644 drivers/misc/xlink-core/xlink-dispatcher.h
create mode 100644 drivers/misc/xlink-core/xlink-ioctl.c
create mode 100644 drivers/misc/xlink-core/xlink-ioctl.h
create mode 100644 drivers/misc/xlink-core/xlink-multiplexer.c
create mode 100644 drivers/misc/xlink-core/xlink-multiplexer.h
create mode 100644 drivers/misc/xlink-core/xlink-platform.c
create mode 100644 drivers/misc/xlink-core/xlink-platform.h
create mode 100644 drivers/misc/xlink-ipc/Kconfig
create mode 100644 drivers/misc/xlink-ipc/Makefile
create mode 100644 drivers/misc/xlink-ipc/xlink-ipc.c
create mode 100644 drivers/misc/xlink-pcie/Kconfig
create mode 100644 drivers/misc/xlink-pcie/Makefile
create mode 100644 drivers/misc/xlink-pcie/common/core.h
create mode 100644 drivers/misc/xlink-pcie/common/interface.c
create mode 100644 drivers/misc/xlink-pcie/common/util.c
create mode 100644 drivers/misc/xlink-pcie/common/util.h
create mode 100644 drivers/misc/xlink-pcie/common/xpcie.h
create mode 100644 drivers/misc/xlink-pcie/local_host/Makefile
create mode 100644 drivers/misc/xlink-pcie/local_host/core.c
create mode 100644 drivers/misc/xlink-pcie/local_host/dma.c
create mode 100644 drivers/misc/xlink-pcie/local_host/epf.c
create mode 100644 drivers/misc/xlink-pcie/local_host/epf.h
create mode 100644 drivers/misc/xlink-pcie/remote_host/Makefile
create mode 100644 drivers/misc/xlink-pcie/remote_host/core.c
create mode 100644 drivers/misc/xlink-pcie/remote_host/main.c
create mode 100644 drivers/misc/xlink-pcie/remote_host/pci.c
create mode 100644 drivers/misc/xlink-pcie/remote_host/pci.h
create mode 100644 drivers/soc/intel/Kconfig
create mode 100644 drivers/soc/intel/Makefile
create mode 100644 drivers/soc/intel/keembay-ipc.c
create mode 100644 drivers/soc/intel/keembay-vpu-ipc.c
create mode 100644 include/linux/soc/intel/keembay-ipc.h
create mode 100644 include/linux/soc/intel/keembay-vpu-ipc.h
create mode 100644 include/linux/xlink-ipc.h
create mode 100644 include/linux/xlink.h
create mode 100644 include/linux/xlink_drv_inf.h
create mode 100644 include/uapi/misc/xlink_uapi.h

--
2.17.1


2020-12-01 22:42:02

by mark gross

[permalink] [raw]
Subject: [PATCH 12/22] misc: xlink-pcie: rh: Add core communication logic

From: Srikanth Thokala <[email protected]>

Add logic to establish communication with the local host which is through
ring buffer management and MSI/Doorbell interrupts

Reviewed-by: Mark Gross <[email protected]>
Signed-off-by: Srikanth Thokala <[email protected]>
---
drivers/misc/xlink-pcie/common/core.h | 11 +-
drivers/misc/xlink-pcie/remote_host/Makefile | 2 +
drivers/misc/xlink-pcie/remote_host/core.c | 647 +++++++++++++++++++
drivers/misc/xlink-pcie/remote_host/pci.c | 48 +-
4 files changed, 696 insertions(+), 12 deletions(-)
create mode 100644 drivers/misc/xlink-pcie/remote_host/core.c

diff --git a/drivers/misc/xlink-pcie/common/core.h b/drivers/misc/xlink-pcie/common/core.h
index 84985ef41a64..eec8566c19d9 100644
--- a/drivers/misc/xlink-pcie/common/core.h
+++ b/drivers/misc/xlink-pcie/common/core.h
@@ -10,15 +10,11 @@
#ifndef XPCIE_CORE_HEADER_
#define XPCIE_CORE_HEADER_

-#include <linux/io.h>
-#include <linux/types.h>
-#include <linux/workqueue.h>
-#include <linux/slab.h>
-#include <linux/mutex.h>
-#include <linux/mempool.h>
#include <linux/dma-mapping.h>
-#include <linux/cache.h>
+#include <linux/mutex.h>
+#include <linux/slab.h>
#include <linux/wait.h>
+#include <linux/workqueue.h>

#include <linux/xlink_drv_inf.h>

@@ -64,6 +60,7 @@ struct xpcie_buf_desc {
struct xpcie_stream {
size_t frag;
struct xpcie_pipe pipe;
+ struct xpcie_buf_desc **ddr;
};

struct xpcie_list {
diff --git a/drivers/misc/xlink-pcie/remote_host/Makefile b/drivers/misc/xlink-pcie/remote_host/Makefile
index 96374a43023e..e8074dbb1161 100644
--- a/drivers/misc/xlink-pcie/remote_host/Makefile
+++ b/drivers/misc/xlink-pcie/remote_host/Makefile
@@ -1,3 +1,5 @@
obj-$(CONFIG_XLINK_PCIE_RH_DRIVER) += mxlk.o
mxlk-objs := main.o
mxlk-objs += pci.o
+mxlk-objs += core.o
+mxlk-objs += ../common/util.o
diff --git a/drivers/misc/xlink-pcie/remote_host/core.c b/drivers/misc/xlink-pcie/remote_host/core.c
new file mode 100644
index 000000000000..668fded17e9c
--- /dev/null
+++ b/drivers/misc/xlink-pcie/remote_host/core.c
@@ -0,0 +1,647 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*****************************************************************************
+ *
+ * Intel Keem Bay XLink PCIe Driver
+ *
+ * Copyright (C) 2020 Intel Corporation
+ *
+ ****************************************************************************/
+
+#include "pci.h"
+
+#include "../common/core.h"
+#include "../common/util.h"
+
+static int rx_pool_size = SZ_32M;
+module_param(rx_pool_size, int, 0664);
+MODULE_PARM_DESC(rx_pool_size, "receive pool size (default 32 MiB)");
+
+static int tx_pool_size = SZ_32M;
+module_param(tx_pool_size, int, 0664);
+MODULE_PARM_DESC(tx_pool_size, "transmit pool size (default 32 MiB)");
+
+static int intel_xpcie_version_check(struct xpcie *xpcie)
+{
+ struct xpcie_version version;
+
+ memcpy_fromio(&version,
+ (void __iomem *)(xpcie->mmio + XPCIE_MMIO_VERSION),
+ sizeof(version));
+
+ dev_dbg(xpcie_to_dev(xpcie), "ver: device %u.%u.%u, host %u.%u.%u\n",
+ version.major, version.minor, version.build,
+ XPCIE_VERSION_MAJOR, XPCIE_VERSION_MINOR, XPCIE_VERSION_BUILD);
+
+ if (intel_xpcie_ioread8(xpcie->mmio + XPCIE_MMIO_LEGACY_A0))
+ xpcie->legacy_a0 = true;
+
+ return 0;
+}
+
+static int intel_xpcie_map_dma(struct xpcie *xpcie, struct xpcie_buf_desc *bd,
+ int direction)
+{
+ struct xpcie_dev *xdev = container_of(xpcie, struct xpcie_dev, xpcie);
+ struct device *dev = &xdev->pci->dev;
+
+ bd->phys = dma_map_single(dev, bd->data, bd->length, direction);
+
+ return dma_mapping_error(dev, bd->phys);
+}
+
+static void intel_xpcie_unmap_dma(struct xpcie *xpcie,
+ struct xpcie_buf_desc *bd,
+ int direction)
+{
+ struct xpcie_dev *xdev = container_of(xpcie, struct xpcie_dev, xpcie);
+ struct device *dev = &xdev->pci->dev;
+
+ dma_unmap_single(dev, bd->phys, bd->length, direction);
+}
+
+static void intel_xpcie_txrx_cleanup(struct xpcie *xpcie)
+{
+ struct xpcie_interface *inf = &xpcie->interfaces[0];
+ struct xpcie_stream *tx = &xpcie->tx;
+ struct xpcie_stream *rx = &xpcie->rx;
+ struct xpcie_buf_desc *bd;
+ int index;
+
+ xpcie->stop_flag = true;
+ xpcie->no_tx_buffer = false;
+ inf->data_avail = true;
+ wake_up_interruptible(&xpcie->tx_waitq);
+ wake_up_interruptible(&inf->rx_waitq);
+ mutex_lock(&xpcie->wlock);
+ mutex_lock(&inf->rlock);
+
+ if (tx->ddr) {
+ for (index = 0; index < tx->pipe.ndesc; index++) {
+ struct xpcie_transfer_desc *td = tx->pipe.tdr + index;
+
+ bd = tx->ddr[index];
+ if (bd) {
+ intel_xpcie_unmap_dma(xpcie, bd, DMA_TO_DEVICE);
+ intel_xpcie_free_tx_bd(xpcie, bd);
+ intel_xpcie_set_td_address(td, 0);
+ intel_xpcie_set_td_length(td, 0);
+ }
+ }
+ kfree(tx->ddr);
+ }
+
+ if (rx->ddr) {
+ for (index = 0; index < rx->pipe.ndesc; index++) {
+ struct xpcie_transfer_desc *td = rx->pipe.tdr + index;
+
+ bd = rx->ddr[index];
+ if (bd) {
+ intel_xpcie_unmap_dma(xpcie,
+ bd, DMA_FROM_DEVICE);
+ intel_xpcie_free_rx_bd(xpcie, bd);
+ intel_xpcie_set_td_address(td, 0);
+ intel_xpcie_set_td_length(td, 0);
+ }
+ }
+ kfree(rx->ddr);
+ }
+
+ intel_xpcie_list_cleanup(&xpcie->tx_pool);
+ intel_xpcie_list_cleanup(&xpcie->rx_pool);
+
+ mutex_unlock(&inf->rlock);
+ mutex_unlock(&xpcie->wlock);
+}
+
+static int intel_xpcie_txrx_init(struct xpcie *xpcie,
+ struct xpcie_cap_txrx *cap)
+{
+ struct xpcie_stream *tx = &xpcie->tx;
+ struct xpcie_stream *rx = &xpcie->rx;
+ struct xpcie_buf_desc *bd;
+ int rc, index, ndesc;
+
+ xpcie->txrx = cap;
+ xpcie->fragment_size = intel_xpcie_ioread32(&cap->fragment_size);
+ xpcie->stop_flag = false;
+
+ tx->pipe.ndesc = intel_xpcie_ioread32(&cap->tx.ndesc);
+ tx->pipe.head = &cap->tx.head;
+ tx->pipe.tail = &cap->tx.tail;
+ tx->pipe.old = intel_xpcie_ioread32(&cap->tx.tail);
+ tx->pipe.tdr = (struct xpcie_transfer_desc *)(xpcie->mmio +
+ intel_xpcie_ioread32(&cap->tx.ring));
+
+ tx->ddr = kcalloc(tx->pipe.ndesc, sizeof(struct xpcie_buf_desc *),
+ GFP_KERNEL);
+ if (!tx->ddr) {
+ rc = -ENOMEM;
+ goto error;
+ }
+
+ rx->pipe.ndesc = intel_xpcie_ioread32(&cap->rx.ndesc);
+ rx->pipe.head = &cap->rx.head;
+ rx->pipe.tail = &cap->rx.tail;
+ rx->pipe.old = intel_xpcie_ioread32(&cap->rx.head);
+ rx->pipe.tdr = (struct xpcie_transfer_desc *)(xpcie->mmio +
+ intel_xpcie_ioread32(&cap->rx.ring));
+
+ rx->ddr = kcalloc(rx->pipe.ndesc, sizeof(struct xpcie_buf_desc *),
+ GFP_KERNEL);
+ if (!rx->ddr) {
+ rc = -ENOMEM;
+ goto error;
+ }
+
+ intel_xpcie_list_init(&xpcie->rx_pool);
+ rx_pool_size = roundup(rx_pool_size, xpcie->fragment_size);
+ ndesc = rx_pool_size / xpcie->fragment_size;
+
+ for (index = 0; index < ndesc; index++) {
+ bd = intel_xpcie_alloc_bd(xpcie->fragment_size);
+ if (bd) {
+ intel_xpcie_list_put(&xpcie->rx_pool, bd);
+ } else {
+ rc = -ENOMEM;
+ goto error;
+ }
+ }
+
+ intel_xpcie_list_init(&xpcie->tx_pool);
+ tx_pool_size = roundup(tx_pool_size, xpcie->fragment_size);
+ ndesc = tx_pool_size / xpcie->fragment_size;
+
+ for (index = 0; index < ndesc; index++) {
+ bd = intel_xpcie_alloc_bd(xpcie->fragment_size);
+ if (bd) {
+ intel_xpcie_list_put(&xpcie->tx_pool, bd);
+ } else {
+ rc = -ENOMEM;
+ goto error;
+ }
+ }
+
+ for (index = 0; index < rx->pipe.ndesc; index++) {
+ struct xpcie_transfer_desc *td = rx->pipe.tdr + index;
+
+ bd = intel_xpcie_alloc_rx_bd(xpcie);
+ if (!bd) {
+ rc = -ENOMEM;
+ goto error;
+ }
+
+ if (intel_xpcie_map_dma(xpcie, bd, DMA_FROM_DEVICE)) {
+ dev_err(xpcie_to_dev(xpcie), "failed to map rx bd\n");
+ rc = -ENOMEM;
+ goto error;
+ }
+
+ rx->ddr[index] = bd;
+ intel_xpcie_set_td_address(td, bd->phys);
+ intel_xpcie_set_td_length(td, bd->length);
+ }
+
+ return 0;
+
+error:
+ intel_xpcie_txrx_cleanup(xpcie);
+
+ return rc;
+}
+
+static int intel_xpcie_discover_txrx(struct xpcie *xpcie)
+{
+ struct xpcie_cap_txrx *cap;
+ int error;
+
+ cap = intel_xpcie_cap_find(xpcie, 0, XPCIE_CAP_TXRX);
+ if (cap)
+ error = intel_xpcie_txrx_init(xpcie, cap);
+ else
+ error = -EIO;
+
+ return error;
+}
+
+static void intel_xpcie_start_tx(struct xpcie *xpcie, unsigned long delay)
+{
+ queue_delayed_work(xpcie->tx_wq, &xpcie->tx_event, delay);
+}
+
+static void intel_xpcie_start_rx(struct xpcie *xpcie, unsigned long delay)
+{
+ queue_delayed_work(xpcie->rx_wq, &xpcie->rx_event, delay);
+}
+
+static void intel_xpcie_rx_event_handler(struct work_struct *work)
+{
+ struct xpcie *xpcie = container_of(work, struct xpcie, rx_event.work);
+ struct xpcie_dev *xdev = container_of(xpcie, struct xpcie_dev, xpcie);
+ struct xpcie_buf_desc *bd, *replacement = NULL;
+ unsigned long delay = msecs_to_jiffies(1);
+ struct xpcie_stream *rx = &xpcie->rx;
+ struct xpcie_transfer_desc *td;
+ u32 head, tail, ndesc, length;
+ u16 status, interface;
+ int rc;
+
+ if (intel_xpcie_get_device_status(xpcie) != XPCIE_STATUS_RUN)
+ return;
+
+ ndesc = rx->pipe.ndesc;
+ tail = intel_xpcie_get_tdr_tail(&rx->pipe);
+ head = intel_xpcie_get_tdr_head(&rx->pipe);
+
+ while (head != tail) {
+ td = rx->pipe.tdr + head;
+ bd = rx->ddr[head];
+
+ replacement = intel_xpcie_alloc_rx_bd(xpcie);
+ if (!replacement) {
+ delay = msecs_to_jiffies(20);
+ break;
+ }
+
+ rc = intel_xpcie_map_dma(xpcie, replacement, DMA_FROM_DEVICE);
+ if (rc) {
+ dev_err(xpcie_to_dev(xpcie),
+ "failed to map rx bd (%d)\n", rc);
+ intel_xpcie_free_rx_bd(xpcie, replacement);
+ break;
+ }
+
+ status = intel_xpcie_get_td_status(td);
+ interface = intel_xpcie_get_td_interface(td);
+ length = intel_xpcie_get_td_length(td);
+ intel_xpcie_unmap_dma(xpcie, bd, DMA_FROM_DEVICE);
+
+ if (unlikely(status != XPCIE_DESC_STATUS_SUCCESS) ||
+ unlikely(interface >= XPCIE_NUM_INTERFACES)) {
+ dev_err(xpcie_to_dev(xpcie),
+ "rx desc failure, status(%u), interface(%u)\n",
+ status, interface);
+ intel_xpcie_free_rx_bd(xpcie, bd);
+ } else {
+ bd->interface = interface;
+ bd->length = length;
+ bd->next = NULL;
+
+ intel_xpcie_add_bd_to_interface(xpcie, bd);
+ }
+
+ rx->ddr[head] = replacement;
+ intel_xpcie_set_td_address(td, replacement->phys);
+ intel_xpcie_set_td_length(td, replacement->length);
+ head = XPCIE_CIRCULAR_INC(head, ndesc);
+ }
+
+ if (intel_xpcie_get_tdr_head(&rx->pipe) != head) {
+ intel_xpcie_set_tdr_head(&rx->pipe, head);
+ intel_xpcie_pci_raise_irq(xdev, DATA_RECEIVED, 1);
+ }
+
+ if (!replacement)
+ intel_xpcie_start_rx(xpcie, delay);
+}
+
+static void intel_xpcie_tx_event_handler(struct work_struct *work)
+{
+ struct xpcie *xpcie = container_of(work, struct xpcie, tx_event.work);
+ struct xpcie_dev *xdev = container_of(xpcie, struct xpcie_dev, xpcie);
+ struct xpcie_stream *tx = &xpcie->tx;
+ struct xpcie_transfer_desc *td;
+ u32 head, tail, old, ndesc;
+ struct xpcie_buf_desc *bd;
+ size_t bytes, buffers;
+ u16 status;
+
+ if (intel_xpcie_get_device_status(xpcie) != XPCIE_STATUS_RUN)
+ return;
+
+ ndesc = tx->pipe.ndesc;
+ old = tx->pipe.old;
+ tail = intel_xpcie_get_tdr_tail(&tx->pipe);
+ head = intel_xpcie_get_tdr_head(&tx->pipe);
+
+ /* clean old entries first */
+ while (old != head) {
+ bd = tx->ddr[old];
+ td = tx->pipe.tdr + old;
+ status = intel_xpcie_get_td_status(td);
+ if (status != XPCIE_DESC_STATUS_SUCCESS)
+ dev_err(xpcie_to_dev(xpcie),
+ "detected tx desc failure (%u)\n", status);
+
+ intel_xpcie_unmap_dma(xpcie, bd, DMA_TO_DEVICE);
+ intel_xpcie_free_tx_bd(xpcie, bd);
+ tx->ddr[old] = NULL;
+ old = XPCIE_CIRCULAR_INC(old, ndesc);
+ }
+ tx->pipe.old = old;
+
+ /* add new entries */
+ while (XPCIE_CIRCULAR_INC(tail, ndesc) != head) {
+ bd = intel_xpcie_list_get(&xpcie->write);
+ if (!bd)
+ break;
+
+ td = tx->pipe.tdr + tail;
+
+ if (intel_xpcie_map_dma(xpcie, bd, DMA_TO_DEVICE)) {
+ dev_err(xpcie_to_dev(xpcie),
+ "dma mapping error bd addr %p, size %zu\n",
+ bd->data, bd->length);
+ break;
+ }
+
+ tx->ddr[tail] = bd;
+ intel_xpcie_set_td_address(td, bd->phys);
+ intel_xpcie_set_td_length(td, bd->length);
+ intel_xpcie_set_td_interface(td, bd->interface);
+ intel_xpcie_set_td_status(td, XPCIE_DESC_STATUS_ERROR);
+
+ tail = XPCIE_CIRCULAR_INC(tail, ndesc);
+ }
+
+ if (intel_xpcie_get_tdr_tail(&tx->pipe) != tail) {
+ intel_xpcie_set_tdr_tail(&tx->pipe, tail);
+ intel_xpcie_pci_raise_irq(xdev, DATA_SENT, 1);
+ }
+
+ intel_xpcie_list_info(&xpcie->write, &bytes, &buffers);
+ if (buffers)
+ xpcie->tx_pending = true;
+ else
+ xpcie->tx_pending = false;
+}
+
+static irqreturn_t intel_xpcie_interrupt(int irq, void *args)
+{
+ struct xpcie_dev *xdev = args;
+ struct xpcie *xpcie;
+
+ xpcie = &xdev->xpcie;
+
+ if (intel_xpcie_get_doorbell(xpcie, FROM_DEVICE, DATA_SENT)) {
+ intel_xpcie_set_doorbell(xpcie, FROM_DEVICE, DATA_SENT, 0);
+ intel_xpcie_start_rx(xpcie, 0);
+ }
+ if (intel_xpcie_get_doorbell(xpcie, FROM_DEVICE, DATA_RECEIVED)) {
+ intel_xpcie_set_doorbell(xpcie, FROM_DEVICE, DATA_RECEIVED, 0);
+ if (xpcie->tx_pending)
+ intel_xpcie_start_tx(xpcie, 0);
+ }
+
+ return IRQ_HANDLED;
+}
+
+static int intel_xpcie_events_init(struct xpcie *xpcie)
+{
+ xpcie->rx_wq = alloc_ordered_workqueue(XPCIE_DRIVER_NAME,
+ WQ_MEM_RECLAIM | WQ_HIGHPRI);
+ if (!xpcie->rx_wq) {
+ dev_err(xpcie_to_dev(xpcie), "failed to allocate workqueue\n");
+ return -ENOMEM;
+ }
+
+ xpcie->tx_wq = alloc_ordered_workqueue(XPCIE_DRIVER_NAME,
+ WQ_MEM_RECLAIM | WQ_HIGHPRI);
+ if (!xpcie->tx_wq) {
+ dev_err(xpcie_to_dev(xpcie), "failed to allocate workqueue\n");
+ destroy_workqueue(xpcie->rx_wq);
+ return -ENOMEM;
+ }
+
+ INIT_DELAYED_WORK(&xpcie->rx_event, intel_xpcie_rx_event_handler);
+ INIT_DELAYED_WORK(&xpcie->tx_event, intel_xpcie_tx_event_handler);
+
+ return 0;
+}
+
+static void intel_xpcie_events_cleanup(struct xpcie *xpcie)
+{
+ cancel_delayed_work_sync(&xpcie->rx_event);
+ cancel_delayed_work_sync(&xpcie->tx_event);
+
+ destroy_workqueue(xpcie->rx_wq);
+ destroy_workqueue(xpcie->tx_wq);
+}
+
+int intel_xpcie_core_init(struct xpcie *xpcie)
+{
+ struct xpcie_dev *xdev = container_of(xpcie, struct xpcie_dev, xpcie);
+ int status, rc;
+
+ status = intel_xpcie_get_device_status(xpcie);
+ if (status != XPCIE_STATUS_RUN) {
+ dev_err(&xdev->pci->dev,
+ "device status not RUNNING (%d)\n", status);
+ rc = -EBUSY;
+ return rc;
+ }
+
+ intel_xpcie_version_check(xpcie);
+
+ rc = intel_xpcie_events_init(xpcie);
+ if (rc)
+ return rc;
+
+ rc = intel_xpcie_discover_txrx(xpcie);
+ if (rc)
+ goto error_txrx;
+
+ intel_xpcie_interfaces_init(xpcie);
+
+ rc = intel_xpcie_pci_register_irq(xdev, &intel_xpcie_interrupt);
+ if (rc)
+ goto error_txrx;
+
+ intel_xpcie_set_host_status(xpcie, XPCIE_STATUS_RUN);
+
+ return 0;
+
+error_txrx:
+ intel_xpcie_events_cleanup(xpcie);
+ intel_xpcie_set_host_status(xpcie, XPCIE_STATUS_ERROR);
+
+ return rc;
+}
+
+void intel_xpcie_core_cleanup(struct xpcie *xpcie)
+{
+ if (xpcie->status == XPCIE_STATUS_RUN) {
+ intel_xpcie_set_host_status(xpcie, XPCIE_STATUS_UNINIT);
+ intel_xpcie_events_cleanup(xpcie);
+ intel_xpcie_interfaces_cleanup(xpcie);
+ intel_xpcie_txrx_cleanup(xpcie);
+ }
+}
+
+int intel_xpcie_core_read(struct xpcie *xpcie, void *buffer, size_t *length,
+ uint32_t timeout_ms)
+{
+ long jiffies_timeout = (long)msecs_to_jiffies(timeout_ms);
+ struct xpcie_interface *inf = &xpcie->interfaces[0];
+ unsigned long jiffies_start = jiffies;
+ struct xpcie_buf_desc *bd;
+ size_t remaining, len;
+ long jiffies_passed = 0;
+ int ret;
+
+ if (*length == 0)
+ return -EINVAL;
+
+ if (xpcie->status != XPCIE_STATUS_RUN)
+ return -ENODEV;
+
+ len = *length;
+ remaining = len;
+ *length = 0;
+
+ ret = mutex_lock_interruptible(&inf->rlock);
+ if (ret < 0)
+ return -EINTR;
+
+ do {
+ while (!inf->data_avail) {
+ mutex_unlock(&inf->rlock);
+ if (timeout_ms == 0) {
+ ret = wait_event_interruptible(inf->rx_waitq,
+ inf->data_avail);
+ } else {
+ ret =
+ wait_event_interruptible_timeout(inf->rx_waitq,
+ inf->data_avail,
+ jiffies_timeout -
+ jiffies_passed);
+ if (ret == 0)
+ return -ETIME;
+ }
+ if (ret < 0 || xpcie->stop_flag)
+ return -EINTR;
+
+ ret = mutex_lock_interruptible(&inf->rlock);
+ if (ret < 0)
+ return -EINTR;
+ }
+
+ bd = (inf->partial_read) ? inf->partial_read :
+ intel_xpcie_list_get(&inf->read);
+ while (remaining && bd) {
+ size_t bcopy;
+
+ bcopy = min(remaining, bd->length);
+ memcpy(buffer, bd->data, bcopy);
+
+ buffer += bcopy;
+ remaining -= bcopy;
+ bd->data += bcopy;
+ bd->length -= bcopy;
+
+ if (bd->length == 0) {
+ intel_xpcie_free_rx_bd(xpcie, bd);
+ bd = intel_xpcie_list_get(&inf->read);
+ }
+ }
+
+ /* save for next time */
+ inf->partial_read = bd;
+
+ if (!bd)
+ inf->data_avail = false;
+
+ *length = len - remaining;
+
+ jiffies_passed = (long)jiffies - (long)jiffies_start;
+ } while (remaining > 0 && (jiffies_passed < jiffies_timeout ||
+ timeout_ms == 0));
+
+ mutex_unlock(&inf->rlock);
+
+ return 0;
+}
+
+int intel_xpcie_core_write(struct xpcie *xpcie, void *buffer, size_t *length,
+ uint32_t timeout_ms)
+{
+ long jiffies_timeout = (long)msecs_to_jiffies(timeout_ms);
+ struct xpcie_interface *inf = &xpcie->interfaces[0];
+ unsigned long jiffies_start = jiffies;
+ struct xpcie_buf_desc *bd, *head;
+ long jiffies_passed = 0;
+ size_t remaining, len;
+ int ret;
+
+ if (*length == 0)
+ return -EINVAL;
+
+ if (xpcie->status != XPCIE_STATUS_RUN)
+ return -ENODEV;
+
+ len = *length;
+ remaining = len;
+ *length = 0;
+
+ ret = mutex_lock_interruptible(&xpcie->wlock);
+ if (ret < 0)
+ return -EINTR;
+
+ do {
+ bd = intel_xpcie_alloc_tx_bd(xpcie);
+ head = bd;
+ while (!head) {
+ mutex_unlock(&xpcie->wlock);
+ if (timeout_ms == 0) {
+ ret =
+ wait_event_interruptible(xpcie->tx_waitq,
+ !xpcie->no_tx_buffer);
+ } else {
+ ret =
+ wait_event_interruptible_timeout(xpcie->tx_waitq,
+ !xpcie->no_tx_buffer,
+ jiffies_timeout -
+ jiffies_passed);
+ if (ret == 0)
+ return -ETIME;
+ }
+ if (ret < 0 || xpcie->stop_flag)
+ return -EINTR;
+
+ ret = mutex_lock_interruptible(&xpcie->wlock);
+ if (ret < 0)
+ return -EINTR;
+
+ bd = intel_xpcie_alloc_tx_bd(xpcie);
+ head = bd;
+ }
+
+ while (remaining && bd) {
+ size_t bcopy;
+
+ bcopy = min(bd->length, remaining);
+ memcpy(bd->data, buffer, bcopy);
+
+ buffer += bcopy;
+ remaining -= bcopy;
+ bd->length = bcopy;
+ bd->interface = inf->id;
+
+ if (remaining) {
+ bd->next = intel_xpcie_alloc_tx_bd(xpcie);
+ bd = bd->next;
+ }
+ }
+
+ intel_xpcie_list_put(&xpcie->write, head);
+ intel_xpcie_start_tx(xpcie, 0);
+
+ *length = len - remaining;
+
+ jiffies_passed = (long)jiffies - (long)jiffies_start;
+ } while (remaining > 0 && (jiffies_passed < jiffies_timeout ||
+ timeout_ms == 0));
+
+ mutex_unlock(&xpcie->wlock);
+
+ return 0;
+}
diff --git a/drivers/misc/xlink-pcie/remote_host/pci.c b/drivers/misc/xlink-pcie/remote_host/pci.c
index 0ca0755b591f..f92a78f41324 100644
--- a/drivers/misc/xlink-pcie/remote_host/pci.c
+++ b/drivers/misc/xlink-pcie/remote_host/pci.c
@@ -208,10 +208,8 @@ static void xpcie_device_poll(struct work_struct *work)
{
struct xpcie_dev *xdev = container_of(work, struct xpcie_dev,
wait_event.work);
- u32 dev_status = intel_xpcie_ioread32(xdev->xpcie.mmio +
- XPCIE_MMIO_DEV_STATUS);

- if (dev_status < XPCIE_STATUS_RUN)
+ if (intel_xpcie_get_device_status(&xdev->xpcie) < XPCIE_STATUS_RUN)
schedule_delayed_work(&xdev->wait_event,
msecs_to_jiffies(100));
else
@@ -224,9 +222,10 @@ static int intel_xpcie_pci_prepare_dev_reset(struct xpcie_dev *xdev,
if (mutex_lock_interruptible(&xdev->lock))
return -EINTR;

- if (xdev->core_irq_callback)
+ if (xdev->core_irq_callback) {
xdev->core_irq_callback = NULL;
-
+ intel_xpcie_core_cleanup(&xdev->xpcie);
+ }
xdev->xpcie.status = XPCIE_STATUS_OFF;
if (notify)
intel_xpcie_pci_raise_irq(xdev, DEV_EVENT, REQUEST_RESET);
@@ -326,6 +325,8 @@ int intel_xpcie_pci_cleanup(struct xpcie_dev *xdev)
xdev->core_irq_callback = NULL;
intel_xpcie_pci_irq_cleanup(xdev);

+ intel_xpcie_core_cleanup(&xdev->xpcie);
+
intel_xpcie_pci_unmap_bar(xdev);
pci_release_regions(xdev->pci);
pci_disable_device(xdev->pci);
@@ -359,6 +360,7 @@ int intel_xpcie_pci_raise_irq(struct xpcie_dev *xdev,
{
u16 pci_status;

+ intel_xpcie_set_doorbell(&xdev->xpcie, TO_DEVICE, type, value);
pci_read_config_word(xdev->pci, PCI_STATUS, &pci_status);

return 0;
@@ -445,7 +447,43 @@ int intel_xpcie_pci_connect_device(u32 id)
goto connect_cleanup;
}

+ rc = intel_xpcie_core_init(&xdev->xpcie);
+ if (rc < 0) {
+ dev_err(&xdev->pci->dev, "failed to sync with device\n");
+ goto connect_cleanup;
+ }
+
connect_cleanup:
mutex_unlock(&xdev->lock);
return rc;
}
+
+int intel_xpcie_pci_read(u32 id, void *data, size_t *size, u32 timeout)
+{
+ struct xpcie_dev *xdev = intel_xpcie_get_device_by_id(id);
+
+ if (!xdev)
+ return -ENODEV;
+
+ return intel_xpcie_core_read(&xdev->xpcie, data, size, timeout);
+}
+
+int intel_xpcie_pci_write(u32 id, void *data, size_t *size, u32 timeout)
+{
+ struct xpcie_dev *xdev = intel_xpcie_get_device_by_id(id);
+
+ if (!xdev)
+ return -ENODEV;
+
+ return intel_xpcie_core_write(&xdev->xpcie, data, size, timeout);
+}
+
+int intel_xpcie_pci_reset_device(u32 id)
+{
+ struct xpcie_dev *xdev = intel_xpcie_get_device_by_id(id);
+
+ if (!xdev)
+ return -ENOMEM;
+
+ return intel_xpcie_pci_prepare_dev_reset(xdev, true);
+}
--
2.17.1

2020-12-01 22:42:13

by mark gross

[permalink] [raw]
Subject: [PATCH 05/22] keembay-vpu-ipc: Add Keem Bay VPU IPC module

From: Paul Murphy <[email protected]>

Intel Keem Bay SoC contains a Vision Processing Unit (VPU) to enable
machine vision and other applications.

Enable Linux to control the VPU processor and provides an interface to
the Keem Bay IPC for communicating with the VPU firmware.

Specifically the driver provides the following functionality to other
kernel components:
- Starting (including loading the VPU firmware) / Stopping / Rebooting
the VPU.
- Getting notifications of VPU events (e.g., WDT events).
- Communicating with the VPU via the Keem Bay IPC mechanism.

In addition to the above, the driver also exposes SoC information (like
stepping, device ID, etc.) to user-space via sysfs. Specifically, the
following sysfs files are provided:
- /sys/firmware/keembay-vpu-ipc/device_id
- /sys/firmware/keembay-vpu-ipc/feature_exclusion
- /sys/firmware/keembay-vpu-ipc/hardware_id
- /sys/firmware/keembay-vpu-ipc/sku
- /sys/firmware/keembay-vpu-ipc/stepping

Reviewed-by: Mark Gross <[email protected]>
Signed-off-by: Paul Murphy <[email protected]>
Co-developed-by: Daniele Alessandrelli <[email protected]>
Signed-off-by: Daniele Alessandrelli <[email protected]>
Signed-off-by: Mark Gross <[email protected]>
---
MAINTAINERS | 9 +
drivers/soc/intel/Kconfig | 15 +
drivers/soc/intel/Makefile | 3 +-
drivers/soc/intel/keembay-vpu-ipc.c | 2036 +++++++++++++++++++++
include/linux/soc/intel/keembay-vpu-ipc.h | 62 +
5 files changed, 2124 insertions(+), 1 deletion(-)
create mode 100644 drivers/soc/intel/keembay-vpu-ipc.c
create mode 100644 include/linux/soc/intel/keembay-vpu-ipc.h

diff --git a/MAINTAINERS b/MAINTAINERS
index 64475ba4ea9d..f95cef927d19 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8963,6 +8963,15 @@ F: Documentation/devicetree/bindings/soc/intel/intel,keembay-ipc.yaml
F: drivers/soc/intel/keembay-ipc.c
F: include/linux/soc/intel/keembay-ipc.h

+INTEL KEEM BAY VPU IPC DRIVER
+M: Paul J Murphy <[email protected]>
+M: Daniele Alessandrelli <[email protected]>
+M: Mark Gross <[email protected]>
+S: Maintained
+F: Documentation/devicetree/bindings/soc/intel/intel,keembay-vpu-ipc.yaml
+F: drivers/soc/intel/keembay-vpu-ipc.c
+F: include/linux/soc/intel/keembay-vpu-ipc.h
+
INTEL MANAGEMENT ENGINE (mei)
M: Tomas Winkler <[email protected]>
L: [email protected]
diff --git a/drivers/soc/intel/Kconfig b/drivers/soc/intel/Kconfig
index 1f5c05b97649..83ce8c4a7851 100644
--- a/drivers/soc/intel/Kconfig
+++ b/drivers/soc/intel/Kconfig
@@ -14,4 +14,19 @@ config KEEMBAY_IPC

Select this if you are compiling the Kernel for an Intel SoC that
includes the Intel Vision Processing Unit (VPU) such as Keem Bay.
+
+config KEEMBAY_VPU_IPC
+ tristate "Intel Keem Bay VPU IPC Driver"
+ depends on KEEMBAY_IPC
+ depends on HAVE_ARM_SMCCC
+ help
+ This option enables support for loading and communicating with
+ the firmware on the Vision Processing Unit (VPU) of the Keem Bay
+ SoC. The driver depends on the Keem Bay IPC driver to do
+ communication, and it depends on secure world monitor software to
+ do the control of the VPU state.
+
+ Select this if you are compiling the Kernel for an Intel SoC that
+ includes the Intel Vision Processing Unit (VPU) such as Keem Bay.
+
endmenu
diff --git a/drivers/soc/intel/Makefile b/drivers/soc/intel/Makefile
index ecf0246e7822..363a81848843 100644
--- a/drivers/soc/intel/Makefile
+++ b/drivers/soc/intel/Makefile
@@ -1,4 +1,5 @@
#
# Makefile for Keem Bay IPC Linux driver
#
-obj-$(CONFIG_KEEMBAY_IPC) += keembay-ipc.o
+obj-$(CONFIG_KEEMBAY_IPC) += keembay-ipc.o
+obj-$(CONFIG_KEEMBAY_VPU_IPC) += keembay-vpu-ipc.o
diff --git a/drivers/soc/intel/keembay-vpu-ipc.c b/drivers/soc/intel/keembay-vpu-ipc.c
new file mode 100644
index 000000000000..bcf9bb4a225a
--- /dev/null
+++ b/drivers/soc/intel/keembay-vpu-ipc.c
@@ -0,0 +1,2036 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * Keem Bay VPU IPC Driver.
+ *
+ * Copyright (c) 2018-2020 Intel Corporation.
+ *
+ * The purpose of this driver is to facilitate booting, control and
+ * communication with the VPU IP on the Keem Bay SoC.
+ *
+ * Specifically the driver provides the following functionality to other kernel
+ * components:
+ * - Loading the VPU firmware into DDR for the VPU to execute.
+ * - Starting / Stopping / Rebooting the VPU.
+ * - Getting notifications of VPU events (e.g., WDT events).
+ * - Communicating with the VPU using the Keem Bay IPC mechanism.
+ *
+ * In addition to the above, the driver also exposes SoC information (like
+ * stepping, device ID, etc.) to user-space via sysfs.
+ *
+ *
+ * VPU Firmware loading
+ * --------------------
+ *
+ * The VPU Firmware consists of both the RTOS and the application code meant to
+ * be run by the VPU.
+ *
+ * The VPU Firmware is loaded into DDR using the Linux Firmware API. The
+ * firmware is loaded into a specific reserved memory region in DDR and
+ * executed by the VPU directly from there.
+ *
+ * The VPU Firmware binary is expected to have the following format:
+ *
+ * +------------------+ 0x0
+ * | Header |
+ * +------------------+ 0x1000
+ * | FW Version Area |
+ * +------------------+ 0x2000
+ * | FW Image |
+ * +------------------+ 0x2000 + FW image size
+ * | x509 Certificate |
+ * +------------------+
+ *
+ * Note: the x509 Certificate is ignored for now.
+ *
+ * As part of the firmware loading process, the driver performs the following
+ * operations:
+ * - It parses the VPU firmware binary.
+ * - It loads the FW version area to the DDR location expected by the VPU
+ * firmware and specified in the FW header.
+ * - It loads the FW image to the location specified in the FW header.
+ * - It prepares the boot parameters to be passed to the VPU firmware and loads
+ * them at the location specified in the FW header.
+ *
+ * VPU Start / Stop / Reboot
+ * -------------------------
+ *
+ * The VPU is started / stopped by the SoC firmware, not by this driver
+ * directly. This driver just calls the VPU_BOOT and VPU_STOP SMC SiP services
+ * provided by the SoC firmware.
+ *
+ * Reboot is performed by stopping and re-starting the VPU, including
+ * re-loading the VPU firmware (this is because the VPU firmware .data and .bss
+ * sections need to be re-initialized).
+ *
+ * Sysfs interface
+ * ---------------
+ *
+ * This module exposes SoC information via sysfs. The following sysfs files are
+ * created by this module:
+ * - /sys/firmware/keembay-vpu-ipc/device_id
+ * - /sys/firmware/keembay-vpu-ipc/feature_exclusion
+ * - /sys/firmware/keembay-vpu-ipc/hardware_id
+ * - /sys/firmware/keembay-vpu-ipc/sku
+ * - /sys/firmware/keembay-vpu-ipc/stepping
+ */
+
+#include <linux/arm-smccc.h>
+#include <linux/clk.h>
+#include <linux/dma-direct.h>
+#include <linux/dma-mapping.h>
+#include <linux/firmware.h>
+#include <linux/interrupt.h>
+#include <linux/io.h>
+#include <linux/kernel.h>
+#include <linux/kthread.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/of_platform.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
+
+#include <linux/soc/intel/keembay-ipc.h>
+#include <linux/soc/intel/keembay-vpu-ipc.h>
+
+/* Function ID for the SiP SMC to boot the VPU */
+#define KMB_SIP_SVC_VPU_BOOT 0xFF10
+
+/* Function ID for the SiP SMC to stop the VPU */
+#define KMB_SIP_SVC_VPU_STOP 0xFF16
+
+/* Device tree "memory-region" for VPU firmware area */
+#define VPU_IPC_FW_AREA_IDX 0
+
+/* Device tree region for VPU driver to store X509 region */
+#define VPU_IPC_X509_AREA_IDX 1
+
+/* Device tree "memory-region" for MSS IPC header area */
+#define MSS_IPC_AREA_IDX 2
+
+/*
+ * These are the parameters of the ready message to be received
+ * from the VPU when it is booted correctly.
+ */
+#define READY_MESSAGE_IPC_CHANNEL 0xA
+
+/* Ready message timeout, in ms */
+#define READY_MESSAGE_TIMEOUT_MS 2000
+
+/* Ready message 'physical data address', which is actually a command. */
+#define READY_MESSAGE_EXPECTED_PADDR 0x424f4f54
+
+/* Ready message size */
+#define READY_MESSAGE_EXPECTED_SIZE 0
+
+/* Size of version information in the header */
+#define VPU_VERSION_SIZE 32
+
+/* Version of header that this driver supports. */
+#define HEADER_VERSION_SUPPORTED 0x1
+
+/* Maximum size allowed for firmware version region */
+#define MAX_FIRMWARE_VERSION_SIZE 0x1000
+
+/* Size allowed for header region */
+#define MAX_HEADER_SIZE 0x1000
+
+/* VPU reset vector must be aligned to 4kB. */
+#define VPU_RESET_VECTOR_ALIGNMENT 0x1000
+
+/* Watchdog timer reset trigger */
+#define TIM_WATCHDOG 0x0
+
+/* Watchdog counter enable register */
+#define TIM_WDOG_EN 0x8
+
+/* Write access to protected registers */
+#define TIM_SAFE 0xC
+
+/* Watchdog timer count value */
+#define TIM_WATCHDOG_RESET_VALUE 0xFFFFFFFF
+
+/* Watchdog timer safe value */
+#define TIM_SAFE_ENABLE 0xf1d0dead
+
+/* Watchdog timeout interrupt clear bit */
+#define TIM_GEN_CONFIG_WDOG_TO_INT_CLR BIT(9)
+
+/* Magic number for the boot parameters. */
+#define BOOT_PARAMS_MAGIC_NUMBER 0x00010000
+
+/* Maximum size of string of form "pll_i_out_j" */
+#define PLL_STRING_SIZE 128
+
+/* Number of PLLs */
+#define NUM_PLLS 3
+
+/* Every PLL has this many outputs. */
+#define NUM_PLL_OUTPUTS 4
+
+/* SoC SKU length, in bytes. */
+#define SOC_INFO_SKU_BYTES 6
+
+/* SoC stepping length, in bytes. */
+#define SOC_INFO_STEPPING_BYTES 2
+
+/**
+ * struct boot_parameters - Boot parameters passed to the VPU.
+ * @magic_number: Magic number to indicate structure populated
+ * @vpu_id: ID to be passed to the VPU firmware.
+ * @reserved_0: Reserved memory for other 'header' information
+ * @cpu_frequency_hz: Frequency that the CPU is running at
+ * @pll0_out: Frequency of each of the outputs of PLL 0
+ * @pll1_out: Frequency of each of the outputs of PLL 1
+ * @pll2_out: Frequency of each of the outputs of PLL 2
+ * @reserved_1: Reserved memory for other clock frequencies
+ * @mss_ipc_header_address: Base address of MSS IPC header region
+ * @mss_ipc_header_area_size: Size of MSS IPC header region
+ * @mmio_address: MMIO region for VPU communication
+ * @mmio_area_size: Size of MMIO region for VPU communication
+ * @reserved_2: Reserved memory for other memory regions
+ * @mss_wdt_to_irq_a53_redir: MSS redirects WDT TO IRQ to this ARM IRQ number
+ * @nce_wdt_to_irq_a53_redir: NCE redirects WDT TO IRQ to this ARM IRQ number
+ * @vpu_to_host_irq: VPU to host notification IRQ
+ * @reserved_3: Reserved memory for other configurations
+ * @a53ss_version_id: SoC A53SS_VERSION_ID register value
+ * @si_stepping: Silicon stepping, 2 characters
+ * @device_id: 64 bits of device ID info from fuses
+ * @feature_exclusion: 64 bits of feature exclusion info from fuses
+ * @sku: 64-bit SKU identifier.
+ * @reserved_4: Reserved memory for other information
+ * @reserved_5: Unused/reserved memory
+ */
+struct boot_parameters {
+ /* Header: 0x0 - 0x1F */
+ u32 magic_number;
+ u32 vpu_id;
+ u32 reserved_0[6];
+ /* Clock frequencies: 0x20 - 0xFF */
+ u32 cpu_frequency_hz;
+ u32 pll0_out[NUM_PLL_OUTPUTS];
+ u32 pll1_out[NUM_PLL_OUTPUTS];
+ u32 pll2_out[NUM_PLL_OUTPUTS];
+ u32 reserved_1[43];
+ /* Memory regions: 0x100 - 0x1FF */
+ u64 mss_ipc_header_address;
+ u32 mss_ipc_header_area_size;
+ u64 mmio_address;
+ u32 mmio_area_size;
+ u32 reserved_2[58];
+ /* IRQ re-direct numbers: 0x200 - 0x2FF */
+ u32 mss_wdt_to_irq_a53_redir;
+ u32 nce_wdt_to_irq_a53_redir;
+ u32 vpu_to_host_irq;
+ u32 reserved_3[61];
+ /* Silicon information: 0x300 - 0x3FF */
+ u32 a53ss_version_id;
+ u32 si_stepping;
+ u64 device_id;
+ u64 feature_exclusion;
+ u64 sku;
+ u32 reserved_4[56];
+ /* Unused/reserved: 0x400 - 0xFFF */
+ u32 reserved_5[0x300];
+} __packed;
+
+/**
+ * struct firmware_header - Firmware header information
+ * @header_ver: This header version dictates content structure of
+ * remainder of firmware image, including the header
+ * itself.
+ * @image_format: Image format defines how the loader will handle the
+ * 'firmware image'.
+ * @image_load_addr: VPU address where the firmware image must be loaded to.
+ * @image_size: Size of the image.
+ * @entry_point: Entry point for the VPU firmware (this is a VPU
+ * address).
+ * @vpu_ver: Version of the VPU firmware.
+ * @compression_type: Type of compression used for the VPU firmware.
+ * @fw_ver_load_addr: VPU address where to load the data in the FW version
+ * area of the binary.
+ * @fw_ver_size: Size of the FW version.
+ * @config_load_addr: VPU IPC driver will populate the 4kB of configuration
+ * data to this address.
+ */
+struct firmware_header {
+ u32 header_ver;
+ u32 image_format;
+ u64 image_load_addr;
+ u32 image_size;
+ u64 entry_point;
+ u8 vpu_ver[VPU_VERSION_SIZE];
+ u32 compression_type;
+ u64 fw_ver_load_addr;
+ u32 fw_ver_size;
+ u64 config_load_addr;
+} __packed;
+
+/**
+ * struct vpu_mem - Information about reserved memory shared with VPU.
+ * @vaddr: The virtual address of the memory region.
+ * @paddr: The (CPU) physical address of the memory region.
+ * @vpu_addr: The VPU address of the memory region.
+ * @size: The size of the memory region.
+ */
+struct vpu_mem {
+ void *vaddr;
+ phys_addr_t paddr;
+ dma_addr_t vpu_addr;
+ size_t size;
+};
+
+/**
+ * struct vpu_mem - Information about reserved memory shared with ATF.
+ * @vaddr: The virtual address of the memory region.
+ * @paddr: The physical address of the memory region.
+ * @size: The size of the memory region.
+ */
+struct atf_mem {
+ void __iomem *vaddr;
+ phys_addr_t paddr;
+ size_t size;
+};
+
+/**
+ * struct vpu_ipc_dev - The VPU IPC device structure.
+ * @pdev: Platform device associated with this VPU IPC device.
+ * @state: The current state of the device's finite state machine.
+ * @reserved_mem: VPU firmware reserved memory region. The VPU firmware,
+ * VPU firmware version and the boot parameters are loaded
+ * inside this region.
+ * @x509_mem: x509 reserved memory region.
+ * @mss_ipc_mem: The reserved memory from which the VPU is expected to
+ * allocate its own IPC buffers.
+ * @boot_vec_paddr: The VPU entry point (as specified in the VPU FW binary).
+ * @boot_params: Pointer to the VPU boot parameters.
+ * @fw_res: The memory region where the VPU FW was loaded.
+ * @ready_message_task: The ktrhead instantiated to handle the reception of the
+ * VPU ready message.
+ * @lock: Spinlock protecting @state.
+ * @cpu_clock: The main clock powering the VPU IP.
+ * @pll: Array of PLL clocks.
+ * @nce_irq: IRQ number of the A53 re-direct IRQ used for receiving
+ * the NCE WDT timeout interrupt.
+ * @mss_irq: IRQ number of the A53 re-direct IRQ which will be used
+ * for receiving the MSS WDT timeout interrupt.
+ * @nce_wdt_redirect: Re-direct IRQ for NCE ICB.
+ * @mss_wdt_redirect: Re-direct IRQ for MSS ICB.
+ * @imr: Isolated Memory Region (IMR) to be used to protect the
+ * loaded VPU firmware.
+ * @vpu_id: The ID of the VPU associated with this device.
+ * @nce_wdt_reg: NCE WDT registers.
+ * @nce_tim_cfg_reg: NCE TIM registers.
+ * @mss_wdt_reg: MSS WDT registers.
+ * @mss_tim_cfg_reg: MSS TIM registers.
+ * @nce_wdt_count: Number of NCE WDT timeout event occurred since device
+ * probing.
+ * @mss_wdt_count: Number of MSS WDT timeout event occurred since device
+ * probing.
+ * @ready_queue: Wait queue for waiting on VPU to be ready.
+ * @ipc_dev: The IPC device to use for IPC communication.
+ * @firmware_name: The name of the firmware binary to be loaded.
+ * @callback: The callback executed when CONNECT / DISCONNECT events
+ * happen.
+ */
+struct vpu_ipc_dev {
+ struct platform_device *pdev;
+ enum intel_keembay_vpu_state state;
+ struct vpu_mem reserved_mem;
+ struct atf_mem x509_mem;
+ struct vpu_mem mss_ipc_mem;
+ u64 boot_vec_paddr;
+ struct boot_parameters *boot_params;
+ struct resource fw_res;
+ struct task_struct *ready_message_task;
+ spinlock_t lock; /* Protects the 'state' field above. */
+ struct clk *cpu_clock;
+ struct clk *pll[NUM_PLLS][NUM_PLL_OUTPUTS];
+ int nce_irq;
+ int mss_irq;
+ u32 nce_wdt_redirect;
+ u32 mss_wdt_redirect;
+ u32 imr;
+ u32 vpu_id;
+ void __iomem *nce_wdt_reg;
+ void __iomem *nce_tim_cfg_reg;
+ void __iomem *mss_wdt_reg;
+ void __iomem *mss_tim_cfg_reg;
+ unsigned int nce_wdt_count;
+ unsigned int mss_wdt_count;
+ wait_queue_head_t ready_queue;
+ struct device *ipc_dev;
+ char *firmware_name;
+ void (*callback)(struct device *dev, enum intel_keembay_vpu_event);
+};
+
+/**
+ * struct vpu_ipc_soc_info - VPU SKU information.
+ * @device_id: Value of device ID e-fuse.
+ * @feature_exclusion: Value of feature exclusion e-fuse.
+ * @hardware_id: Hardware identifier.
+ * @stepping: Silicon stepping.
+ * @sku: SKU identifier.
+ *
+ * SoC information read from the device-tree and exported via sysfs.
+ */
+struct vpu_ipc_soc_info {
+ u64 device_id;
+ u64 feature_exclusion;
+ u32 hardware_id;
+ u8 stepping[SOC_INFO_STEPPING_BYTES];
+ u8 sku[SOC_INFO_SKU_BYTES];
+};
+
+/**
+ * enum keembay_vpu_event - Internal events handled by the driver state machine.
+ * @KEEMBAY_VPU_EVENT_BOOT: VPU booted.
+ * @KEEMBAY_VPU_EVENT_BOOT_FAILED: VPU boot failed.
+ * @KEEMBAY_VPU_EVENT_STOP: VPU stop initiated.
+ * @KEEMBAY_VPU_EVENT_STOP_COMPLETE: VPU stop completed.
+ * @KEEMBAY_VPU_EVENT_NCE_WDT_TIMEOUT: NCE watchdog triggered.
+ * @KEEMBAY_VPU_EVENT_MSS_WDT_TIMEOUT: MSS watchdog triggered.
+ * @KEEMBAY_VPU_EVENT_MSS_READY_OK: VPU ready message successfully received.
+ * @KEEMBAY_VPU_EVENT_MSS_READY_FAIL: Failed to receive VPU ready message.
+ */
+enum keembay_vpu_event {
+ KEEMBAY_VPU_EVENT_BOOT = 0,
+ KEEMBAY_VPU_EVENT_BOOT_FAILED,
+ KEEMBAY_VPU_EVENT_STOP,
+ KEEMBAY_VPU_EVENT_STOP_COMPLETE,
+ KEEMBAY_VPU_EVENT_NCE_WDT_TIMEOUT,
+ KEEMBAY_VPU_EVENT_MSS_WDT_TIMEOUT,
+ KEEMBAY_VPU_EVENT_MSS_READY_OK,
+ KEEMBAY_VPU_EVENT_MSS_READY_FAIL
+};
+
+static struct vpu_ipc_dev *to_vpu_dev(struct device *dev);
+
+/* Variable containing SoC information. */
+static struct vpu_ipc_soc_info *vpu_ipc_soc_info;
+
+/**
+ * vpu_ipc_notify_event() - Trigger callback
+ * @vpu_dev: Private data
+ * @event: Event to notify
+ *
+ * This function is called when an event has occurred. If a callback has
+ * been registered it is called with the device and event as arguments.
+ *
+ * This function can be called from interrupt context.
+ */
+static void vpu_ipc_notify_event(struct vpu_ipc_dev *vpu_dev,
+ enum intel_keembay_vpu_event event)
+{
+ struct device *dev = &vpu_dev->pdev->dev;
+
+ if (vpu_dev->callback)
+ vpu_dev->callback(dev, event);
+}
+
+/**
+ * vpu_ipc_handle_event() - Handle events and optionally update state
+ *
+ * @vpu_dev: Private data
+ * @event: Event that has occurred
+ *
+ * This function is called in the case that an event has occurred. This
+ * function tells the calling code if the event is valid for the current state
+ * and also updates the internal state accordingly to the event.
+ *
+ * This function can be called from interrupt context.
+ *
+ * Returns: 0 for success otherwise negative error value
+ */
+static int vpu_ipc_handle_event(struct vpu_ipc_dev *vpu_dev,
+ enum keembay_vpu_event event)
+{
+ struct device *dev = &vpu_dev->pdev->dev;
+ unsigned long flags;
+ int rc = -EINVAL;
+
+ /*
+ * Atomic update of state.
+ * Note: this function is called by the WDT IRQ handlers; therefore
+ * we must use the spin_lock_irqsave().
+ */
+ spin_lock_irqsave(&vpu_dev->lock, flags);
+
+ switch (vpu_dev->state) {
+ case KEEMBAY_VPU_OFF:
+ if (event == KEEMBAY_VPU_EVENT_BOOT) {
+ vpu_dev->state = KEEMBAY_VPU_BUSY;
+ rc = 0;
+ }
+ break;
+ case KEEMBAY_VPU_BUSY:
+ if (event == KEEMBAY_VPU_EVENT_MSS_READY_OK) {
+ vpu_dev->state = KEEMBAY_VPU_READY;
+ vpu_ipc_notify_event(vpu_dev,
+ KEEMBAY_VPU_NOTIFY_CONNECT);
+ rc = 0;
+ break;
+ }
+ if (event == KEEMBAY_VPU_EVENT_MSS_READY_FAIL ||
+ event == KEEMBAY_VPU_EVENT_BOOT_FAILED) {
+ vpu_dev->state = KEEMBAY_VPU_ERROR;
+ rc = 0;
+ }
+ break;
+ case KEEMBAY_VPU_READY:
+ if (event != KEEMBAY_VPU_EVENT_MSS_READY_OK)
+ vpu_ipc_notify_event(vpu_dev,
+ KEEMBAY_VPU_NOTIFY_DISCONNECT);
+
+ if (event == KEEMBAY_VPU_EVENT_MSS_READY_FAIL ||
+ event == KEEMBAY_VPU_EVENT_BOOT_FAILED) {
+ vpu_dev->state = KEEMBAY_VPU_ERROR;
+ rc = 0;
+ break;
+ }
+ if (event == KEEMBAY_VPU_EVENT_NCE_WDT_TIMEOUT ||
+ event == KEEMBAY_VPU_EVENT_MSS_WDT_TIMEOUT) {
+ vpu_dev->state = KEEMBAY_VPU_ERROR;
+ rc = 0;
+ break;
+ }
+ fallthrough;
+ case KEEMBAY_VPU_ERROR:
+ if (event == KEEMBAY_VPU_EVENT_BOOT) {
+ vpu_dev->state = KEEMBAY_VPU_BUSY;
+ rc = 0;
+ break;
+ }
+ if (event == KEEMBAY_VPU_EVENT_STOP) {
+ vpu_dev->state = KEEMBAY_VPU_STOPPING;
+ rc = 0;
+ }
+ break;
+ case KEEMBAY_VPU_STOPPING:
+ if (event == KEEMBAY_VPU_EVENT_STOP_COMPLETE) {
+ vpu_dev->state = KEEMBAY_VPU_OFF;
+ rc = 0;
+ }
+ break;
+ default:
+ break;
+ }
+
+ spin_unlock_irqrestore(&vpu_dev->lock, flags);
+
+ if (rc)
+ dev_err(dev, "Can't handle event %d in state %d\n",
+ event, vpu_dev->state);
+
+ return rc;
+}
+
+/**
+ * clear_and_disable_vpu_wdt() - Clear and disable VPU WDT.
+ * @wdt_base: Base address of the WDT register.
+ * @tim_cfg_base: Base address of the associated TIM configuration
+ * register.
+ */
+static void clear_and_disable_vpu_wdt(u8 __iomem *wdt_base,
+ u8 __iomem *tim_cfg_base)
+{
+ u32 tim_gen_config;
+
+ /* Enable writing and set non-zero WDT value */
+ iowrite32(TIM_SAFE_ENABLE, wdt_base + TIM_SAFE);
+ iowrite32(TIM_WATCHDOG_RESET_VALUE, wdt_base + TIM_WATCHDOG);
+
+ /* Enable writing and disable watchdog timer */
+ iowrite32(TIM_SAFE_ENABLE, wdt_base + TIM_SAFE);
+ iowrite32(0, wdt_base + TIM_WDOG_EN);
+
+ /* Now clear the timeout interrupt */
+ tim_gen_config = ioread32(tim_cfg_base);
+ tim_gen_config &= ~(TIM_GEN_CONFIG_WDOG_TO_INT_CLR);
+ iowrite32(tim_gen_config, tim_cfg_base);
+}
+
+static irqreturn_t nce_wdt_irq_handler(int irq, void *ptr)
+{
+ struct vpu_ipc_dev *vpu_dev = ptr;
+ struct device *dev = &vpu_dev->pdev->dev;
+ int rc;
+
+ vpu_ipc_notify_event(vpu_dev, KEEMBAY_VPU_NOTIFY_NCE_WDT);
+ dev_dbg_ratelimited(dev, "NCE WDT IRQ occurred.\n");
+
+ clear_and_disable_vpu_wdt(vpu_dev->nce_wdt_reg,
+ vpu_dev->nce_tim_cfg_reg);
+ /* Update driver state machine. */
+ rc = vpu_ipc_handle_event(vpu_dev, KEEMBAY_VPU_EVENT_NCE_WDT_TIMEOUT);
+ if (rc < 0)
+ dev_warn_ratelimited(dev, "Unexpected NCE WDT event.\n");
+
+ vpu_dev->nce_wdt_count++;
+
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t mss_wdt_irq_handler(int irq, void *ptr)
+{
+ struct vpu_ipc_dev *vpu_dev = ptr;
+ struct device *dev = &vpu_dev->pdev->dev;
+ int rc;
+
+ vpu_ipc_notify_event(vpu_dev, KEEMBAY_VPU_NOTIFY_MSS_WDT);
+ dev_dbg_ratelimited(dev, "MSS WDT IRQ occurred.\n");
+
+ clear_and_disable_vpu_wdt(vpu_dev->mss_wdt_reg,
+ vpu_dev->mss_tim_cfg_reg);
+ /* Update driver state machine. */
+ rc = vpu_ipc_handle_event(vpu_dev, KEEMBAY_VPU_EVENT_MSS_WDT_TIMEOUT);
+ if (rc < 0)
+ dev_warn_ratelimited(dev, "Unexpected MSS WDT event.\n");
+
+ vpu_dev->mss_wdt_count++;
+
+ return IRQ_HANDLED;
+}
+
+static resource_size_t get_reserved_mem_size(struct device *dev,
+ unsigned int idx)
+{
+ struct resource mem;
+ struct device_node *np;
+ int rc;
+
+ np = of_parse_phandle(dev->of_node, "memory-region", idx);
+ if (!np) {
+ pr_err("Couldn't find memory-region %d\n", idx);
+ return 0;
+ }
+
+ rc = of_address_to_resource(np, 0, &mem);
+ if (rc) {
+ pr_err("Couldn't map address to resource\n");
+ return 0;
+ }
+
+ return resource_size(&mem);
+}
+
+static int setup_vpu_fw_region(struct vpu_ipc_dev *vpu_dev)
+{
+ struct device *dev = &vpu_dev->pdev->dev;
+ struct vpu_mem *rsvd_mem = &vpu_dev->reserved_mem;
+ int rc;
+
+ rc = of_reserved_mem_device_init(dev);
+ if (rc) {
+ dev_err(dev, "Failed to initialise device reserved memory.\n");
+ return rc;
+ }
+
+ rsvd_mem->size = get_reserved_mem_size(dev, VPU_IPC_FW_AREA_IDX);
+ if (rsvd_mem->size == 0) {
+ dev_err(dev, "Couldn't get size of reserved memory region.\n");
+ rc = -ENODEV;
+ goto setup_vpu_fw_fail;
+ }
+
+ rsvd_mem->vaddr = dmam_alloc_coherent(dev, rsvd_mem->size,
+ &rsvd_mem->vpu_addr, GFP_KERNEL);
+ /* Get the physical address of the reserved memory region. */
+ rsvd_mem->paddr = dma_to_phys(dev, vpu_dev->reserved_mem.vpu_addr);
+
+ if (!rsvd_mem->vaddr) {
+ dev_err(dev, "Failed to allocate memory for firmware.\n");
+ rc = -ENOMEM;
+ goto setup_vpu_fw_fail;
+ }
+
+ return 0;
+
+setup_vpu_fw_fail:
+ of_reserved_mem_device_release(dev);
+
+ return rc;
+}
+
+static int setup_x509_region(struct vpu_ipc_dev *vpu_dev)
+{
+ struct device *dev = &vpu_dev->pdev->dev;
+ struct device_node *node;
+ struct resource res;
+ int rc;
+
+ node = of_parse_phandle(dev->of_node, "memory-region",
+ VPU_IPC_X509_AREA_IDX);
+ if (!node) {
+ dev_err(dev, "Couldn't find X509 region.\n");
+ return -EINVAL;
+ }
+
+ rc = of_address_to_resource(node, 0, &res);
+
+ /* Release node first as we will not use it anymore */
+ of_node_put(node);
+
+ if (rc) {
+ dev_err(dev, "Couldn't resolve X509 region.\n");
+ return rc;
+ }
+
+ vpu_dev->x509_mem.vaddr =
+ devm_ioremap(dev, res.start, resource_size(&res));
+ if (!vpu_dev->x509_mem.vaddr) {
+ dev_err(dev, "Couldn't ioremap x509 region.\n");
+ return -EADDRNOTAVAIL;
+ }
+
+ vpu_dev->x509_mem.paddr = res.start;
+ vpu_dev->x509_mem.size = resource_size(&res);
+
+ return 0;
+}
+
+static int setup_mss_ipc_region(struct vpu_ipc_dev *vpu_dev)
+{
+ struct device *dev = &vpu_dev->pdev->dev;
+ struct device_node *node;
+ struct resource res;
+ int rc;
+
+ node = of_parse_phandle(dev->of_node, "memory-region",
+ MSS_IPC_AREA_IDX);
+ if (!node) {
+ dev_err(dev, "Didn't find MSS IPC region.\n");
+ return -EINVAL;
+ }
+
+ rc = of_address_to_resource(node, 0, &res);
+ if (rc) {
+ dev_err(dev, "Couldn't resolve MSS IPC region.\n");
+ return rc;
+ }
+ of_node_put(node);
+
+ vpu_dev->mss_ipc_mem.paddr = res.start;
+ vpu_dev->mss_ipc_mem.vpu_addr = phys_to_dma(dev, res.start);
+ vpu_dev->mss_ipc_mem.size = resource_size(&res);
+
+ return 0;
+}
+
+static int setup_reserved_memory(struct vpu_ipc_dev *vpu_dev)
+{
+ struct device *dev = &vpu_dev->pdev->dev;
+ int rc;
+
+ /*
+ * Find the VPU firmware area described in the device tree,
+ * and allocate it for our usage.
+ */
+ rc = setup_vpu_fw_region(vpu_dev);
+ if (rc) {
+ dev_err(dev, "Failed to init FW memory.\n");
+ return rc;
+ }
+
+ /*
+ * Find the X509 area described in the device tree,
+ * and allocate it for our usage.
+ */
+ rc = setup_x509_region(vpu_dev);
+ if (rc) {
+ dev_err(dev, "Failed to setup X509 region.\n");
+ goto res_mem_setup_fail;
+ }
+
+ /*
+ * Find the MSS IPC area in the device tree and get the location and
+ * size information
+ */
+ rc = setup_mss_ipc_region(vpu_dev);
+ if (rc) {
+ dev_err(dev, "Couldn't setup MSS IPC region.\n");
+ goto res_mem_setup_fail;
+ }
+
+ return 0;
+
+res_mem_setup_fail:
+ of_reserved_mem_device_release(dev);
+
+ return rc;
+}
+
+static void ipc_device_put(struct vpu_ipc_dev *vpu_dev)
+{
+ put_device(vpu_dev->ipc_dev);
+}
+
+static int ipc_device_get(struct vpu_ipc_dev *vpu_dev)
+{
+ struct device *dev = &vpu_dev->pdev->dev;
+ struct platform_device *pdev;
+ struct device_node *np;
+
+ np = of_parse_phandle(dev->of_node, "intel,keembay-ipc", 0);
+ if (!np) {
+ dev_err(dev, "Cannot find phandle to IPC device\n");
+ return -ENODEV;
+ }
+
+ pdev = of_find_device_by_node(np);
+ if (!pdev) {
+ dev_info(dev, "IPC device not probed\n");
+ of_node_put(np);
+ return -EPROBE_DEFER;
+ }
+
+ vpu_dev->ipc_dev = get_device(&pdev->dev);
+ of_node_put(np);
+
+ return 0;
+}
+
+static int retrieve_clocks(struct vpu_ipc_dev *vpu_dev)
+{
+ struct device *dev = &vpu_dev->pdev->dev;
+ char pll_string[PLL_STRING_SIZE];
+ struct clk *clk;
+ int pll;
+ int out;
+
+ clk = devm_clk_get(dev, "cpu_clock");
+ if (IS_ERR(clk)) {
+ dev_err(dev, "cpu_clock not found.\n");
+ return PTR_ERR(clk);
+ }
+ vpu_dev->cpu_clock = clk;
+
+ for (pll = 0; pll < NUM_PLLS; pll++) {
+ for (out = 0; out < NUM_PLL_OUTPUTS; out++) {
+ sprintf(pll_string, "pll_%d_out_%d", pll, out);
+ clk = devm_clk_get(dev, pll_string);
+ if (IS_ERR(clk)) {
+ dev_err(dev, "%s not found.\n", pll_string);
+ return PTR_ERR(clk);
+ }
+ vpu_dev->pll[pll][out] = clk;
+ }
+ }
+
+ return 0;
+}
+
+/* Get register resource from device tree and re-map as I/O memory. */
+static int get_pdev_res_and_ioremap(struct platform_device *pdev,
+ const char *reg_name,
+ void __iomem **target_reg)
+{
+ struct device *dev = &pdev->dev;
+ struct resource *res;
+ void __iomem *reg;
+
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM, reg_name);
+ if (!res) {
+ dev_err(dev, "Couldn't get resource for %s\n", reg_name);
+ return -EINVAL;
+ }
+
+ reg = devm_ioremap_resource(dev, res);
+ if (IS_ERR(reg)) {
+ dev_err(dev, "Couldn't ioremap resource for %s\n", reg_name);
+ return PTR_ERR(reg);
+ }
+
+ *target_reg = reg;
+
+ return 0;
+}
+
+static int setup_watchdog_resources(struct vpu_ipc_dev *vpu_dev)
+{
+ struct platform_device *pdev = vpu_dev->pdev;
+ struct device *dev = &vpu_dev->pdev->dev;
+ int rc;
+
+ /* Get registers */
+ rc = get_pdev_res_and_ioremap(pdev, "nce_wdt", &vpu_dev->nce_wdt_reg);
+ if (rc) {
+ dev_err(dev, "Failed to get NCE WDT registers\n");
+ return rc;
+ }
+ rc = get_pdev_res_and_ioremap(pdev, "nce_tim_cfg",
+ &vpu_dev->nce_tim_cfg_reg);
+ if (rc) {
+ dev_err(dev, "Failed to get NCE TIM_GEN_CONFIG register\n");
+ return rc;
+ }
+ rc = get_pdev_res_and_ioremap(pdev, "mss_wdt", &vpu_dev->mss_wdt_reg);
+ if (rc) {
+ dev_err(dev, "Failed to get MSS WDT registers\n");
+ return rc;
+ }
+ rc = get_pdev_res_and_ioremap(pdev, "mss_tim_cfg",
+ &vpu_dev->mss_tim_cfg_reg);
+ if (rc) {
+ dev_err(dev, "Failed to get MSS TIM_GEN_CONFIG register\n");
+ return rc;
+ }
+
+ /* Request interrupts */
+ vpu_dev->nce_irq = platform_get_irq_byname(pdev, "nce_wdt");
+ if (vpu_dev->nce_irq < 0)
+ return vpu_dev->nce_irq;
+ vpu_dev->mss_irq = platform_get_irq_byname(pdev, "mss_wdt");
+ if (vpu_dev->mss_irq < 0)
+ return vpu_dev->mss_irq;
+ rc = devm_request_irq(dev, vpu_dev->nce_irq, nce_wdt_irq_handler, 0,
+ "keembay-vpu-ipc", vpu_dev);
+ if (rc) {
+ dev_err(dev, "failed to request NCE IRQ.\n");
+ return rc;
+ }
+ rc = devm_request_irq(dev, vpu_dev->mss_irq, mss_wdt_irq_handler, 0,
+ "keembay-vpu-ipc", vpu_dev);
+ if (rc) {
+ dev_err(dev, "failed to request MSS IRQ.\n");
+ return rc;
+ }
+
+ /* Request interrupt re-direct numbers */
+ rc = of_property_read_u32(dev->of_node,
+ "intel,keembay-vpu-ipc-nce-wdt-redirect",
+ &vpu_dev->nce_wdt_redirect);
+ if (rc) {
+ dev_err(dev, "failed to get NCE WDT redirect number.\n");
+ return rc;
+ }
+ rc = of_property_read_u32(dev->of_node,
+ "intel,keembay-vpu-ipc-mss-wdt-redirect",
+ &vpu_dev->mss_wdt_redirect);
+ if (rc) {
+ dev_err(dev, "failed to get MSS WDT redirect number.\n");
+ return rc;
+ }
+
+ return 0;
+}
+
+/* Populate the boot parameters to be passed to the VPU. */
+static int setup_boot_parameters(struct vpu_ipc_dev *vpu_dev)
+{
+ int i;
+
+ /* Set all values to zero. This will disable most clocks/devices */
+ memset(vpu_dev->boot_params, 0, sizeof(*vpu_dev->boot_params));
+
+ /*
+ * Set magic number, so VPU knows that the parameters are
+ * populated correctly
+ */
+ vpu_dev->boot_params->magic_number = BOOT_PARAMS_MAGIC_NUMBER;
+
+ /* Set VPU ID. */
+ vpu_dev->boot_params->vpu_id = vpu_dev->vpu_id;
+
+ /* Inform VPU of clock frequencies */
+ vpu_dev->boot_params->cpu_frequency_hz =
+ clk_get_rate(vpu_dev->cpu_clock);
+ for (i = 0; i < NUM_PLL_OUTPUTS; i++) {
+ vpu_dev->boot_params->pll0_out[i] =
+ clk_get_rate(vpu_dev->pll[0][i]);
+ vpu_dev->boot_params->pll1_out[i] =
+ clk_get_rate(vpu_dev->pll[1][i]);
+ vpu_dev->boot_params->pll2_out[i] =
+ clk_get_rate(vpu_dev->pll[2][i]);
+ }
+
+ /*
+ * Fill in IPC buffer information: the VPU needs to know where it
+ * should allocate IPC buffer from.
+ */
+ vpu_dev->boot_params->mss_ipc_header_address =
+ vpu_dev->mss_ipc_mem.vpu_addr;
+ vpu_dev->boot_params->mss_ipc_header_area_size =
+ vpu_dev->mss_ipc_mem.size;
+
+ /* Fill in IRQ re-direct request information */
+ vpu_dev->boot_params->mss_wdt_to_irq_a53_redir =
+ vpu_dev->mss_wdt_redirect;
+ vpu_dev->boot_params->nce_wdt_to_irq_a53_redir =
+ vpu_dev->nce_wdt_redirect;
+
+ /* Setup A53SS_VERSION_ID */
+ vpu_dev->boot_params->a53ss_version_id = vpu_ipc_soc_info->hardware_id;
+
+ /* Setup Silicon stepping */
+ vpu_dev->boot_params->si_stepping = vpu_ipc_soc_info->stepping[0] |
+ vpu_ipc_soc_info->stepping[1] << 8;
+
+ /* Set feature exclude and device id information. */
+ vpu_dev->boot_params->device_id = vpu_ipc_soc_info->device_id;
+ vpu_dev->boot_params->feature_exclusion =
+ vpu_ipc_soc_info->feature_exclusion;
+
+ /* Set SKU information */
+ vpu_dev->boot_params->sku = (u64)vpu_ipc_soc_info->sku[0] |
+ (u64)vpu_ipc_soc_info->sku[1] << 8 |
+ (u64)vpu_ipc_soc_info->sku[2] << 16 |
+ (u64)vpu_ipc_soc_info->sku[3] << 24 |
+ (u64)vpu_ipc_soc_info->sku[4] << 32 |
+ (u64)vpu_ipc_soc_info->sku[5] << 40;
+ return 0;
+}
+
+/* Request SoC firmware to boot the VPU. */
+static int request_vpu_boot(struct vpu_ipc_dev *vpu_dev)
+{
+ u64 function_id;
+ struct arm_smccc_res res;
+ u16 function_number = KMB_SIP_SVC_VPU_BOOT;
+
+ function_id = ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_32,
+ ARM_SMCCC_OWNER_SIP, function_number);
+
+ /*
+ * Arguments are as follows:
+ * 1. Reserved
+ * 2. Reserved region size
+ * 3. Firmware image physical base address
+ * 4. Firmware image size
+ * 5. Entry point for VPU
+ * 6. X509 certificate location
+ * 7. IMR driver number
+ */
+ arm_smccc_smc(function_id, 0,
+ vpu_dev->reserved_mem.size, vpu_dev->fw_res.start,
+ resource_size(&vpu_dev->fw_res), vpu_dev->boot_vec_paddr,
+ vpu_dev->x509_mem.paddr, vpu_dev->imr, &res);
+
+ if (res.a0) {
+ dev_info(&vpu_dev->pdev->dev, "Boot failed: 0x%lx.\n", res.a0);
+ return -EIO;
+ }
+
+ dev_info(&vpu_dev->pdev->dev,
+ "VPU Boot successfully requested to secure monitor.\n");
+
+ return 0;
+}
+
+/* Request SoC firmware to stop the VPU. */
+static int request_vpu_stop(struct vpu_ipc_dev *vpu_dev)
+{
+ u64 function_id;
+ struct arm_smccc_res res;
+ u16 function_number = KMB_SIP_SVC_VPU_STOP;
+
+ function_id = ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_32,
+ ARM_SMCCC_OWNER_SIP, function_number);
+
+ arm_smccc_smc(function_id, 0, 0, 0, 0, 0, 0, 0, &res);
+
+ if (res.a0) {
+ dev_info(&vpu_dev->pdev->dev, "Stop failed: 0x%lx.\n", res.a0);
+ return -EIO;
+ }
+
+ dev_info(&vpu_dev->pdev->dev,
+ "VPU Stop successfully requested to secure monitor.\n");
+
+ return 0;
+}
+
+/*
+ * Get kernel virtual address of resource inside the VPU reserved memory
+ * region.
+ */
+static void *get_vpu_dev_vaddr(struct vpu_ipc_dev *vpu_dev,
+ struct resource *res)
+{
+ unsigned long offset;
+
+ /* Given the calculation below, this must not be true. */
+ if (res->start < vpu_dev->reserved_mem.vpu_addr)
+ return NULL;
+
+ offset = res->start - vpu_dev->reserved_mem.vpu_addr;
+
+ /* Cast to (u8 *) since void pointer arithmetic is undefined. */
+ return (u8 *)vpu_dev->reserved_mem.vaddr + offset;
+}
+
+static int parse_fw_header(struct vpu_ipc_dev *vpu_dev,
+ const struct firmware *fw)
+{
+ struct resource config_res, version_res, total_reserved_res;
+ struct device *dev = &vpu_dev->pdev->dev;
+ struct firmware_header *fw_header;
+ void *version_region;
+ void *config_region;
+ void *fw_region;
+
+ /* Is the fw size big enough to read the header? */
+ if (fw->size < sizeof(struct firmware_header)) {
+ dev_err(dev, "Firmware was too small for header.\n");
+ return -EINVAL;
+ }
+
+ fw_header = (struct firmware_header *)fw->data;
+
+ /* Check header version */
+ if (fw_header->header_ver != HEADER_VERSION_SUPPORTED) {
+ dev_err(dev, "Header version check expected 0x%x, got 0x%x\n",
+ HEADER_VERSION_SUPPORTED, fw_header->header_ver);
+ return -EINVAL;
+ }
+
+ /* Check firmware version size is allowed */
+ if (fw_header->fw_ver_size > MAX_FIRMWARE_VERSION_SIZE) {
+ dev_err(dev, "Firmware version area larger than allowed: %d\n",
+ fw_header->fw_ver_size);
+ return -EINVAL;
+ }
+
+ /*
+ * Check the firmware binary is at least large enough for the
+ * firmware image size described in the header.
+ */
+ if (fw->size < (MAX_HEADER_SIZE + MAX_FIRMWARE_VERSION_SIZE +
+ fw_header->image_size)) {
+ dev_err(dev, "Real firmware size is not large enough.\n");
+ return -EINVAL;
+ }
+
+ /*
+ * Make sure that the final address is aligned correctly. If not, the
+ * boot will never work.
+ */
+ if (!IS_ALIGNED(fw_header->entry_point, VPU_RESET_VECTOR_ALIGNMENT)) {
+ dev_err(dev,
+ "Entry point for firmware (0x%llX) is not correctly aligned.\n",
+ fw_header->entry_point);
+ return -EINVAL;
+ }
+
+ /*
+ * Generate the resource describing the region containing the actual
+ * firmware data.
+ */
+ vpu_dev->fw_res.start = fw_header->image_load_addr;
+ vpu_dev->fw_res.end = fw_header->image_size +
+ fw_header->image_load_addr - 1;
+ vpu_dev->fw_res.flags = IORESOURCE_MEM;
+
+ /*
+ * Generate the resource describing the region containing the
+ * configuration data for the VPU.
+ */
+ config_res.start = fw_header->config_load_addr;
+ config_res.end = sizeof(struct boot_parameters) +
+ fw_header->config_load_addr - 1;
+ config_res.flags = IORESOURCE_MEM;
+
+ /*
+ * Generate the resource describing the region containing the
+ * version information for the VPU.
+ */
+ version_res.start = fw_header->fw_ver_load_addr;
+ version_res.end = fw_header->fw_ver_size +
+ fw_header->fw_ver_load_addr - 1;
+ version_res.flags = IORESOURCE_MEM;
+
+ /*
+ * Generate the resource describing the region of memory
+ * completely dedicated to the VPU.
+ */
+ total_reserved_res.start = vpu_dev->reserved_mem.vpu_addr;
+ total_reserved_res.end = vpu_dev->reserved_mem.vpu_addr +
+ vpu_dev->reserved_mem.size - 1;
+ total_reserved_res.flags = IORESOURCE_MEM;
+
+ /*
+ * Check all pieces to be copied reside completely in the reserved
+ * region
+ */
+ if (!resource_contains(&total_reserved_res, &vpu_dev->fw_res)) {
+ dev_err(dev, "Can't fit firmware in reserved region.\n");
+ return -EINVAL;
+ }
+ if (!resource_contains(&total_reserved_res, &version_res)) {
+ dev_err(dev,
+ "Can't fit firmware version data in reserved region.\n");
+ return -EINVAL;
+ }
+ if (!resource_contains(&total_reserved_res, &config_res)) {
+ dev_err(dev,
+ "Can't fit configuration information in reserved region.\n");
+ return -EINVAL;
+ }
+
+ /* Check for overlapping regions */
+ if (resource_overlaps(&vpu_dev->fw_res, &version_res)) {
+ dev_err(dev, "FW and version regions overlap.\n");
+ return -EINVAL;
+ }
+ if (resource_overlaps(&vpu_dev->fw_res, &config_res)) {
+ dev_err(dev, "FW and config regions overlap.\n");
+ return -EINVAL;
+ }
+ if (resource_overlaps(&config_res, &version_res)) {
+ dev_err(dev, "Version and config regions overlap.\n");
+ return -EINVAL;
+ }
+
+ /* Setup boot parameter region */
+ config_region = get_vpu_dev_vaddr(vpu_dev, &config_res);
+ if (!config_region) {
+ dev_err(dev,
+ "Couldn't map boot configuration area to CPU virtual address.\n");
+ return -EINVAL;
+ }
+ version_region = get_vpu_dev_vaddr(vpu_dev, &version_res);
+ if (!version_region) {
+ dev_err(dev,
+ "Couldn't map version area to CPU virtual address.\n");
+ return -EINVAL;
+ }
+ fw_region = get_vpu_dev_vaddr(vpu_dev, &vpu_dev->fw_res);
+ if (!fw_region) {
+ dev_err(dev,
+ "Couldn't map firmware area to CPU virtual address.\n");
+ return -EINVAL;
+ }
+
+ /*
+ * Copy version region: the region is located in the file @ offset of
+ * MAX_HEADER_SIZE, size was specified in the header and has been
+ * checked to not be larger than that allowed.
+ */
+ memcpy(version_region, &fw->data[MAX_HEADER_SIZE],
+ fw_header->fw_ver_size);
+
+ /*
+ * Copy firmware region: the region is located in the file @ offset of
+ * MAX_HEADER_SIZE + MAX_FIRMWARE_VERSION_SIZE, size was specified in
+ * the header and has been checked to not be larger than that allowed.
+ */
+ memcpy(fw_region,
+ &fw->data[MAX_HEADER_SIZE + MAX_FIRMWARE_VERSION_SIZE],
+ fw_header->image_size);
+
+ /* Save off boot parameters region vaddr */
+ vpu_dev->boot_params = config_region;
+
+ /* Save off boot vector physical address */
+ vpu_dev->boot_vec_paddr = fw_header->entry_point;
+
+ return 0;
+}
+
+static int ready_message_wait_thread(void *arg)
+{
+ struct vpu_ipc_dev *vpu_dev = arg;
+ struct device *dev = &vpu_dev->pdev->dev;
+ size_t size = 0;
+ u32 paddr = 0;
+ int close_rc;
+ int rc;
+
+ /*
+ * We will wait a few seconds for the message. We will complete earlier
+ * if the message is received earlier.
+ * NOTE: this is not a busy wait, we sleep until message is received.
+ */
+ rc = intel_keembay_ipc_recv(vpu_dev->ipc_dev, KMB_IPC_NODE_LEON_MSS,
+ READY_MESSAGE_IPC_CHANNEL, &paddr, &size,
+ READY_MESSAGE_TIMEOUT_MS);
+ /*
+ * IPC channel must be closed regardless of 'rc' value, so close the
+ * channel now and then process 'rc' value.
+ */
+ close_rc = intel_keembay_ipc_close_channel(vpu_dev->ipc_dev,
+ KMB_IPC_NODE_LEON_MSS,
+ READY_MESSAGE_IPC_CHANNEL);
+ if (close_rc < 0) {
+ dev_warn(dev, "Couldn't close IPC channel.\n");
+ /* Continue, as this is not a critical issue. */
+ }
+
+ /* Now process recv() return code. */
+ if (rc < 0) {
+ dev_err(dev,
+ "Failed to receive ready message within %d ms: %d.\n",
+ READY_MESSAGE_TIMEOUT_MS, rc);
+ goto ready_message_thread_failure;
+ }
+
+ if (paddr != READY_MESSAGE_EXPECTED_PADDR ||
+ size != READY_MESSAGE_EXPECTED_SIZE) {
+ dev_err(dev, "Bad ready message: (paddr, size) = (0x%x, %zu)\n",
+ paddr, size);
+ goto ready_message_thread_failure;
+ }
+
+ dev_info(dev, "VPU ready message received successfully!\n");
+
+ rc = vpu_ipc_handle_event(vpu_dev, KEEMBAY_VPU_EVENT_MSS_READY_OK);
+ if (rc < 0)
+ dev_err(dev, "Fatal error: failed to set state (ready ok).\n");
+
+ /* Wake up anyone waiting for READY. */
+ wake_up_all(&vpu_dev->ready_queue);
+
+ return 0;
+
+ready_message_thread_failure:
+ rc = vpu_ipc_handle_event(vpu_dev, KEEMBAY_VPU_EVENT_MSS_READY_FAIL);
+ if (rc < 0)
+ dev_err(dev,
+ "Fatal error: failed to set state (ready timeout).\n");
+
+ return 0;
+}
+
+static int create_ready_message_thread(struct vpu_ipc_dev *vpu_dev)
+{
+ struct device *dev = &vpu_dev->pdev->dev;
+ struct task_struct *task;
+
+ task = kthread_run(&ready_message_wait_thread, (void *)vpu_dev,
+ "keembay-vpu-ipc-ready");
+ if (IS_ERR(task)) {
+ dev_err(dev, "Couldn't start thread to receive message.\n");
+ return -EIO;
+ }
+
+ vpu_dev->ready_message_task = task;
+
+ return 0;
+}
+
+static int kickoff_vpu_sequence(struct vpu_ipc_dev *vpu_dev,
+ int (*boot_fn)(struct vpu_ipc_dev *))
+{
+ struct device *dev = &vpu_dev->pdev->dev;
+ int err_rc;
+ int rc;
+
+ /*
+ * Open the IPC channel. If we don't do it before booting
+ * the VPU, we may miss the message, as the IPC driver will
+ * discard messages for unopened channels.
+ */
+ rc = intel_keembay_ipc_open_channel(vpu_dev->ipc_dev,
+ KMB_IPC_NODE_LEON_MSS,
+ READY_MESSAGE_IPC_CHANNEL);
+ if (rc < 0) {
+ dev_err(dev,
+ "Couldn't open IPC channel to receive ready message.\n");
+ goto kickoff_failed;
+ }
+
+ /* Request boot */
+ rc = boot_fn(vpu_dev);
+ if (rc < 0) {
+ dev_err(dev, "Failed to do request to boot.\n");
+ goto close_and_kickoff_failed;
+ }
+
+ /*
+ * Start thread waiting for message, and update state
+ * if the request was successful.
+ */
+ rc = create_ready_message_thread(vpu_dev);
+ if (rc < 0) {
+ dev_err(dev,
+ "Failed to create thread to wait for ready message.\n");
+ goto close_and_kickoff_failed;
+ }
+
+ return 0;
+
+close_and_kickoff_failed:
+ /* Close the channel. */
+ err_rc = intel_keembay_ipc_close_channel(vpu_dev->ipc_dev,
+ KMB_IPC_NODE_LEON_MSS,
+ READY_MESSAGE_IPC_CHANNEL);
+ if (err_rc < 0) {
+ dev_err(dev, "Couldn't close IPC channel: %d\n", err_rc);
+ /*
+ * We have had a more serious failure - don't update the
+ * original 'rc' and continue.
+ */
+ }
+
+kickoff_failed:
+ return rc;
+}
+
+/*
+ * Try to boot the VPU using the firmware name stored in
+ * vpu_dev->firmware_name (which when this function is called is expected to be
+ * not NULL).
+ */
+static int do_boot_sequence(struct vpu_ipc_dev *vpu_dev)
+{
+ struct device *dev = &vpu_dev->pdev->dev;
+ const struct firmware *fw;
+ int event_rc;
+ int rc;
+
+ /* Update state machine. */
+ rc = vpu_ipc_handle_event(vpu_dev, KEEMBAY_VPU_EVENT_BOOT);
+ if (rc < 0) {
+ dev_err(dev, "Can't start in this state.\n");
+ return rc;
+ }
+
+ /* Stop the VPU running */
+ rc = request_vpu_stop(vpu_dev);
+ if (rc < 0)
+ dev_err(dev, "Failed stop - continue sequence anyway.\n");
+
+ dev_info(dev, "Keem Bay VPU IPC start with %s.\n",
+ vpu_dev->firmware_name);
+
+ /* Request firmware and wait for it. */
+ rc = request_firmware(&fw, vpu_dev->firmware_name, &vpu_dev->pdev->dev);
+ if (rc < 0) {
+ dev_err(dev, "Couldn't find firmware: %d\n", rc);
+ goto boot_failed_no_fw;
+ }
+
+ /* Do checks on the firmware header. */
+ rc = parse_fw_header(vpu_dev, fw);
+ if (rc < 0) {
+ dev_err(dev, "Firmware checks failed.\n");
+ goto boot_failed;
+ }
+
+ /* Write configuration data. */
+ rc = setup_boot_parameters(vpu_dev);
+ if (rc < 0) {
+ dev_err(dev, "Failed to set up boot parameters.\n");
+ goto boot_failed;
+ }
+
+ /* Try 'boot' sequence */
+ rc = kickoff_vpu_sequence(vpu_dev, request_vpu_boot);
+ if (rc < 0) {
+ dev_err(dev, "Failed to boot VPU.\n");
+ goto boot_failed;
+ }
+
+ release_firmware(fw);
+ return 0;
+
+boot_failed:
+ release_firmware(fw);
+
+boot_failed_no_fw:
+ /* Update state machine after failure. */
+ event_rc = vpu_ipc_handle_event(vpu_dev,
+ KEEMBAY_VPU_EVENT_BOOT_FAILED);
+ if (event_rc < 0) {
+ dev_err(dev,
+ "Unexpected error: failed to handle fail event: %d.\n",
+ event_rc);
+ /* Continue: prefer original 'rc' to 'event_rc'. */
+ }
+
+ return rc;
+}
+
+/**
+ * intel_keembay_vpu_ipc_open_channel() - Open an IPC channel.
+ * @dev: The VPU IPC device to use.
+ * @node_id: The node ID of the remote node (used to identify the link the
+ * channel must be added to). KMB_IPC_NODE_LEON_MSS is the only
+ * allowed value for now.
+ * @chan_id: The ID of the channel to be opened.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int intel_keembay_vpu_ipc_open_channel(struct device *dev, u8 node_id,
+ u16 chan_id)
+{
+ struct vpu_ipc_dev *vpu_dev = to_vpu_dev(dev);
+
+ if (IS_ERR(vpu_dev))
+ return -EINVAL;
+
+ return intel_keembay_ipc_open_channel(vpu_dev->ipc_dev, node_id,
+ chan_id);
+}
+EXPORT_SYMBOL(intel_keembay_vpu_ipc_open_channel);
+
+/**
+ * intel_keembay_vpu_ipc_close_channel() - Close an IPC channel.
+ * @dev: The VPU IPC device to use.
+ * @node_id: The node ID of the remote node (used to identify the link the
+ * channel must be added to). KMB_IPC_NODE_LEON_MSS is the only
+ * allowed value for now.
+ * @chan_id: The ID of the channel to be closed.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+
+int intel_keembay_vpu_ipc_close_channel(struct device *dev, u8 node_id,
+ u16 chan_id)
+{
+ struct vpu_ipc_dev *vpu_dev = to_vpu_dev(dev);
+
+ if (IS_ERR(vpu_dev))
+ return -EINVAL;
+
+ return intel_keembay_ipc_close_channel(vpu_dev->ipc_dev,
+ node_id, chan_id);
+}
+EXPORT_SYMBOL(intel_keembay_vpu_ipc_close_channel);
+
+/**
+ * intel_keembay_vpu_ipc_send() - Send data via IPC.
+ * @dev: The VPU IPC device to use.
+ * @node_id: The node ID of the remote node (used to identify the link the
+ * channel must be added to). KMB_IPC_NODE_LEON_MSS is the only
+ * allowed value for now.
+ * @chan_id: The IPC channel to be used to send the message.
+ * @vpu_addr: The VPU address of the data to be transferred.
+ * @size: The size of the data to be transferred.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int intel_keembay_vpu_ipc_send(struct device *dev, u8 node_id, u16 chan_id,
+ u32 vpu_addr, size_t size)
+{
+ struct vpu_ipc_dev *vpu_dev = to_vpu_dev(dev);
+
+ if (IS_ERR(vpu_dev))
+ return -EINVAL;
+
+ return intel_keembay_ipc_send(vpu_dev->ipc_dev, node_id, chan_id,
+ vpu_addr, size);
+}
+EXPORT_SYMBOL(intel_keembay_vpu_ipc_send);
+
+/**
+ * intel_keembay_vpu_ipc_recv() - Read data via IPC
+ * @dev: The VPU IPC device to use.
+ * @node_id: The node ID of the remote node (used to identify the link the
+ * channel must be added to). KMB_IPC_NODE_LEON_MSS is the only
+ * allowed value for now.
+ * @chan_id: The IPC channel to read from.
+ * @vpu_addr: [out] The VPU address of the received data.
+ * @size: [out] Where to store the size of the received data.
+ * @timeout: How long (in ms) the function will block waiting for an IPC
+ * message; if UINT32_MAX it will block indefinitely; if 0 it
+ * will not block.
+ *
+ * Return: 0 on success, negative error code otherwise
+ */
+int intel_keembay_vpu_ipc_recv(struct device *dev, u8 node_id, u16 chan_id,
+ u32 *vpu_addr, size_t *size, u32 timeout)
+{
+ struct vpu_ipc_dev *vpu_dev = to_vpu_dev(dev);
+
+ if (IS_ERR(vpu_dev))
+ return -EINVAL;
+
+ return intel_keembay_ipc_recv(vpu_dev->ipc_dev, node_id, chan_id,
+ vpu_addr, size, timeout);
+}
+EXPORT_SYMBOL(intel_keembay_vpu_ipc_recv);
+
+/**
+ * intel_keembay_vpu_startup() - Boot the VPU
+ * @dev: The VPU device to boot.
+ * @firmware_name: Name of firmware file
+ *
+ * This API is only valid while the VPU is OFF.
+ *
+ * The firmware called "firmware_name" will be searched for using the
+ * kernel firmware API. The firmware header will then be parsed. This driver
+ * will load requested information to the reserved memory region, including
+ * initialisation data. Lastly, we will request the secure world to do the
+ * boot sequence. If the boot sequence is successful, the
+ * VPU state will become BUSY. The caller should then wait for the status to
+ * become READY before starting to communicate with the VPU. If the boot
+ * sequence failed, this function will fail and the caller may try again,
+ * the VPU status will still be OFF.
+ *
+ * If we fail to get to READY, because the VPU did not send us the 'ready'
+ * message, the VPU state will go to ERROR.
+ *
+ * Return: 0 on success, negative error code otherwise
+ */
+int intel_keembay_vpu_startup(struct device *dev, const char *firmware_name)
+{
+ struct vpu_ipc_dev *vpu_dev = to_vpu_dev(dev);
+
+ if (IS_ERR(vpu_dev))
+ return PTR_ERR(vpu_dev);
+
+ if (!firmware_name)
+ return -EINVAL;
+
+ /* Free old vpu_dev->firmware_name value (if any). */
+ kfree(vpu_dev->firmware_name);
+
+ /* Set new value. */
+ vpu_dev->firmware_name = kstrdup(firmware_name, GFP_KERNEL);
+
+ return do_boot_sequence(vpu_dev);
+}
+EXPORT_SYMBOL(intel_keembay_vpu_startup);
+
+/**
+ * intel_keembay_vpu_reset() - Reset the VPU
+ * @dev: The VPU device to reset.
+ *
+ * Resets the VPU. Only valid when the VPU is in the READY or ERROR state.
+ * The state of the VPU will become BUSY.
+ *
+ * Return: 0 on success, negative error code otherwise
+ */
+int intel_keembay_vpu_reset(struct device *dev)
+{
+ struct vpu_ipc_dev *vpu_dev = to_vpu_dev(dev);
+
+ if (IS_ERR(vpu_dev))
+ return PTR_ERR(vpu_dev);
+
+ /*
+ * If vpu_dev->firmware_name == NULL then the VPU is not running
+ * (either it was never booted or vpu_stop() was called). So, calling
+ * reset is not allowed.
+ */
+ if (!vpu_dev->firmware_name)
+ return -EINVAL;
+
+ return do_boot_sequence(vpu_dev);
+}
+EXPORT_SYMBOL(intel_keembay_vpu_reset);
+
+/**
+ * intel_keembay_vpu_stop() - Stop the VPU
+ * @dev: The VPU device to stop.
+ *
+ * Stops the VPU and restores to the OFF state. Only valid when the VPU is in
+ * the READY or ERROR state.
+ *
+ * Return: 0 on success, negative error code otherwise
+ */
+int intel_keembay_vpu_stop(struct device *dev)
+{
+ struct vpu_ipc_dev *vpu_dev = to_vpu_dev(dev);
+ int event_rc;
+ int rc;
+
+ if (IS_ERR(vpu_dev))
+ return -EINVAL;
+
+ rc = vpu_ipc_handle_event(vpu_dev, KEEMBAY_VPU_EVENT_STOP);
+ if (rc < 0) {
+ dev_err(dev, "Can't stop in this state.\n");
+ return rc;
+ }
+
+ dev_info(dev, "Keem Bay VPU IPC stop.\n");
+
+ /* Request stop */
+ rc = request_vpu_stop(vpu_dev);
+ if (rc < 0) {
+ dev_err(dev,
+ "Failed to do request to stop - resetting state to OFF anyway.\n");
+ }
+
+ /* Remove any saved-off name */
+ kfree(vpu_dev->firmware_name);
+ vpu_dev->firmware_name = NULL;
+
+ event_rc = vpu_ipc_handle_event(vpu_dev,
+ KEEMBAY_VPU_EVENT_STOP_COMPLETE);
+ if (event_rc < 0) {
+ dev_err(dev,
+ "Failed to handle 'stop complete' event, probably fatal.\n");
+ return event_rc;
+ }
+
+ return rc;
+}
+EXPORT_SYMBOL(intel_keembay_vpu_stop);
+
+/**
+ * intel_keembay_vpu_status() - Get the VPU state.
+ * @dev: The VPU device to retrieve the status for.
+ *
+ * Returns the state of the VPU as tracked by this driver.
+ *
+ * Return: Relevant value of enum intel_keembay_vpu_state
+ */
+enum intel_keembay_vpu_state intel_keembay_vpu_status(struct device *dev)
+{
+ struct vpu_ipc_dev *vpu_dev = to_vpu_dev(dev);
+
+ if (IS_ERR(vpu_dev))
+ return -EINVAL;
+
+ return vpu_dev->state;
+}
+EXPORT_SYMBOL(intel_keembay_vpu_status);
+
+/**
+ * intel_keembay_vpu_get_wdt_count() - Get the WDT count
+ * @dev: The VPU device to get the WDT count for.
+ * @id: ID of WDT events we wish to get
+ *
+ * Returns: Number of WDT timeout occurrences for given ID, or negative
+ * error value for invalid ID.
+ */
+int intel_keembay_vpu_get_wdt_count(struct device *dev,
+ enum intel_keembay_wdt_cpu_id id)
+{
+ struct vpu_ipc_dev *vpu_dev = to_vpu_dev(dev);
+ int rc = -EINVAL;
+
+ if (IS_ERR(vpu_dev))
+ return -EINVAL;
+
+ switch (id) {
+ case KEEMBAY_VPU_NCE:
+ rc = vpu_dev->nce_wdt_count;
+ break;
+ case KEEMBAY_VPU_MSS:
+ rc = vpu_dev->mss_wdt_count;
+ break;
+ default:
+ break;
+ }
+ return rc;
+}
+EXPORT_SYMBOL(intel_keembay_vpu_get_wdt_count);
+
+/**
+ * intel_keembay_vpu_wait_for_ready() - Sleep until VPU is READY
+ * @dev: The VPU device for which we are waiting the ready message.
+ * @timeout: How long (in ms) the function will block waiting for the VPU
+ * to become ready.
+ *
+ * The caller may ask the VPU IPC driver to notify it when the VPU
+ * is READY. The driver performs no checks on the current state, so it
+ * is up to the caller to confirm that the state is correct before starting
+ * the wait.
+ *
+ * Returns: 0 on success negative error code otherwise
+ */
+int intel_keembay_vpu_wait_for_ready(struct device *dev, u32 timeout)
+{
+ struct vpu_ipc_dev *vpu_dev = to_vpu_dev(dev);
+ int rc;
+
+ if (IS_ERR(vpu_dev))
+ return -EINVAL;
+
+ /*
+ * If we are in ERROR state, we will not get to READY
+ * state without some other transitions, so return
+ * error immediately for caller to handle.
+ */
+ if (vpu_dev->state == KEEMBAY_VPU_ERROR)
+ return -EIO;
+
+ rc = wait_event_interruptible_timeout(vpu_dev->ready_queue,
+ vpu_dev->state == KEEMBAY_VPU_READY,
+ msecs_to_jiffies(timeout));
+
+ /* Condition was false after timeout elapsed */
+ if (!rc)
+ rc = -ETIME;
+
+ /* Condition was true, so rc == 1 */
+ if (rc > 0)
+ rc = 0;
+
+ return rc;
+}
+EXPORT_SYMBOL(intel_keembay_vpu_wait_for_ready);
+
+/**
+ * intel_keembay_vpu_register_for_events() - Register callback for event notification
+ * @dev: The VPU device.
+ * @callback: Callback function pointer
+ *
+ * Only a single callback can be registered at a time.
+ *
+ * Callback can be triggered from any context, so needs to be able to be run
+ * from IRQ context.
+ *
+ * Return: 0 on success, negative error code otherwise
+ */
+int intel_keembay_vpu_register_for_events(struct device *dev,
+ void (*callback)(struct device *dev,
+ enum intel_keembay_vpu_event))
+{
+ struct vpu_ipc_dev *vpu_dev = to_vpu_dev(dev);
+
+ if (IS_ERR(vpu_dev))
+ return PTR_ERR(vpu_dev);
+
+ if (vpu_dev->callback)
+ return -EEXIST;
+
+ vpu_dev->callback = callback;
+
+ return 0;
+}
+EXPORT_SYMBOL(intel_keembay_vpu_register_for_events);
+
+/**
+ * intel_keembay_vpu_unregister_for_events() - Unregister callback for event notification
+ * @dev: The VPU device.
+ *
+ * Return: 0 on success, negative error code otherwise
+ */
+int intel_keembay_vpu_unregister_for_events(struct device *dev)
+{
+ struct vpu_ipc_dev *vpu_dev = to_vpu_dev(dev);
+
+ if (IS_ERR(vpu_dev))
+ return PTR_ERR(vpu_dev);
+
+ vpu_dev->callback = NULL;
+
+ return 0;
+}
+EXPORT_SYMBOL(intel_keembay_vpu_unregister_for_events);
+
+/* Probe() function for the VPU IPC platform driver. */
+static int keembay_vpu_ipc_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct vpu_ipc_dev *vpu_dev;
+ int rc;
+
+ vpu_dev = devm_kzalloc(dev, sizeof(*vpu_dev), GFP_KERNEL);
+ if (!vpu_dev)
+ return -ENOMEM;
+
+ vpu_dev->pdev = pdev;
+ vpu_dev->state = KEEMBAY_VPU_OFF;
+ vpu_dev->ready_message_task = NULL;
+ vpu_dev->firmware_name = NULL;
+ vpu_dev->nce_wdt_count = 0;
+ vpu_dev->mss_wdt_count = 0;
+ spin_lock_init(&vpu_dev->lock);
+ init_waitqueue_head(&vpu_dev->ready_queue);
+
+ /* Retrieve clocks */
+ rc = retrieve_clocks(vpu_dev);
+ if (rc) {
+ dev_err(dev, "Failed to retrieve clocks %d\n", rc);
+ return rc;
+ }
+
+ /* Retrieve memory regions, allocate memory */
+ rc = setup_reserved_memory(vpu_dev);
+ if (rc) {
+ dev_err(dev,
+ "Failed to set up reserved memory regions: %d\n", rc);
+ return rc;
+ }
+
+ /* Request watchdog timer resources */
+ rc = setup_watchdog_resources(vpu_dev);
+ if (rc) {
+ dev_err(dev, "Failed to setup watchdog resources %d\n", rc);
+ goto probe_fail_post_resmem_setup;
+ }
+
+ /* Request the IMR number to be used */
+ rc = of_property_read_u32(dev->of_node, "intel,keembay-vpu-ipc-imr",
+ &vpu_dev->imr);
+ if (rc) {
+ dev_err(dev, "failed to get IMR number.\n");
+ goto probe_fail_post_resmem_setup;
+ }
+
+ /* Get VPU ID. */
+ rc = of_property_read_u32(dev->of_node, "intel,keembay-vpu-ipc-id",
+ &vpu_dev->vpu_id);
+ if (rc) {
+ /* Only warn for now; we will enforce this in the future. */
+ dev_err(dev, "VPU ID not defined in Device Tree\n");
+ goto probe_fail_post_resmem_setup;
+ }
+
+ /* Get IPC device to be used for IPC communication. */
+ rc = ipc_device_get(vpu_dev);
+ if (rc) {
+ dev_err(dev, "Failed to get IPC device\n");
+ goto probe_fail_post_resmem_setup;
+ }
+
+ /* Set platform data reference. */
+ platform_set_drvdata(pdev, vpu_dev);
+
+ return rc;
+
+probe_fail_post_resmem_setup:
+ of_reserved_mem_device_release(dev);
+
+ return rc;
+}
+
+/* Remove() function for the VPU IPC platform driver. */
+static int keembay_vpu_ipc_remove(struct platform_device *pdev)
+{
+ struct vpu_ipc_dev *vpu_dev = platform_get_drvdata(pdev);
+ struct device *dev = &pdev->dev;
+
+ if (vpu_dev->ready_message_task) {
+ kthread_stop(vpu_dev->ready_message_task);
+ vpu_dev->ready_message_task = NULL;
+ }
+
+ of_reserved_mem_device_release(dev);
+
+ ipc_device_put(vpu_dev);
+
+ return 0;
+}
+
+/* Compatible string for the VPU/IPC driver. */
+static const struct of_device_id keembay_vpu_ipc_of_match[] = {
+ {
+ .compatible = "intel,keembay-vpu-ipc",
+ },
+ {}
+};
+
+/* The VPU IPC platform driver. */
+static struct platform_driver keem_bay_vpu_ipc_driver = {
+ .driver = {
+ .name = "keembay-vpu-ipc",
+ .of_match_table = keembay_vpu_ipc_of_match,
+ },
+ .probe = keembay_vpu_ipc_probe,
+ .remove = keembay_vpu_ipc_remove,
+};
+
+/* Helper function to get a vpu_dev struct from a generic device pointer. */
+static struct vpu_ipc_dev *to_vpu_dev(struct device *dev)
+{
+ struct platform_device *pdev;
+
+ if (!dev || dev->driver != &keem_bay_vpu_ipc_driver.driver)
+ return ERR_PTR(-EINVAL);
+ pdev = to_platform_device(dev);
+
+ return platform_get_drvdata(pdev);
+}
+
+/*
+ * Retrieve SoC information from the '/soc/version-info' device tree node and
+ * store it into 'vpu_ipc_soc_info' global variable.
+ */
+static int retrieve_dt_soc_information(void)
+{
+ struct device_node *soc_info_dn;
+ int ret;
+
+ soc_info_dn = of_find_node_by_path("/soc/version-info");
+ if (!soc_info_dn)
+ return -ENOENT;
+
+ ret = of_property_read_u64(soc_info_dn, "feature-exclusion",
+ &vpu_ipc_soc_info->feature_exclusion);
+ if (ret) {
+ pr_err("Property 'feature-exclusion' can't be read.\n");
+ return ret;
+ }
+ ret = of_property_read_u64(soc_info_dn, "device-id",
+ &vpu_ipc_soc_info->device_id);
+ if (ret) {
+ pr_err("Property 'device-id' can't be read.\n");
+ return ret;
+ }
+ ret = of_property_read_u32(soc_info_dn, "hardware-id",
+ &vpu_ipc_soc_info->hardware_id);
+ if (ret) {
+ pr_err("Property 'hardware-id' can't be read.\n");
+ return ret;
+ }
+ /*
+ * Note: the SKU and stepping information from the device tree is
+ * not a string, but an array of u8/chars. Therefore, we cannot
+ * parse it as a string.
+ */
+ ret = of_property_read_u8_array(soc_info_dn, "sku",
+ vpu_ipc_soc_info->sku,
+ sizeof(vpu_ipc_soc_info->sku));
+ if (ret) {
+ pr_err("Property 'sku' can't be read.\n");
+ return ret;
+ }
+ ret = of_property_read_u8_array(soc_info_dn, "stepping",
+ vpu_ipc_soc_info->stepping,
+ sizeof(vpu_ipc_soc_info->stepping));
+ if (ret) {
+ pr_err("Property 'stepping' can't be read.\n");
+ return ret;
+ }
+
+ return 0;
+}
+
+/*
+ * Init VPU IPC module:
+ * - Retrieve SoC information from device tree.
+ * - Register the VPU IPC platform driver.
+ */
+static int __init vpu_ipc_init(void)
+{
+ int rc;
+
+ vpu_ipc_soc_info = kzalloc(sizeof(*vpu_ipc_soc_info), GFP_KERNEL);
+ if (!vpu_ipc_soc_info)
+ return -ENOMEM;
+
+ rc = retrieve_dt_soc_information();
+ if (rc < 0)
+ pr_warn("VPU IPC failed to find SoC info, using defaults.\n");
+
+ rc = platform_driver_register(&keem_bay_vpu_ipc_driver);
+ if (rc < 0) {
+ pr_err("Failed to register platform driver for VPU IPC.\n");
+ goto cleanup_soc_info;
+ }
+
+ return 0;
+
+cleanup_soc_info:
+ kfree(vpu_ipc_soc_info);
+
+ return rc;
+}
+
+/*
+ * Remove VPU IPC module.
+ * - Un-register the VPU IPC platform driver.
+ * - Remove sysfs exposing SoC information.
+ * - Free allocated memory.
+ */
+static void __exit vpu_ipc_exit(void)
+{
+ platform_driver_unregister(&keem_bay_vpu_ipc_driver);
+ kfree(vpu_ipc_soc_info);
+}
+
+module_init(vpu_ipc_init);
+module_exit(vpu_ipc_exit);
+
+MODULE_DESCRIPTION("Keem Bay VPU IPC Driver");
+MODULE_AUTHOR("Paul Murphy <[email protected]>");
+MODULE_AUTHOR("Daniele Alessandrelli <[email protected]>");
+MODULE_LICENSE("GPL");
diff --git a/include/linux/soc/intel/keembay-vpu-ipc.h b/include/linux/soc/intel/keembay-vpu-ipc.h
new file mode 100644
index 000000000000..81d132186482
--- /dev/null
+++ b/include/linux/soc/intel/keembay-vpu-ipc.h
@@ -0,0 +1,62 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * Keem Bay VPU IPC Linux Kernel API
+ *
+ * Copyright (c) 2018-2020 Intel Corporation.
+ */
+
+#ifndef __KEEMBAY_VPU_IPC_H
+#define __KEEMBAY_VPU_IPC_H
+
+#include "linux/types.h"
+
+/* The possible node IDs. */
+enum {
+ KMB_VPU_IPC_NODE_ARM_CSS = 0,
+ KMB_VPU_IPC_NODE_LEON_MSS,
+};
+
+/* Possible states of VPU. */
+enum intel_keembay_vpu_state {
+ KEEMBAY_VPU_OFF = 0,
+ KEEMBAY_VPU_BUSY,
+ KEEMBAY_VPU_READY,
+ KEEMBAY_VPU_ERROR,
+ KEEMBAY_VPU_STOPPING
+};
+
+/* Possible CPU IDs for which we receive WDT timeout events. */
+enum intel_keembay_wdt_cpu_id {
+ KEEMBAY_VPU_MSS = 0,
+ KEEMBAY_VPU_NCE
+};
+
+/* Events that can be notified via callback, when registered. */
+enum intel_keembay_vpu_event {
+ KEEMBAY_VPU_NOTIFY_DISCONNECT = 0,
+ KEEMBAY_VPU_NOTIFY_CONNECT,
+ KEEMBAY_VPU_NOTIFY_MSS_WDT,
+ KEEMBAY_VPU_NOTIFY_NCE_WDT,
+};
+
+int intel_keembay_vpu_ipc_open_channel(struct device *dev, u8 node_id,
+ u16 chan_id);
+int intel_keembay_vpu_ipc_close_channel(struct device *dev, u8 node_id,
+ u16 chan_id);
+int intel_keembay_vpu_ipc_send(struct device *dev, u8 node_id, u16 chan_id,
+ u32 paddr, size_t size);
+int intel_keembay_vpu_ipc_recv(struct device *dev, u8 node_id, u16 chan_id,
+ u32 *paddr, size_t *size, u32 timeout);
+int intel_keembay_vpu_startup(struct device *dev, const char *firmware_name);
+int intel_keembay_vpu_reset(struct device *dev);
+int intel_keembay_vpu_stop(struct device *dev);
+enum intel_keembay_vpu_state intel_keembay_vpu_status(struct device *dev);
+int intel_keembay_vpu_get_wdt_count(struct device *dev,
+ enum intel_keembay_wdt_cpu_id id);
+int intel_keembay_vpu_wait_for_ready(struct device *dev, u32 timeout);
+int intel_keembay_vpu_register_for_events(struct device *dev,
+ void (*callback)(struct device *dev,
+ enum intel_keembay_vpu_event event));
+int intel_keembay_vpu_unregister_for_events(struct device *dev);
+
+#endif /* __KEEMBAY_VPU_IPC_H */
--
2.17.1

2020-12-01 22:43:19

by mark gross

[permalink] [raw]
Subject: [PATCH 16/22] xlink-ipc: Add xlink ipc driver

From: Seamus Kelly <[email protected]>

Add xLink driver, which interfaces the xLink Core driver with the Keem
Bay VPU IPC driver, thus enabling xLink to control and communicate with
the VPU IP present on the Intel Keem Bay SoC.

Specifically the driver enables xLink Core to:

* Boot / Reset the VPU IP
* Register to VPU IP event notifications (device connected, device
disconnected, WDT event)
* Query the status of the VPU IP (OFF, BUSY, READY, ERROR, RECOVERY)
* Exchange data with the VPU IP, using the Keem Bay IPC mechanism
- Including the ability to send 'volatile' data (i.e., small amount of
data, up to 128-bytes that was not allocated in the CPU/VPU shared
memory region)

Cc: [email protected]
Reviewed-by: Mark Gross <[email protected]>
Signed-off-by: Seamus Kelly <[email protected]>
Signed-off-by: Ryan Carnaghi <[email protected]>
---
Documentation/vpu/index.rst | 1 +
Documentation/vpu/xlink-ipc.rst | 50 ++
MAINTAINERS | 6 +
drivers/misc/Kconfig | 1 +
drivers/misc/Makefile | 1 +
drivers/misc/xlink-ipc/Kconfig | 7 +
drivers/misc/xlink-ipc/Makefile | 4 +
drivers/misc/xlink-ipc/xlink-ipc.c | 879 +++++++++++++++++++++++++++++
include/linux/xlink-ipc.h | 48 ++
9 files changed, 997 insertions(+)
create mode 100644 Documentation/vpu/xlink-ipc.rst
create mode 100644 drivers/misc/xlink-ipc/Kconfig
create mode 100644 drivers/misc/xlink-ipc/Makefile
create mode 100644 drivers/misc/xlink-ipc/xlink-ipc.c
create mode 100644 include/linux/xlink-ipc.h

diff --git a/Documentation/vpu/index.rst b/Documentation/vpu/index.rst
index 661cc700ee45..49c78bb65b83 100644
--- a/Documentation/vpu/index.rst
+++ b/Documentation/vpu/index.rst
@@ -15,3 +15,4 @@ This documentation contains information for the Intel VPU stack.

vpu-stack-overview
xlink-pcie
+ xlink-ipc
diff --git a/Documentation/vpu/xlink-ipc.rst b/Documentation/vpu/xlink-ipc.rst
new file mode 100644
index 000000000000..af583579e70d
--- /dev/null
+++ b/Documentation/vpu/xlink-ipc.rst
@@ -0,0 +1,50 @@
+.. SPDX-License-Identifier: GPL-2.0-only
+
+Kernel driver: xLink IPC driver
+=================================
+Supported chips:
+
+* | Intel Edge.AI Computer Vision platforms: Keem Bay
+ | Suffix: Bay
+ | Datasheet: (not yet publicly available)
+
+------------
+Introduction
+------------
+
+The xLink IPC driver interfaces the xLink Core driver with the Keem Bay VPU IPC
+driver, thus enabling xLink to control and communicate with the VPU IP present
+on the Intel Keem Bay SoC.
+
+Specifically the driver enables xLink Core to:
+
+* Boot / Reset the VPU IP
+* Register to VPU IP event notifications (device connected, device disconnected,
+ WDT event)
+* Query the status of the VPU IP (OFF, BUSY, READY, ERROR, RECOVERY)
+* Exchange data with the VPU IP, using the Keem Bay IPC mechanism
+
+ * Including the ability to send 'volatile' data (i.e., small amount of data,
+ up to 128-bytes that was not allocated in the CPU/VPU shared memory region)
+
+Sending / Receiving 'volatile' data
+-----------------------------------
+
+Data to be exchanged with Keem Bay IPC needs to be allocated in the portion of
+DDR shared between the CPU and VPU.
+
+This can be impractical for small amount of data that user code can allocate
+on the stack.
+
+To reduce the burden on user code, xLink Core provides special send / receive
+functions to send up to 128 bytes of 'volatile data', i.e., data that is not
+allocated in the shared memory and that might also disappear after the xLink
+API is called (e.g., because allocated on the stack).
+
+The xLink IPC driver implements support for transferring such 'volatile data'
+to the VPU using Keem Bay IPC. To this end, the driver reserved some memory in
+the shared memory region.
+
+When volatile data is to be sent, xLink IPC allocates a buffer from the
+reserved memory region and copies the volatile data to the buffer. The buffer
+is then transferred to the VPU using Keem Bay IPC.
diff --git a/MAINTAINERS b/MAINTAINERS
index b3468dea6557..ebc2cce7251f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1952,6 +1952,12 @@ F: Documentation/devicetree/bindings/arm/intel,keembay.yaml
F: arch/arm64/boot/dts/intel/keembay-evm.dts
F: arch/arm64/boot/dts/intel/keembay-soc.dtsi

+ARM/INTEL XLINK IPC SUPPORT
+M: Seamus Kelly <[email protected]>
+M: Mark Gross <[email protected]>
+S: Maintained
+F: drivers/misc/xlink-ipc/
+
ARM/INTEL KEEMBAY XLINK PCIE SUPPORT
M: Srikanth Thokala <[email protected]>
M: Mark Gross <[email protected]>
diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig
index dfb98e444c6e..1f81ea915b95 100644
--- a/drivers/misc/Kconfig
+++ b/drivers/misc/Kconfig
@@ -482,4 +482,5 @@ source "drivers/misc/cardreader/Kconfig"
source "drivers/misc/habanalabs/Kconfig"
source "drivers/misc/uacce/Kconfig"
source "drivers/misc/xlink-pcie/Kconfig"
+source "drivers/misc/xlink-ipc/Kconfig"
endmenu
diff --git a/drivers/misc/Makefile b/drivers/misc/Makefile
index d17621fc43d5..b51495a2f1e0 100644
--- a/drivers/misc/Makefile
+++ b/drivers/misc/Makefile
@@ -58,3 +58,4 @@ obj-$(CONFIG_UACCE) += uacce/
obj-$(CONFIG_XILINX_SDFEC) += xilinx_sdfec.o
obj-$(CONFIG_HISI_HIKEY_USB) += hisi_hikey_usb.o
obj-y += xlink-pcie/
+obj-$(CONFIG_XLINK_IPC) += xlink-ipc/
diff --git a/drivers/misc/xlink-ipc/Kconfig b/drivers/misc/xlink-ipc/Kconfig
new file mode 100644
index 000000000000..6aa2592fe9a3
--- /dev/null
+++ b/drivers/misc/xlink-ipc/Kconfig
@@ -0,0 +1,7 @@
+config XLINK_IPC
+ tristate "Support for XLINK IPC"
+ depends on KEEMBAY_VPU_IPC
+ help
+ XLINK IPC enables the communication/control IPC Sub-System.
+
+ Select M if you have an Intel SoC with a Vision Processing Unit (VPU)
diff --git a/drivers/misc/xlink-ipc/Makefile b/drivers/misc/xlink-ipc/Makefile
new file mode 100644
index 000000000000..f92c95525e23
--- /dev/null
+++ b/drivers/misc/xlink-ipc/Makefile
@@ -0,0 +1,4 @@
+#
+# Makefile for xlink IPC Linux driver
+#
+obj-$(CONFIG_XLINK_IPC) += xlink-ipc.o
diff --git a/drivers/misc/xlink-ipc/xlink-ipc.c b/drivers/misc/xlink-ipc/xlink-ipc.c
new file mode 100644
index 000000000000..088bb7b3da46
--- /dev/null
+++ b/drivers/misc/xlink-ipc/xlink-ipc.c
@@ -0,0 +1,879 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * xlink Linux Kernel Platform API
+ *
+ * Copyright (C) 2018-2020 Intel Corporation
+ *
+ */
+#include <linux/device.h>
+#include <linux/dma-mapping.h>
+#include <linux/kernel.h>
+#include <linux/init.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/of_platform.h>
+#include <linux/of_reserved_mem.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
+
+#include <linux/soc/intel/keembay-vpu-ipc.h>
+
+#include <linux/xlink-ipc.h>
+
+#define XLINK_IPC_MAX_DEVICE_NAME_SIZE 12
+
+/* used to extract fields from and create xlink sw device id */
+#define SW_DEVICE_ID_INTERFACE_SHIFT 24U
+#define SW_DEVICE_ID_INTERFACE_MASK 0x7
+#define SW_DEVICE_ID_INTERFACE_SMASK \
+ (SW_DEVICE_ID_INTERFACE_MASK << SW_DEVICE_ID_INTERFACE_SHIFT)
+#define SW_DEVICE_ID_INTERFACE_IPC_VALUE 0x0
+#define SW_DEVICE_ID_INTERFACE_IPC_SVALUE \
+ (SW_DEVICE_ID_INTERFACE_IPC_VALUE << SW_DEVICE_ID_INTERFACE_SHIFT)
+#define SW_DEVICE_ID_VPU_ID_SHIFT 1U
+#define SW_DEVICE_ID_VPU_ID_MASK 0x7
+#define SW_DEVICE_ID_VPU_ID_SMASK \
+ (SW_DEVICE_ID_VPU_ID_MASK << SW_DEVICE_ID_VPU_ID_SHIFT)
+#define GET_VPU_ID_FROM_SW_DEVICE_ID(id) \
+ (((id) >> SW_DEVICE_ID_VPU_ID_SHIFT) & SW_DEVICE_ID_VPU_ID_MASK)
+#define GET_SW_DEVICE_ID_FROM_VPU_ID(id) \
+ ((((id) << SW_DEVICE_ID_VPU_ID_SHIFT) & SW_DEVICE_ID_VPU_ID_SMASK) \
+ | SW_DEVICE_ID_INTERFACE_IPC_SVALUE)
+
+/* the maximum buffer size for volatile xlink operations */
+#define XLINK_MAX_BUF_SIZE 128U
+
+/* indices used to retrieve reserved memory from the dt */
+#define LOCAL_XLINK_IPC_BUFFER_IDX 0
+#define REMOTE_XLINK_IPC_BUFFER_IDX 1
+
+/* index used to retrieve the vpu ipc device phandle from the dt */
+#define VPU_IPC_DEVICE_PHANDLE_IDX 1
+
+/* the timeout (in ms) used to wait for the vpu ready message */
+#define XLINK_VPU_WAIT_FOR_READY_MS 3000
+
+/* xlink buffer memory region */
+struct xlink_buf_mem {
+ struct device *dev; /* child device managing the memory region */
+ void *vaddr; /* the virtual address of the memory region */
+ dma_addr_t dma_handle; /* the physical address of the memory region */
+ size_t size; /* the size of the memory region */
+};
+
+/* xlink buffer pool */
+struct xlink_buf_pool {
+ void *buf; /* pointer to the start of pool area */
+ size_t buf_cnt; /* pool size (i.e., number of buffers) */
+ size_t idx; /* current index */
+ spinlock_t lock; /* the lock protecting this pool */
+};
+
+/* xlink ipc device */
+struct xlink_ipc_dev {
+ struct platform_device *pdev; /* pointer to platform device */
+ u32 vpu_id; /* The VPU ID defined in the device tree */
+ u32 sw_device_id; /* the sw device id */
+ const char *device_name; /* the vpu device name */
+ struct xlink_buf_mem local_xlink_mem; /* tx buffer memory region */
+ struct xlink_buf_mem remote_xlink_mem; /* rx buffer memory region */
+ struct xlink_buf_pool xlink_buf_pool; /* tx buffer pool */
+ struct device *vpu_dev; /* pointer to vpu ipc device */
+ int (*callback)(u32 sw_device_id, u32 event);
+};
+
+/* Events that can be notified via callback, when registered. */
+enum xlink_vpu_event {
+ XLINK_VPU_NOTIFY_DISCONNECT = 0,
+ XLINK_VPU_NOTIFY_CONNECT,
+ XLINK_VPU_NOTIFY_MSS_WDT,
+ XLINK_VPU_NOTIFY_NCE_WDT,
+ NUM_EVENT_TYPE
+};
+
+#define VPU_ID 0
+#define VPU_DEVICE_NAME "vpu-0"
+#define VPU_SW_DEVICE_ID 0
+static struct xlink_ipc_dev *xlink_dev;
+
+/*
+ * Functions related to reserved-memory sub-devices.
+ */
+
+/*
+ * xlink_reserved_memory_remove() - Removes the reserved memory sub-devices.
+ *
+ * @xlink_dev: [in] The xlink ipc device with reserved memory sub-devices.
+ */
+static void xlink_reserved_memory_remove(struct xlink_ipc_dev *xlink_dev)
+{
+ device_unregister(xlink_dev->local_xlink_mem.dev);
+ device_unregister(xlink_dev->remote_xlink_mem.dev);
+}
+
+/*
+ * xlink_reserved_mem_release() - Reserved memory release callback function.
+ *
+ * @dev: [in] The reserved memory sub-device.
+ */
+static void xlink_reserved_mem_release(struct device *dev)
+{
+ of_reserved_mem_device_release(dev);
+}
+
+/*
+ * get_xlink_reserved_mem_size() - Gets the size of the reserved memory region.
+ *
+ * @dev: [in] The device the reserved memory region is allocated to.
+ * @idx: [in] The reserved memory region's index in the phandle table.
+ *
+ * Return: The reserved memory size, 0 on failure.
+ */
+static resource_size_t get_xlink_reserved_mem_size(struct device *dev, int idx)
+{
+ struct device_node *np;
+ struct resource mem;
+ int rc;
+
+ np = of_parse_phandle(dev->of_node, "memory-region", idx);
+ if (!np) {
+ dev_err(dev, "Couldn't find memory-region %d\n", idx);
+ return 0;
+ }
+
+ rc = of_address_to_resource(np, 0, &mem);
+ if (rc) {
+ dev_err(dev, "Couldn't map address to resource %d\n", idx);
+ return 0;
+ }
+ return resource_size(&mem);
+}
+
+/*
+ * init_xlink_reserved_mem_dev() - Initializes the reserved memory sub-devices.
+ *
+ * @dev: [in] The parent device of the reserved memory sub-device.
+ * @name: [in] The name to assign to the memory region.
+ * @idx: [in] The reserved memory region index in the phandle table.
+ *
+ * Return: The initialized sub-device, NULL on failure.
+ */
+static struct device *init_xlink_reserved_mem_dev(struct device *dev,
+ const char *name, int idx)
+{
+ struct device *child;
+ int rc;
+
+ child = devm_kzalloc(dev, sizeof(*child), GFP_KERNEL);
+ if (!child)
+ return NULL;
+
+ device_initialize(child);
+ dev_set_name(child, "%s:%s", dev_name(dev), name);
+ dev_err(dev, " dev_name %s, name %s\n", dev_name(dev), name);
+ child->parent = dev;
+ child->dma_mask = dev->dma_mask;
+ child->coherent_dma_mask = dev->coherent_dma_mask;
+ /* set up dma configuration using information from parent's dt node */
+ rc = of_dma_configure(child, dev->of_node, true);
+ if (rc)
+ return NULL;
+ child->release = xlink_reserved_mem_release;
+
+ rc = device_add(child);
+ if (rc)
+ goto err;
+ rc = of_reserved_mem_device_init_by_idx(child, dev->of_node, idx);
+ if (rc) {
+ dev_err(dev, "Couldn't get reserved memory with idx = %d, %d\n",
+ idx, rc);
+ device_del(child);
+ goto err;
+ }
+ return child;
+
+err:
+ put_device(child);
+ return NULL;
+}
+
+/*
+ * xlink_reserved_memory_init() - Initialize reserved memory for the device.
+ *
+ * @xlink_dev: [in] The xlink ipc device the reserved memory is allocated to.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+static int xlink_reserved_memory_init(struct xlink_ipc_dev *xlink_dev)
+{
+ struct device *dev = &xlink_dev->pdev->dev;
+
+ xlink_dev->local_xlink_mem.dev = init_xlink_reserved_mem_dev(dev,
+ "xlink_local_reserved",
+ LOCAL_XLINK_IPC_BUFFER_IDX);
+ if (!xlink_dev->local_xlink_mem.dev)
+ return -ENOMEM;
+
+ xlink_dev->local_xlink_mem.size = get_xlink_reserved_mem_size(dev,
+ LOCAL_XLINK_IPC_BUFFER_IDX);
+
+ xlink_dev->remote_xlink_mem.dev = init_xlink_reserved_mem_dev(dev,
+ "xlink_remote_reserved",
+ REMOTE_XLINK_IPC_BUFFER_IDX);
+ if (!xlink_dev->remote_xlink_mem.dev) {
+ device_unregister(xlink_dev->local_xlink_mem.dev);
+ return -ENOMEM;
+ }
+
+ xlink_dev->remote_xlink_mem.size = get_xlink_reserved_mem_size(dev,
+ REMOTE_XLINK_IPC_BUFFER_IDX);
+
+ return 0;
+}
+
+/*
+ * xlink_reserved_memory_alloc() - Allocate reserved memory for the device.
+ *
+ * @xlink_dev: [in] The xlink ipc device.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+static int xlink_reserved_memory_alloc(struct xlink_ipc_dev *xlink_dev)
+{
+ xlink_dev->local_xlink_mem.vaddr =
+ dmam_alloc_coherent(xlink_dev->local_xlink_mem.dev,
+ xlink_dev->local_xlink_mem.size,
+ &xlink_dev->local_xlink_mem.dma_handle,
+ GFP_KERNEL);
+ if (!xlink_dev->local_xlink_mem.vaddr) {
+ dev_err(&xlink_dev->pdev->dev,
+ "Failed to allocate from local reserved memory.\n");
+ return -ENOMEM;
+ }
+ xlink_dev->remote_xlink_mem.vaddr = dmam_alloc_coherent
+ (xlink_dev->remote_xlink_mem.dev,
+ xlink_dev->remote_xlink_mem.size,
+ &xlink_dev->remote_xlink_mem.dma_handle,
+ GFP_KERNEL);
+ if (!xlink_dev->remote_xlink_mem.vaddr) {
+ dev_err(&xlink_dev->pdev->dev,
+ "Failed to allocate from remote reserved memory.\n");
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+/*
+ * init_xlink_buf_pool() - Initialize the device's tx buffer pool.
+ *
+ * @xlink_dev: [in] The xlink ipc device.
+ *
+ * Return: 0 on success.
+ */
+static int init_xlink_buf_pool(struct xlink_ipc_dev *xlink_dev)
+{
+ struct xlink_buf_mem *mem = &xlink_dev->local_xlink_mem;
+
+ memset(mem->vaddr, 0, mem->size);
+ xlink_dev->xlink_buf_pool.buf = mem->vaddr;
+ xlink_dev->xlink_buf_pool.buf_cnt =
+ mem->size / XLINK_MAX_BUF_SIZE;
+ xlink_dev->xlink_buf_pool.idx = 0;
+ dev_info(&xlink_dev->pdev->dev, "xlink Buffer Pool size: %zX\n",
+ xlink_dev->xlink_buf_pool.buf_cnt);
+ spin_lock_init(&xlink_dev->xlink_buf_pool.lock);
+
+ return 0;
+}
+
+/*
+ * xlink_phys_to_virt() - Convert an xlink physical addresses to a virtual one.
+ *
+ * @xlink_mem: [in] The memory region where the physical address is located.
+ * @paddr: [in] The physical address to convert to a virtual one.
+ *
+ * Return: The corresponding virtual address, or NULL if the
+ * physical address is not in the expected memory
+ * range.
+ */
+static void *xlink_phys_to_virt(const struct xlink_buf_mem *xlink_mem,
+ u32 paddr)
+{
+ if (unlikely(paddr < xlink_mem->dma_handle) ||
+ paddr >= (xlink_mem->dma_handle + xlink_mem->size))
+ return NULL;
+
+ return xlink_mem->vaddr + (paddr - xlink_mem->dma_handle);
+}
+
+/*
+ * xlink_virt_to_phys() - Convert an xlink virtual addresses to a physical one.
+ *
+ * @xlink_mem: [in] The memory region where the physical address is located.
+ * @vaddr: [in] The virtual address to convert to a physical one.
+ * @paddr: [out] Where to store the computed physical address.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+static int xlink_virt_to_phys(struct xlink_buf_mem *xlink_mem, void *vaddr,
+ u32 *paddr)
+{
+ if (unlikely((xlink_mem->dma_handle + xlink_mem->size) > 0xFFFFFFFF))
+ return -EINVAL;
+ if (unlikely(vaddr < xlink_mem->vaddr ||
+ vaddr >= (xlink_mem->vaddr + xlink_mem->size)))
+ return -EINVAL;
+ *paddr = xlink_mem->dma_handle + (vaddr - xlink_mem->vaddr);
+
+ return 0;
+}
+
+/*
+ * get_next_xlink_buf() - Get next xlink buffer from an xlink device's pool.
+ *
+ * @xlink_dev: [in] The xlink ipc device to get a buffer from.
+ * @buf: [out] Where to store the reference to the next buffer.
+ * @size: [in] The size of the buffer to get.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+static int get_next_xlink_buf(struct xlink_ipc_dev *xlink_dev, void **buf,
+ int size)
+{
+ struct xlink_buf_pool *pool;
+ unsigned long flags;
+
+ if (!xlink_dev)
+ return -ENODEV;
+
+ if (size > XLINK_MAX_BUF_SIZE)
+ return -EINVAL;
+
+ pool = &xlink_dev->xlink_buf_pool;
+
+ spin_lock_irqsave(&pool->lock, flags);
+ if (pool->idx == pool->buf_cnt) {
+ /* reached end of buffers - wrap around */
+ pool->idx = 0;
+ }
+ *buf = pool->buf + (pool->idx * XLINK_MAX_BUF_SIZE);
+ pool->idx++;
+ spin_unlock_irqrestore(&pool->lock, flags);
+ return 0;
+}
+
+/*
+ * Functions related to the vpu ipc device reference.
+ */
+
+/*
+ * vpu_ipc_device_put() - Release the vpu ipc device held by the xlink device.
+ *
+ * @xlink_dev: [in] The xlink ipc device.
+ */
+static void vpu_ipc_device_put(struct xlink_ipc_dev *xlink_dev)
+{
+ put_device(xlink_dev->vpu_dev);
+}
+
+/*
+ * vpu_ipc_device_get() - Get the vpu ipc device reference for the xlink device.
+ *
+ * @xlink_dev: [in] The xlink ipc device.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+static int vpu_ipc_device_get(struct xlink_ipc_dev *xlink_dev)
+{
+ struct device *dev = &xlink_dev->pdev->dev;
+ struct platform_device *pdev;
+ struct device_node *np;
+
+ np = of_parse_phandle(dev->of_node, "intel,keembay-vpu-ipc", 0);
+ if (!np)
+ return -ENODEV;
+
+ pdev = of_find_device_by_node(np);
+ if (!pdev) {
+ dev_info(dev, "IPC device not probed\n");
+ of_node_put(np);
+ return -EPROBE_DEFER;
+ }
+
+ xlink_dev->vpu_dev = get_device(&pdev->dev);
+ of_node_put(np);
+
+ dev_info(dev, "Using IPC device: %s\n", dev_name(xlink_dev->vpu_dev));
+ return 0;
+}
+
+/*
+ * xlink platform api - ipc interface functions
+ */
+
+/*
+ * xlink_ipc_connect() - platform connect interface.
+ *
+ * @sw_device_id: [in] The sw device id of the device to connect to.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int xlink_ipc_connect(u32 sw_device_id)
+{
+ if (!xlink_dev)
+ return -ENODEV;
+
+ return 0;
+}
+EXPORT_SYMBOL(xlink_ipc_connect);
+
+/*
+ * xlink_ipc_write() - platform write interface.
+ *
+ * @sw_device_id: [in] The sw device id of the device to write to.
+ * @data: [in] The data buffer to write.
+ * @size: [in-out] The amount of data to write/written.
+ * @timeout: [in] The time (in ms) to wait before timing out.
+ * @context: [in] The ipc operation context.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int xlink_ipc_write(u32 sw_device_id, void *data, size_t * const size,
+ u32 timeout, void *context)
+{
+ struct xlink_ipc_context *ctx = context;
+ void *vaddr = NULL;
+ u32 paddr;
+ int rc;
+
+ if (!ctx)
+ return -EINVAL;
+
+ if (!xlink_dev)
+ return -ENODEV;
+
+ if (ctx->is_volatile) {
+ rc = get_next_xlink_buf(xlink_dev, &vaddr, XLINK_MAX_BUF_SIZE);
+ if (rc)
+ return rc;
+ memcpy(vaddr, data, *size);
+ rc = xlink_virt_to_phys(&xlink_dev->local_xlink_mem, vaddr,
+ &paddr);
+ if (rc)
+ return rc;
+ } else {
+ paddr = *(u32 *)data;
+ }
+ rc = intel_keembay_vpu_ipc_send(xlink_dev->vpu_dev,
+ KMB_VPU_IPC_NODE_LEON_MSS, ctx->chan,
+ paddr, *size);
+
+ return rc;
+}
+EXPORT_SYMBOL(xlink_ipc_write);
+
+/*
+ * xlink_ipc_read() - platform read interface.
+ *
+ * @sw_device_id: [in] The sw device id of the device to read from.
+ * @data: [out] The data buffer to read into.
+ * @size: [in-out] The amount of data to read/was read.
+ * @timeout: [in] The time (in ms) to wait before timing out.
+ * @context: [in] The ipc operation context.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int xlink_ipc_read(u32 sw_device_id, void *data, size_t * const size,
+ u32 timeout, void *context)
+{
+ struct xlink_ipc_context *ctx = context;
+ u32 addr = 0;
+ void *vaddr;
+ int rc;
+
+ if (!ctx)
+ return -EINVAL;
+
+ if (!xlink_dev)
+ return -ENODEV;
+
+ rc = intel_keembay_vpu_ipc_recv(xlink_dev->vpu_dev,
+ KMB_VPU_IPC_NODE_LEON_MSS, ctx->chan,
+ &addr, size, timeout);
+
+ if (ctx->is_volatile) {
+ vaddr = xlink_phys_to_virt(&xlink_dev->remote_xlink_mem, addr);
+ if (vaddr)
+ memcpy(data, vaddr, *size);
+ else
+ return -ENXIO;
+ } else {
+ *(u32 *)data = addr;
+ }
+ return rc;
+}
+EXPORT_SYMBOL(xlink_ipc_read);
+
+/*
+ * xlink_ipc_get_device_list() - platform get device list interface.
+ *
+ * @sw_device_id_list: [out] The list of devices found.
+ * @num_devices: [out] The number of devices found.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int xlink_ipc_get_device_list(u32 *sw_device_id_list, u32 *num_devices)
+{
+ int i = 0;
+
+ if (!sw_device_id_list || !num_devices)
+ return -EINVAL;
+
+ if (xlink_dev) {
+ *sw_device_id_list = xlink_dev->sw_device_id;
+ i++;
+ }
+
+ *num_devices = i;
+ return 0;
+}
+EXPORT_SYMBOL(xlink_ipc_get_device_list);
+
+/*
+ * xlink_ipc_get_device_name() - platform get device name interface.
+ *
+ * @sw_device_id: [in] The sw device id of the device to get name of.
+ * @device_name: [out] The name of the xlink ipc device.
+ * @name_size: [in] The maximum size of the name.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int xlink_ipc_get_device_name(u32 sw_device_id, char *device_name,
+ size_t name_size)
+{
+ size_t size;
+
+ if (!device_name)
+ return -EINVAL;
+
+ if (!xlink_dev)
+ return -ENODEV;
+
+ size = (name_size > XLINK_IPC_MAX_DEVICE_NAME_SIZE)
+ ? XLINK_IPC_MAX_DEVICE_NAME_SIZE
+ : name_size;
+ strncpy(device_name, xlink_dev->device_name, size);
+ return 0;
+}
+EXPORT_SYMBOL(xlink_ipc_get_device_name);
+
+/*
+ * xlink_ipc_get_device_status() - platform get device status interface.
+ *
+ * @sw_device_id: [in] The sw device id of the device to get status of.
+ * @device_status: [out] The device status.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int xlink_ipc_get_device_status(u32 sw_device_id, u32 *device_status)
+{
+ if (!device_status)
+ return -EINVAL;
+
+ if (!xlink_dev)
+ return -ENODEV;
+
+ *device_status = intel_keembay_vpu_status(xlink_dev->vpu_dev);
+ return 0;
+}
+EXPORT_SYMBOL(xlink_ipc_get_device_status);
+
+static void kernel_callback(struct device *dev, enum intel_keembay_vpu_event event)
+{
+ if ((enum xlink_vpu_event)event >= NUM_EVENT_TYPE)
+ return;
+
+ if (xlink_dev) {
+ if (xlink_dev->callback)
+ xlink_dev->callback(xlink_dev->sw_device_id, event);
+ }
+}
+
+/*
+ * xlink_ipc_register_for_events() - platform register for events
+ *
+ * @sw_device_id: [in] The sw device id of the device to get status of.
+ * @callback: [in] Callback function for events
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int xlink_ipc_register_for_events(u32 sw_device_id,
+ int (*callback)(u32 sw_device_id, enum xlink_vpu_event event))
+{
+ int rc;
+
+ if (!xlink_dev)
+ return -ENODEV;
+ xlink_dev->callback = callback;
+ rc = intel_keembay_vpu_register_for_events(xlink_dev->vpu_dev, kernel_callback);
+ return rc;
+}
+EXPORT_SYMBOL(xlink_ipc_register_for_events);
+/*
+ * xlink_ipc_unregister_for_events() - platform register for events
+ *
+ * @sw_device_id: [in] The sw device id of the device to get status of.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int xlink_ipc_unregister_for_events(u32 sw_device_id)
+{
+ int rc;
+
+ if (!xlink_dev)
+ return -ENODEV;
+ rc = intel_keembay_vpu_unregister_for_events(xlink_dev->vpu_dev);
+ return rc;
+}
+EXPORT_SYMBOL(xlink_ipc_unregister_for_events);
+
+/*
+ * xlink_ipc_boot_device() - platform boot device interface.
+ *
+ * @sw_device_id: [in] The sw device id of the device to boot.
+ * @binary_name: [in] The file name of the firmware binary to boot.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int xlink_ipc_boot_device(u32 sw_device_id, const char *binary_name)
+{
+ enum intel_keembay_vpu_state state;
+ int rc;
+
+ if (!binary_name)
+ return -EINVAL;
+
+ if (!xlink_dev)
+ return -ENODEV;
+
+ pr_info("\nStart VPU 0x%x - %s\n", sw_device_id, binary_name);
+ rc = intel_keembay_vpu_startup(xlink_dev->vpu_dev, binary_name);
+ if (rc) {
+ pr_err("Failed to start VPU: %d\n", rc);
+ return -EBUSY;
+ }
+ pr_info("Successfully started VPU!\n");
+
+ /* Wait for VPU to be READY */
+ rc = intel_keembay_vpu_wait_for_ready(xlink_dev->vpu_dev,
+ XLINK_VPU_WAIT_FOR_READY_MS);
+ if (rc) {
+ pr_err("Tried to start VPU but never got READY.\n");
+ return -EBUSY;
+ }
+ pr_info("Successfully synchronised state with VPU!\n");
+
+ /* Check state */
+ state = intel_keembay_vpu_status(xlink_dev->vpu_dev);
+ if (state != KEEMBAY_VPU_READY) {
+ pr_err("VPU was not ready, it was %d\n", state);
+ return -EBUSY;
+ }
+ pr_info("VPU was ready.\n");
+ return 0;
+}
+EXPORT_SYMBOL(xlink_ipc_boot_device);
+
+/*
+ * xlink_ipc_reset_device() - platform reset device interface.
+ *
+ * @sw_device_id: [in] The sw device id of the device to reset.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int xlink_ipc_reset_device(u32 sw_device_id)
+{
+ enum intel_keembay_vpu_state state;
+ int rc;
+
+ if (!xlink_dev)
+ return -ENODEV;
+
+ /* stop the vpu */
+ rc = intel_keembay_vpu_stop(xlink_dev->vpu_dev);
+ if (rc) {
+ pr_err("Failed to stop VPU: %d\n", rc);
+ return -EBUSY;
+ }
+ pr_info("Successfully stopped VPU!\n");
+
+ /* check state */
+ state = intel_keembay_vpu_status(xlink_dev->vpu_dev);
+ if (state != KEEMBAY_VPU_OFF) {
+ pr_err("VPU was not OFF after stop request, it was %d\n",
+ state);
+ return -EBUSY;
+ }
+ return 0;
+}
+EXPORT_SYMBOL(xlink_ipc_reset_device);
+
+/*
+ * xlink_ipc_open_channel() - platform open channel interface.
+ *
+ * @sw_device_id: [in] The sw device id of the device to open channel to.
+ * @channel: [in] The channel id to open.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int xlink_ipc_open_channel(u32 sw_device_id, u32 channel)
+{
+ int rc;
+
+ if (!xlink_dev)
+ return -ENODEV;
+
+ rc = intel_keembay_vpu_ipc_open_channel(xlink_dev->vpu_dev,
+ KMB_VPU_IPC_NODE_LEON_MSS, channel);
+ return rc;
+}
+EXPORT_SYMBOL(xlink_ipc_open_channel);
+
+/*
+ * xlink_ipc_close_channel() - platform close channel interface.
+ *
+ * @sw_device_id: [in] The sw device id of the device to close channel to.
+ * @channel: [in] The channel id to close.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int xlink_ipc_close_channel(u32 sw_device_id, u32 channel)
+{
+ int rc;
+
+ if (!xlink_dev)
+ return -ENODEV;
+
+ rc = intel_keembay_vpu_ipc_close_channel(xlink_dev->vpu_dev,
+ KMB_VPU_IPC_NODE_LEON_MSS, channel);
+ return rc;
+}
+EXPORT_SYMBOL(xlink_ipc_close_channel);
+
+/*
+ * xlink ipc driver functions
+ */
+
+static int keembay_xlink_ipc_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ int rc;
+
+ /* allocate device data structure */
+ xlink_dev = kzalloc(sizeof(*xlink_dev), GFP_KERNEL);
+ if (!xlink_dev)
+ return -ENOMEM;
+
+ xlink_dev->pdev = pdev;
+ dev_info(dev, "Keem Bay xlink IPC driver probed.\n");
+
+ /* grab reserved memory regions and assign to child devices */
+ rc = xlink_reserved_memory_init(xlink_dev);
+ if (rc < 0) {
+ dev_err(&pdev->dev,
+ "Failed to set up reserved memory regions.\n");
+ goto r_cleanup;
+ }
+
+ /* allocate memory from the reserved memory regions */
+ rc = xlink_reserved_memory_alloc(xlink_dev);
+ if (rc < 0) {
+ dev_err(&pdev->dev,
+ "Failed to allocate reserved memory regions.\n");
+ goto r_cleanup;
+ }
+
+ /* init the xlink buffer pool used for rx/tx */
+ init_xlink_buf_pool(xlink_dev);
+
+ /* get reference to vpu ipc device */
+ rc = vpu_ipc_device_get(xlink_dev);
+ if (rc)
+ goto r_cleanup;
+
+ /* get device id */
+ rc = of_property_read_u32(dev->of_node, "intel,keembay-vpu-ipc-id",
+ &xlink_dev->vpu_id);
+ if (rc) {
+ dev_err(dev, "Cannot get VPU ID from DT.\n");
+ goto r_cleanup;
+ }
+
+ /* assign a sw device id */
+ xlink_dev->sw_device_id = GET_SW_DEVICE_ID_FROM_VPU_ID
+ (xlink_dev->vpu_id);
+
+ /* assign a device name */
+ rc = of_property_read_string(dev->of_node, "intel,keembay-vpu-ipc-name",
+ &xlink_dev->device_name);
+ if (rc) {
+ /* only warn for now; we will enforce this in the future */
+ dev_warn(dev, "VPU name not defined in DT, using %s as default.\n",
+ VPU_DEVICE_NAME);
+ dev_warn(dev, "WARNING: additional VPU devices may fail probing.\n");
+ xlink_dev->device_name = VPU_DEVICE_NAME;
+ }
+
+ /* get platform data reference */
+ platform_set_drvdata(pdev, xlink_dev);
+
+ dev_info(dev, "Device id=%u sw_device_id=0x%x name=%s probe complete.\n",
+ xlink_dev->vpu_id, xlink_dev->sw_device_id,
+ xlink_dev->device_name);
+ return 0;
+
+r_cleanup:
+ xlink_reserved_memory_remove(xlink_dev);
+ return rc;
+}
+
+static int keembay_xlink_ipc_remove(struct platform_device *pdev)
+{
+ struct xlink_ipc_dev *xlink_dev = platform_get_drvdata(pdev);
+ struct device *dev = &pdev->dev;
+
+ /*
+ * no need to de-alloc xlink mem (local_xlink_mem and remote_xlink_mem)
+ * since it was allocated with dmam_alloc
+ */
+ xlink_reserved_memory_remove(xlink_dev);
+
+ /* release vpu ipc device */
+ vpu_ipc_device_put(xlink_dev);
+
+ dev_info(dev, "Keem Bay xlink IPC driver removed.\n");
+ return 0;
+}
+
+static const struct of_device_id keembay_xlink_ipc_of_match[] = {
+ {
+ .compatible = "intel,keembay-xlink-ipc",
+ },
+ {}
+};
+
+static struct platform_driver keembay_xlink_ipc_driver = {
+ .driver = {
+ .name = "keembay-xlink-ipc",
+ .of_match_table = keembay_xlink_ipc_of_match,
+ },
+ .probe = keembay_xlink_ipc_probe,
+ .remove = keembay_xlink_ipc_remove,
+};
+module_platform_driver(keembay_xlink_ipc_driver);
+
+MODULE_DESCRIPTION("Keem Bay xlink IPC Driver");
+MODULE_AUTHOR("Ryan Carnaghi <[email protected]>");
+MODULE_LICENSE("GPL v2");
diff --git a/include/linux/xlink-ipc.h b/include/linux/xlink-ipc.h
new file mode 100644
index 000000000000..f26b53bf6506
--- /dev/null
+++ b/include/linux/xlink-ipc.h
@@ -0,0 +1,48 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*****************************************************************************
+ *
+ * Intel Keem Bay xlink IPC Driver
+ *
+ * Copyright (C) 2020 Intel Corporation
+ *
+ ****************************************************************************/
+
+#ifndef _XLINK_IPC_H_
+#define _XLINK_IPC_H_
+
+#include <linux/types.h>
+
+struct xlink_ipc_context {
+ u16 chan;
+ u8 is_volatile;
+};
+
+int xlink_ipc_connect(u32 sw_device_id);
+
+int xlink_ipc_write(u32 sw_device_id, void *data, size_t * const size,
+ u32 timeout, void *context);
+
+int xlink_ipc_read(u32 sw_device_id, void *data, size_t * const size,
+ u32 timeout, void *context);
+
+int xlink_ipc_get_device_list(u32 *sw_device_id_list, u32 *num_devices);
+
+int xlink_ipc_get_device_name(u32 sw_device_id, char *device_name,
+ size_t name_size);
+
+int xlink_ipc_get_device_status(u32 sw_device_id, u32 *device_status);
+
+int xlink_ipc_boot_device(u32 sw_device_id, const char *binary_name);
+
+int xlink_ipc_reset_device(u32 sw_device_id);
+
+int xlink_ipc_open_channel(u32 sw_device_id, u32 channel);
+
+int xlink_ipc_close_channel(u32 sw_device_id, u32 channel);
+
+int xlink_ipc_register_for_events(u32 sw_device_id,
+ int (*callback)(u32 sw_device_id, u32 event));
+
+int xlink_ipc_unregister_for_events(u32 sw_device_id);
+
+#endif /* _XLINK_IPC_H_ */
--
2.17.1

2020-12-01 22:43:24

by mark gross

[permalink] [raw]
Subject: [PATCH 09/22] misc: xlink-pcie: lh: Add core communication logic

From: Srikanth Thokala <[email protected]>

Add logic to establish communication with the remote host which is through
ring buffer management and MSI/Doorbell interrupts

Reviewed-by: Mark Gross <[email protected]>
Signed-off-by: Srikanth Thokala <[email protected]>
---
drivers/misc/xlink-pcie/local_host/Makefile | 2 +
drivers/misc/xlink-pcie/local_host/core.c | 894 ++++++++++++++++++++
drivers/misc/xlink-pcie/local_host/core.h | 247 ++++++
drivers/misc/xlink-pcie/local_host/epf.c | 116 ++-
drivers/misc/xlink-pcie/local_host/epf.h | 26 +
drivers/misc/xlink-pcie/local_host/util.c | 375 ++++++++
drivers/misc/xlink-pcie/local_host/util.h | 70 ++
drivers/misc/xlink-pcie/local_host/xpcie.h | 65 ++
include/linux/xlink_drv_inf.h | 60 ++
9 files changed, 1847 insertions(+), 8 deletions(-)
create mode 100644 drivers/misc/xlink-pcie/local_host/core.c
create mode 100644 drivers/misc/xlink-pcie/local_host/core.h
create mode 100644 drivers/misc/xlink-pcie/local_host/util.c
create mode 100644 drivers/misc/xlink-pcie/local_host/util.h
create mode 100644 include/linux/xlink_drv_inf.h

diff --git a/drivers/misc/xlink-pcie/local_host/Makefile b/drivers/misc/xlink-pcie/local_host/Makefile
index 54fc118e2dd1..28761751d43b 100644
--- a/drivers/misc/xlink-pcie/local_host/Makefile
+++ b/drivers/misc/xlink-pcie/local_host/Makefile
@@ -1,3 +1,5 @@
obj-$(CONFIG_XLINK_PCIE_LH_DRIVER) += mxlk_ep.o
mxlk_ep-objs := epf.o
mxlk_ep-objs += dma.o
+mxlk_ep-objs += core.o
+mxlk_ep-objs += util.o
diff --git a/drivers/misc/xlink-pcie/local_host/core.c b/drivers/misc/xlink-pcie/local_host/core.c
new file mode 100644
index 000000000000..aecaaa783153
--- /dev/null
+++ b/drivers/misc/xlink-pcie/local_host/core.c
@@ -0,0 +1,894 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*****************************************************************************
+ *
+ * Intel Keem Bay XLink PCIe Driver
+ *
+ * Copyright (C) 2020 Intel Corporation
+ *
+ ****************************************************************************/
+
+#include <linux/of_reserved_mem.h>
+
+#include "epf.h"
+#include "core.h"
+#include "util.h"
+
+static struct xpcie *global_xpcie;
+
+static int rx_pool_size = SZ_32M;
+module_param(rx_pool_size, int, 0664);
+MODULE_PARM_DESC(rx_pool_size, "receiving pool size (default 32 MiB)");
+
+static int tx_pool_size = SZ_32M;
+module_param(tx_pool_size, int, 0664);
+MODULE_PARM_DESC(tx_pool_size, "transmitting pool size (default 32 MiB)");
+
+static int fragment_size = XPCIE_FRAGMENT_SIZE;
+module_param(fragment_size, int, 0664);
+MODULE_PARM_DESC(fragment_size, "transfer descriptor size (default 128 KiB)");
+
+static bool tx_pool_coherent = true;
+module_param(tx_pool_coherent, bool, 0664);
+MODULE_PARM_DESC(tx_pool_coherent,
+ "transmitting pool using coherent memory (default true)");
+
+static bool rx_pool_coherent;
+module_param(rx_pool_coherent, bool, 0664);
+MODULE_PARM_DESC(rx_pool_coherent,
+ "receiving pool using coherent memory (default false)");
+
+static struct xpcie *intel_xpcie_core_get_by_id(u32 sw_device_id)
+{
+ return (sw_device_id == xlink_sw_id) ? global_xpcie : NULL;
+}
+
+static int intel_xpcie_map_dma(struct xpcie *xpcie, struct xpcie_buf_desc *bd,
+ int direction)
+{
+ struct xpcie_epf *xpcie_epf = container_of(xpcie,
+ struct xpcie_epf, xpcie);
+ struct pci_epf *epf = xpcie_epf->epf;
+ struct device *dma_dev = epf->epc->dev.parent;
+
+ bd->phys = dma_map_single(dma_dev, bd->data, bd->length, direction);
+
+ return dma_mapping_error(dma_dev, bd->phys);
+}
+
+static void intel_xpcie_unmap_dma(struct xpcie *xpcie,
+ struct xpcie_buf_desc *bd, int direction)
+{
+ struct xpcie_epf *xpcie_epf = container_of(xpcie,
+ struct xpcie_epf, xpcie);
+ struct pci_epf *epf = xpcie_epf->epf;
+ struct device *dma_dev = epf->epc->dev.parent;
+
+ dma_unmap_single(dma_dev, bd->phys, bd->length, direction);
+}
+
+static void intel_xpcie_set_cap_txrx(struct xpcie *xpcie)
+{
+ size_t tx_len = sizeof(struct xpcie_transfer_desc) *
+ XPCIE_NUM_TX_DESCS;
+ size_t rx_len = sizeof(struct xpcie_transfer_desc) *
+ XPCIE_NUM_RX_DESCS;
+ size_t hdr_len = sizeof(struct xpcie_cap_txrx);
+ u32 start = sizeof(struct xpcie_mmio);
+ struct xpcie_cap_txrx *cap;
+ struct xpcie_cap_hdr *hdr;
+ u16 next;
+
+ next = (u16)(start + hdr_len + tx_len + rx_len);
+ intel_xpcie_iowrite32(start, xpcie->mmio + XPCIE_MMIO_CAP_OFF);
+ cap = (void *)xpcie->mmio + start;
+ memset(cap, 0, sizeof(struct xpcie_cap_txrx));
+ cap->hdr.id = XPCIE_CAP_TXRX;
+ cap->hdr.next = next;
+ cap->fragment_size = fragment_size;
+ cap->tx.ring = start + hdr_len;
+ cap->tx.ndesc = XPCIE_NUM_TX_DESCS;
+ cap->rx.ring = start + hdr_len + tx_len;
+ cap->rx.ndesc = XPCIE_NUM_RX_DESCS;
+
+ hdr = (struct xpcie_cap_hdr *)((void *)xpcie->mmio + next);
+ hdr->id = XPCIE_CAP_NULL;
+}
+
+static int intel_xpcie_set_version(struct xpcie *xpcie)
+{
+ struct xpcie_version version;
+
+ version.major = XPCIE_VERSION_MAJOR;
+ version.minor = XPCIE_VERSION_MINOR;
+ version.build = XPCIE_VERSION_BUILD;
+
+ memcpy(xpcie->mmio + XPCIE_MMIO_VERSION, &version, sizeof(version));
+
+ dev_dbg(xpcie_to_dev(xpcie), "ver: device %u.%u.%u\n",
+ version.major, version.minor, version.build);
+
+ return 0;
+}
+
+static void intel_xpcie_txrx_cleanup(struct xpcie *xpcie)
+{
+ struct xpcie_epf *xpcie_epf = container_of(xpcie,
+ struct xpcie_epf, xpcie);
+ struct device *dma_dev = xpcie_epf->epf->epc->dev.parent;
+ struct xpcie_interface *inf = &xpcie->interfaces[0];
+ struct xpcie_stream *tx = &xpcie->tx;
+ struct xpcie_stream *rx = &xpcie->rx;
+ struct xpcie_transfer_desc *td;
+ int index;
+
+ xpcie->stop_flag = true;
+ xpcie->no_tx_buffer = false;
+ inf->data_avail = true;
+ wake_up_interruptible(&xpcie->tx_waitq);
+ wake_up_interruptible(&inf->rx_waitq);
+ mutex_lock(&xpcie->wlock);
+ mutex_lock(&inf->rlock);
+
+ for (index = 0; index < rx->pipe.ndesc; index++) {
+ td = rx->pipe.tdr + index;
+ intel_xpcie_set_td_address(td, 0);
+ intel_xpcie_set_td_length(td, 0);
+ }
+ for (index = 0; index < tx->pipe.ndesc; index++) {
+ td = tx->pipe.tdr + index;
+ intel_xpcie_set_td_address(td, 0);
+ intel_xpcie_set_td_length(td, 0);
+ }
+
+ intel_xpcie_list_cleanup(&xpcie->tx_pool);
+ intel_xpcie_list_cleanup(&xpcie->rx_pool);
+
+ if (rx_pool_coherent && xpcie_epf->rx_virt) {
+ dma_free_coherent(dma_dev, xpcie_epf->rx_size,
+ xpcie_epf->rx_virt, xpcie_epf->rx_phys);
+ }
+
+ if (tx_pool_coherent && xpcie_epf->tx_virt) {
+ dma_free_coherent(dma_dev, xpcie_epf->tx_size,
+ xpcie_epf->tx_virt, xpcie_epf->tx_phys);
+ }
+
+ mutex_unlock(&inf->rlock);
+ mutex_unlock(&xpcie->wlock);
+}
+
+/*
+ * The RX/TX are named for Remote Host, in Local Host
+ * RX/TX is reversed.
+ */
+static int intel_xpcie_txrx_init(struct xpcie *xpcie,
+ struct xpcie_cap_txrx *cap)
+{
+ struct xpcie_epf *xpcie_epf = container_of(xpcie,
+ struct xpcie_epf, xpcie);
+ struct device *dma_dev = xpcie_epf->epf->epc->dev.parent;
+ struct xpcie_stream *tx = &xpcie->tx;
+ struct xpcie_stream *rx = &xpcie->rx;
+ struct xpcie_buf_desc *bd;
+ int index, ndesc, rc;
+
+ xpcie->txrx = cap;
+ xpcie->fragment_size = cap->fragment_size;
+ xpcie->stop_flag = false;
+
+ rx->pipe.ndesc = cap->tx.ndesc;
+ rx->pipe.head = &cap->tx.head;
+ rx->pipe.tail = &cap->tx.tail;
+ rx->pipe.tdr = (void *)xpcie->mmio + cap->tx.ring;
+
+ tx->pipe.ndesc = cap->rx.ndesc;
+ tx->pipe.head = &cap->rx.head;
+ tx->pipe.tail = &cap->rx.tail;
+ tx->pipe.tdr = (void *)xpcie->mmio + cap->rx.ring;
+
+ intel_xpcie_list_init(&xpcie->rx_pool);
+ rx_pool_size = roundup(rx_pool_size, xpcie->fragment_size);
+ ndesc = rx_pool_size / xpcie->fragment_size;
+
+ /* Initialize reserved memory resources */
+ rc = of_reserved_mem_device_init(dma_dev);
+ if (rc) {
+ dev_err(dma_dev, "Could not get reserved memory\n");
+ goto error;
+ }
+
+ if (rx_pool_coherent) {
+ xpcie_epf->rx_size = rx_pool_size;
+ xpcie_epf->rx_virt = dma_alloc_coherent(dma_dev,
+ xpcie_epf->rx_size,
+ &xpcie_epf->rx_phys,
+ GFP_KERNEL);
+ if (!xpcie_epf->rx_virt)
+ goto error;
+ }
+
+ for (index = 0; index < ndesc; index++) {
+ if (rx_pool_coherent) {
+ bd =
+ intel_xpcie_alloc_bd_reuse(xpcie->fragment_size,
+ xpcie_epf->rx_virt +
+ (index *
+ xpcie->fragment_size),
+ xpcie_epf->rx_phys +
+ (index *
+ xpcie->fragment_size));
+ } else {
+ bd = intel_xpcie_alloc_bd(xpcie->fragment_size);
+ }
+ if (bd) {
+ intel_xpcie_list_put(&xpcie->rx_pool, bd);
+ } else {
+ dev_err(xpcie_to_dev(xpcie),
+ "failed to alloc all rx pool descriptors\n");
+ goto error;
+ }
+ }
+
+ intel_xpcie_list_init(&xpcie->tx_pool);
+ tx_pool_size = roundup(tx_pool_size, xpcie->fragment_size);
+ ndesc = tx_pool_size / xpcie->fragment_size;
+
+ if (tx_pool_coherent) {
+ xpcie_epf->tx_size = tx_pool_size;
+ xpcie_epf->tx_virt = dma_alloc_coherent(dma_dev,
+ xpcie_epf->tx_size,
+ &xpcie_epf->tx_phys,
+ GFP_KERNEL);
+ if (!xpcie_epf->tx_virt)
+ goto error;
+ }
+
+ for (index = 0; index < ndesc; index++) {
+ if (tx_pool_coherent) {
+ bd =
+ intel_xpcie_alloc_bd_reuse(xpcie->fragment_size,
+ xpcie_epf->tx_virt +
+ (index *
+ xpcie->fragment_size),
+ xpcie_epf->tx_phys +
+ (index *
+ xpcie->fragment_size));
+ } else {
+ bd = intel_xpcie_alloc_bd(xpcie->fragment_size);
+ }
+ if (bd) {
+ intel_xpcie_list_put(&xpcie->tx_pool, bd);
+ } else {
+ dev_err(xpcie_to_dev(xpcie),
+ "failed to alloc all tx pool descriptors\n");
+ goto error;
+ }
+ }
+
+ return 0;
+
+error:
+ intel_xpcie_txrx_cleanup(xpcie);
+
+ return -ENOMEM;
+}
+
+static int intel_xpcie_discover_txrx(struct xpcie *xpcie)
+{
+ struct xpcie_cap_txrx *cap;
+ int error;
+
+ cap = intel_xpcie_cap_find(xpcie, 0, XPCIE_CAP_TXRX);
+ if (cap) {
+ error = intel_xpcie_txrx_init(xpcie, cap);
+ } else {
+ dev_err(xpcie_to_dev(xpcie), "xpcie txrx info not found\n");
+ error = -EIO;
+ }
+
+ return error;
+}
+
+static void intel_xpcie_start_tx(struct xpcie *xpcie, unsigned long delay)
+{
+ /*
+ * Use only one WQ for both Rx and Tx
+ *
+ * Synchronous Read and Writes to DDR is found to result in memory
+ * mismatch errors in stability tests due to silicon bug in A0 SoC.
+ */
+ if (xpcie->legacy_a0)
+ queue_delayed_work(xpcie->rx_wq, &xpcie->tx_event, delay);
+ else
+ queue_delayed_work(xpcie->tx_wq, &xpcie->tx_event, delay);
+}
+
+static void intel_xpcie_start_rx(struct xpcie *xpcie, unsigned long delay)
+{
+ queue_delayed_work(xpcie->rx_wq, &xpcie->rx_event, delay);
+}
+
+static void intel_xpcie_rx_event_handler(struct work_struct *work)
+{
+ struct xpcie *xpcie = container_of(work, struct xpcie, rx_event.work);
+ struct xpcie_epf *xpcie_epf = container_of(xpcie,
+ struct xpcie_epf, xpcie);
+ struct xpcie_buf_desc *bd_head, *bd_tail, *bd;
+ u32 head, tail, ndesc, length, initial_head;
+ unsigned long delay = msecs_to_jiffies(1);
+ struct xpcie_stream *rx = &xpcie->rx;
+ int descs_num = 0, chan = 0, rc;
+ struct xpcie_dma_ll_desc *desc;
+ struct xpcie_transfer_desc *td;
+ bool reset_work = false;
+ u16 interface;
+ u64 address;
+
+ if (intel_xpcie_get_host_status(xpcie) != XPCIE_STATUS_RUN)
+ return;
+
+ bd_head = NULL;
+ bd_tail = NULL;
+ ndesc = rx->pipe.ndesc;
+ tail = intel_xpcie_get_tdr_tail(&rx->pipe);
+ initial_head = intel_xpcie_get_tdr_head(&rx->pipe);
+ head = initial_head;
+
+ while (head != tail) {
+ td = rx->pipe.tdr + head;
+
+ bd = intel_xpcie_alloc_rx_bd(xpcie);
+ if (!bd) {
+ reset_work = true;
+ if (descs_num == 0) {
+ delay = msecs_to_jiffies(10);
+ goto task_exit;
+ }
+ break;
+ }
+
+ interface = intel_xpcie_get_td_interface(td);
+ length = intel_xpcie_get_td_length(td);
+ address = intel_xpcie_get_td_address(td);
+
+ bd->length = length;
+ bd->interface = interface;
+ if (!rx_pool_coherent) {
+ rc = intel_xpcie_map_dma(xpcie, bd, DMA_FROM_DEVICE);
+ if (rc) {
+ dev_err(xpcie_to_dev(xpcie),
+ "failed to map rx bd (%d)\n", rc);
+ intel_xpcie_free_rx_bd(xpcie, bd);
+ break;
+ }
+ }
+
+ desc = &xpcie_epf->rx_desc_buf[chan].virt[descs_num++];
+ desc->dma_transfer_size = length;
+ desc->dst_addr = bd->phys;
+ desc->src_addr = address;
+
+ if (bd_head)
+ bd_tail->next = bd;
+ else
+ bd_head = bd;
+ bd_tail = bd;
+
+ head = XPCIE_CIRCULAR_INC(head, ndesc);
+ }
+
+ if (descs_num == 0)
+ goto task_exit;
+
+ rc = intel_xpcie_copy_from_host_ll(xpcie, chan, descs_num);
+
+ bd = bd_head;
+ while (bd && !rx_pool_coherent) {
+ intel_xpcie_unmap_dma(xpcie, bd, DMA_FROM_DEVICE);
+ bd = bd->next;
+ }
+
+ if (rc) {
+ dev_err(xpcie_to_dev(xpcie),
+ "failed to DMA from host (%d)\n", rc);
+ intel_xpcie_free_rx_bd(xpcie, bd_head);
+ delay = msecs_to_jiffies(5);
+ reset_work = true;
+ goto task_exit;
+ }
+
+ head = initial_head;
+ bd = bd_head;
+ while (bd) {
+ td = rx->pipe.tdr + head;
+ bd_head = bd_head->next;
+ bd->next = NULL;
+
+ if (likely(bd->interface < XPCIE_NUM_INTERFACES)) {
+ intel_xpcie_set_td_status(td,
+ XPCIE_DESC_STATUS_SUCCESS);
+ intel_xpcie_add_bd_to_interface(xpcie, bd);
+ } else {
+ dev_err(xpcie_to_dev(xpcie),
+ "detected rx desc interface failure (%u)\n",
+ bd->interface);
+ intel_xpcie_set_td_status(td, XPCIE_DESC_STATUS_ERROR);
+ intel_xpcie_free_rx_bd(xpcie, bd);
+ }
+
+ bd = bd_head;
+ head = XPCIE_CIRCULAR_INC(head, ndesc);
+ }
+
+ if (head != initial_head) {
+ intel_xpcie_set_tdr_head(&rx->pipe, head);
+ intel_xpcie_raise_irq(xpcie, DATA_RECEIVED);
+ }
+
+task_exit:
+ if (reset_work)
+ intel_xpcie_start_rx(xpcie, delay);
+}
+
+static void intel_xpcie_tx_event_handler(struct work_struct *work)
+{
+ struct xpcie *xpcie = container_of(work, struct xpcie, tx_event.work);
+ struct xpcie_epf *xpcie_epf = container_of(xpcie,
+ struct xpcie_epf, xpcie);
+ struct xpcie_buf_desc *bd_head, *bd_tail, *bd;
+ struct xpcie_stream *tx = &xpcie->tx;
+ u32 head, tail, ndesc, initial_tail;
+ struct xpcie_dma_ll_desc *desc;
+ struct xpcie_transfer_desc *td;
+ int descs_num = 0, chan = 0, rc;
+ size_t buffers = 0, bytes = 0;
+ u64 address;
+
+ if (intel_xpcie_get_host_status(xpcie) != XPCIE_STATUS_RUN)
+ return;
+
+ bd_head = NULL;
+ bd_tail = NULL;
+ ndesc = tx->pipe.ndesc;
+ initial_tail = intel_xpcie_get_tdr_tail(&tx->pipe);
+ tail = initial_tail;
+ head = intel_xpcie_get_tdr_head(&tx->pipe);
+
+ /* add new entries */
+ while (XPCIE_CIRCULAR_INC(tail, ndesc) != head) {
+ bd = intel_xpcie_list_get(&xpcie->write);
+ if (!bd)
+ break;
+
+ if (!tx_pool_coherent) {
+ if (intel_xpcie_map_dma(xpcie, bd, DMA_TO_DEVICE)) {
+ dev_err(xpcie_to_dev(xpcie),
+ "dma map error bd addr %p, size %zu\n",
+ bd->data, bd->length);
+ intel_xpcie_list_put_head(&xpcie->write, bd);
+ break;
+ }
+ }
+
+ td = tx->pipe.tdr + tail;
+ address = intel_xpcie_get_td_address(td);
+
+ desc = &xpcie_epf->tx_desc_buf[chan].virt[descs_num++];
+ desc->dma_transfer_size = bd->length;
+ desc->src_addr = bd->phys;
+ desc->dst_addr = address;
+
+ if (bd_head)
+ bd_tail->next = bd;
+ else
+ bd_head = bd;
+ bd_tail = bd;
+
+ tail = XPCIE_CIRCULAR_INC(tail, ndesc);
+ }
+
+ if (descs_num == 0)
+ goto task_exit;
+
+ rc = intel_xpcie_copy_to_host_ll(xpcie, chan, descs_num);
+
+ tail = initial_tail;
+ bd = bd_head;
+ while (bd) {
+ if (!tx_pool_coherent)
+ intel_xpcie_unmap_dma(xpcie, bd, DMA_TO_DEVICE);
+
+ if (rc) {
+ bd = bd->next;
+ continue;
+ }
+
+ td = tx->pipe.tdr + tail;
+ intel_xpcie_set_td_status(td, XPCIE_DESC_STATUS_SUCCESS);
+ intel_xpcie_set_td_length(td, bd->length);
+ intel_xpcie_set_td_interface(td, bd->interface);
+
+ bd = bd->next;
+ tail = XPCIE_CIRCULAR_INC(tail, ndesc);
+ }
+
+ if (rc) {
+ dev_err(xpcie_to_dev(xpcie),
+ "failed to DMA to host (%d)\n", rc);
+ intel_xpcie_list_put_head(&xpcie->write, bd_head);
+ return;
+ }
+
+ intel_xpcie_free_tx_bd(xpcie, bd_head);
+
+ if (intel_xpcie_get_tdr_tail(&tx->pipe) != tail) {
+ intel_xpcie_set_tdr_tail(&tx->pipe, tail);
+ intel_xpcie_raise_irq(xpcie, DATA_SENT);
+ }
+
+task_exit:
+ intel_xpcie_list_info(&xpcie->write, &bytes, &buffers);
+ if (buffers) {
+ xpcie->tx_pending = true;
+ head = intel_xpcie_get_tdr_head(&tx->pipe);
+ if (XPCIE_CIRCULAR_INC(tail, ndesc) != head)
+ intel_xpcie_start_tx(xpcie, 0);
+ } else {
+ xpcie->tx_pending = false;
+ }
+}
+
+static irqreturn_t intel_xpcie_core_irq_cb(int irq, void *args)
+{
+ struct xpcie *xpcie = args;
+
+ if (intel_xpcie_get_doorbell(xpcie, TO_DEVICE, DATA_SENT)) {
+ intel_xpcie_set_doorbell(xpcie, TO_DEVICE, DATA_SENT, 0);
+ intel_xpcie_start_rx(xpcie, 0);
+ }
+ if (intel_xpcie_get_doorbell(xpcie, TO_DEVICE, DATA_RECEIVED)) {
+ intel_xpcie_set_doorbell(xpcie, TO_DEVICE, DATA_RECEIVED, 0);
+ if (xpcie->tx_pending)
+ intel_xpcie_start_tx(xpcie, 0);
+ }
+
+ return IRQ_HANDLED;
+}
+
+static int intel_xpcie_events_init(struct xpcie *xpcie)
+{
+ xpcie->rx_wq = alloc_ordered_workqueue(XPCIE_DRIVER_NAME,
+ WQ_MEM_RECLAIM | WQ_HIGHPRI);
+ if (!xpcie->rx_wq) {
+ dev_err(xpcie_to_dev(xpcie), "failed to allocate workqueue\n");
+ return -ENOMEM;
+ }
+
+ if (!xpcie->legacy_a0) {
+ xpcie->tx_wq = alloc_ordered_workqueue(XPCIE_DRIVER_NAME,
+ WQ_MEM_RECLAIM |
+ WQ_HIGHPRI);
+ if (!xpcie->tx_wq) {
+ dev_err(xpcie_to_dev(xpcie),
+ "failed to allocate workqueue\n");
+ destroy_workqueue(xpcie->rx_wq);
+ return -ENOMEM;
+ }
+ }
+
+ INIT_DELAYED_WORK(&xpcie->rx_event, intel_xpcie_rx_event_handler);
+ INIT_DELAYED_WORK(&xpcie->tx_event, intel_xpcie_tx_event_handler);
+
+ return 0;
+}
+
+static void intel_xpcie_events_cleanup(struct xpcie *xpcie)
+{
+ cancel_delayed_work_sync(&xpcie->rx_event);
+ cancel_delayed_work_sync(&xpcie->tx_event);
+
+ destroy_workqueue(xpcie->rx_wq);
+ if (!xpcie->legacy_a0)
+ destroy_workqueue(xpcie->tx_wq);
+}
+
+int intel_xpcie_core_init(struct xpcie *xpcie)
+{
+ int error;
+
+ global_xpcie = xpcie;
+
+ intel_xpcie_set_version(xpcie);
+ intel_xpcie_set_cap_txrx(xpcie);
+
+ error = intel_xpcie_events_init(xpcie);
+ if (error)
+ return error;
+
+ error = intel_xpcie_discover_txrx(xpcie);
+ if (error)
+ goto error_txrx;
+
+ intel_xpcie_interfaces_init(xpcie);
+
+ intel_xpcie_set_doorbell(xpcie, TO_DEVICE, DATA_SENT, 0);
+ intel_xpcie_set_doorbell(xpcie, TO_DEVICE, DATA_RECEIVED, 0);
+ intel_xpcie_set_doorbell(xpcie, TO_DEVICE, DEV_EVENT, NO_OP);
+ intel_xpcie_set_doorbell(xpcie, FROM_DEVICE, DATA_SENT, 0);
+ intel_xpcie_set_doorbell(xpcie, FROM_DEVICE, DATA_RECEIVED, 0);
+ intel_xpcie_set_doorbell(xpcie, FROM_DEVICE, DEV_EVENT, NO_OP);
+
+ intel_xpcie_register_host_irq(xpcie, intel_xpcie_core_irq_cb);
+
+ return 0;
+
+error_txrx:
+ intel_xpcie_events_cleanup(xpcie);
+
+ return error;
+}
+
+void intel_xpcie_core_cleanup(struct xpcie *xpcie)
+{
+ if (xpcie->status == XPCIE_STATUS_RUN) {
+ intel_xpcie_events_cleanup(xpcie);
+ intel_xpcie_interfaces_cleanup(xpcie);
+ intel_xpcie_txrx_cleanup(xpcie);
+ }
+}
+
+int intel_xpcie_core_read(struct xpcie *xpcie, void *buffer,
+ size_t *length, u32 timeout_ms)
+{
+ long jiffies_timeout = (long)msecs_to_jiffies(timeout_ms);
+ struct xpcie_interface *inf = &xpcie->interfaces[0];
+ unsigned long jiffies_start = jiffies;
+ struct xpcie_buf_desc *bd;
+ long jiffies_passed = 0;
+ size_t len, remaining;
+ int ret;
+
+ if (*length == 0)
+ return -EINVAL;
+
+ if (xpcie->status != XPCIE_STATUS_RUN)
+ return -ENODEV;
+
+ len = *length;
+ remaining = len;
+ *length = 0;
+
+ ret = mutex_lock_interruptible(&inf->rlock);
+ if (ret < 0)
+ return -EINTR;
+
+ do {
+ while (!inf->data_avail) {
+ mutex_unlock(&inf->rlock);
+ if (timeout_ms == 0) {
+ ret =
+ wait_event_interruptible(inf->rx_waitq,
+ inf->data_avail);
+ } else {
+ ret =
+ wait_event_interruptible_timeout(inf->rx_waitq,
+ inf->data_avail,
+ jiffies_timeout -
+ jiffies_passed);
+ if (ret == 0)
+ return -ETIME;
+ }
+ if (ret < 0 || xpcie->stop_flag)
+ return -EINTR;
+
+ ret = mutex_lock_interruptible(&inf->rlock);
+ if (ret < 0)
+ return -EINTR;
+ }
+
+ bd = (inf->partial_read) ? inf->partial_read :
+ intel_xpcie_list_get(&inf->read);
+
+ while (remaining && bd) {
+ size_t bcopy;
+
+ bcopy = min(remaining, bd->length);
+ memcpy(buffer, bd->data, bcopy);
+
+ buffer += bcopy;
+ remaining -= bcopy;
+ bd->data += bcopy;
+ bd->length -= bcopy;
+
+ if (bd->length == 0) {
+ intel_xpcie_free_rx_bd(xpcie, bd);
+ bd = intel_xpcie_list_get(&inf->read);
+ }
+ }
+
+ /* save for next time */
+ inf->partial_read = bd;
+
+ if (!bd)
+ inf->data_avail = false;
+
+ *length = len - remaining;
+
+ jiffies_passed = (long)jiffies - (long)jiffies_start;
+ } while (remaining > 0 && (jiffies_passed < jiffies_timeout ||
+ timeout_ms == 0));
+
+ mutex_unlock(&inf->rlock);
+
+ return 0;
+}
+
+int intel_xpcie_core_write(struct xpcie *xpcie, void *buffer,
+ size_t *length, u32 timeout_ms)
+{
+ long jiffies_timeout = (long)msecs_to_jiffies(timeout_ms);
+ struct xpcie_interface *inf = &xpcie->interfaces[0];
+ unsigned long jiffies_start = jiffies;
+ struct xpcie_buf_desc *bd, *head;
+ long jiffies_passed = 0;
+ size_t remaining, len;
+ int ret;
+
+ if (*length == 0)
+ return -EINVAL;
+
+ if (xpcie->status != XPCIE_STATUS_RUN)
+ return -ENODEV;
+
+ if (intel_xpcie_get_host_status(xpcie) != XPCIE_STATUS_RUN)
+ return -ENODEV;
+
+ len = *length;
+ remaining = len;
+ *length = 0;
+
+ ret = mutex_lock_interruptible(&xpcie->wlock);
+ if (ret < 0)
+ return -EINTR;
+
+ do {
+ bd = intel_xpcie_alloc_tx_bd(xpcie);
+ head = bd;
+ while (!head) {
+ mutex_unlock(&xpcie->wlock);
+ if (timeout_ms == 0) {
+ ret =
+ wait_event_interruptible(xpcie->tx_waitq,
+ !xpcie->no_tx_buffer);
+ } else {
+ ret =
+ wait_event_interruptible_timeout(xpcie->tx_waitq,
+ !xpcie->no_tx_buffer,
+ jiffies_timeout -
+ jiffies_passed);
+ if (ret == 0)
+ return -ETIME;
+ }
+ if (ret < 0 || xpcie->stop_flag)
+ return -EINTR;
+
+ ret = mutex_lock_interruptible(&xpcie->wlock);
+ if (ret < 0)
+ return -EINTR;
+
+ bd = intel_xpcie_alloc_tx_bd(xpcie);
+ head = bd;
+ }
+
+ while (remaining && bd) {
+ size_t bcopy;
+
+ bcopy = min(bd->length, remaining);
+ memcpy(bd->data, buffer, bcopy);
+
+ buffer += bcopy;
+ remaining -= bcopy;
+ bd->length = bcopy;
+ bd->interface = inf->id;
+
+ if (remaining) {
+ bd->next = intel_xpcie_alloc_tx_bd(xpcie);
+ bd = bd->next;
+ }
+ }
+
+ intel_xpcie_list_put(&inf->xpcie->write, head);
+ intel_xpcie_start_tx(xpcie, 0);
+
+ *length = len - remaining;
+
+ jiffies_passed = (long)jiffies - (long)jiffies_start;
+ } while (remaining > 0 && (jiffies_passed < jiffies_timeout ||
+ timeout_ms == 0));
+
+ mutex_unlock(&xpcie->wlock);
+
+ return 0;
+}
+
+int intel_xpcie_get_device_status_by_id(u32 id, u32 *status)
+{
+ struct xpcie *xpcie = intel_xpcie_core_get_by_id(id);
+
+ if (!xpcie)
+ return -ENODEV;
+
+ *status = xpcie->status;
+
+ return 0;
+}
+
+u32 intel_xpcie_get_device_num(u32 *id_list)
+{
+ u32 num_devices = 0;
+
+ if (xlink_sw_id) {
+ num_devices = 1;
+ *id_list = xlink_sw_id;
+ }
+
+ return num_devices;
+}
+
+int intel_xpcie_get_device_name_by_id(u32 id,
+ char *device_name, size_t name_size)
+{
+ struct xpcie *xpcie;
+
+ xpcie = intel_xpcie_core_get_by_id(id);
+ if (!xpcie)
+ return -ENODEV;
+
+ memset(device_name, 0, name_size);
+ if (name_size > strlen(XPCIE_DRIVER_NAME))
+ name_size = strlen(XPCIE_DRIVER_NAME);
+ memcpy(device_name, XPCIE_DRIVER_NAME, name_size);
+
+ return 0;
+}
+
+int intel_xpcie_pci_connect_device(u32 id)
+{
+ struct xpcie *xpcie;
+
+ xpcie = intel_xpcie_core_get_by_id(id);
+ if (!xpcie)
+ return -ENODEV;
+
+ if (xpcie->status != XPCIE_STATUS_RUN)
+ return -EIO;
+
+ return 0;
+}
+
+int intel_xpcie_pci_read(u32 id, void *data, size_t *size, u32 timeout)
+{
+ struct xpcie *xpcie;
+
+ xpcie = intel_xpcie_core_get_by_id(id);
+ if (!xpcie)
+ return -ENODEV;
+
+ return intel_xpcie_core_read(xpcie, data, size, timeout);
+}
+
+int intel_xpcie_pci_write(u32 id, void *data, size_t *size, u32 timeout)
+{
+ struct xpcie *xpcie;
+
+ xpcie = intel_xpcie_core_get_by_id(id);
+ if (!xpcie)
+ return -ENODEV;
+
+ return intel_xpcie_core_write(xpcie, data, size, timeout);
+}
+
+int intel_xpcie_pci_reset_device(u32 id)
+{
+ return 0;
+}
diff --git a/drivers/misc/xlink-pcie/local_host/core.h b/drivers/misc/xlink-pcie/local_host/core.h
new file mode 100644
index 000000000000..84985ef41a64
--- /dev/null
+++ b/drivers/misc/xlink-pcie/local_host/core.h
@@ -0,0 +1,247 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*****************************************************************************
+ *
+ * Intel Keem Bay XLink PCIe Driver
+ *
+ * Copyright (C) 2020 Intel Corporation
+ *
+ ****************************************************************************/
+
+#ifndef XPCIE_CORE_HEADER_
+#define XPCIE_CORE_HEADER_
+
+#include <linux/io.h>
+#include <linux/types.h>
+#include <linux/workqueue.h>
+#include <linux/slab.h>
+#include <linux/mutex.h>
+#include <linux/mempool.h>
+#include <linux/dma-mapping.h>
+#include <linux/cache.h>
+#include <linux/wait.h>
+
+#include <linux/xlink_drv_inf.h>
+
+/* Number of interfaces to statically allocate resources for */
+#define XPCIE_NUM_INTERFACES (1)
+
+/* max should be always power of '2' */
+#define XPCIE_CIRCULAR_INC(val, max) (((val) + 1) & ((max) - 1))
+
+#define XPCIE_FRAGMENT_SIZE SZ_128K
+
+/* Status encoding of the transfer descriptors */
+#define XPCIE_DESC_STATUS_SUCCESS (0)
+#define XPCIE_DESC_STATUS_ERROR (0xFFFF)
+
+/* Layout transfer descriptors used by device and host */
+struct xpcie_transfer_desc {
+ u64 address;
+ u32 length;
+ u16 status;
+ u16 interface;
+} __packed;
+
+struct xpcie_pipe {
+ u32 old;
+ u32 ndesc;
+ u32 *head;
+ u32 *tail;
+ struct xpcie_transfer_desc *tdr;
+};
+
+struct xpcie_buf_desc {
+ struct xpcie_buf_desc *next;
+ void *head;
+ dma_addr_t phys;
+ size_t true_len;
+ void *data;
+ size_t length;
+ int interface;
+ bool own_mem;
+};
+
+struct xpcie_stream {
+ size_t frag;
+ struct xpcie_pipe pipe;
+};
+
+struct xpcie_list {
+ spinlock_t lock; /* list lock */
+ size_t bytes;
+ size_t buffers;
+ struct xpcie_buf_desc *head;
+ struct xpcie_buf_desc *tail;
+};
+
+struct xpcie_interface {
+ int id;
+ struct xpcie *xpcie;
+ struct mutex rlock; /* read lock */
+ struct xpcie_list read;
+ struct xpcie_buf_desc *partial_read;
+ bool data_avail;
+ wait_queue_head_t rx_waitq;
+};
+
+struct xpcie_debug_stats {
+ struct {
+ size_t cnts;
+ size_t bytes;
+ } tx_krn, rx_krn, tx_usr, rx_usr;
+ size_t send_ints;
+ size_t interrupts;
+ size_t rx_event_runs;
+ size_t tx_event_runs;
+};
+
+/* Defined capabilities located in mmio space */
+#define XPCIE_CAP_NULL (0)
+#define XPCIE_CAP_TXRX (1)
+
+#define XPCIE_CAP_TTL (32)
+#define XPCIE_CAP_HDR_ID (offsetof(struct xpcie_cap_hdr, id))
+#define XPCIE_CAP_HDR_NEXT (offsetof(struct xpcie_cap_hdr, next))
+
+/* Header at the beginning of each capability to define and link to next */
+struct xpcie_cap_hdr {
+ u16 id;
+ u16 next;
+} __packed;
+
+struct xpcie_cap_pipe {
+ u32 ring;
+ u32 ndesc;
+ u32 head;
+ u32 tail;
+} __packed;
+
+/* Transmit and Receive capability */
+struct xpcie_cap_txrx {
+ struct xpcie_cap_hdr hdr;
+ u32 fragment_size;
+ struct xpcie_cap_pipe tx;
+ struct xpcie_cap_pipe rx;
+} __packed;
+
+static inline u64 _ioread64(void __iomem *addr)
+{
+ u64 low, high;
+
+ low = ioread32(addr);
+ high = ioread32(addr + sizeof(u32));
+
+ return low | (high << 32);
+}
+
+static inline void _iowrite64(u64 value, void __iomem *addr)
+{
+ iowrite32(value, addr);
+ iowrite32(value >> 32, addr + sizeof(u32));
+}
+
+#define intel_xpcie_iowrite64(value, addr) \
+ _iowrite64(value, (void __iomem *)addr)
+#define intel_xpcie_iowrite32(value, addr) \
+ iowrite32(value, (void __iomem *)addr)
+#define intel_xpcie_iowrite16(value, addr) \
+ iowrite16(value, (void __iomem *)addr)
+#define intel_xpcie_iowrite8(value, addr) \
+ iowrite8(value, (void __iomem *)addr)
+#define intel_xpcie_ioread64(addr) \
+ _ioread64((void __iomem *)addr)
+#define intel_xpcie_ioread32(addr) \
+ ioread32((void __iomem *)addr)
+#define intel_xpcie_ioread16(addr) \
+ ioread16((void __iomem *)addr)
+#define intel_xpcie_ioread8(addr) \
+ ioread8((void __iomem *)addr)
+
+static inline
+void intel_xpcie_set_td_address(struct xpcie_transfer_desc *td, u64 address)
+{
+ intel_xpcie_iowrite64(address, &td->address);
+}
+
+static inline
+u64 intel_xpcie_get_td_address(struct xpcie_transfer_desc *td)
+{
+ return intel_xpcie_ioread64(&td->address);
+}
+
+static inline
+void intel_xpcie_set_td_length(struct xpcie_transfer_desc *td, u32 length)
+{
+ intel_xpcie_iowrite32(length, &td->length);
+}
+
+static inline
+u32 intel_xpcie_get_td_length(struct xpcie_transfer_desc *td)
+{
+ return intel_xpcie_ioread32(&td->length);
+}
+
+static inline
+void intel_xpcie_set_td_interface(struct xpcie_transfer_desc *td, u16 interface)
+{
+ intel_xpcie_iowrite16(interface, &td->interface);
+}
+
+static inline
+u16 intel_xpcie_get_td_interface(struct xpcie_transfer_desc *td)
+{
+ return intel_xpcie_ioread16(&td->interface);
+}
+
+static inline
+void intel_xpcie_set_td_status(struct xpcie_transfer_desc *td, u16 status)
+{
+ intel_xpcie_iowrite16(status, &td->status);
+}
+
+static inline
+u16 intel_xpcie_get_td_status(struct xpcie_transfer_desc *td)
+{
+ return intel_xpcie_ioread16(&td->status);
+}
+
+static inline
+void intel_xpcie_set_tdr_head(struct xpcie_pipe *p, u32 head)
+{
+ intel_xpcie_iowrite32(head, p->head);
+}
+
+static inline
+u32 intel_xpcie_get_tdr_head(struct xpcie_pipe *p)
+{
+ return intel_xpcie_ioread32(p->head);
+}
+
+static inline
+void intel_xpcie_set_tdr_tail(struct xpcie_pipe *p, u32 tail)
+{
+ intel_xpcie_iowrite32(tail, p->tail);
+}
+
+static inline
+u32 intel_xpcie_get_tdr_tail(struct xpcie_pipe *p)
+{
+ return intel_xpcie_ioread32(p->tail);
+}
+
+int intel_xpcie_core_init(struct xpcie *xpcie);
+void intel_xpcie_core_cleanup(struct xpcie *xpcie);
+int intel_xpcie_core_read(struct xpcie *xpcie, void *buffer, size_t *length,
+ u32 timeout_ms);
+int intel_xpcie_core_write(struct xpcie *xpcie, void *buffer, size_t *length,
+ u32 timeout_ms);
+u32 intel_xpcie_get_device_num(u32 *id_list);
+struct xpcie_dev *intel_xpcie_get_device_by_id(u32 id);
+int intel_xpcie_get_device_name_by_id(u32 id, char *device_name,
+ size_t name_size);
+int intel_xpcie_get_device_status_by_id(u32 id, u32 *status);
+int intel_xpcie_pci_connect_device(u32 id);
+int intel_xpcie_pci_read(u32 id, void *data, size_t *size, u32 timeout);
+int intel_xpcie_pci_write(u32 id, void *data, size_t *size, u32 timeout);
+int intel_xpcie_pci_reset_device(u32 id);
+#endif /* XPCIE_CORE_HEADER_ */
diff --git a/drivers/misc/xlink-pcie/local_host/epf.c b/drivers/misc/xlink-pcie/local_host/epf.c
index fc29180aa508..f334d4749cce 100644
--- a/drivers/misc/xlink-pcie/local_host/epf.c
+++ b/drivers/misc/xlink-pcie/local_host/epf.c
@@ -9,6 +9,7 @@

#include <linux/of.h>
#include <linux/platform_device.h>
+#include <linux/reboot.h>

#include "epf.h"

@@ -21,6 +22,12 @@
#define PCIE_REGS_PCIE_ERR_INTR_FLAGS 0x24
#define LINK_REQ_RST_FLG BIT(15)

+#define PCIE_REGS_PCIE_SYS_CFG_CORE 0x7C
+#define PCIE_CFG_PBUS_NUM_OFFSET 8
+#define PCIE_CFG_PBUS_NUM_MASK 0xFF
+#define PCIE_CFG_PBUS_DEV_NUM_OFFSET 16
+#define PCIE_CFG_PBUS_DEV_NUM_MASK 0x1F
+
static struct pci_epf_header xpcie_header = {
.vendorid = PCI_VENDOR_ID_INTEL,
.deviceid = PCI_DEVICE_ID_INTEL_KEEMBAY,
@@ -37,6 +44,45 @@ static const struct pci_epf_device_id xpcie_epf_ids[] = {
{},
};

+u32 xlink_sw_id;
+
+int intel_xpcie_copy_from_host_ll(struct xpcie *xpcie, int chan, int descs_num)
+{
+ struct xpcie_epf *xpcie_epf = container_of(xpcie,
+ struct xpcie_epf, xpcie);
+ struct pci_epf *epf = xpcie_epf->epf;
+
+ return intel_xpcie_ep_dma_read_ll(epf, chan, descs_num);
+}
+
+int intel_xpcie_copy_to_host_ll(struct xpcie *xpcie, int chan, int descs_num)
+{
+ struct xpcie_epf *xpcie_epf = container_of(xpcie,
+ struct xpcie_epf, xpcie);
+ struct pci_epf *epf = xpcie_epf->epf;
+
+ return intel_xpcie_ep_dma_write_ll(epf, chan, descs_num);
+}
+
+void intel_xpcie_register_host_irq(struct xpcie *xpcie, irq_handler_t func)
+{
+ struct xpcie_epf *xpcie_epf = container_of(xpcie,
+ struct xpcie_epf, xpcie);
+
+ xpcie_epf->core_irq_callback = func;
+}
+
+int intel_xpcie_raise_irq(struct xpcie *xpcie, enum xpcie_doorbell_type type)
+{
+ struct xpcie_epf *xpcie_epf = container_of(xpcie,
+ struct xpcie_epf, xpcie);
+ struct pci_epf *epf = xpcie_epf->epf;
+
+ intel_xpcie_set_doorbell(xpcie, FROM_DEVICE, type, 1);
+
+ return pci_epc_raise_irq(epf->epc, epf->func_no, PCI_EPC_IRQ_MSI, 1);
+}
+
static irqreturn_t intel_xpcie_err_interrupt(int irq, void *args)
{
struct xpcie_epf *xpcie_epf;
@@ -57,6 +103,7 @@ static irqreturn_t intel_xpcie_host_interrupt(int irq, void *args)
{
struct xpcie_epf *xpcie_epf;
struct xpcie *xpcie = args;
+ u8 event;
u32 val;

xpcie_epf = container_of(xpcie, struct xpcie_epf, xpcie);
@@ -64,6 +111,18 @@ static irqreturn_t intel_xpcie_host_interrupt(int irq, void *args)
if (val & LBC_CII_EVENT_FLAG) {
iowrite32(LBC_CII_EVENT_FLAG,
xpcie_epf->apb_base + PCIE_REGS_PCIE_INTR_FLAGS);
+
+ event = intel_xpcie_get_doorbell(xpcie, TO_DEVICE, DEV_EVENT);
+ if (unlikely(event != NO_OP)) {
+ intel_xpcie_set_doorbell(xpcie, TO_DEVICE,
+ DEV_EVENT, NO_OP);
+ if (event == REQUEST_RESET)
+ orderly_reboot();
+ return IRQ_HANDLED;
+ }
+
+ if (likely(xpcie_epf->core_irq_callback))
+ xpcie_epf->core_irq_callback(irq, xpcie);
}

return IRQ_HANDLED;
@@ -269,6 +328,7 @@ static int intel_xpcie_epf_bind(struct pci_epf *epf)
struct xpcie_epf *xpcie_epf = epf_get_drvdata(epf);
const struct pci_epc_features *features;
struct pci_epc *epc = epf->epc;
+ u32 bus_num, dev_num;
struct device *dev;
size_t align = SZ_16K;
int ret;
@@ -300,12 +360,12 @@ static int intel_xpcie_epf_bind(struct pci_epf *epf)

if (!strcmp(xpcie_epf->stepping, "A0")) {
xpcie_epf->xpcie.legacy_a0 = true;
- iowrite32(1, (void __iomem *)xpcie_epf->xpcie.mmio +
- XPCIE_MMIO_LEGACY_A0);
+ intel_xpcie_iowrite32(1, xpcie_epf->xpcie.mmio +
+ XPCIE_MMIO_LEGACY_A0);
} else {
xpcie_epf->xpcie.legacy_a0 = false;
- iowrite32(0, (void __iomem *)xpcie_epf->xpcie.mmio +
- XPCIE_MMIO_LEGACY_A0);
+ intel_xpcie_iowrite32(0, xpcie_epf->xpcie.mmio +
+ XPCIE_MMIO_LEGACY_A0);
}

/* Enable interrupt */
@@ -330,13 +390,46 @@ static int intel_xpcie_epf_bind(struct pci_epf *epf)
ret = intel_xpcie_ep_dma_init(epf);
if (ret) {
dev_err(&epf->dev, "DMA initialization failed\n");
- goto err_free_err_irq;
+ goto err_cleanup_bars;
}

+ intel_xpcie_set_device_status(&xpcie_epf->xpcie, XPCIE_STATUS_READY);
+
+ ret = ioread32(xpcie_epf->apb_base + PCIE_REGS_PCIE_SYS_CFG_CORE);
+ bus_num = (ret >> PCIE_CFG_PBUS_NUM_OFFSET) & PCIE_CFG_PBUS_NUM_MASK;
+ dev_num = (ret >> PCIE_CFG_PBUS_DEV_NUM_OFFSET) &
+ PCIE_CFG_PBUS_DEV_NUM_MASK;
+
+ xlink_sw_id = FIELD_PREP(XLINK_DEV_INF_TYPE_MASK,
+ XLINK_DEV_INF_PCIE) |
+ FIELD_PREP(XLINK_DEV_PHYS_ID_MASK,
+ bus_num << 8 | dev_num) |
+ FIELD_PREP(XLINK_DEV_TYPE_MASK, XLINK_DEV_TYPE_KMB) |
+ FIELD_PREP(XLINK_DEV_PCIE_ID_MASK, XLINK_DEV_PCIE_0) |
+ FIELD_PREP(XLINK_DEV_FUNC_MASK, XLINK_DEV_FUNC_VPU);
+
+ ret = intel_xpcie_core_init(&xpcie_epf->xpcie);
+ if (ret) {
+ dev_err(&epf->dev, "Core component configuration failed\n");
+ goto err_uninit_dma;
+ }
+
+ intel_xpcie_iowrite32(XPCIE_STATUS_UNINIT,
+ xpcie_epf->xpcie.mmio + XPCIE_MMIO_HOST_STATUS);
+ intel_xpcie_set_device_status(&xpcie_epf->xpcie, XPCIE_STATUS_RUN);
+ intel_xpcie_set_doorbell(&xpcie_epf->xpcie, FROM_DEVICE,
+ DEV_EVENT, NO_OP);
+ memcpy(xpcie_epf->xpcie.mmio + XPCIE_MMIO_MAGIC_OFF, XPCIE_MAGIC_YOCTO,
+ strlen(XPCIE_MAGIC_YOCTO));
+
return 0;

-err_free_err_irq:
- free_irq(xpcie_epf->irq_err, &xpcie_epf->xpcie);
+err_uninit_dma:
+ intel_xpcie_set_device_status(&xpcie_epf->xpcie, XPCIE_STATUS_ERROR);
+ memcpy(xpcie_epf->xpcie.mmio + XPCIE_MMIO_MAGIC_OFF, XPCIE_MAGIC_YOCTO,
+ strlen(XPCIE_MAGIC_YOCTO));
+
+ intel_xpcie_ep_dma_uninit(epf);

err_cleanup_bars:
intel_xpcie_cleanup_bars(epf);
@@ -346,8 +439,12 @@ static int intel_xpcie_epf_bind(struct pci_epf *epf)

static void intel_xpcie_epf_unbind(struct pci_epf *epf)
{
+ struct xpcie_epf *xpcie_epf = epf_get_drvdata(epf);
struct pci_epc *epc = epf->epc;

+ intel_xpcie_core_cleanup(&xpcie_epf->xpcie);
+ intel_xpcie_set_device_status(&xpcie_epf->xpcie, XPCIE_STATUS_READY);
+
intel_xpcie_ep_dma_uninit(epf);

pci_epc_stop(epc);
@@ -379,8 +476,11 @@ static void intel_xpcie_epf_shutdown(struct device *dev)
xpcie_epf = epf_get_drvdata(epf);

/* Notify host in case PCIe hot plug not supported */
- if (xpcie_epf)
+ if (xpcie_epf && xpcie_epf->xpcie.status == XPCIE_STATUS_RUN) {
+ intel_xpcie_set_doorbell(&xpcie_epf->xpcie, FROM_DEVICE,
+ DEV_EVENT, DEV_SHUTDOWN);
pci_epc_raise_irq(epf->epc, epf->func_no, PCI_EPC_IRQ_MSI, 1);
+ }
}

static struct pci_epf_ops ops = {
diff --git a/drivers/misc/xlink-pcie/local_host/epf.h b/drivers/misc/xlink-pcie/local_host/epf.h
index 6ce5260e67be..675c07455ffa 100644
--- a/drivers/misc/xlink-pcie/local_host/epf.h
+++ b/drivers/misc/xlink-pcie/local_host/epf.h
@@ -14,6 +14,7 @@
#include <linux/pci-epf.h>

#include "xpcie.h"
+#include "util.h"

#define XPCIE_DRIVER_NAME "mxlk_pcie_epf"
#define XPCIE_DRIVER_DESC "Intel(R) xLink PCIe endpoint function driver"
@@ -26,6 +27,7 @@
#define XPCIE_NUM_RX_DESCS (64)

extern bool dma_ll_mode;
+extern u32 xlink_sw_id;

struct xpcie_dma_ll_desc {
u32 dma_ch_control1;
@@ -67,14 +69,38 @@ struct xpcie_epf {
void __iomem *dbi_base;
char stepping[KEEMBAY_XPCIE_STEPPING_MAXLEN];

+ irq_handler_t core_irq_callback;
+ dma_addr_t tx_phys;
+ void *tx_virt;
+ size_t tx_size;
+ dma_addr_t rx_phys;
+ void *rx_virt;
+ size_t rx_size;
+
struct xpcie_dma_ll_desc_buf tx_desc_buf[DMA_CHAN_NUM];
struct xpcie_dma_ll_desc_buf rx_desc_buf[DMA_CHAN_NUM];
};

+static inline struct device *xpcie_to_dev(struct xpcie *xpcie)
+{
+ struct xpcie_epf *xpcie_epf = container_of(xpcie,
+ struct xpcie_epf, xpcie);
+
+ return &xpcie_epf->epf->dev;
+}
+
int intel_xpcie_ep_dma_init(struct pci_epf *epf);
int intel_xpcie_ep_dma_uninit(struct pci_epf *epf);
int intel_xpcie_ep_dma_reset(struct pci_epf *epf);
int intel_xpcie_ep_dma_read_ll(struct pci_epf *epf, int chan, int descs_num);
int intel_xpcie_ep_dma_write_ll(struct pci_epf *epf, int chan, int descs_num);

+void intel_xpcie_register_host_irq(struct xpcie *xpcie,
+ irq_handler_t func);
+int intel_xpcie_raise_irq(struct xpcie *xpcie,
+ enum xpcie_doorbell_type type);
+int intel_xpcie_copy_from_host_ll(struct xpcie *xpcie,
+ int chan, int descs_num);
+int intel_xpcie_copy_to_host_ll(struct xpcie *xpcie,
+ int chan, int descs_num);
#endif /* XPCIE_EPF_HEADER_ */
diff --git a/drivers/misc/xlink-pcie/local_host/util.c b/drivers/misc/xlink-pcie/local_host/util.c
new file mode 100644
index 000000000000..ec808b0cd72b
--- /dev/null
+++ b/drivers/misc/xlink-pcie/local_host/util.c
@@ -0,0 +1,375 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*****************************************************************************
+ *
+ * Intel Keem Bay XLink PCIe Driver
+ *
+ * Copyright (C) 2020 Intel Corporation
+ *
+ ****************************************************************************/
+
+#include "util.h"
+
+void intel_xpcie_set_device_status(struct xpcie *xpcie, u32 status)
+{
+ xpcie->status = status;
+ intel_xpcie_iowrite32(status, xpcie->mmio + XPCIE_MMIO_DEV_STATUS);
+}
+
+u32 intel_xpcie_get_device_status(struct xpcie *xpcie)
+{
+ return intel_xpcie_ioread32(xpcie->mmio + XPCIE_MMIO_DEV_STATUS);
+}
+
+static size_t intel_xpcie_doorbell_offset(struct xpcie *xpcie,
+ enum xpcie_doorbell_direction dirt,
+ enum xpcie_doorbell_type type)
+{
+ if (dirt == TO_DEVICE && type == DATA_SENT)
+ return XPCIE_MMIO_HTOD_TX_DOORBELL;
+ if (dirt == TO_DEVICE && type == DATA_RECEIVED)
+ return XPCIE_MMIO_HTOD_RX_DOORBELL;
+ if (dirt == TO_DEVICE && type == DEV_EVENT)
+ return XPCIE_MMIO_HTOD_EVENT_DOORBELL;
+ if (dirt == FROM_DEVICE && type == DATA_SENT)
+ return XPCIE_MMIO_DTOH_TX_DOORBELL;
+ if (dirt == FROM_DEVICE && type == DATA_RECEIVED)
+ return XPCIE_MMIO_DTOH_RX_DOORBELL;
+ if (dirt == FROM_DEVICE && type == DEV_EVENT)
+ return XPCIE_MMIO_DTOH_EVENT_DOORBELL;
+
+ return 0;
+}
+
+void intel_xpcie_set_doorbell(struct xpcie *xpcie,
+ enum xpcie_doorbell_direction dirt,
+ enum xpcie_doorbell_type type, u8 value)
+{
+ size_t offset = intel_xpcie_doorbell_offset(xpcie, dirt, type);
+
+ intel_xpcie_iowrite8(value, xpcie->mmio + offset);
+}
+
+u8 intel_xpcie_get_doorbell(struct xpcie *xpcie,
+ enum xpcie_doorbell_direction dirt,
+ enum xpcie_doorbell_type type)
+{
+ size_t offset = intel_xpcie_doorbell_offset(xpcie, dirt, type);
+
+ return intel_xpcie_ioread8(xpcie->mmio + offset);
+}
+
+u32 intel_xpcie_get_host_status(struct xpcie *xpcie)
+{
+ return intel_xpcie_ioread32(xpcie->mmio + XPCIE_MMIO_HOST_STATUS);
+}
+
+void intel_xpcie_set_host_status(struct xpcie *xpcie, u32 status)
+{
+ xpcie->status = status;
+ intel_xpcie_iowrite32(status, xpcie->mmio + XPCIE_MMIO_HOST_STATUS);
+}
+
+struct xpcie_buf_desc *intel_xpcie_alloc_bd(size_t length)
+{
+ struct xpcie_buf_desc *bd;
+
+ bd = kzalloc(sizeof(*bd), GFP_KERNEL);
+ if (!bd)
+ return NULL;
+
+ bd->head = kzalloc(roundup(length, cache_line_size()), GFP_KERNEL);
+ if (!bd->head) {
+ kfree(bd);
+ return NULL;
+ }
+
+ bd->data = bd->head;
+ bd->length = length;
+ bd->true_len = length;
+ bd->next = NULL;
+ bd->own_mem = true;
+
+ return bd;
+}
+
+struct xpcie_buf_desc *intel_xpcie_alloc_bd_reuse(size_t length, void *virt,
+ dma_addr_t phys)
+{
+ struct xpcie_buf_desc *bd;
+
+ bd = kzalloc(sizeof(*bd), GFP_KERNEL);
+ if (!bd)
+ return NULL;
+
+ bd->head = virt;
+ bd->phys = phys;
+ bd->data = bd->head;
+ bd->length = length;
+ bd->true_len = length;
+ bd->next = NULL;
+ bd->own_mem = false;
+
+ return bd;
+}
+
+void intel_xpcie_free_bd(struct xpcie_buf_desc *bd)
+{
+ if (bd) {
+ if (bd->own_mem)
+ kfree(bd->head);
+ kfree(bd);
+ }
+}
+
+int intel_xpcie_list_init(struct xpcie_list *list)
+{
+ spin_lock_init(&list->lock);
+ list->bytes = 0;
+ list->buffers = 0;
+ list->head = NULL;
+ list->tail = NULL;
+
+ return 0;
+}
+
+void intel_xpcie_list_cleanup(struct xpcie_list *list)
+{
+ struct xpcie_buf_desc *bd;
+
+ spin_lock(&list->lock);
+ while (list->head) {
+ bd = list->head;
+ list->head = bd->next;
+ intel_xpcie_free_bd(bd);
+ }
+
+ list->head = NULL;
+ list->tail = NULL;
+ spin_unlock(&list->lock);
+}
+
+int intel_xpcie_list_put(struct xpcie_list *list, struct xpcie_buf_desc *bd)
+{
+ if (!bd)
+ return -EINVAL;
+
+ spin_lock(&list->lock);
+ if (list->head)
+ list->tail->next = bd;
+ else
+ list->head = bd;
+
+ while (bd) {
+ list->tail = bd;
+ list->bytes += bd->length;
+ list->buffers++;
+ bd = bd->next;
+ }
+ spin_unlock(&list->lock);
+
+ return 0;
+}
+
+int intel_xpcie_list_put_head(struct xpcie_list *list,
+ struct xpcie_buf_desc *bd)
+{
+ struct xpcie_buf_desc *old_head;
+
+ if (!bd)
+ return -EINVAL;
+
+ spin_lock(&list->lock);
+ old_head = list->head;
+ list->head = bd;
+ while (bd) {
+ list->bytes += bd->length;
+ list->buffers++;
+ if (!bd->next) {
+ list->tail = list->tail ? list->tail : bd;
+ bd->next = old_head;
+ break;
+ }
+ bd = bd->next;
+ }
+ spin_unlock(&list->lock);
+
+ return 0;
+}
+
+struct xpcie_buf_desc *intel_xpcie_list_get(struct xpcie_list *list)
+{
+ struct xpcie_buf_desc *bd;
+
+ spin_lock(&list->lock);
+ bd = list->head;
+ if (list->head) {
+ list->head = list->head->next;
+ if (!list->head)
+ list->tail = NULL;
+ bd->next = NULL;
+ list->bytes -= bd->length;
+ list->buffers--;
+ }
+ spin_unlock(&list->lock);
+
+ return bd;
+}
+
+void intel_xpcie_list_info(struct xpcie_list *list,
+ size_t *bytes, size_t *buffers)
+{
+ spin_lock(&list->lock);
+ *bytes = list->bytes;
+ *buffers = list->buffers;
+ spin_unlock(&list->lock);
+}
+
+struct xpcie_buf_desc *intel_xpcie_alloc_rx_bd(struct xpcie *xpcie)
+{
+ struct xpcie_buf_desc *bd;
+
+ bd = intel_xpcie_list_get(&xpcie->rx_pool);
+ if (bd) {
+ bd->data = bd->head;
+ bd->length = bd->true_len;
+ bd->next = NULL;
+ bd->interface = 0;
+ }
+
+ return bd;
+}
+
+void intel_xpcie_free_rx_bd(struct xpcie *xpcie, struct xpcie_buf_desc *bd)
+{
+ if (bd)
+ intel_xpcie_list_put(&xpcie->rx_pool, bd);
+}
+
+struct xpcie_buf_desc *intel_xpcie_alloc_tx_bd(struct xpcie *xpcie)
+{
+ struct xpcie_buf_desc *bd;
+
+ bd = intel_xpcie_list_get(&xpcie->tx_pool);
+ if (bd) {
+ bd->data = bd->head;
+ bd->length = bd->true_len;
+ bd->next = NULL;
+ bd->interface = 0;
+ } else {
+ xpcie->no_tx_buffer = true;
+ }
+
+ return bd;
+}
+
+void intel_xpcie_free_tx_bd(struct xpcie *xpcie, struct xpcie_buf_desc *bd)
+{
+ if (!bd)
+ return;
+
+ intel_xpcie_list_put(&xpcie->tx_pool, bd);
+
+ xpcie->no_tx_buffer = false;
+ wake_up_interruptible(&xpcie->tx_waitq);
+}
+
+int intel_xpcie_interface_init(struct xpcie *xpcie, int id)
+{
+ struct xpcie_interface *inf = xpcie->interfaces + id;
+
+ inf->id = id;
+ inf->xpcie = xpcie;
+
+ inf->partial_read = NULL;
+ intel_xpcie_list_init(&inf->read);
+ mutex_init(&inf->rlock);
+ inf->data_avail = false;
+ init_waitqueue_head(&inf->rx_waitq);
+
+ return 0;
+}
+
+void intel_xpcie_interface_cleanup(struct xpcie_interface *inf)
+{
+ struct xpcie_buf_desc *bd;
+
+ intel_xpcie_free_rx_bd(inf->xpcie, inf->partial_read);
+ while ((bd = intel_xpcie_list_get(&inf->read)))
+ intel_xpcie_free_rx_bd(inf->xpcie, bd);
+
+ mutex_destroy(&inf->rlock);
+}
+
+void intel_xpcie_interfaces_cleanup(struct xpcie *xpcie)
+{
+ int index;
+
+ for (index = 0; index < XPCIE_NUM_INTERFACES; index++)
+ intel_xpcie_interface_cleanup(xpcie->interfaces + index);
+
+ intel_xpcie_list_cleanup(&xpcie->write);
+ mutex_destroy(&xpcie->wlock);
+}
+
+int intel_xpcie_interfaces_init(struct xpcie *xpcie)
+{
+ int index;
+
+ mutex_init(&xpcie->wlock);
+ intel_xpcie_list_init(&xpcie->write);
+ init_waitqueue_head(&xpcie->tx_waitq);
+ xpcie->no_tx_buffer = false;
+
+ for (index = 0; index < XPCIE_NUM_INTERFACES; index++)
+ intel_xpcie_interface_init(xpcie, index);
+
+ return 0;
+}
+
+void intel_xpcie_add_bd_to_interface(struct xpcie *xpcie,
+ struct xpcie_buf_desc *bd)
+{
+ struct xpcie_interface *inf;
+
+ inf = xpcie->interfaces + bd->interface;
+
+ intel_xpcie_list_put(&inf->read, bd);
+
+ mutex_lock(&inf->rlock);
+ inf->data_avail = true;
+ mutex_unlock(&inf->rlock);
+ wake_up_interruptible(&inf->rx_waitq);
+}
+
+void *intel_xpcie_cap_find(struct xpcie *xpcie, u32 start, u16 id)
+{
+ int ttl = XPCIE_CAP_TTL;
+ void *hdr;
+ u16 id_out, next;
+
+ /* If user didn't specify start, assume start of mmio */
+ if (!start)
+ start = intel_xpcie_ioread32(xpcie->mmio + XPCIE_MMIO_CAP_OFF);
+
+ /* Read header info */
+ hdr = xpcie->mmio + start;
+
+ /* Check if we still have time to live */
+ while (ttl--) {
+ id_out = intel_xpcie_ioread16(hdr + XPCIE_CAP_HDR_ID);
+ next = intel_xpcie_ioread16(hdr + XPCIE_CAP_HDR_NEXT);
+
+ /* If cap matches, return header */
+ if (id_out == id)
+ return hdr;
+ /* If cap is NULL, we are at the end of the list */
+ else if (id_out == XPCIE_CAP_NULL)
+ return NULL;
+ /* If no match and no end of list, traverse the linked list */
+ else
+ hdr = xpcie->mmio + next;
+ }
+
+ /* If we reached here, the capability list is corrupted */
+ return NULL;
+}
diff --git a/drivers/misc/xlink-pcie/local_host/util.h b/drivers/misc/xlink-pcie/local_host/util.h
new file mode 100644
index 000000000000..908be897a61d
--- /dev/null
+++ b/drivers/misc/xlink-pcie/local_host/util.h
@@ -0,0 +1,70 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*****************************************************************************
+ *
+ * Intel Keem Bay XLink PCIe Driver
+ *
+ * Copyright (C) 2020 Intel Corporation
+ *
+ ****************************************************************************/
+
+#ifndef XPCIE_UTIL_HEADER_
+#define XPCIE_UTIL_HEADER_
+
+#include "xpcie.h"
+
+enum xpcie_doorbell_direction {
+ TO_DEVICE,
+ FROM_DEVICE
+};
+
+enum xpcie_doorbell_type {
+ DATA_SENT,
+ DATA_RECEIVED,
+ DEV_EVENT
+};
+
+enum xpcie_event_type {
+ NO_OP,
+ REQUEST_RESET,
+ DEV_SHUTDOWN
+};
+
+void intel_xpcie_set_doorbell(struct xpcie *xpcie,
+ enum xpcie_doorbell_direction dirt,
+ enum xpcie_doorbell_type type, u8 value);
+u8 intel_xpcie_get_doorbell(struct xpcie *xpcie,
+ enum xpcie_doorbell_direction dirt,
+ enum xpcie_doorbell_type type);
+
+void intel_xpcie_set_device_status(struct xpcie *xpcie, u32 status);
+u32 intel_xpcie_get_device_status(struct xpcie *xpcie);
+u32 intel_xpcie_get_host_status(struct xpcie *xpcie);
+void intel_xpcie_set_host_status(struct xpcie *xpcie, u32 status);
+
+struct xpcie_buf_desc *intel_xpcie_alloc_bd(size_t length);
+struct xpcie_buf_desc *intel_xpcie_alloc_bd_reuse(size_t length, void *virt,
+ dma_addr_t phys);
+void intel_xpcie_free_bd(struct xpcie_buf_desc *bd);
+
+int intel_xpcie_list_init(struct xpcie_list *list);
+void intel_xpcie_list_cleanup(struct xpcie_list *list);
+int intel_xpcie_list_put(struct xpcie_list *list, struct xpcie_buf_desc *bd);
+int intel_xpcie_list_put_head(struct xpcie_list *list,
+ struct xpcie_buf_desc *bd);
+struct xpcie_buf_desc *intel_xpcie_list_get(struct xpcie_list *list);
+void intel_xpcie_list_info(struct xpcie_list *list, size_t *bytes,
+ size_t *buffers);
+
+struct xpcie_buf_desc *intel_xpcie_alloc_rx_bd(struct xpcie *xpcie);
+void intel_xpcie_free_rx_bd(struct xpcie *xpcie, struct xpcie_buf_desc *bd);
+struct xpcie_buf_desc *intel_xpcie_alloc_tx_bd(struct xpcie *xpcie);
+void intel_xpcie_free_tx_bd(struct xpcie *xpcie, struct xpcie_buf_desc *bd);
+
+int intel_xpcie_interface_init(struct xpcie *xpcie, int id);
+void intel_xpcie_interface_cleanup(struct xpcie_interface *inf);
+void intel_xpcie_interfaces_cleanup(struct xpcie *xpcie);
+int intel_xpcie_interfaces_init(struct xpcie *xpcie);
+void intel_xpcie_add_bd_to_interface(struct xpcie *xpcie,
+ struct xpcie_buf_desc *bd);
+void *intel_xpcie_cap_find(struct xpcie *xpcie, u32 start, u16 id);
+#endif /* XPCIE_UTIL_HEADER_ */
diff --git a/drivers/misc/xlink-pcie/local_host/xpcie.h b/drivers/misc/xlink-pcie/local_host/xpcie.h
index 932ec5d5c718..5ae0b3dbd503 100644
--- a/drivers/misc/xlink-pcie/local_host/xpcie.h
+++ b/drivers/misc/xlink-pcie/local_host/xpcie.h
@@ -14,6 +14,8 @@
#include <linux/module.h>
#include <linux/pci_ids.h>

+#include "core.h"
+
#ifndef PCI_DEVICE_ID_INTEL_KEEMBAY
#define PCI_DEVICE_ID_INTEL_KEEMBAY 0x6240
#endif
@@ -35,20 +37,83 @@ struct xpcie_version {
u16 build;
} __packed;

+/* Status encoding of both device and host */
+#define XPCIE_STATUS_ERROR (0xFFFFFFFF)
+#define XPCIE_STATUS_UNINIT (0)
+#define XPCIE_STATUS_READY (1)
+#define XPCIE_STATUS_RECOVERY (2)
+#define XPCIE_STATUS_OFF (3)
+#define XPCIE_STATUS_RUN (4)
+
+#define XPCIE_MAGIC_STRLEN (16)
+#define XPCIE_MAGIC_YOCTO "VPUYOCTO"
+
/* MMIO layout and offsets shared between device and host */
struct xpcie_mmio {
struct xpcie_version version;
+ u32 device_status;
+ u32 host_status;
u8 legacy_a0;
+ u8 htod_tx_doorbell;
+ u8 htod_rx_doorbell;
+ u8 htod_event_doorbell;
+ u8 dtoh_tx_doorbell;
+ u8 dtoh_rx_doorbell;
+ u8 dtoh_event_doorbell;
+ u8 reserved;
+ u32 cap_offset;
+ u8 magic[XPCIE_MAGIC_STRLEN];
} __packed;

#define XPCIE_MMIO_VERSION (offsetof(struct xpcie_mmio, version))
#define XPCIE_MMIO_LEGACY_A0 (offsetof(struct xpcie_mmio, legacy_a0))
+#define XPCIE_MMIO_DEV_STATUS (offsetof(struct xpcie_mmio, device_status))
+#define XPCIE_MMIO_LEGACY_A0 (offsetof(struct xpcie_mmio, legacy_a0))
+#define XPCIE_MMIO_HOST_STATUS (offsetof(struct xpcie_mmio, host_status))
+#define XPCIE_MMIO_LEGACY_A0 (offsetof(struct xpcie_mmio, legacy_a0))
+#define XPCIE_MMIO_HTOD_TX_DOORBELL \
+ (offsetof(struct xpcie_mmio, htod_tx_doorbell))
+#define XPCIE_MMIO_HTOD_RX_DOORBELL \
+ (offsetof(struct xpcie_mmio, htod_rx_doorbell))
+#define XPCIE_MMIO_HTOD_EVENT_DOORBELL \
+ (offsetof(struct xpcie_mmio, htod_event_doorbell))
+#define XPCIE_MMIO_DTOH_TX_DOORBELL \
+ (offsetof(struct xpcie_mmio, dtoh_tx_doorbell))
+#define XPCIE_MMIO_DTOH_RX_DOORBELL \
+ (offsetof(struct xpcie_mmio, dtoh_rx_doorbell))
+#define XPCIE_MMIO_DTOH_EVENT_DOORBELL \
+ (offsetof(struct xpcie_mmio, dtoh_event_doorbell))
+#define XPCIE_MMIO_CAP_OFF (offsetof(struct xpcie_mmio, cap_offset))
+#define XPCIE_MMIO_MAGIC_OFF (offsetof(struct xpcie_mmio, magic))

struct xpcie {
u32 status;
bool legacy_a0;
void *mmio;
void *bar4;
+
+ struct workqueue_struct *rx_wq;
+ struct workqueue_struct *tx_wq;
+
+ struct xpcie_interface interfaces[XPCIE_NUM_INTERFACES];
+
+ size_t fragment_size;
+ struct xpcie_cap_txrx *txrx;
+ struct xpcie_stream tx;
+ struct xpcie_stream rx;
+
+ struct mutex wlock; /* write lock */
+ struct xpcie_list write;
+ bool no_tx_buffer;
+ wait_queue_head_t tx_waitq;
+ bool tx_pending;
+ bool stop_flag;
+
+ struct xpcie_list rx_pool;
+ struct xpcie_list tx_pool;
+
+ struct delayed_work rx_event;
+ struct delayed_work tx_event;
};

#endif /* XPCIE_HEADER_ */
diff --git a/include/linux/xlink_drv_inf.h b/include/linux/xlink_drv_inf.h
new file mode 100644
index 000000000000..ffe8f4c253e6
--- /dev/null
+++ b/include/linux/xlink_drv_inf.h
@@ -0,0 +1,60 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*****************************************************************************
+ *
+ * Intel Keem Bay XLink PCIe Driver
+ *
+ * Copyright (C) 2020 Intel Corporation
+ *
+ ****************************************************************************/
+
+#ifndef _XLINK_DRV_INF_H_
+#define _XLINK_DRV_INF_H_
+
+#include <linux/bitfield.h>
+#include <linux/bits.h>
+#include <linux/types.h>
+
+#define XLINK_DEV_INF_TYPE_MASK GENMASK(27, 24)
+#define XLINK_DEV_PHYS_ID_MASK GENMASK(23, 8)
+#define XLINK_DEV_TYPE_MASK GENMASK(6, 4)
+#define XLINK_DEV_PCIE_ID_MASK GENMASK(3, 1)
+#define XLINK_DEV_FUNC_MASK GENMASK(0, 0)
+
+enum xlink_device_inf_type {
+ XLINK_DEV_INF_PCIE = 1,
+};
+
+enum xlink_device_type {
+ XLINK_DEV_TYPE_KMB = 0,
+};
+
+enum xlink_device_pcie {
+ XLINK_DEV_PCIE_0 = 0,
+};
+
+enum xlink_device_func {
+ XLINK_DEV_FUNC_VPU = 0,
+};
+
+enum _xlink_device_status {
+ _XLINK_DEV_OFF,
+ _XLINK_DEV_ERROR,
+ _XLINK_DEV_BUSY,
+ _XLINK_DEV_RECOVERY,
+ _XLINK_DEV_READY
+};
+
+int xlink_pcie_get_device_list(u32 *sw_device_id_list,
+ u32 *num_devices);
+int xlink_pcie_get_device_name(u32 sw_device_id, char *device_name,
+ size_t name_size);
+int xlink_pcie_get_device_status(u32 sw_device_id,
+ u32 *device_status);
+int xlink_pcie_boot_device(u32 sw_device_id, const char *binary_name);
+int xlink_pcie_connect(u32 sw_device_id);
+int xlink_pcie_read(u32 sw_device_id, void *data, size_t *const size,
+ u32 timeout);
+int xlink_pcie_write(u32 sw_device_id, void *data, size_t *const size,
+ u32 timeout);
+int xlink_pcie_reset_device(u32 sw_device_id);
+#endif
--
2.17.1

2020-12-02 06:22:00

by Greg Kroah-Hartman

[permalink] [raw]
Subject: Re: [PATCH 09/22] misc: xlink-pcie: lh: Add core communication logic

On Tue, Dec 01, 2020 at 02:34:58PM -0800, [email protected] wrote:
> From: Srikanth Thokala <[email protected]>
>
> Add logic to establish communication with the remote host which is through
> ring buffer management and MSI/Doorbell interrupts
>
> Reviewed-by: Mark Gross <[email protected]>
> Signed-off-by: Srikanth Thokala <[email protected]>
> ---
> drivers/misc/xlink-pcie/local_host/Makefile | 2 +
> drivers/misc/xlink-pcie/local_host/core.c | 894 ++++++++++++++++++++
> drivers/misc/xlink-pcie/local_host/core.h | 247 ++++++
> drivers/misc/xlink-pcie/local_host/epf.c | 116 ++-
> drivers/misc/xlink-pcie/local_host/epf.h | 26 +
> drivers/misc/xlink-pcie/local_host/util.c | 375 ++++++++
> drivers/misc/xlink-pcie/local_host/util.h | 70 ++
> drivers/misc/xlink-pcie/local_host/xpcie.h | 65 ++
> include/linux/xlink_drv_inf.h | 60 ++
> 9 files changed, 1847 insertions(+), 8 deletions(-)
> create mode 100644 drivers/misc/xlink-pcie/local_host/core.c
> create mode 100644 drivers/misc/xlink-pcie/local_host/core.h
> create mode 100644 drivers/misc/xlink-pcie/local_host/util.c
> create mode 100644 drivers/misc/xlink-pcie/local_host/util.h
> create mode 100644 include/linux/xlink_drv_inf.h
>
> diff --git a/drivers/misc/xlink-pcie/local_host/Makefile b/drivers/misc/xlink-pcie/local_host/Makefile
> index 54fc118e2dd1..28761751d43b 100644
> --- a/drivers/misc/xlink-pcie/local_host/Makefile
> +++ b/drivers/misc/xlink-pcie/local_host/Makefile
> @@ -1,3 +1,5 @@
> obj-$(CONFIG_XLINK_PCIE_LH_DRIVER) += mxlk_ep.o
> mxlk_ep-objs := epf.o
> mxlk_ep-objs += dma.o
> +mxlk_ep-objs += core.o
> +mxlk_ep-objs += util.o
> diff --git a/drivers/misc/xlink-pcie/local_host/core.c b/drivers/misc/xlink-pcie/local_host/core.c
> new file mode 100644
> index 000000000000..aecaaa783153
> --- /dev/null
> +++ b/drivers/misc/xlink-pcie/local_host/core.c
> @@ -0,0 +1,894 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*****************************************************************************
> + *
> + * Intel Keem Bay XLink PCIe Driver
> + *
> + * Copyright (C) 2020 Intel Corporation
> + *
> + ****************************************************************************/
> +
> +#include <linux/of_reserved_mem.h>
> +
> +#include "epf.h"
> +#include "core.h"
> +#include "util.h"
> +
> +static struct xpcie *global_xpcie;
> +
> +static int rx_pool_size = SZ_32M;
> +module_param(rx_pool_size, int, 0664);
> +MODULE_PARM_DESC(rx_pool_size, "receiving pool size (default 32 MiB)");
> +
> +static int tx_pool_size = SZ_32M;
> +module_param(tx_pool_size, int, 0664);
> +MODULE_PARM_DESC(tx_pool_size, "transmitting pool size (default 32 MiB)");
> +
> +static int fragment_size = XPCIE_FRAGMENT_SIZE;
> +module_param(fragment_size, int, 0664);
> +MODULE_PARM_DESC(fragment_size, "transfer descriptor size (default 128 KiB)");
> +
> +static bool tx_pool_coherent = true;
> +module_param(tx_pool_coherent, bool, 0664);
> +MODULE_PARM_DESC(tx_pool_coherent,
> + "transmitting pool using coherent memory (default true)");
> +
> +static bool rx_pool_coherent;
> +module_param(rx_pool_coherent, bool, 0664);
> +MODULE_PARM_DESC(rx_pool_coherent,
> + "receiving pool using coherent memory (default false)");

This isn't the 1990's anymore. Please make these dynamic such that they
are never needed (the code figures out the best values), or on some
per-device basis using configfs or sysfs.

thanks,

greg k-h

2020-12-02 16:50:33

by Thokala, Srikanth

[permalink] [raw]
Subject: RE: [PATCH 09/22] misc: xlink-pcie: lh: Add core communication logic

Hi Greg,

> -----Original Message-----
> From: Greg KH <[email protected]>
> Sent: Wednesday, December 2, 2020 11:48 AM
> To: [email protected]
> Cc: [email protected]; [email protected]; [email protected];
> [email protected]; [email protected]; [email protected];
> [email protected]; [email protected];
> [email protected]; [email protected]; [email protected];
> [email protected]; [email protected]; Thokala, Srikanth
> <[email protected]>
> Subject: Re: [PATCH 09/22] misc: xlink-pcie: lh: Add core communication
> logic
>
> On Tue, Dec 01, 2020 at 02:34:58PM -0800, [email protected] wrote:
> > From: Srikanth Thokala <[email protected]>
> >
> > Add logic to establish communication with the remote host which is
> through
> > ring buffer management and MSI/Doorbell interrupts
> >
> > Reviewed-by: Mark Gross <[email protected]>
> > Signed-off-by: Srikanth Thokala <[email protected]>
> > ---
> > drivers/misc/xlink-pcie/local_host/Makefile | 2 +
> > drivers/misc/xlink-pcie/local_host/core.c | 894 ++++++++++++++++++++
> > drivers/misc/xlink-pcie/local_host/core.h | 247 ++++++
> > drivers/misc/xlink-pcie/local_host/epf.c | 116 ++-
> > drivers/misc/xlink-pcie/local_host/epf.h | 26 +
> > drivers/misc/xlink-pcie/local_host/util.c | 375 ++++++++
> > drivers/misc/xlink-pcie/local_host/util.h | 70 ++
> > drivers/misc/xlink-pcie/local_host/xpcie.h | 65 ++
> > include/linux/xlink_drv_inf.h | 60 ++
> > 9 files changed, 1847 insertions(+), 8 deletions(-)
> > create mode 100644 drivers/misc/xlink-pcie/local_host/core.c
> > create mode 100644 drivers/misc/xlink-pcie/local_host/core.h
> > create mode 100644 drivers/misc/xlink-pcie/local_host/util.c
> > create mode 100644 drivers/misc/xlink-pcie/local_host/util.h
> > create mode 100644 include/linux/xlink_drv_inf.h
> >
> > diff --git a/drivers/misc/xlink-pcie/local_host/Makefile
> b/drivers/misc/xlink-pcie/local_host/Makefile
> > index 54fc118e2dd1..28761751d43b 100644
> > --- a/drivers/misc/xlink-pcie/local_host/Makefile
> > +++ b/drivers/misc/xlink-pcie/local_host/Makefile
> > @@ -1,3 +1,5 @@
> > obj-$(CONFIG_XLINK_PCIE_LH_DRIVER) += mxlk_ep.o
> > mxlk_ep-objs := epf.o
> > mxlk_ep-objs += dma.o
> > +mxlk_ep-objs += core.o
> > +mxlk_ep-objs += util.o
> > diff --git a/drivers/misc/xlink-pcie/local_host/core.c
> b/drivers/misc/xlink-pcie/local_host/core.c
> > new file mode 100644
> > index 000000000000..aecaaa783153
> > --- /dev/null
> > +++ b/drivers/misc/xlink-pcie/local_host/core.c
> > @@ -0,0 +1,894 @@
> > +// SPDX-License-Identifier: GPL-2.0-only
> >
> +/************************************************************************
> *****
> > + *
> > + * Intel Keem Bay XLink PCIe Driver
> > + *
> > + * Copyright (C) 2020 Intel Corporation
> > + *
> > +
> **************************************************************************
> **/
> > +
> > +#include <linux/of_reserved_mem.h>
> > +
> > +#include "epf.h"
> > +#include "core.h"
> > +#include "util.h"
> > +
> > +static struct xpcie *global_xpcie;
> > +
> > +static int rx_pool_size = SZ_32M;
> > +module_param(rx_pool_size, int, 0664);
> > +MODULE_PARM_DESC(rx_pool_size, "receiving pool size (default 32 MiB)");
> > +
> > +static int tx_pool_size = SZ_32M;
> > +module_param(tx_pool_size, int, 0664);
> > +MODULE_PARM_DESC(tx_pool_size, "transmitting pool size (default 32
> MiB)");
> > +
> > +static int fragment_size = XPCIE_FRAGMENT_SIZE;
> > +module_param(fragment_size, int, 0664);
> > +MODULE_PARM_DESC(fragment_size, "transfer descriptor size (default 128
> KiB)");
> > +
> > +static bool tx_pool_coherent = true;
> > +module_param(tx_pool_coherent, bool, 0664);
> > +MODULE_PARM_DESC(tx_pool_coherent,
> > + "transmitting pool using coherent memory (default true)");
> > +
> > +static bool rx_pool_coherent;
> > +module_param(rx_pool_coherent, bool, 0664);
> > +MODULE_PARM_DESC(rx_pool_coherent,
> > + "receiving pool using coherent memory (default false)");
>
> This isn't the 1990's anymore. Please make these dynamic such that they
> are never needed (the code figures out the best values), or on some
> per-device basis using configfs or sysfs.

Sure, I will fix it in my v2.

thanks!
Srikanth

> thanks,
>
> greg k-h

2020-12-07 02:35:25

by Joe Perches

[permalink] [raw]
Subject: Re: [PATCH 16/22] xlink-ipc: Add xlink ipc driver

On Tue, 2020-12-01 at 14:35 -0800, [email protected] wrote:
> From: Seamus Kelly <[email protected]>
>
> Add xLink driver, which interfaces the xLink Core driver with the Keem
> Bay VPU IPC driver, thus enabling xLink to control and communicate with
> the VPU IP present on the Intel Keem Bay SoC.

Trivial style comments:

> diff --git a/drivers/misc/xlink-ipc/xlink-ipc.c b/drivers/misc/xlink-ipc/xlink-ipc.c

[]

> +/*
> + * xlink_reserved_memory_init() - Initialize reserved memory for the device.
> + *
> + * @xlink_dev: [in] The xlink ipc device the reserved memory is allocated to.
> + *
> + * Return: 0 on success, negative error code otherwise.
> + */
> +static int xlink_reserved_memory_init(struct xlink_ipc_dev *xlink_dev)
> +{
> + struct device *dev = &xlink_dev->pdev->dev;
> +
> + xlink_dev->local_xlink_mem.dev = init_xlink_reserved_mem_dev(dev,
> + "xlink_local_reserved",
> + LOCAL_XLINK_IPC_BUFFER_IDX);
> + if (!xlink_dev->local_xlink_mem.dev)
> + return -ENOMEM;

This sort of code, with a repeated struct dereference, generally reads
better using a temporary and is also less prone to typo use.

struct device *dev = &xlink_dev->pdev->dev;
struct xlink_buf_mem *lxm = &xlink_dev->local_xlink_mem;

lxm->dev = init_xlink_reserved_mem_dev(dev, "xlink_local_reserved",
LOCAL_XLINK_IPC_BUFFER_IDX);
if (!lxm->dev)
return -ENOMEM;

> + xlink_dev->local_xlink_mem.size = get_xlink_reserved_mem_size(dev,
> + LOCAL_XLINK_IPC_BUFFER_IDX);

lxm->size = get_xlink_reserved_mem_size(dev, LOCAL_XLINK_IPC_BUFFER_IDX);

etc...



2020-12-07 19:57:53

by Randy Dunlap

[permalink] [raw]
Subject: Re: [PATCH 16/22] xlink-ipc: Add xlink ipc driver

Ahoy--

On 12/1/20 2:35 PM, [email protected] wrote:
> From: Seamus Kelly <[email protected]>
>
> Add xLink driver, which interfaces the xLink Core driver with the Keem
> Bay VPU IPC driver, thus enabling xLink to control and communicate with
> the VPU IP present on the Intel Keem Bay SoC.
>
> Specifically the driver enables xLink Core to:
>
> * Boot / Reset the VPU IP
> * Register to VPU IP event notifications (device connected, device
> disconnected, WDT event)
> * Query the status of the VPU IP (OFF, BUSY, READY, ERROR, RECOVERY)
> * Exchange data with the VPU IP, using the Keem Bay IPC mechanism
> - Including the ability to send 'volatile' data (i.e., small amount of
> data, up to 128-bytes that was not allocated in the CPU/VPU shared
> memory region)
>
> Cc: [email protected]
> Reviewed-by: Mark Gross <[email protected]>
> Signed-off-by: Seamus Kelly <[email protected]>
> Signed-off-by: Ryan Carnaghi <[email protected]>
> ---
> Documentation/vpu/index.rst | 1 +
> Documentation/vpu/xlink-ipc.rst | 50 ++
> MAINTAINERS | 6 +
> drivers/misc/Kconfig | 1 +
> drivers/misc/Makefile | 1 +
> drivers/misc/xlink-ipc/Kconfig | 7 +
> drivers/misc/xlink-ipc/Makefile | 4 +
> drivers/misc/xlink-ipc/xlink-ipc.c | 879 +++++++++++++++++++++++++++++
> include/linux/xlink-ipc.h | 48 ++
> 9 files changed, 997 insertions(+)
> create mode 100644 Documentation/vpu/xlink-ipc.rst
> create mode 100644 drivers/misc/xlink-ipc/Kconfig
> create mode 100644 drivers/misc/xlink-ipc/Makefile
> create mode 100644 drivers/misc/xlink-ipc/xlink-ipc.c
> create mode 100644 include/linux/xlink-ipc.h

> diff --git a/Documentation/vpu/xlink-ipc.rst b/Documentation/vpu/xlink-ipc.rst
> new file mode 100644
> index 000000000000..af583579e70d
> --- /dev/null
> +++ b/Documentation/vpu/xlink-ipc.rst
> @@ -0,0 +1,50 @@
> +.. SPDX-License-Identifier: GPL-2.0-only
> +
> +Kernel driver: xLink IPC driver
> +=================================
> +Supported chips:
> +
> +* | Intel Edge.AI Computer Vision platforms: Keem Bay
> + | Suffix: Bay
> + | Datasheet: (not yet publicly available)
> +
> +------------
> +Introduction
> +------------
> +
> +The xLink IPC driver interfaces the xLink Core driver with the Keem Bay VPU IPC
> +driver, thus enabling xLink to control and communicate with the VPU IP present
> +on the Intel Keem Bay SoC.
> +
> +Specifically the driver enables xLink Core to:
> +
> +* Boot / Reset the VPU IP
> +* Register to VPU IP event notifications (device connected, device disconnected,
> + WDT event)
> +* Query the status of the VPU IP (OFF, BUSY, READY, ERROR, RECOVERY)
> +* Exchange data with the VPU IP, using the Keem Bay IPC mechanism
> +
> + * Including the ability to send 'volatile' data (i.e., small amount of data,
> + up to 128-bytes that was not allocated in the CPU/VPU shared memory region)
> +
> +Sending / Receiving 'volatile' data
> +-----------------------------------
> +
> +Data to be exchanged with Keem Bay IPC needs to be allocated in the portion of
> +DDR shared between the CPU and VPU.
> +
> +This can be impractical for small amount of data that user code can allocate

amounts

> +on the stack.
> +
> +To reduce the burden on user code, xLink Core provides special send / receive
> +functions to send up to 128 bytes of 'volatile data', i.e., data that is not
> +allocated in the shared memory and that might also disappear after the xLink
> +API is called (e.g., because allocated on the stack).
> +
> +The xLink IPC driver implements support for transferring such 'volatile data'
> +to the VPU using Keem Bay IPC. To this end, the driver reserved some memory in

better: reserves

> +the shared memory region.
> +
> +When volatile data is to be sent, xLink IPC allocates a buffer from the
> +reserved memory region and copies the volatile data to the buffer. The buffer
> +is then transferred to the VPU using Keem Bay IPC.

> diff --git a/drivers/misc/xlink-ipc/Kconfig b/drivers/misc/xlink-ipc/Kconfig
> new file mode 100644
> index 000000000000..6aa2592fe9a3
> --- /dev/null
> +++ b/drivers/misc/xlink-ipc/Kconfig
> @@ -0,0 +1,7 @@
> +config XLINK_IPC
> + tristate "Support for XLINK IPC"
> + depends on KEEMBAY_VPU_IPC
> + help
> + XLINK IPC enables the communication/control IPC Sub-System.
> +
> + Select M if you have an Intel SoC with a Vision Processing Unit (VPU)

End that sentence with a '.', please.


--
~Randy

2020-12-11 13:54:48

by Kelly, Seamus

[permalink] [raw]
Subject: RE: [PATCH 16/22] xlink-ipc: Add xlink ipc driver


From: Joe Perches <[email protected]>
Sent: Monday, December 7, 2020 2:33 AM
To: [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Cc: [email protected]; Kelly, Seamus <[email protected]>; [email protected]; Ryan Carnaghi <[email protected]>
Subject: Re: [PATCH 16/22] xlink-ipc: Add xlink ipc driver

On Tue, 2020-12-01 at 14:35 -0800, [email protected] wrote:
> From: Seamus Kelly <[email protected]>
>
> Add xLink driver, which interfaces the xLink Core driver with the Keem
> Bay VPU IPC driver, thus enabling xLink to control and communicate
> with the VPU IP present on the Intel Keem Bay SoC.

Trivial style comments:

> diff --git a/drivers/misc/xlink-ipc/xlink-ipc.c
> b/drivers/misc/xlink-ipc/xlink-ipc.c

[]

> +/*
> + * xlink_reserved_memory_init() - Initialize reserved memory for the device.
> + *
> + * @xlink_dev: [in] The xlink ipc device the reserved memory is allocated to.
> + *
> + * Return: 0 on success, negative error code otherwise.
> + */
> +static int xlink_reserved_memory_init(struct xlink_ipc_dev
> +*xlink_dev) {
> + struct device *dev = &xlink_dev->pdev->dev;
> +
> + xlink_dev->local_xlink_mem.dev = init_xlink_reserved_mem_dev(dev,
> + "xlink_local_reserved",
> + LOCAL_XLINK_IPC_BUFFER_IDX);
> + if (!xlink_dev->local_xlink_mem.dev)
> + return -ENOMEM;

This sort of code, with a repeated struct dereference, generally reads better using a temporary and is also less prone to typo use.

struct device *dev = &xlink_dev->pdev->dev;
struct xlink_buf_mem *lxm = &xlink_dev->local_xlink_mem;

lxm->dev = init_xlink_reserved_mem_dev(dev, "xlink_local_reserved",
LOCAL_XLINK_IPC_BUFFER_IDX);
if (!lxm->dev)
return -ENOMEM;

> + xlink_dev->local_xlink_mem.size = get_xlink_reserved_mem_size(dev,
> + LOCAL_XLINK_IPC_BUFFER_IDX);

lxm->size = get_xlink_reserved_mem_size(dev, LOCAL_XLINK_IPC_BUFFER_IDX);

etc...

[Kelly, Seamus] Thank you! will do.



--------------------------------------------------------------
Intel Research and Development Ireland Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263


This e-mail and any attachments may contain confidential material for the sole
use of the intended recipient(s). Any review or distribution by others is
strictly prohibited. If you are not the intended recipient, please contact the
sender and delete all copies.

2020-12-11 14:06:11

by Greg Kroah-Hartman

[permalink] [raw]
Subject: Re: [PATCH 16/22] xlink-ipc: Add xlink ipc driver

On Fri, Dec 11, 2020 at 11:33:02AM +0000, Kelly, Seamus wrote:
> This e-mail and any attachments may contain confidential material for the sole
> use of the intended recipient(s). Any review or distribution by others is
> strictly prohibited. If you are not the intended recipient, please contact the
> sender and delete all copies.

Now deleted!

This footer is incompatible with Linux kernel development, please remove
it in order for us to read your emails and do anything with them.

I can't believe that Intel still has this problem, after all of these
years...

greg k-h

2020-12-11 18:59:43

by Gross, Mark

[permalink] [raw]
Subject: RE: [PATCH 16/22] xlink-ipc: Add xlink ipc driver



> -----Original Message-----
> From: [email protected] <[email protected]>
> Sent: Friday, December 11, 2020 4:14 AM
> To: Kelly, Seamus <[email protected]>
> Cc: Joe Perches <[email protected]>; [email protected];
> [email protected]; [email protected]; [email protected];
> [email protected]; [email protected]; [email protected];
> [email protected]; [email protected];
> [email protected]; [email protected]; [email protected];
> [email protected]; [email protected]; linux-
> [email protected]; Ryan Carnaghi <[email protected]>
> Subject: Re: [PATCH 16/22] xlink-ipc: Add xlink ipc driver
>
> On Fri, Dec 11, 2020 at 11:33:02AM +0000, Kelly, Seamus wrote:
> > This e-mail and any attachments may contain confidential material for
> > the sole use of the intended recipient(s). Any review or distribution
> > by others is strictly prohibited. If you are not the intended
> > recipient, please contact the sender and delete all copies.

Must be something Seamus is using that knows when emails go outside. I would have caught this on our internal mock reviews otherwise.

>
> Now deleted!
>
> This footer is incompatible with Linux kernel development, please remove it in
> order for us to read your emails and do anything with them.
Yes, I know.

>
> I can't believe that Intel still has this problem, after all of these years...
I'll add a warning about this in my training collateral and have all engineers I work with on this type of upstreaming send a test email to [email protected] so I can see if they have such legal clauses appended to out-bound email that is not visible to internal recipients. I'll fix this.

Sorry,

--mark