Changes since v2 [1]
- Rebased onto v5.11-rc2
- fixes for sparse (Ben)
- Add (and modify) IOCTL number to ioctl-number.rst. (Randy)
- Change @enable to @flags with enable field. (Dan)
- Move command "name" to UAPI enum. (Dan)
- Make command array use designated initializer. (Dan)
- Avoid copies to/from payload registers (Ben)
- Actually write the input payload length (Ben)
- Return ENOTTY instead of EINVAL for missing Send command. (Dan)
- Keep device locked and pinned during send commands. (Dan)
- Validate input payload size doesn't exceed hardware's limit. (Ben)
- Get rid of TAINT flag in UAPI. (Dan)
- Spelling fixes. (Dan)
- Document things that must be zero. (Dan)
- Use new TAINT (Dan)
- Replace cxl_debug with dev_debug (Dan)
- Squash debug info into relevant patches (Ben)
- Put CXL_CMD on a diet, #define the opcodes (Dan)
- Switch mailbox sync from spinlock to mutex (Ben)
- Store off (and debug print) hardware's payload size (Ben)
- Fix length of GENMASK for mailbox payload (Ben)
- Create concept of enabled commands. (Ben)
- Use CEL to enable commands based on hardware. (Ben)
- Move command struct definitions into mem.c (Ben)
- Create concept of hidden commands, and kernel only commands (Ben)
- Add payload min to sysfs (Ben)
---
The patches can be found here:
https://gitlab.com/bwidawsk/linux/-/commits/cxl-2.0v3
I've also created #cxl on oftc for any discussion.
---
Introduce support for “type-3” memory devices defined in the recently released
Compute Express Link (CXL) 2.0 specification[2]. Specifically, these are the
memory devices defined by section 8.2.8.5 of the CXL 2.0 spec. A reference
implementation emulating these devices has been submitted to the QEMU mailing
list [3] and is available on gitlab [4]. “Type-3” is a CXL device that acts as a
memory expander for RAM or PMEM. It might be interleaved with other CXL devices
in a given physical address range.
These changes allow for foundational enumeration of CXL 2.0 memory devices as
well as basic userspace interaction. The functionality present is:
- Initial driver bring-up
- Device enumeration and an initial sysfs representation
- Submit a basic firmware command via ‘mailbox’ to an emulated memory device
with non-volatile capacity.
- Provide an interface to send commands to the hardware.
Some of the functionality that is still missing includes:
- Memory interleaving at the host bridge, root port, or switch level
- CXL 1.1 Root Complex Integrated Endpoint Support
- CXL 2.0 Hot plug support
- A bevy of supported device commands
In addition to the core functionality of discovering the spec defined registers
and resources, introduce a CXL device model that will be the foundation for
translating CXL capabilities into existing Linux infrastructure for Persistent
Memory and other memory devices. For now, this only includes support for the
management command mailbox that type-3 devices surface. These control devices
fill the role of “DIMMs” / nmemX memory-devices in LIBNVDIMM terms.
Now, while implementing the driver some feedback for the specification was
generated to cover perceived gaps and address conflicts. The feedback is
presented as a reference implementation in the driver and QEMU emulation.
Specifically the following concepts are original to the Linux implementation and
feedback / collaboration is requested to develop these into specification
proposals:
1. Top level ACPI object (ACPI0017)
2. HW imposed address space and interleave constraints
ACPI0017
--------
Introduce a new ACPI namespace device with an _HID of ACPI0017. The purpose of
this object is twofold, support a legacy OS with a set of out-of-tree CXL
modules, and establish an attach point for a driver that knows about
interleaving. Both of these boil down to the same point, to centralize Operating
System support for resources described by the CXL Early Discovery Table (CEDT).
The legacy OS problem stems from the spec's description of a host bridge,
ACPI0016 is denoted as the _HID for host bridges, with a _CID of PNP0A08. In a
CXL unaware version of Linux, the core ACPI subsystem will bind a driver to
PNP0A08 and preclude a CXL-aware driver from binding to ACPI0016. An ACPI0017
device allows a standalone CXL-aware driver to register for handling /
coordinating CEDT and CXL-specific _OSC control.
Similarly when managing interleaving there needs to be some management layer
above the ACPI0016 device that is capable of assembling leaf nodes into
interleave sets. As is the case with ACPI0012 that does this central
coordination for NFIT defined resources, ACPI0017 does the same for CEDT
described resources.
Memory Windows
-------
For CXL.mem capable platforms, there is a need for a mechanism for platform
firmware to make the Operating System aware of any restrictions that hardware
might have in address space. For example, in a system with 4 host bridges all
participating in an interleave set, the firmware needs to provide some
description of this. That information is missing from the CXL 2.0 spec as of
today and it also is not implemented in the driver. A variety of ACPI based
mechanisms, for example _CRS fields on the ACPI0017 device, were considered.
Next steps after this basic foundation is expanded command support and LIBNVDIMM
integration. This is the initial “release early / release often” version of the
Linux CXL enabling.
[1]: https://lore.kernel.org/linux-cxl/[email protected]/
[2]: https://www.computeexpresslink.org/
[3]: https://lore.kernel.org/qemu-devel/[email protected]/T/#t
[4]: https://gitlab.com/bwidawsk/qemu/-/tree/cxl-2.0v2
Ben Widawsky (12):
docs: cxl: Add basic documentation
cxl/mem: Map memory device registers
cxl/mem: Find device capabilities
cxl/mem: Implement polled mode mailbox
cxl/mem: Add basic IOCTL interface
cxl/mem: Add send command
taint: add taint for direct hardware access
cxl/mem: Add a "RAW" send command
cxl/mem: Create concept of enabled commands
cxl/mem: Use CEL for enabling commands
cxl/mem: Add limited Get Log command (0401h)
MAINTAINERS: Add maintainers of the CXL driver
Dan Williams (2):
cxl/mem: Introduce a driver for CXL-2.0-Type-3 endpoints
cxl/mem: Register CXL memX devices
Vishal Verma (2):
cxl/acpi: Add an acpi_cxl module for the CXL interconnect
cxl/acpi: add OSC support
.clang-format | 1 +
Documentation/ABI/testing/sysfs-bus-cxl | 26 +
Documentation/admin-guide/sysctl/kernel.rst | 1 +
Documentation/admin-guide/tainted-kernels.rst | 6 +-
Documentation/cxl/index.rst | 12 +
Documentation/cxl/memory-devices.rst | 51 +
Documentation/index.rst | 1 +
.../userspace-api/ioctl/ioctl-number.rst | 1 +
MAINTAINERS | 10 +
drivers/Kconfig | 1 +
drivers/Makefile | 1 +
drivers/base/core.c | 14 +
drivers/cxl/Kconfig | 58 +
drivers/cxl/Makefile | 9 +
drivers/cxl/acpi.c | 351 ++++
drivers/cxl/acpi.h | 35 +
drivers/cxl/bus.c | 54 +
drivers/cxl/bus.h | 8 +
drivers/cxl/cxl.h | 147 ++
drivers/cxl/mem.c | 1475 +++++++++++++++++
drivers/cxl/pci.h | 34 +
include/acpi/actbl1.h | 50 +
include/linux/device.h | 1 +
include/linux/kernel.h | 3 +-
include/uapi/linux/cxl_mem.h | 168 ++
kernel/panic.c | 1 +
26 files changed, 2517 insertions(+), 2 deletions(-)
create mode 100644 Documentation/ABI/testing/sysfs-bus-cxl
create mode 100644 Documentation/cxl/index.rst
create mode 100644 Documentation/cxl/memory-devices.rst
create mode 100644 drivers/cxl/Kconfig
create mode 100644 drivers/cxl/Makefile
create mode 100644 drivers/cxl/acpi.c
create mode 100644 drivers/cxl/acpi.h
create mode 100644 drivers/cxl/bus.c
create mode 100644 drivers/cxl/bus.h
create mode 100644 drivers/cxl/cxl.h
create mode 100644 drivers/cxl/mem.c
create mode 100644 drivers/cxl/pci.h
create mode 100644 include/uapi/linux/cxl_mem.h
--
2.30.0
From: Vishal Verma <[email protected]>
Add support to advertise OS capabilities, and request OS control for CXL
features using the ACPI _OSC mechanism. Advertise support for all
possible CXL features, and attempt to request control for all possible
features.
Based on a patch by Sean Kelley.
Signed-off-by: Vishal Verma <[email protected]>
Signed-off-by: Ben Widawsky <[email protected]>
---
Documentation/cxl/memory-devices.rst | 15 ++
drivers/cxl/acpi.c | 260 ++++++++++++++++++++++++++-
drivers/cxl/acpi.h | 20 +++
3 files changed, 292 insertions(+), 3 deletions(-)
diff --git a/Documentation/cxl/memory-devices.rst b/Documentation/cxl/memory-devices.rst
index aa4262280c67..6ce88f9d5f4f 100644
--- a/Documentation/cxl/memory-devices.rst
+++ b/Documentation/cxl/memory-devices.rst
@@ -13,3 +13,18 @@ Driver Infrastructure
=====================
This sections covers the driver infrastructure for a CXL memory device.
+
+ACPI CXL
+--------
+
+.. kernel-doc:: drivers/cxl/acpi.c
+ :doc: cxl acpi
+
+.. kernel-doc:: drivers/cxl/acpi.c
+ :internal:
+
+External Interfaces
+===================
+
+.. kernel-doc:: drivers/cxl/acpi.c
+ :export:
diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c
index 0f1ba9b3f1ed..af9c0dfdee20 100644
--- a/drivers/cxl/acpi.c
+++ b/drivers/cxl/acpi.c
@@ -11,7 +11,258 @@
#include <linux/pci.h>
#include "acpi.h"
-/*
+/**
+ * DOC: cxl acpi
+ *
+ * ACPI _OSC setup: The exported function cxl_bus_acquire() sets up ACPI
+ * Operating System Capabilities (_OSC) for the CXL bus. It declares support
+ * for all CXL capabilities, and attempts to request control for all possible
+ * capabilities. The resulting support and control sets are saved in global
+ * variables cxl_osc_support_set and cxl_osc_control_set. The internal
+ * functions cxl_osc_declare_support(), and cxl_osc_request_control() can be
+ * used to update the support and control sets in accordance with the ACPI
+ * rules for _OSC evaluation - most importantly, capabilities already granted
+ * should not be rescinded by either the OS or firmware.
+ */
+
+static u32 cxl_osc_support_set;
+static u32 cxl_osc_control_set;
+static DEFINE_MUTEX(acpi_desc_lock);
+
+struct pci_osc_bit_struct {
+ u32 bit;
+ char *desc;
+};
+
+static struct pci_osc_bit_struct cxl_osc_support_bit[] = {
+ { CXL_OSC_PORT_REG_ACCESS_SUPPORT, "CXLPortRegAccess" },
+ { CXL_OSC_PORT_DEV_REG_ACCESS_SUPPORT, "CXLPortDevRegAccess" },
+ { CXL_OSC_PER_SUPPORT, "CXLProtocolErrorReporting" },
+ { CXL_OSC_NATIVE_HP_SUPPORT, "CXLNativeHotPlug" },
+};
+
+static struct pci_osc_bit_struct cxl_osc_control_bit[] = {
+ { CXL_OSC_MEM_ERROR_CONTROL, "CXLMemErrorReporting" },
+};
+
+static u8 cxl_osc_uuid_str[] = "68F2D50B-C469-4d8A-BD3D-941A103FD3FC";
+
+static void decode_osc_bits(struct device *dev, char *msg, u32 word,
+ struct pci_osc_bit_struct *table, int size)
+{
+ char buf[80];
+ int i, len = 0;
+ struct pci_osc_bit_struct *entry;
+
+ buf[0] = '\0';
+ for (i = 0, entry = table; i < size; i++, entry++)
+ if (word & entry->bit)
+ len += scnprintf(buf + len, sizeof(buf) - len, "%s%s",
+ len ? " " : "", entry->desc);
+
+ dev_info(dev, "_OSC: %s [%s]\n", msg, buf);
+}
+
+static void decode_cxl_osc_support(struct device *dev, char *msg, u32 word)
+{
+ decode_osc_bits(dev, msg, word, cxl_osc_support_bit,
+ ARRAY_SIZE(cxl_osc_support_bit));
+}
+
+static void decode_cxl_osc_control(struct device *dev, char *msg, u32 word)
+{
+ decode_osc_bits(dev, msg, word, cxl_osc_control_bit,
+ ARRAY_SIZE(cxl_osc_control_bit));
+}
+
+static acpi_status acpi_cap_run_osc(acpi_handle handle, const u32 *capbuf,
+ u8 *uuid_str, u32 *retval)
+{
+ struct acpi_osc_context context = {
+ .uuid_str = uuid_str,
+ .rev = 1,
+ .cap.length = 20,
+ .cap.pointer = (void *)capbuf,
+ };
+ acpi_status status;
+
+ status = acpi_run_osc(handle, &context);
+ if (ACPI_SUCCESS(status)) {
+ /* pointer + offset to DWORD 5 */
+ *retval = *((u32 *)(context.ret.pointer + 16));
+ kfree(context.ret.pointer);
+ }
+ return status;
+}
+
+static acpi_status cxl_query_osc(acpi_handle handle, u32 support, u32 *control)
+{
+ struct acpi_pci_root *root;
+ acpi_status status;
+ u32 result, capbuf[5];
+
+ root = acpi_pci_find_root(handle);
+ if (!root)
+ return -ENXIO;
+
+ support &= CXL_OSC_SUPPORT_VALID_MASK;
+ support |= cxl_osc_support_set;
+
+ capbuf[OSC_QUERY_DWORD] = OSC_QUERY_ENABLE;
+ capbuf[PCI_OSC_SUPPORT_DWORD] = root->osc_support_set;
+ capbuf[PCI_OSC_CONTROL_DWORD] = root->osc_control_set;
+ capbuf[CXL_OSC_SUPPORT_DWORD] = support;
+ if (control) {
+ *control &= CXL_OSC_CONTROL_VALID_MASK;
+ capbuf[CXL_OSC_CONTROL_DWORD] = *control | cxl_osc_control_set;
+ } else {
+ /* Run _OSC query only with existing controls. */
+ capbuf[CXL_OSC_CONTROL_DWORD] = cxl_osc_control_set;
+ }
+
+ status = acpi_cap_run_osc(handle, capbuf, cxl_osc_uuid_str, &result);
+ if (ACPI_SUCCESS(status)) {
+ cxl_osc_support_set = support;
+ if (control)
+ *control = result;
+ }
+ return status;
+}
+
+/**
+ * cxl_osc_declare_support - Declare support for CXL root _OSC features.
+ * @handle: ACPI device handle for the PCI root bridge (or PCIe Root Complex).
+ * @flags: The _OSC bits to declare support for.
+ *
+ * The PCI specific DWORDS are obtained from the pci root port device, and
+ * used as-is. The resulting CXL support set is saved in cxl_osc_support_set.
+ * Any future calls to this are forced to be strictly incremental from the
+ * existing cxl_osc_support_set.
+ */
+static acpi_status cxl_osc_declare_support(acpi_handle handle, u32 flags)
+{
+ acpi_status status;
+
+ mutex_lock(&acpi_desc_lock);
+ status = cxl_query_osc(handle, flags, NULL);
+ mutex_unlock(&acpi_desc_lock);
+ return status;
+}
+
+/**
+ * cxl_osc_request_control - Request control of CXL root _OSC features.
+ * @adev: ACPI device for the PCI root bridge (or PCIe Root Complex).
+ * @mask: Mask of _OSC bits to request control of, place to store control mask.
+ * @req: Mask of _OSC bits the control of is essential to the caller.
+ *
+ * The PCI specific DWORDS are obtained from the pci root port device, and
+ * used as-is. The resulting CXL control set is saved in cxl_osc_control_set.
+ * Any future calls to this are forced to be strictly incremental from the
+ * existing cxl_osc_control_set.
+ */
+static acpi_status cxl_osc_request_control(struct acpi_device *adev, u32 *mask,
+ u32 req)
+{
+ acpi_handle handle = adev->handle;
+ struct device *dev = &adev->dev;
+ struct acpi_pci_root *root;
+ acpi_status status = AE_OK;
+ u32 ctrl, capbuf[5];
+
+ if (!mask)
+ return AE_BAD_PARAMETER;
+
+ ctrl = *mask & CXL_OSC_MEM_ERROR_CONTROL;
+ if ((ctrl & req) != req)
+ return AE_TYPE;
+
+ root = acpi_pci_find_root(handle);
+ if (!root)
+ return -ENXIO;
+
+ mutex_lock(&acpi_desc_lock);
+
+ *mask = ctrl | cxl_osc_control_set;
+ /* No need to evaluate _OSC if the control was already granted. */
+ if ((cxl_osc_control_set & ctrl) == ctrl)
+ goto out;
+
+ /* Need to check the available controls bits before requesting them. */
+ while (*mask) {
+ status = cxl_query_osc(handle, cxl_osc_support_set, mask);
+ if (ACPI_FAILURE(status))
+ goto out;
+ if (ctrl == *mask)
+ break;
+ decode_cxl_osc_control(dev, "platform does not support",
+ ctrl & ~(*mask));
+ ctrl = *mask;
+ }
+
+ if ((ctrl & req) != req) {
+ decode_cxl_osc_control(dev,
+ "not requesting control; platform does not support",
+ req & ~(ctrl));
+ status = AE_SUPPORT;
+ goto out;
+ }
+
+ capbuf[OSC_QUERY_DWORD] = 0;
+ capbuf[PCI_OSC_SUPPORT_DWORD] = root->osc_support_set;
+ capbuf[PCI_OSC_CONTROL_DWORD] = root->osc_control_set;
+ capbuf[CXL_OSC_SUPPORT_DWORD] = cxl_osc_support_set;
+ capbuf[CXL_OSC_CONTROL_DWORD] = ctrl;
+ status = acpi_cap_run_osc(handle, capbuf, cxl_osc_uuid_str, mask);
+ if (ACPI_SUCCESS(status))
+ cxl_osc_control_set = *mask;
+out:
+ mutex_unlock(&acpi_desc_lock);
+ return status;
+}
+
+static int cxl_negotiate_osc(struct acpi_device *adev)
+{
+ u32 cxl_support, cxl_control, requested;
+ acpi_handle handle = adev->handle;
+ struct device *dev = &adev->dev;
+ acpi_status status;
+
+ /* Declare support for everything */
+ cxl_support = CXL_OSC_SUPPORT_VALID_MASK;
+ decode_cxl_osc_support(dev, "OS supports", cxl_support);
+ status = cxl_osc_declare_support(handle, cxl_support);
+ if (ACPI_FAILURE(status)) {
+ dev_info(dev, "CXL_OSC failed (%s)\n",
+ acpi_format_exception(status));
+ return -ENXIO;
+ }
+
+ /* Request control for everything */
+ cxl_control = CXL_OSC_CONTROL_VALID_MASK;
+ requested = cxl_control;
+ status = cxl_osc_request_control(adev, &cxl_control,
+ CXL_OSC_MEM_ERROR_CONTROL);
+ if (ACPI_SUCCESS(status)) {
+ decode_cxl_osc_control(dev, "OS now controls", cxl_control);
+ } else {
+ decode_cxl_osc_control(dev, "OS requested", requested);
+ decode_cxl_osc_control(dev, "platform willing to grant",
+ cxl_control);
+ dev_info(dev, "_OSC failed (%s)\n",
+ acpi_format_exception(status));
+ }
+ return 0;
+}
+
+/**
+ * cxl_bus_acquire - Perform platform-specific bus operations
+ * @pdev: pci_dev associated with the CXL device
+ *
+ * This performs bus-specific operations such as ACPI _OSC to ensure that
+ * the bus is 'prepared'. Since the CXL definition of _OSC depends on the
+ * existing PCI _OSC DWORDS in 'Arg3', pull those in from the pci root port
+ * device, and merge those into the new CXL-augmented _OSC calls.
+ *
* If/when CXL support is defined by other platform firmware the kernel
* will need a mechanism to select between the platform specific version
* of this routine, until then, hard-code ACPI assumptions
@@ -21,6 +272,7 @@ int cxl_bus_acquire(struct pci_dev *pdev)
struct acpi_device *adev;
struct pci_dev *root_port;
struct device *root;
+ int rc;
root_port = pcie_find_root_port(pdev);
if (!root_port)
@@ -34,9 +286,11 @@ int cxl_bus_acquire(struct pci_dev *pdev)
if (!adev)
return -ENXIO;
- /* TODO: OSC enabling */
+ rc = cxl_negotiate_osc(adev);
+ if (rc)
+ dev_err(&pdev->dev, "Failed to negotiate OSC\n");
- return 0;
+ return rc;
}
EXPORT_SYMBOL_GPL(cxl_bus_acquire);
diff --git a/drivers/cxl/acpi.h b/drivers/cxl/acpi.h
index d638f8886ab7..6ef154021745 100644
--- a/drivers/cxl/acpi.h
+++ b/drivers/cxl/acpi.h
@@ -12,4 +12,24 @@ struct cxl_acpi_desc {
int cxl_bus_acquire(struct pci_dev *pci_dev);
+/* Indexes into _OSC Capabilities Buffer */
+#define PCI_OSC_SUPPORT_DWORD 1 /* DWORD 2 */
+#define PCI_OSC_CONTROL_DWORD 2 /* DWORD 3 */
+#define CXL_OSC_SUPPORT_DWORD 3 /* DWORD 4 */
+#define CXL_OSC_CONTROL_DWORD 4 /* DWORD 5 */
+
+/* CXL Host Bridge _OSC: Capabilities DWORD 4: Support Field */
+#define CXL_OSC_PORT_REG_ACCESS_SUPPORT 0x00000001
+#define CXL_OSC_PORT_DEV_REG_ACCESS_SUPPORT 0x00000002
+#define CXL_OSC_PER_SUPPORT 0x00000004
+#define CXL_OSC_NATIVE_HP_SUPPORT 0x00000008
+#define CXL_OSC_SUPPORT_VALID_MASK (CXL_OSC_PORT_REG_ACCESS_SUPPORT | \
+ CXL_OSC_PORT_DEV_REG_ACCESS_SUPPORT | \
+ CXL_OSC_PER_SUPPORT | \
+ CXL_OSC_NATIVE_HP_SUPPORT)
+
+/* CXL Host Bridge _OSC: Capabilities DWORD 5: Control Field */
+#define CXL_OSC_MEM_ERROR_CONTROL 0x00000001
+#define CXL_OSC_CONTROL_VALID_MASK (CXL_OSC_MEM_ERROR_CONTROL)
+
#endif /* __CXL_ACPI_H__ */
--
2.30.0
This starts a new cxl directory because CXL is a new bus and it is
expected that the documentation will grow large over time. Currently
only memory devices are documented as they are all that's supported.
Signed-off-by: Ben Widawsky <[email protected]>
---
Documentation/cxl/index.rst | 12 ++++++++++++
Documentation/cxl/memory-devices.rst | 15 +++++++++++++++
Documentation/index.rst | 1 +
3 files changed, 28 insertions(+)
create mode 100644 Documentation/cxl/index.rst
create mode 100644 Documentation/cxl/memory-devices.rst
diff --git a/Documentation/cxl/index.rst b/Documentation/cxl/index.rst
new file mode 100644
index 000000000000..036e49553542
--- /dev/null
+++ b/Documentation/cxl/index.rst
@@ -0,0 +1,12 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+====================
+Compute Express Link
+====================
+
+.. toctree::
+ :maxdepth: 1
+
+ memory-devices
+
+.. only:: subproject and html
diff --git a/Documentation/cxl/memory-devices.rst b/Documentation/cxl/memory-devices.rst
new file mode 100644
index 000000000000..aa4262280c67
--- /dev/null
+++ b/Documentation/cxl/memory-devices.rst
@@ -0,0 +1,15 @@
+.. SPDX-License-Identifier: GPL-2.0
+.. include:: <isonum.txt>
+
+===================================
+Compute Express Link Memory Devices
+===================================
+
+A Compute Express Link Memory Device is a CXL component that implements the
+CXL.mem protocol. It contains some amount of volatile memory, persistent memory,
+or both.
+
+Driver Infrastructure
+=====================
+
+This sections covers the driver infrastructure for a CXL memory device.
diff --git a/Documentation/index.rst b/Documentation/index.rst
index 5888e8a7272f..7e5ab77264e9 100644
--- a/Documentation/index.rst
+++ b/Documentation/index.rst
@@ -137,6 +137,7 @@ needed).
misc-devices/index
scheduler/index
mhi/index
+ cxl/index
Architecture-agnostic documentation
-----------------------------------
--
2.30.0
From: Vishal Verma <[email protected]>
Add an acpi_cxl module to coordinate the ACPI portions of the CXL
(Compute eXpress Link) interconnect. This driver binds to ACPI0017
objects in the ACPI tree, and coordinates access to the resources
provided by the ACPI CEDT (CXL Early Discovery Table).
It also coordinates operations of the root port _OSC object to notify
platform firmware that the OS has native support for the CXL
capabilities of endpoints.
Note: the actbl1.h changes are speculative. The expectation is that they
will arrive through the ACPICA tree in due time.
Cc: Ben Widawsky <[email protected]>
Cc: Dan Williams <[email protected]>
Signed-off-by: Vishal Verma <[email protected]>
Signed-off-by: Ben Widawsky <[email protected]>
---
drivers/Kconfig | 1 +
drivers/Makefile | 1 +
drivers/cxl/Kconfig | 36 ++++++++++++++++
drivers/cxl/Makefile | 5 +++
drivers/cxl/acpi.c | 97 +++++++++++++++++++++++++++++++++++++++++++
drivers/cxl/acpi.h | 15 +++++++
include/acpi/actbl1.h | 50 ++++++++++++++++++++++
7 files changed, 205 insertions(+)
create mode 100644 drivers/cxl/Kconfig
create mode 100644 drivers/cxl/Makefile
create mode 100644 drivers/cxl/acpi.c
create mode 100644 drivers/cxl/acpi.h
diff --git a/drivers/Kconfig b/drivers/Kconfig
index dcecc9f6e33f..62c753a73651 100644
--- a/drivers/Kconfig
+++ b/drivers/Kconfig
@@ -6,6 +6,7 @@ menu "Device Drivers"
source "drivers/amba/Kconfig"
source "drivers/eisa/Kconfig"
source "drivers/pci/Kconfig"
+source "drivers/cxl/Kconfig"
source "drivers/pcmcia/Kconfig"
source "drivers/rapidio/Kconfig"
diff --git a/drivers/Makefile b/drivers/Makefile
index fd11b9ac4cc3..678ea810410f 100644
--- a/drivers/Makefile
+++ b/drivers/Makefile
@@ -73,6 +73,7 @@ obj-$(CONFIG_NVM) += lightnvm/
obj-y += base/ block/ misc/ mfd/ nfc/
obj-$(CONFIG_LIBNVDIMM) += nvdimm/
obj-$(CONFIG_DAX) += dax/
+obj-$(CONFIG_CXL_BUS) += cxl/
obj-$(CONFIG_DMA_SHARED_BUFFER) += dma-buf/
obj-$(CONFIG_NUBUS) += nubus/
obj-y += macintosh/
diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig
new file mode 100644
index 000000000000..68da926ba5b1
--- /dev/null
+++ b/drivers/cxl/Kconfig
@@ -0,0 +1,36 @@
+# SPDX-License-Identifier: GPL-2.0-only
+menuconfig CXL_BUS
+ tristate "CXL (Compute Express Link) Devices Support"
+ help
+ CXL is a bus that is electrically compatible with PCI-E, but layers
+ three protocols on that signalling (CXL.io, CXL.cache, and CXL.mem).
+ The CXL.cache protocol allows devices to hold cachelines locally, the
+ CXL.mem protocol allows devices to be fully coherent memory targets,
+ the CXL.io protocol is equivalent to PCI-E. Say 'y' to enable support
+ for the configuration and management of devices supporting these
+ protocols.
+
+if CXL_BUS
+
+config CXL_BUS_PROVIDER
+ tristate
+
+config CXL_ACPI
+ tristate "CXL ACPI: Platform Support"
+ depends on ACPI
+ default CXL_BUS
+ select CXL_BUS_PROVIDER
+ help
+ Say 'y/m' to enable a driver (named "cxl_acpi.ko" when built
+ as a module) that will enable support for CXL.mem endpoint
+ devices. In general, CXL Platform Support is a prerequisite
+ for any CXL device driver that wants to claim ownership of a
+ component register space. By default platform firmware assumes
+ Linux is unaware of CXL capabilities and requires explicit
+ opt-in. This platform component also mediates resources
+ described by the CEDT (CXL Early Discovery Table). See
+ Chapter 9.14.1 CXL Early Discovery Table (CEDT) in the CXL 2.0
+ specification.
+
+ If unsure say 'm'
+endif
diff --git a/drivers/cxl/Makefile b/drivers/cxl/Makefile
new file mode 100644
index 000000000000..d38cd34a2582
--- /dev/null
+++ b/drivers/cxl/Makefile
@@ -0,0 +1,5 @@
+# SPDX-License-Identifier: GPL-2.0
+obj-$(CONFIG_CXL_ACPI) += cxl_acpi.o
+
+ccflags-y += -DDEFAULT_SYMBOL_NAMESPACE=CXL
+cxl_acpi-y := acpi.o
diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c
new file mode 100644
index 000000000000..0f1ba9b3f1ed
--- /dev/null
+++ b/drivers/cxl/acpi.c
@@ -0,0 +1,97 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright(c) 2020 Intel Corporation. All rights reserved. */
+#include <linux/platform_device.h>
+#include <linux/list_sort.h>
+#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/sysfs.h>
+#include <linux/list.h>
+#include <linux/acpi.h>
+#include <linux/sort.h>
+#include <linux/pci.h>
+#include "acpi.h"
+
+/*
+ * If/when CXL support is defined by other platform firmware the kernel
+ * will need a mechanism to select between the platform specific version
+ * of this routine, until then, hard-code ACPI assumptions
+ */
+int cxl_bus_acquire(struct pci_dev *pdev)
+{
+ struct acpi_device *adev;
+ struct pci_dev *root_port;
+ struct device *root;
+
+ root_port = pcie_find_root_port(pdev);
+ if (!root_port)
+ return -ENXIO;
+
+ root = root_port->dev.parent;
+ if (!root)
+ return -ENXIO;
+
+ adev = ACPI_COMPANION(root);
+ if (!adev)
+ return -ENXIO;
+
+ /* TODO: OSC enabling */
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(cxl_bus_acquire);
+
+static void acpi_cedt_put_table(void *table)
+{
+ acpi_put_table(table);
+}
+
+static int cxl_acpi_probe(struct platform_device *pdev)
+{
+ struct acpi_device *adev = ACPI_COMPANION(&pdev->dev);
+ struct cxl_acpi_desc *acpi_desc;
+ struct device *dev = &adev->dev;
+ struct acpi_table_header *tbl;
+ acpi_status status;
+ acpi_size sz;
+ int rc;
+
+ status = acpi_get_table(ACPI_SIG_CEDT, 0, &tbl);
+ if (ACPI_FAILURE(status)) {
+ dev_err(dev, "failed to find CEDT at startup\n");
+ return 0;
+ }
+
+ rc = devm_add_action_or_reset(dev, acpi_cedt_put_table, tbl);
+ if (rc)
+ return rc;
+
+ sz = tbl->length;
+ dev_info(dev, "found CEDT at startup: %lld bytes\n", sz);
+
+ acpi_desc = devm_kzalloc(dev, sizeof(*acpi_desc), GFP_KERNEL);
+ if (!acpi_desc)
+ return -ENOMEM;
+
+ dev_set_drvdata(&adev->dev, acpi_desc);
+ acpi_desc->dev = &adev->dev;
+ acpi_desc->acpi_header = *tbl;
+
+ return 0;
+}
+
+static const struct acpi_device_id cxl_acpi_ids[] = {
+ { "ACPI0017", 0 },
+ { "", 0 },
+};
+MODULE_DEVICE_TABLE(acpi, cxl_acpi_ids);
+
+static struct platform_driver cxl_acpi_driver = {
+ .probe = cxl_acpi_probe,
+ .driver = {
+ .name = KBUILD_MODNAME,
+ .acpi_match_table = cxl_acpi_ids,
+ },
+};
+
+module_platform_driver(cxl_acpi_driver);
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/cxl/acpi.h b/drivers/cxl/acpi.h
new file mode 100644
index 000000000000..d638f8886ab7
--- /dev/null
+++ b/drivers/cxl/acpi.h
@@ -0,0 +1,15 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright(c) 2020 Intel Corporation. All rights reserved. */
+
+#ifndef __CXL_ACPI_H__
+#define __CXL_ACPI_H__
+#include <linux/acpi.h>
+
+struct cxl_acpi_desc {
+ struct acpi_table_header acpi_header;
+ struct device *dev;
+};
+
+int cxl_bus_acquire(struct pci_dev *pci_dev);
+
+#endif /* __CXL_ACPI_H__ */
diff --git a/include/acpi/actbl1.h b/include/acpi/actbl1.h
index 43549547ed3e..f3a0303984c8 100644
--- a/include/acpi/actbl1.h
+++ b/include/acpi/actbl1.h
@@ -28,6 +28,7 @@
#define ACPI_SIG_BERT "BERT" /* Boot Error Record Table */
#define ACPI_SIG_BGRT "BGRT" /* Boot Graphics Resource Table */
#define ACPI_SIG_BOOT "BOOT" /* Simple Boot Flag Table */
+#define ACPI_SIG_CEDT "CEDT" /* CXL Early Discovery Table */
#define ACPI_SIG_CPEP "CPEP" /* Corrected Platform Error Polling table */
#define ACPI_SIG_CSRT "CSRT" /* Core System Resource Table */
#define ACPI_SIG_DBG2 "DBG2" /* Debug Port table type 2 */
@@ -1624,6 +1625,55 @@ struct acpi_ibft_target {
u16 reverse_chap_secret_offset;
};
+/*******************************************************************************
+ *
+ * CEDT - CXL Early Discovery Table (ACPI 6.4)
+ * Version 1
+ *
+ ******************************************************************************/
+
+struct acpi_table_cedt {
+ struct acpi_table_header header; /* Common ACPI table header */
+ u32 reserved;
+};
+
+/* Values for CEDT structure types */
+
+enum acpi_cedt_type {
+ ACPI_CEDT_TYPE_HOST_BRIDGE = 0, /* CHBS - CXL Host Bridge Structure */
+};
+
+struct acpi_cedt_structure {
+ u8 type;
+ u8 reserved;
+ u16 length;
+};
+
+/*
+ * CEDT Structures, correspond to Type in struct acpi_cedt_structure
+ */
+
+/* 0: CXL Host Bridge Structure */
+
+struct acpi_cedt_chbs {
+ struct acpi_cedt_structure header;
+ u32 uid;
+ u32 version;
+ u32 reserved1;
+ u64 base;
+ u64 length;
+};
+
+/* Values for version field above */
+
+#define ACPI_CEDT_CHBS_VERSION_CXL11 (0)
+#define ACPI_CEDT_CHBS_VERSION_CXL20 (1)
+
+/* Values for length field above */
+
+#define ACPI_CEDT_CHBS_LENGTH_CXL11 (0x2000)
+#define ACPI_CEDT_CHBS_LENGTH_CXL20 (0x10000)
+
/* Reset to default packing */
#pragma pack()
--
2.30.0
Provide enough functionality to utilize the mailbox of a memory device.
The mailbox is used to interact with the firmware running on the memory
device.
The CXL specification defines separate capabilities for the mailbox and
the memory device. While we can confirm the mailbox is ready, in order
to actually interact with the memory device, you must also confirm the
device's firmware is ready.
Create a function to handle sending a command, optionally with a
payload, to the memory device, polling on a result, and then optionally
copying out the payload. The algorithm for doing this comes straight out
of the CXL 2.0 specification.
Primary mailboxes are capable of generating an interrupt when submitting
a command in the background. That implementation is saved for a later
time.
Secondary mailboxes aren't implemented at this time.
The flow is proven with one implemented command, "identify". Because the
class code has already told the driver this is a memory device and the
identify command is mandatory, it's safe to assume for sane devices that
everything here will work.
Signed-off-by: Ben Widawsky <[email protected]>
---
drivers/cxl/cxl.h | 43 +++++++
drivers/cxl/mem.c | 312 ++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 355 insertions(+)
diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
index a77286d04ce4..ca3fa496e21c 100644
--- a/drivers/cxl/cxl.h
+++ b/drivers/cxl/cxl.h
@@ -32,9 +32,40 @@
#define CXLDEV_MB_CAP_PAYLOAD_SIZE_MASK GENMASK(4, 0)
#define CXLDEV_MB_CAP_PAYLOAD_SIZE_SHIFT 0
#define CXLDEV_MB_CTRL_OFFSET 0x04
+#define CXLDEV_MB_CTRL_DOORBELL BIT(0)
#define CXLDEV_MB_CMD_OFFSET 0x08
+#define CXLDEV_MB_CMD_COMMAND_OPCODE_SHIFT 0
+#define CXLDEV_MB_CMD_COMMAND_OPCODE_MASK GENMASK(15, 0)
+#define CXLDEV_MB_CMD_PAYLOAD_LENGTH_SHIFT 16
+#define CXLDEV_MB_CMD_PAYLOAD_LENGTH_MASK GENMASK(36, 16)
#define CXLDEV_MB_STATUS_OFFSET 0x10
+#define CXLDEV_MB_STATUS_RET_CODE_SHIFT 32
+#define CXLDEV_MB_STATUS_RET_CODE_MASK GENMASK(47, 32)
#define CXLDEV_MB_BG_CMD_STATUS_OFFSET 0x18
+#define CXLDEV_MB_PAYLOAD_OFFSET 0x20
+
+/* Memory Device (CXL 2.0 - 8.2.8.5.1.1) */
+#define CXLMDEV_STATUS_OFFSET 0x0
+#define CXLMDEV_DEV_FATAL BIT(0)
+#define CXLMDEV_FW_HALT BIT(1)
+#define CXLMDEV_STATUS_MEDIA_STATUS_SHIFT 2
+#define CXLMDEV_STATUS_MEDIA_STATUS_MASK GENMASK(3, 2)
+#define CXLMDEV_MS_NOT_READY 0
+#define CXLMDEV_MS_READY 1
+#define CXLMDEV_MS_ERROR 2
+#define CXLMDEV_MS_DISABLED 3
+#define CXLMDEV_READY(status) \
+ (CXL_GET_FIELD(status, CXLMDEV_STATUS_MEDIA_STATUS) == CXLMDEV_MS_READY)
+#define CXLMDEV_MBOX_IF_READY BIT(4)
+#define CXLMDEV_RESET_NEEDED_SHIFT 5
+#define CXLMDEV_RESET_NEEDED_MASK GENMASK(7, 5)
+#define CXLMDEV_RESET_NEEDED_NOT 0
+#define CXLMDEV_RESET_NEEDED_COLD 1
+#define CXLMDEV_RESET_NEEDED_WARM 2
+#define CXLMDEV_RESET_NEEDED_HOT 3
+#define CXLMDEV_RESET_NEEDED_CXL 4
+#define CXLMDEV_RESET_NEEDED(status) \
+ (CXL_GET_FIELD(status, CXLMDEV_RESET_NEEDED) != CXLMDEV_RESET_NEEDED_NOT)
/**
* struct cxl_mem - A CXL memory device
@@ -45,6 +76,16 @@ struct cxl_mem {
struct pci_dev *pdev;
void __iomem *regs;
+ struct {
+ struct range range;
+ } pmem;
+
+ struct {
+ struct range range;
+ } ram;
+
+ char firmware_version[0x10];
+
/* Cap 0001h - CXL_CAP_CAP_ID_DEVICE_STATUS */
struct {
void __iomem *regs;
@@ -52,6 +93,7 @@ struct cxl_mem {
/* Cap 0002h - CXL_CAP_CAP_ID_PRIMARY_MAILBOX */
struct {
+ struct mutex mutex; /* Protects device mailbox and firmware */
void __iomem *regs;
size_t payload_size;
} mbox;
@@ -90,6 +132,7 @@ struct cxl_mem {
cxl_reg(status);
cxl_reg(mbox);
+cxl_reg(mem);
#define cxl_payload_regs(cxlm) \
((void __iomem *)(cxlm)->mbox.regs + CXLDEV_MB_PAYLOAD_OFFSET)
diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
index 8da9f4a861ea..e9ba97bbd7b9 100644
--- a/drivers/cxl/mem.c
+++ b/drivers/cxl/mem.c
@@ -1,5 +1,6 @@
// SPDX-License-Identifier: GPL-2.0-only
/* Copyright(c) 2020 Intel Corporation. All rights reserved. */
+#include <linux/sched/clock.h>
#include <linux/module.h>
#include <linux/pci.h>
#include <linux/io.h>
@@ -7,6 +8,248 @@
#include "pci.h"
#include "cxl.h"
+#define cxl_doorbell_busy(cxlm) \
+ (cxl_read_mbox_reg32(cxlm, CXLDEV_MB_CTRL_OFFSET) & \
+ CXLDEV_MB_CTRL_DOORBELL)
+
+#define CXL_MAILBOX_TIMEOUT_US 2000
+
+enum opcode {
+ CXL_MBOX_OP_IDENTIFY = 0x4000,
+ CXL_MBOX_OP_MAX = 0x10000
+};
+
+/**
+ * struct mbox_cmd - A command to be submitted to hardware.
+ * @opcode: (input) The command set and command submitted to hardware.
+ * @payload: (input/output) Pointer to the input and output payload.
+ * Payload can be NULL if the caller wants to populate the payload
+ * registers themselves (potentially avoiding a copy).
+ * @size_in: (input) Number of bytes to load from @payload.
+ * @size_out:
+ * - (input) Number of bytes allocated to load into @payload.
+ * - (output) Number of bytes loaded into @payload.
+ * @return_code: (output) Error code returned from hardware.
+ *
+ * This is the primary mechanism used to send commands to the hardware.
+ * All the fields except @payload correspond exactly to the fields described in
+ * Command Register section of the CXL 2.0 spec (8.2.8.4.5). @payload
+ * corresponds to the Command Payload Registers (8.2.8.4.8).
+ */
+struct mbox_cmd {
+ u16 opcode;
+ void *payload;
+ size_t size_in;
+ size_t size_out;
+ u16 return_code;
+};
+
+static int cxl_mem_wait_for_doorbell(struct cxl_mem *cxlm)
+{
+ const int timeout = msecs_to_jiffies(CXL_MAILBOX_TIMEOUT_US);
+ const unsigned long start = jiffies;
+ unsigned long end = start;
+
+ while (cxl_doorbell_busy(cxlm)) {
+ end = jiffies;
+
+ if (time_after(end, start + timeout)) {
+ /* Check again in case preempted before timeout test */
+ if (!cxl_doorbell_busy(cxlm))
+ break;
+ return -ETIMEDOUT;
+ }
+ cpu_relax();
+ }
+
+ dev_dbg(&cxlm->pdev->dev, "Doorbell wait took %dms",
+ jiffies_to_msecs(end) - jiffies_to_msecs(start));
+ return 0;
+}
+
+static void cxl_mem_mbox_timeout(struct cxl_mem *cxlm,
+ struct mbox_cmd *mbox_cmd)
+{
+ dev_warn(&cxlm->pdev->dev, "Mailbox command timed out\n");
+ dev_info(&cxlm->pdev->dev,
+ "\topcode: 0x%04x\n"
+ "\tpayload size: %zub\n",
+ mbox_cmd->opcode, mbox_cmd->size_in);
+ print_hex_dump_debug("Payload ", DUMP_PREFIX_OFFSET, 16, 1,
+ mbox_cmd->payload, mbox_cmd->size_in, true);
+
+ /* Here's a good place to figure out if a device reset is needed */
+}
+
+/**
+ * cxl_mem_mbox_send_cmd() - Send a mailbox command to a memory device.
+ * @cxlm: The CXL memory device to communicate with.
+ * @mbox_cmd: Command to send to the memory device.
+ *
+ * Context: Any context. Expects mbox_lock to be held.
+ * Return: -ETIMEDOUT if timeout occurred waiting for completion. 0 on success.
+ * Caller should check the return code in @mbox_cmd to make sure it
+ * succeeded.
+ *
+ * This is a generic form of the CXL mailbox send command, thus the only I/O
+ * operations used are cxl_read_mbox_reg(). Memory devices, and perhaps other
+ * types of CXL devices may have further information available upon error
+ * conditions.
+ *
+ * FIXME: As stated above, references to &struct cxl_mem should be changed to a
+ * more generic cxl structure when needed.
+ */
+static int cxl_mem_mbox_send_cmd(struct cxl_mem *cxlm,
+ struct mbox_cmd *mbox_cmd)
+{
+ u64 cmd_reg, status_reg;
+ size_t out_len;
+ int rc;
+
+ lockdep_assert_held(&cxlm->mbox.mutex);
+
+ /*
+ * Here are the steps from 8.2.8.4 of the CXL 2.0 spec.
+ * 1. Caller reads MB Control Register to verify doorbell is clear
+ * 2. Caller writes Command Register
+ * 3. Caller writes Command Payload Registers if input payload is non-empty
+ * 4. Caller writes MB Control Register to set doorbell
+ * 5. Caller either polls for doorbell to be clear or waits for interrupt if configured
+ * 6. Caller reads MB Status Register to fetch Return code
+ * 7. If command successful, Caller reads Command Register to get Payload Length
+ * 8. If output payload is non-empty, host reads Command Payload Registers
+ */
+
+ /* #1 */
+ WARN_ON(cxl_doorbell_busy(cxlm));
+
+ cmd_reg = CXL_SET_FIELD(mbox_cmd->opcode, CXLDEV_MB_CMD_COMMAND_OPCODE);
+ if (mbox_cmd->size_in) {
+ cmd_reg |= CXL_SET_FIELD(mbox_cmd->size_in,
+ CXLDEV_MB_CMD_PAYLOAD_LENGTH);
+ if (mbox_cmd->payload)
+ memcpy_toio(cxl_payload_regs(cxlm), mbox_cmd->payload,
+ mbox_cmd->size_in);
+ }
+
+ /* #2, #3 */
+ cxl_write_mbox_reg64(cxlm, CXLDEV_MB_CMD_OFFSET, cmd_reg);
+
+ /* #4 */
+ dev_dbg(&cxlm->pdev->dev, "Sending command\n");
+ cxl_write_mbox_reg32(cxlm, CXLDEV_MB_CTRL_OFFSET,
+ CXLDEV_MB_CTRL_DOORBELL);
+
+ /* #5 */
+ rc = cxl_mem_wait_for_doorbell(cxlm);
+ if (rc == -ETIMEDOUT) {
+ cxl_mem_mbox_timeout(cxlm, mbox_cmd);
+ return rc;
+ }
+
+ /* #6 */
+ status_reg = cxl_read_mbox_reg64(cxlm, CXLDEV_MB_STATUS_OFFSET);
+ mbox_cmd->return_code =
+ CXL_GET_FIELD(status_reg, CXLDEV_MB_STATUS_RET_CODE);
+
+ if (mbox_cmd->return_code != 0) {
+ dev_dbg(&cxlm->pdev->dev, "Mailbox operation had an error\n");
+ return 0;
+ }
+
+ /* #7 */
+ cmd_reg = cxl_read_mbox_reg64(cxlm, CXLDEV_MB_CMD_OFFSET);
+ out_len = CXL_GET_FIELD(cmd_reg, CXLDEV_MB_CMD_PAYLOAD_LENGTH);
+ mbox_cmd->size_out = out_len;
+
+ /* #8 */
+ if (out_len && mbox_cmd->payload)
+ memcpy_fromio(mbox_cmd->payload, cxl_payload_regs(cxlm),
+ mbox_cmd->size_out);
+
+ return 0;
+}
+
+/**
+ * cxl_mem_mbox_get() - Acquire exclusive access to the mailbox.
+ * @cxlm: The memory device to gain access to.
+ *
+ * Context: Any context. Takes the mbox_lock.
+ * Return: 0 if exclusive access was acquired.
+ */
+static int cxl_mem_mbox_get(struct cxl_mem *cxlm)
+{
+ u64 md_status;
+ int rc = -EBUSY;
+
+ mutex_lock_io(&cxlm->mbox.mutex);
+
+ /*
+ * XXX: There is some amount of ambiguity in the 2.0 version of the spec
+ * around the mailbox interface ready (8.2.8.5.1.1). The purpose of the
+ * bit is to allow firmware running on the device to notify us that it's
+ * ready to receive commands. It is unclear if the bit needs to be read
+ * every time one tries to use the mailbox, ie. the firmware can switch
+ * it on and off as needed. Second, there is no defined timeout for
+ * mailbox ready, like there is for the doorbell interface.
+ *
+ * As such, we make the following assumptions:
+ * 1. The firmware might toggle the Mailbox Interface Ready bit, and so
+ * we check it for every command.
+ * 2. If the doorbell is clear, the firmware should have first set the
+ * Mailbox Interface Ready bit. Therefore, waiting for the doorbell
+ * to be ready is a sufficient amount of time.
+ */
+ rc = cxl_mem_wait_for_doorbell(cxlm);
+ if (rc) {
+ dev_warn(&cxlm->pdev->dev, "Mailbox interface not ready\n");
+ goto out;
+ }
+
+ md_status = cxl_read_mem_reg64(cxlm, CXLMDEV_STATUS_OFFSET);
+ if (md_status & CXLMDEV_MBOX_IF_READY && CXLMDEV_READY(md_status)) {
+ /*
+ * Hardware shouldn't allow a ready status but also have failure
+ * bits set. Spit out an error, this should be a bug report
+ */
+ if (md_status & CXLMDEV_DEV_FATAL) {
+ dev_err(&cxlm->pdev->dev,
+ "CXL device reporting ready and fatal\n");
+ rc = -EFAULT;
+ goto out;
+ }
+ if (md_status & CXLMDEV_FW_HALT) {
+ dev_err(&cxlm->pdev->dev,
+ "CXL device reporting ready and halted\n");
+ rc = -EFAULT;
+ goto out;
+ }
+ if (CXLMDEV_RESET_NEEDED(md_status)) {
+ dev_err(&cxlm->pdev->dev,
+ "CXL device reporting ready and reset needed\n");
+ rc = -EFAULT;
+ goto out;
+ }
+
+ return 0;
+ }
+
+out:
+ mutex_unlock(&cxlm->mbox.mutex);
+ return rc;
+}
+
+/**
+ * cxl_mem_mbox_put() - Release exclusive access to the mailbox.
+ * @cxlm: The CXL memory device to communicate with.
+ *
+ * Context: Any context. Expects mbox_lock to be held.
+ */
+static void cxl_mem_mbox_put(struct cxl_mem *cxlm)
+{
+ mutex_unlock(&cxlm->mbox.mutex);
+}
+
/**
* cxl_mem_setup_regs() - Setup necessary MMIO.
* @cxlm: The CXL memory device to communicate with.
@@ -135,6 +378,8 @@ static struct cxl_mem *cxl_mem_create(struct pci_dev *pdev, u32 reg_lo,
return NULL;
}
+ mutex_init(&cxlm->mbox.mutex);
+
regs = pcim_iomap_table(pdev)[bar];
cxlm->pdev = pdev;
cxlm->regs = regs + offset;
@@ -167,6 +412,69 @@ static int cxl_mem_dvsec(struct pci_dev *pdev, int dvsec)
return 0;
}
+/**
+ * cxl_mem_identify() - Send the IDENTIFY command to the device.
+ * @cxlm: The device to identify.
+ *
+ * Return: 0 if identify was executed successfully.
+ *
+ * This will dispatch the identify command to the device and on success populate
+ * structures to be exported to sysfs.
+ */
+static int cxl_mem_identify(struct cxl_mem *cxlm)
+{
+ struct cxl_mbox_identify {
+ char fw_revision[0x10];
+ __le64 total_capacity;
+ __le64 volatile_capacity;
+ __le64 persistent_capacity;
+ __le64 partition_align;
+ __le16 info_event_log_size;
+ __le16 warning_event_log_size;
+ __le16 failure_event_log_size;
+ __le16 fatal_event_log_size;
+ __le32 lsa_size;
+ u8 poison_list_max_mer[3];
+ __le16 inject_poison_limit;
+ u8 poison_caps;
+ u8 qos_telemetry_caps;
+ } __packed id;
+ struct mbox_cmd mbox_cmd;
+ int rc;
+
+ /* Retrieve initial device memory map */
+ rc = cxl_mem_mbox_get(cxlm);
+ if (rc)
+ return rc;
+
+ mbox_cmd = (struct mbox_cmd){
+ .opcode = CXL_MBOX_OP_IDENTIFY,
+ .payload = &id,
+ .size_in = 0,
+ };
+ rc = cxl_mem_mbox_send_cmd(cxlm, &mbox_cmd);
+ cxl_mem_mbox_put(cxlm);
+ if (rc)
+ return rc;
+
+ if (mbox_cmd.size_out != sizeof(id))
+ return -ENXIO;
+
+ /*
+ * TODO: enumerate DPA map, as 'ram' and 'pmem' do not alias.
+ * For now, only the capacity is exported in sysfs
+ */
+ cxlm->ram.range.start = 0;
+ cxlm->ram.range.end = le64_to_cpu(id.volatile_capacity) - 1;
+
+ cxlm->pmem.range.start = 0;
+ cxlm->pmem.range.end = le64_to_cpu(id.persistent_capacity) - 1;
+
+ memcpy(cxlm->firmware_version, id.fw_revision, sizeof(id.fw_revision));
+
+ return rc;
+}
+
static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id)
{
struct device *dev = &pdev->dev;
@@ -222,6 +530,10 @@ static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id)
if (rc)
goto err;
+ rc = cxl_mem_identify(cxlm);
+ if (rc)
+ goto err;
+
pci_set_drvdata(pdev, cxlm);
return 0;
--
2.30.0
Make notes about why we use this.
Signed-off-by: Ben Widawsky <[email protected]>
---
Documentation/admin-guide/sysctl/kernel.rst | 1 +
Documentation/admin-guide/tainted-kernels.rst | 6 +++++-
include/linux/kernel.h | 3 ++-
kernel/panic.c | 1 +
4 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/Documentation/admin-guide/sysctl/kernel.rst b/Documentation/admin-guide/sysctl/kernel.rst
index 1d56a6b73a4e..3e1eada53504 100644
--- a/Documentation/admin-guide/sysctl/kernel.rst
+++ b/Documentation/admin-guide/sysctl/kernel.rst
@@ -1352,6 +1352,7 @@ ORed together. The letters are seen in "Tainted" line of Oops reports.
32768 `(K)` kernel has been live patched
65536 `(X)` Auxiliary taint, defined and used by for distros
131072 `(T)` The kernel was built with the struct randomization plugin
+262144 `(H)` The kernel has allowed vendor shenanigans
====== ===== ==============================================================
See :doc:`/admin-guide/tainted-kernels` for more information.
diff --git a/Documentation/admin-guide/tainted-kernels.rst b/Documentation/admin-guide/tainted-kernels.rst
index ceeed7b0798d..ee2913316344 100644
--- a/Documentation/admin-guide/tainted-kernels.rst
+++ b/Documentation/admin-guide/tainted-kernels.rst
@@ -74,7 +74,7 @@ a particular type of taint. It's best to leave that to the aforementioned
script, but if you need something quick you can use this shell command to check
which bits are set::
- $ for i in $(seq 18); do echo $(($i-1)) $(($(cat /proc/sys/kernel/tainted)>>($i-1)&1));done
+ $ for i in $(seq 19); do echo $(($i-1)) $(($(cat /proc/sys/kernel/tainted)>>($i-1)&1));done
Table for decoding tainted state
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -100,6 +100,7 @@ Bit Log Number Reason that got the kernel tainted
15 _/K 32768 kernel has been live patched
16 _/X 65536 auxiliary taint, defined for and used by distros
17 _/T 131072 kernel was built with the struct randomization plugin
+ 18 _/H 262144 kernel has allowed vendor shenanigans
=== === ====== ========================================================
Note: The character ``_`` is representing a blank in this table to make reading
@@ -175,3 +176,6 @@ More detailed explanation for tainting
produce extremely unusual kernel structure layouts (even performance
pathological ones), which is important to know when debugging. Set at
build time.
+
+ 18) ``H`` Kernel has allowed direct access to hardware and can no longer make
+ any guarantees about the stability of the device or driver.
diff --git a/include/linux/kernel.h b/include/linux/kernel.h
index f7902d8c1048..bc95486f817e 100644
--- a/include/linux/kernel.h
+++ b/include/linux/kernel.h
@@ -443,7 +443,8 @@ extern enum system_states {
#define TAINT_LIVEPATCH 15
#define TAINT_AUX 16
#define TAINT_RANDSTRUCT 17
-#define TAINT_FLAGS_COUNT 18
+#define TAINT_RAW_PASSTHROUGH 18
+#define TAINT_FLAGS_COUNT 19
#define TAINT_FLAGS_MAX ((1UL << TAINT_FLAGS_COUNT) - 1)
struct taint_flag {
diff --git a/kernel/panic.c b/kernel/panic.c
index 332736a72a58..dff22bd80eaf 100644
--- a/kernel/panic.c
+++ b/kernel/panic.c
@@ -386,6 +386,7 @@ const struct taint_flag taint_flags[TAINT_FLAGS_COUNT] = {
[ TAINT_LIVEPATCH ] = { 'K', ' ', true },
[ TAINT_AUX ] = { 'X', ' ', true },
[ TAINT_RANDSTRUCT ] = { 'T', ' ', true },
+ [ TAINT_RAW_PASSTHROUGH ] = { 'H', ' ', true },
};
/**
--
2.30.0
For drivers that moderate access to the underlying hardware it is
sometimes desirable to allow userspace to bypass restrictions. Once
userspace has done this, the driver can no longer guarantee the sanctity
of either the OS or the hardware. When in this state, it is helpful for
kernel developers to be made aware (via this taint flag) of this fact
for subsequent bug reports.
Example usage:
- Hardware xyzzy accepts 2 commands, waldo and fred.
- The xyzzy driver provides an interface for using waldo, but not fred.
- quux is convinced they really need the fred command.
- xyzzy driver allows quux to frob hardware to initiate fred.
- kernel gets tainted.
- turns out fred command is borked, and scribbles over memory.
- developers laugh while closing quux's subsequent bug report.
Signed-off-by: Ben Widawsky <[email protected]>
---
Documentation/admin-guide/sysctl/kernel.rst | 1 +
Documentation/admin-guide/tainted-kernels.rst | 6 +++++-
include/linux/kernel.h | 3 ++-
kernel/panic.c | 1 +
4 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/Documentation/admin-guide/sysctl/kernel.rst b/Documentation/admin-guide/sysctl/kernel.rst
index 1d56a6b73a4e..3e1eada53504 100644
--- a/Documentation/admin-guide/sysctl/kernel.rst
+++ b/Documentation/admin-guide/sysctl/kernel.rst
@@ -1352,6 +1352,7 @@ ORed together. The letters are seen in "Tainted" line of Oops reports.
32768 `(K)` kernel has been live patched
65536 `(X)` Auxiliary taint, defined and used by for distros
131072 `(T)` The kernel was built with the struct randomization plugin
+262144 `(H)` The kernel has allowed vendor shenanigans
====== ===== ==============================================================
See :doc:`/admin-guide/tainted-kernels` for more information.
diff --git a/Documentation/admin-guide/tainted-kernels.rst b/Documentation/admin-guide/tainted-kernels.rst
index ceeed7b0798d..ee2913316344 100644
--- a/Documentation/admin-guide/tainted-kernels.rst
+++ b/Documentation/admin-guide/tainted-kernels.rst
@@ -74,7 +74,7 @@ a particular type of taint. It's best to leave that to the aforementioned
script, but if you need something quick you can use this shell command to check
which bits are set::
- $ for i in $(seq 18); do echo $(($i-1)) $(($(cat /proc/sys/kernel/tainted)>>($i-1)&1));done
+ $ for i in $(seq 19); do echo $(($i-1)) $(($(cat /proc/sys/kernel/tainted)>>($i-1)&1));done
Table for decoding tainted state
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -100,6 +100,7 @@ Bit Log Number Reason that got the kernel tainted
15 _/K 32768 kernel has been live patched
16 _/X 65536 auxiliary taint, defined for and used by distros
17 _/T 131072 kernel was built with the struct randomization plugin
+ 18 _/H 262144 kernel has allowed vendor shenanigans
=== === ====== ========================================================
Note: The character ``_`` is representing a blank in this table to make reading
@@ -175,3 +176,6 @@ More detailed explanation for tainting
produce extremely unusual kernel structure layouts (even performance
pathological ones), which is important to know when debugging. Set at
build time.
+
+ 18) ``H`` Kernel has allowed direct access to hardware and can no longer make
+ any guarantees about the stability of the device or driver.
diff --git a/include/linux/kernel.h b/include/linux/kernel.h
index f7902d8c1048..bc95486f817e 100644
--- a/include/linux/kernel.h
+++ b/include/linux/kernel.h
@@ -443,7 +443,8 @@ extern enum system_states {
#define TAINT_LIVEPATCH 15
#define TAINT_AUX 16
#define TAINT_RANDSTRUCT 17
-#define TAINT_FLAGS_COUNT 18
+#define TAINT_RAW_PASSTHROUGH 18
+#define TAINT_FLAGS_COUNT 19
#define TAINT_FLAGS_MAX ((1UL << TAINT_FLAGS_COUNT) - 1)
struct taint_flag {
diff --git a/kernel/panic.c b/kernel/panic.c
index 332736a72a58..dff22bd80eaf 100644
--- a/kernel/panic.c
+++ b/kernel/panic.c
@@ -386,6 +386,7 @@ const struct taint_flag taint_flags[TAINT_FLAGS_COUNT] = {
[ TAINT_LIVEPATCH ] = { 'K', ' ', true },
[ TAINT_AUX ] = { 'X', ' ', true },
[ TAINT_RANDSTRUCT ] = { 'T', ' ', true },
+ [ TAINT_RAW_PASSTHROUGH ] = { 'H', ' ', true },
};
/**
--
2.30.0
From: Dan Williams <[email protected]>
The CXL.mem protocol allows a device to act as a provider of "System
RAM" and/or "Persistent Memory" that is fully coherent as if the memory
was attached to the typical CPU memory controller.
With the CXL-2.0 specification a PCI endpoint can implement a "Type-3"
device interface and give the operating system control over "Host
Managed Device Memory". See section 2.3 Type 3 CXL Device.
The memory range exported by the device may optionally be described by
the platform firmware memory map, or by infrastructure like LIBNVDIMM to
provision persistent memory capacity from one, or more, CXL.mem devices.
A pre-requisite for Linux-managed memory-capacity provisioning is this
cxl_mem driver that can speak the mailbox protocol defined in section
8.2.8.4 Mailbox Registers.
For now just land the driver boiler-plate and fill it in with
functionality in subsequent commits.
Link: https://www.computeexpresslink.org/download-the-specification
Signed-off-by: Dan Williams <[email protected]>
Signed-off-by: Ben Widawsky <[email protected]>
---
Documentation/cxl/memory-devices.rst | 9 ++++
drivers/cxl/Kconfig | 22 +++++++++
drivers/cxl/Makefile | 2 +
drivers/cxl/mem.c | 69 ++++++++++++++++++++++++++++
drivers/cxl/pci.h | 20 ++++++++
5 files changed, 122 insertions(+)
create mode 100644 drivers/cxl/mem.c
create mode 100644 drivers/cxl/pci.h
diff --git a/Documentation/cxl/memory-devices.rst b/Documentation/cxl/memory-devices.rst
index 6ce88f9d5f4f..134c9b6b4ff4 100644
--- a/Documentation/cxl/memory-devices.rst
+++ b/Documentation/cxl/memory-devices.rst
@@ -23,6 +23,15 @@ ACPI CXL
.. kernel-doc:: drivers/cxl/acpi.c
:internal:
+CXL Memory Device
+-----------------
+
+.. kernel-doc:: drivers/cxl/mem.c
+ :doc: cxl mem
+
+.. kernel-doc:: drivers/cxl/mem.c
+ :internal:
+
External Interfaces
===================
diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig
index 68da926ba5b1..0ac5080cd6e0 100644
--- a/drivers/cxl/Kconfig
+++ b/drivers/cxl/Kconfig
@@ -33,4 +33,26 @@ config CXL_ACPI
specification.
If unsure say 'm'
+
+config CXL_MEM
+ tristate "CXL.mem: Endpoint Support"
+ depends on PCI && CXL_BUS_PROVIDER
+ default CXL_BUS_PROVIDER
+ help
+ The CXL.mem protocol allows a device to act as a provider of
+ "System RAM" and/or "Persistent Memory" that is fully coherent
+ as if the memory was attached to the typical CPU memory
+ controller.
+
+ Say 'y/m' to enable a driver (named "cxl_mem.ko" when built as
+ a module) that will attach to CXL.mem devices for
+ configuration, provisioning, and health monitoring. This
+ driver is required for dynamic provisioning of CXL.mem
+ attached memory which is a pre-requisite for persistent memory
+ support. Typically volatile memory is mapped by platform
+ firmware and included in the platform memory map, but in some
+ cases the OS is responsible for mapping that memory. See
+ Chapter 2.3 Type 3 CXL Device in the CXL 2.0 specification.
+
+ If unsure say 'm'.
endif
diff --git a/drivers/cxl/Makefile b/drivers/cxl/Makefile
index d38cd34a2582..97fdffb00f2d 100644
--- a/drivers/cxl/Makefile
+++ b/drivers/cxl/Makefile
@@ -1,5 +1,7 @@
# SPDX-License-Identifier: GPL-2.0
obj-$(CONFIG_CXL_ACPI) += cxl_acpi.o
+obj-$(CONFIG_CXL_MEM) += cxl_mem.o
ccflags-y += -DDEFAULT_SYMBOL_NAMESPACE=CXL
cxl_acpi-y := acpi.o
+cxl_mem-y := mem.o
diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
new file mode 100644
index 000000000000..005404888942
--- /dev/null
+++ b/drivers/cxl/mem.c
@@ -0,0 +1,69 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright(c) 2020 Intel Corporation. All rights reserved. */
+#include <linux/module.h>
+#include <linux/pci.h>
+#include <linux/io.h>
+#include "acpi.h"
+#include "pci.h"
+
+static int cxl_mem_dvsec(struct pci_dev *pdev, int dvsec)
+{
+ int pos;
+
+ pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_DVSEC);
+ if (!pos)
+ return 0;
+
+ while (pos) {
+ u16 vendor, id;
+
+ pci_read_config_word(pdev, pos + PCI_DVSEC_VENDOR_ID_OFFSET,
+ &vendor);
+ pci_read_config_word(pdev, pos + PCI_DVSEC_ID_OFFSET, &id);
+ if (vendor == PCI_DVSEC_VENDOR_ID_CXL && dvsec == id)
+ return pos;
+
+ pos = pci_find_next_ext_capability(pdev, pos,
+ PCI_EXT_CAP_ID_DVSEC);
+ }
+
+ return 0;
+}
+
+static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id)
+{
+ struct device *dev = &pdev->dev;
+ int rc, regloc;
+
+ rc = cxl_bus_acquire(pdev);
+ if (rc != 0) {
+ dev_err(dev, "failed to acquire interface\n");
+ return rc;
+ }
+
+ regloc = cxl_mem_dvsec(pdev, PCI_DVSEC_ID_CXL_REGLOC);
+ if (!regloc) {
+ dev_err(dev, "register location dvsec not found\n");
+ return -ENXIO;
+ }
+
+ return 0;
+}
+
+static const struct pci_device_id cxl_mem_pci_tbl[] = {
+ /* PCI class code for CXL.mem Type-3 Devices */
+ { PCI_ANY_ID, PCI_ANY_ID, PCI_ANY_ID, PCI_ANY_ID,
+ PCI_CLASS_MEMORY_CXL, 0xffffff, 0 },
+ { /* terminate list */ },
+};
+MODULE_DEVICE_TABLE(pci, cxl_mem_pci_tbl);
+
+static struct pci_driver cxl_mem_driver = {
+ .name = KBUILD_MODNAME,
+ .id_table = cxl_mem_pci_tbl,
+ .probe = cxl_mem_probe,
+};
+
+MODULE_LICENSE("GPL v2");
+module_pci_driver(cxl_mem_driver);
+MODULE_IMPORT_NS(CXL);
diff --git a/drivers/cxl/pci.h b/drivers/cxl/pci.h
new file mode 100644
index 000000000000..a8a9935fa90b
--- /dev/null
+++ b/drivers/cxl/pci.h
@@ -0,0 +1,20 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright(c) 2020 Intel Corporation. All rights reserved. */
+#ifndef __CXL_PCI_H__
+#define __CXL_PCI_H__
+
+#define PCI_CLASS_MEMORY_CXL 0x050210
+
+/*
+ * See section 8.1 Configuration Space Registers in the CXL 2.0
+ * Specification
+ */
+#define PCI_EXT_CAP_ID_DVSEC 0x23
+#define PCI_DVSEC_VENDOR_ID_CXL 0x1E98
+#define PCI_DVSEC_VENDOR_ID_OFFSET 0x4
+#define PCI_DVSEC_ID_CXL 0x0
+#define PCI_DVSEC_ID_OFFSET 0x8
+
+#define PCI_DVSEC_ID_CXL_REGLOC 0x8
+
+#endif /* __CXL_PCI_H__ */
--
2.30.0
The send command allows userspace to issue mailbox commands directly to
the hardware. The driver will verify basic properties of the command and
possible inspect the input (or output) payload to determine whether or
not the command is allowed (or might taint the kernel).
The list of allowed commands and their properties can be determined by
using the QUERY IOCTL for CXL memory devices.
Signed-off-by: Ben Widawsky <[email protected]>
---
drivers/cxl/mem.c | 204 ++++++++++++++++++++++++++++++++++-
include/uapi/linux/cxl_mem.h | 39 +++++++
2 files changed, 239 insertions(+), 4 deletions(-)
diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
index d4eb3f5b9469..f979788b4d9f 100644
--- a/drivers/cxl/mem.c
+++ b/drivers/cxl/mem.c
@@ -84,6 +84,13 @@ static DEFINE_IDR(cxl_mem_idr);
/* protect cxl_mem_idr allocations */
static DEFINE_MUTEX(cxl_memdev_lock);
+#undef C
+#define C(a, b) { b }
+static struct {
+ const char *name;
+} command_names[] = { CMDS };
+#undef C
+
#define CXL_CMD(_id, _flags, sin, sout, f) \
[CXL_MEM_COMMAND_ID_##_id] = { \
{ \
@@ -319,16 +326,25 @@ static int cxl_mem_mbox_send_cmd(struct cxl_mem *cxlm,
/**
* cxl_mem_mbox_get() - Acquire exclusive access to the mailbox.
* @cxlm: The memory device to gain access to.
+ * @interruptible: Allow interrupting mutex acquisition.
*
* Context: Any context. Takes the mbox_lock.
* Return: 0 if exclusive access was acquired.
*/
-static int cxl_mem_mbox_get(struct cxl_mem *cxlm)
+static int cxl_mem_mbox_get(struct cxl_mem *cxlm, bool interruptible)
{
u64 md_status;
- int rc = -EBUSY;
+ int rc;
+
+ if (interruptible) {
+ rc = mutex_lock_interruptible(&cxlm->mbox.mutex);
+ if (rc)
+ return rc;
+ } else {
+ mutex_lock_io(&cxlm->mbox.mutex);
+ }
- mutex_lock_io(&cxlm->mbox.mutex);
+ rc = -EBUSY;
/*
* XXX: There is some amount of ambiguity in the 2.0 version of the spec
@@ -443,6 +459,155 @@ static int cxl_mem_count_commands(void)
}
return n;
+};
+
+/**
+ * handle_mailbox_cmd_from_user() - Dispatch a mailbox command.
+ * @cxlmd: The CXL memory device to communicate with.
+ * @cmd: The validated command.
+ * @in_payload: Pointer to userspace's input payload.
+ * @out_payload: Pointer to userspace's output payload.
+ * @u: The command submitted by userspace. Has output fields.
+ *
+ * Return:
+ * * %0 - Mailbox transaction succeeded.
+ * * %-EFAULT - Something happened with copy_to/from_user.
+ * * %-EINTR - Mailbox acquisition interrupted.
+ * * %-E2BIG - Output payload would overrun buffer.
+ *
+ * Creates the appropriate mailbox command on behalf of a userspace request.
+ * Return value, size, and output payload are all copied out to @u. The
+ * parameters for the command must be validated before calling this function.
+ *
+ * A 0 return code indicates the command executed successfully, not that it was
+ * itself successful. IOW, the retval should always be checked if wanting to
+ * determine the actual result.
+ */
+static int handle_mailbox_cmd_from_user(struct cxl_memdev *cxlmd,
+ const struct cxl_mem_command *cmd,
+ u64 in_payload,
+ u64 out_payload,
+ struct cxl_send_command __user *u)
+{
+ struct mbox_cmd mbox_cmd = {
+ .opcode = cmd->opcode,
+ .size_in = cmd->info.size_in,
+ .payload = NULL, /* Copied by copy_to|from_user() */
+ };
+ int rc;
+
+ if (cmd->info.size_in) {
+ /*
+ * Directly copy the userspace payload into the hardware. UAPI
+ * states that the buffer must already be little endian.
+ */
+ if (copy_from_user((__force void *)cxl_payload_regs(cxlmd->cxlm),
+ u64_to_user_ptr(in_payload),
+ cmd->info.size_in)) {
+ cxl_mem_mbox_put(cxlmd->cxlm);
+ return -EFAULT;
+ }
+ }
+
+ rc = cxl_mem_mbox_get(cxlmd->cxlm, true);
+ if (rc)
+ return rc;
+
+ dev_dbg(&cxlmd->dev,
+ "Submitting %s command for user\n"
+ "\topcode: %x\n"
+ "\tsize: %ub\n",
+ command_names[cmd->info.id].name, mbox_cmd.opcode,
+ cmd->info.size_in);
+
+ rc = cxl_mem_mbox_send_cmd(cxlmd->cxlm, &mbox_cmd);
+ cxl_mem_mbox_put(cxlmd->cxlm);
+ if (rc)
+ return rc;
+
+ if (mbox_cmd.size_out > cmd->info.size_out)
+ return -E2BIG;
+
+ rc = put_user(mbox_cmd.return_code, &u->retval);
+ if (rc)
+ return rc;
+
+ rc = put_user(mbox_cmd.size_out, &u->size_out);
+ if (rc)
+ return rc;
+
+ if (mbox_cmd.size_out)
+ if (copy_to_user(u64_to_user_ptr(out_payload),
+ (__force void *)cxl_payload_regs(cxlmd->cxlm),
+ mbox_cmd.size_out))
+ return -EFAULT;
+
+ return 0;
+}
+
+/**
+ * cxl_validate_cmd_from_user() - Check fields for CXL_MEM_SEND_COMMAND.
+ * @cxlm: &struct cxl_mem device whose mailbox will be used.
+ * @send_cmd: &struct cxl_send_command copied in from userspace.
+ * @out_cmd: Sanitized and populated &struct cxl_mem_command.
+ *
+ * Return:
+ * * %0 - @out_cmd is ready to send.
+ * * %-ENOTTY - Invalid command specified.
+ * * %-EINVAL - Reserved fields or invalid values were used.
+ * * %-EPERM - Attempted to use a protected command.
+ * * %-ENOMEM - Input or output buffer wasn't sized properly.
+ *
+ * The result of this command is a fully validated command in @out_cmd that is
+ * safe to send to the hardware.
+ *
+ * See handle_mailbox_cmd_from_user()
+ */
+static int cxl_validate_cmd_from_user(struct cxl_mem *cxlm,
+ const struct cxl_send_command *send_cmd,
+ struct cxl_mem_command *out_cmd)
+{
+ const struct cxl_command_info *info;
+ struct cxl_mem_command *c;
+
+ if (send_cmd->id == 0 || send_cmd->id >= CXL_MEM_COMMAND_ID_MAX)
+ return -ENOTTY;
+
+ /*
+ * The user can never specify an input payload larger than our hardware
+ * supports, but output can be arbitrarily large, we simply write out as
+ * much data as the hardware provides.
+ */
+ if (send_cmd->size_in > cxlm->mbox.payload_size)
+ return -EINVAL;
+
+ if (send_cmd->flags & CXL_MEM_COMMAND_FLAG_MASK)
+ return -EINVAL;
+
+ if (send_cmd->rsvd)
+ return -EINVAL;
+
+ /* Convert user's command into the internal representation */
+ c = &mem_commands[send_cmd->id];
+ info = &c->info;
+
+ if (info->flags & CXL_MEM_COMMAND_FLAG_KERNEL)
+ return -EPERM;
+
+ /* Check the input buffer is the expected size */
+ if (info->size_in >= 0 && info->size_in != send_cmd->size_in)
+ return -ENOMEM;
+
+ /* Check the output buffer is at least large enough */
+ if (info->size_out >= 0 && send_cmd->size_out < info->size_out)
+ return -ENOMEM;
+
+ /* Setting a few const fields here... */
+ memcpy(out_cmd, c, sizeof(*c));
+ *(s32 *)&out_cmd->info.size_in = send_cmd->size_in;
+ *(s32 *)&out_cmd->info.size_out = send_cmd->size_out;
+
+ return 0;
}
static long cxl_mem_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
@@ -479,6 +644,37 @@ static long cxl_mem_ioctl(struct file *file, unsigned int cmd, unsigned long arg
if (j == n_commands)
break;
}
+
+ return 0;
+ } else if (cmd == CXL_MEM_SEND_COMMAND) {
+ struct cxl_send_command send, __user *u = (void __user *)arg;
+ struct cxl_mem_command c;
+ int rc;
+
+ dev_dbg(dev, "Send IOCTL\n");
+
+ if (copy_from_user(&send, u, sizeof(send)))
+ return -EFAULT;
+
+ rc = device_lock_interruptible(dev);
+ if (rc)
+ return rc;
+
+ if (!get_live_device(dev)) {
+ device_unlock(dev);
+ return -ENXIO;
+ }
+
+ rc = cxl_validate_cmd_from_user(cxlmd->cxlm, &send, &c);
+ if (!rc)
+ rc = handle_mailbox_cmd_from_user(cxlmd, &c,
+ send.in_payload,
+ send.out_payload, u);
+
+ put_device(dev);
+ device_unlock(dev);
+
+ return rc;
}
return -ENOTTY;
@@ -837,7 +1033,7 @@ static int cxl_mem_identify(struct cxl_mem *cxlm)
int rc;
/* Retrieve initial device memory map */
- rc = cxl_mem_mbox_get(cxlm);
+ rc = cxl_mem_mbox_get(cxlm, false);
if (rc)
return rc;
diff --git a/include/uapi/linux/cxl_mem.h b/include/uapi/linux/cxl_mem.h
index 847f825bbe18..cb4e2bee5228 100644
--- a/include/uapi/linux/cxl_mem.h
+++ b/include/uapi/linux/cxl_mem.h
@@ -26,6 +26,7 @@ extern "C" {
*/
#define CXL_MEM_QUERY_COMMANDS _IOR(0xCE, 1, struct cxl_mem_query_commands)
+#define CXL_MEM_SEND_COMMAND _IOWR(0xCE, 2, struct cxl_send_command)
#undef CMDS
#define CMDS \
@@ -69,6 +70,7 @@ struct cxl_command_info {
#define CXL_MEM_COMMAND_FLAG_NONE 0
#define CXL_MEM_COMMAND_FLAG_KERNEL BIT(0)
#define CXL_MEM_COMMAND_FLAG_MUTEX BIT(1)
+#define CXL_MEM_COMMAND_FLAG_MASK GENMASK(31, 2)
__s32 size_in;
__s32 size_out;
@@ -110,6 +112,43 @@ struct cxl_mem_query_commands {
struct cxl_command_info __user commands[]; /* out: supported commands */
};
+/**
+ * struct cxl_send_command - Send a command to a memory device.
+ * @id: The command to send to the memory device. This must be one of the
+ * commands returned by the query command.
+ * @flags: Flags for the command (input).
+ * @rsvd: Must be zero.
+ * @retval: Return value from the memory device (output).
+ * @size_in: Size of the payload to provide to the device (input).
+ * @size_out: Size of the payload received from the device (input/output). This
+ * field is filled in by userspace to let the driver know how much
+ * space was allocated for output. It is populated by the driver to
+ * let userspace know how large the output payload actually was.
+ * @in_payload: Pointer to memory for payload input (little endian order).
+ * @out_payload: Pointer to memory for payload output (little endian order).
+ *
+ * Mechanism for userspace to send a command to the hardware for processing. The
+ * driver will do basic validation on the command sizes, but the payload input
+ * and output are not introspected. Userspace is required to allocate large
+ * enough buffers for max(size_in, size_out).
+ */
+struct cxl_send_command {
+ __u32 id;
+ __u32 flags;
+ __u32 rsvd;
+ __u32 retval;
+
+ struct {
+ __s32 size_in;
+ __u64 in_payload;
+ };
+
+ struct {
+ __s32 size_out;
+ __u64 out_payload;
+ };
+};
+
#if defined(__cplusplus)
}
#endif
--
2.30.0
From: Dan Williams <[email protected]>
Create the /sys/bus/cxl hierarchy to enumerate memory devices
(per-endpoint control devices), memory address space devices (platform
address ranges with interleaving, performance, and persistence
attributes), and memory regions (active provisioned memory from an
address space device that is in use as System RAM or delegated to
libnvdimm as Persistent Memory regions).
For now, only the per-endpoint control devices are registered on the
'cxl' bus.
Signed-off-by: Dan Williams <[email protected]>
Signed-off-by: Ben Widawsky <[email protected]>
---
Documentation/ABI/testing/sysfs-bus-cxl | 26 +++
Documentation/cxl/memory-devices.rst | 3 +
drivers/base/core.c | 14 ++
drivers/cxl/Makefile | 2 +
drivers/cxl/bus.c | 54 +++++
drivers/cxl/bus.h | 8 +
drivers/cxl/cxl.h | 3 +
drivers/cxl/mem.c | 282 +++++++++++++++++++++++-
include/linux/device.h | 1 +
9 files changed, 392 insertions(+), 1 deletion(-)
create mode 100644 Documentation/ABI/testing/sysfs-bus-cxl
create mode 100644 drivers/cxl/bus.c
create mode 100644 drivers/cxl/bus.h
diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl
new file mode 100644
index 000000000000..fe7b87eba988
--- /dev/null
+++ b/Documentation/ABI/testing/sysfs-bus-cxl
@@ -0,0 +1,26 @@
+What: /sys/bus/cxl/devices/memX/firmware_version
+Date: December, 2020
+KernelVersion: v5.12
+Contact: [email protected]
+Description:
+ (RO) "FW Revision" string as reported by the Identify
+ Memory Device Output Payload in the CXL-2.0
+ specification.
+
+What: /sys/bus/cxl/devices/memX/ram/size
+Date: December, 2020
+KernelVersion: v5.12
+Contact: [email protected]
+Description:
+ (RO) "Volatile Only Capacity" as reported by the
+ Identify Memory Device Output Payload in the CXL-2.0
+ specification.
+
+What: /sys/bus/cxl/devices/memX/pmem/size
+Date: December, 2020
+KernelVersion: v5.12
+Contact: [email protected]
+Description:
+ (RO) "Persistent Only Capacity" as reported by the
+ Identify Memory Device Output Payload in the CXL-2.0
+ specification.
diff --git a/Documentation/cxl/memory-devices.rst b/Documentation/cxl/memory-devices.rst
index 134c9b6b4ff4..5f723c25382b 100644
--- a/Documentation/cxl/memory-devices.rst
+++ b/Documentation/cxl/memory-devices.rst
@@ -37,3 +37,6 @@ External Interfaces
.. kernel-doc:: drivers/cxl/acpi.c
:export:
+
+.. kernel-doc:: drivers/cxl/bus.c
+ :export:
diff --git a/drivers/base/core.c b/drivers/base/core.c
index 25e08e5f40bd..33432a4cbe23 100644
--- a/drivers/base/core.c
+++ b/drivers/base/core.c
@@ -3179,6 +3179,20 @@ struct device *get_device(struct device *dev)
}
EXPORT_SYMBOL_GPL(get_device);
+/**
+ * get_live_device() - increment reference count for device iff !dead
+ * @dev: device.
+ *
+ * Forward the call to get_device() if the device is still alive. If
+ * this is called with the device_lock() held then the device is
+ * guaranteed to not die until the device_lock() is dropped.
+ */
+struct device *get_live_device(struct device *dev)
+{
+ return dev && !dev->p->dead ? get_device(dev) : NULL;
+}
+EXPORT_SYMBOL_GPL(get_live_device);
+
/**
* put_device - decrement reference count.
* @dev: device in question.
diff --git a/drivers/cxl/Makefile b/drivers/cxl/Makefile
index 97fdffb00f2d..f9baaeb94bda 100644
--- a/drivers/cxl/Makefile
+++ b/drivers/cxl/Makefile
@@ -1,7 +1,9 @@
# SPDX-License-Identifier: GPL-2.0
+obj-$(CONFIG_CXL_BUS_PROVIDER) += cxl_bus.o
obj-$(CONFIG_CXL_ACPI) += cxl_acpi.o
obj-$(CONFIG_CXL_MEM) += cxl_mem.o
ccflags-y += -DDEFAULT_SYMBOL_NAMESPACE=CXL
+cxl_bus-y := bus.o
cxl_acpi-y := acpi.o
cxl_mem-y := mem.o
diff --git a/drivers/cxl/bus.c b/drivers/cxl/bus.c
new file mode 100644
index 000000000000..91fe765bea94
--- /dev/null
+++ b/drivers/cxl/bus.c
@@ -0,0 +1,54 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/* Copyright(c) 2020 Intel Corporation. All rights reserved. */
+#include <linux/device.h>
+#include <linux/module.h>
+#include "bus.h"
+
+static struct bus_type cxl_bus_type = {
+ .name = "cxl",
+};
+
+static void cxl_unregister(void *dev)
+{
+ device_unregister(dev);
+}
+
+/**
+ * cxl_register() - Register a device on the CXL bus.
+ * @dev: The device to register onto the CXL bus.
+ *
+ * A CXL device driver must call this in order to have the device be a part of
+ * the CXL bus. All endpoint device drivers should utilize this function.
+ *
+ * Return: 0 on success.
+ */
+int cxl_register(struct device *dev)
+{
+ int rc;
+
+ if (!dev->parent || !dev->parent->driver)
+ return -EINVAL;
+
+ dev->bus = &cxl_bus_type;
+ rc = device_add(dev);
+ if (rc)
+ put_device(dev);
+ else
+ rc = devm_add_action_or_reset(dev->parent, cxl_unregister, dev);
+ return rc;
+}
+EXPORT_SYMBOL_GPL(cxl_register);
+
+static __init int cxl_bus_init(void)
+{
+ return bus_register(&cxl_bus_type);
+}
+
+static void cxl_bus_exit(void)
+{
+ bus_unregister(&cxl_bus_type);
+}
+
+module_init(cxl_bus_init);
+module_exit(cxl_bus_exit);
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/cxl/bus.h b/drivers/cxl/bus.h
new file mode 100644
index 000000000000..5fc75ebd3650
--- /dev/null
+++ b/drivers/cxl/bus.h
@@ -0,0 +1,8 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright(c) 2020 Intel Corporation. All rights reserved. */
+#ifndef __CXL_BUS_H__
+#define __CXL_BUS_H__
+
+int cxl_register(struct device *dev);
+
+#endif /* __CXL_BUS_H__ */
diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
index ca3fa496e21c..537ac4d8e6bd 100644
--- a/drivers/cxl/cxl.h
+++ b/drivers/cxl/cxl.h
@@ -3,6 +3,7 @@
#ifndef __CXL_H__
#define __CXL_H__
+#include <linux/range.h>
#include <linux/bitfield.h>
#include <linux/bitops.h>
@@ -67,6 +68,7 @@
#define CXLMDEV_RESET_NEEDED(status) \
(CXL_GET_FIELD(status, CXLMDEV_RESET_NEEDED) != CXLMDEV_RESET_NEEDED_NOT)
+struct cxl_memdev;
/**
* struct cxl_mem - A CXL memory device
* @pdev: The PCI device associated with this CXL device.
@@ -75,6 +77,7 @@
struct cxl_mem {
struct pci_dev *pdev;
void __iomem *regs;
+ struct cxl_memdev *cxlmd;
struct {
struct range range;
diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
index e9ba97bbd7b9..da2bf941fe92 100644
--- a/drivers/cxl/mem.c
+++ b/drivers/cxl/mem.c
@@ -2,11 +2,37 @@
/* Copyright(c) 2020 Intel Corporation. All rights reserved. */
#include <linux/sched/clock.h>
#include <linux/module.h>
+#include <linux/mutex.h>
+#include <linux/cdev.h>
+#include <linux/idr.h>
#include <linux/pci.h>
#include <linux/io.h>
#include "acpi.h"
#include "pci.h"
#include "cxl.h"
+#include "bus.h"
+
+/**
+ * DOC: cxl mem
+ *
+ * This implements a CXL memory device ("type-3") as it is defined by the
+ * Compute Express Link specification.
+ *
+ * The driver has several responsibilities, mainly:
+ * - Create the memX device and register on the CXL bus.
+ * - Enumerate device's register interface and map them.
+ * - Probe the device attributes to establish sysfs interface.
+ * - Provide an IOCTL interface to userspace to communicate with the device for
+ * things like firmware update.
+ * - Support management of interleave sets.
+ * - Handle and manage error conditions.
+ */
+
+/*
+ * An entire PCI topology full of devices should be enough for any
+ * config
+ */
+#define CXL_MEM_MAX_DEVS 65536
#define cxl_doorbell_busy(cxlm) \
(cxl_read_mbox_reg32(cxlm, CXLDEV_MB_CTRL_OFFSET) & \
@@ -44,6 +70,18 @@ struct mbox_cmd {
u16 return_code;
};
+struct cxl_memdev {
+ struct device dev;
+ struct cxl_mem *cxlm;
+ int id;
+};
+
+static int cxl_mem_major;
+static struct cdev cxl_mem_cdev;
+static DEFINE_IDR(cxl_mem_idr);
+/* protect cxl_mem_idr allocations */
+static DEFINE_MUTEX(cxl_memdev_lock);
+
static int cxl_mem_wait_for_doorbell(struct cxl_mem *cxlm)
{
const int timeout = msecs_to_jiffies(CXL_MAILBOX_TIMEOUT_US);
@@ -250,6 +288,55 @@ static void cxl_mem_mbox_put(struct cxl_mem *cxlm)
mutex_unlock(&cxlm->mbox.mutex);
}
+static int cxl_mem_open(struct inode *inode, struct file *file)
+{
+ long minor = iminor(inode);
+ struct cxl_memdev *cxlmd;
+ struct device *dev;
+ int rc = -ENXIO;
+
+ mutex_lock(&cxl_memdev_lock);
+ cxlmd = idr_find(&cxl_mem_idr, minor);
+ if (!cxlmd)
+ goto out;
+
+ dev = get_live_device(&cxlmd->dev);
+ if (!dev)
+ goto out;
+
+ rc = 0;
+ dev_dbg(dev, "Opened %pD\n", file);
+
+ file->private_data = cxlmd;
+
+out:
+ mutex_unlock(&cxl_memdev_lock);
+ return rc;
+}
+
+static int cxl_mem_release(struct inode *inode, struct file *file)
+{
+ struct cxl_memdev *cxlmd = file->private_data;
+
+ put_device(&cxlmd->dev);
+
+ return 0;
+}
+
+static long cxl_mem_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
+{
+ return -ENOTTY;
+}
+
+static const struct file_operations cxl_mem_fops = {
+ .owner = THIS_MODULE,
+ .open = cxl_mem_open,
+ .release = cxl_mem_release,
+ .unlocked_ioctl = cxl_mem_ioctl,
+ .compat_ioctl = compat_ptr_ioctl,
+ .llseek = noop_llseek,
+};
+
/**
* cxl_mem_setup_regs() - Setup necessary MMIO.
* @cxlm: The CXL memory device to communicate with.
@@ -412,6 +499,157 @@ static int cxl_mem_dvsec(struct pci_dev *pdev, int dvsec)
return 0;
}
+static struct cxl_memdev *to_cxl_memdev(struct device *dev)
+{
+ return container_of(dev, struct cxl_memdev, dev);
+}
+
+static void cxl_memdev_release(struct device *dev)
+{
+ struct cxl_memdev *cxlmd = to_cxl_memdev(dev);
+
+ mutex_lock(&cxl_memdev_lock);
+ idr_remove(&cxl_mem_idr, cxlmd->id);
+ mutex_unlock(&cxl_memdev_lock);
+
+ kfree(cxlmd);
+}
+
+static char *cxl_memdev_devnode(struct device *dev, umode_t *mode, kuid_t *uid,
+ kgid_t *gid)
+{
+ return kasprintf(GFP_KERNEL, "cxl/%s", dev_name(dev));
+}
+
+static ssize_t firmware_version_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct cxl_memdev *cxlmd = to_cxl_memdev(dev);
+ struct cxl_mem *cxlm = cxlmd->cxlm;
+
+ return sprintf(buf, "%.16s\n", cxlm->firmware_version);
+}
+static DEVICE_ATTR_RO(firmware_version);
+
+static ssize_t payload_max_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct cxl_memdev *cxlmd = to_cxl_memdev(dev);
+ struct cxl_mem *cxlm = cxlmd->cxlm;
+
+ return sprintf(buf, "%zu\n", cxlm->mbox.payload_size);
+}
+static DEVICE_ATTR_RO(payload_max);
+
+static ssize_t ram_size_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ struct cxl_memdev *cxlmd = to_cxl_memdev(dev);
+ struct cxl_mem *cxlm = cxlmd->cxlm;
+ unsigned long long len = range_len(&cxlm->ram.range);
+
+ return sprintf(buf, "%#llx\n", len);
+}
+
+static struct device_attribute dev_attr_ram_size =
+ __ATTR(size, 0444, ram_size_show, NULL);
+
+static ssize_t pmem_size_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ struct cxl_memdev *cxlmd = to_cxl_memdev(dev);
+ struct cxl_mem *cxlm = cxlmd->cxlm;
+ unsigned long long len = range_len(&cxlm->pmem.range);
+
+ return sprintf(buf, "%#llx\n", len);
+}
+
+static struct device_attribute dev_attr_pmem_size =
+ __ATTR(size, 0444, pmem_size_show, NULL);
+
+static struct attribute *cxl_memdev_attributes[] = {
+ &dev_attr_firmware_version.attr,
+ &dev_attr_payload_max.attr,
+ NULL,
+};
+
+static struct attribute *cxl_memdev_pmem_attributes[] = {
+ &dev_attr_pmem_size.attr,
+ NULL,
+};
+
+static struct attribute *cxl_memdev_ram_attributes[] = {
+ &dev_attr_ram_size.attr,
+ NULL,
+};
+
+static struct attribute_group cxl_memdev_attribute_group = {
+ .attrs = cxl_memdev_attributes,
+};
+
+static struct attribute_group cxl_memdev_ram_attribute_group = {
+ .name = "ram",
+ .attrs = cxl_memdev_ram_attributes,
+};
+
+static struct attribute_group cxl_memdev_pmem_attribute_group = {
+ .name = "pmem",
+ .attrs = cxl_memdev_pmem_attributes,
+};
+
+static const struct attribute_group *cxl_memdev_attribute_groups[] = {
+ &cxl_memdev_attribute_group,
+ &cxl_memdev_ram_attribute_group,
+ &cxl_memdev_pmem_attribute_group,
+ NULL,
+};
+
+static const struct device_type cxl_memdev_type = {
+ .name = "cxl_memdev",
+ .release = cxl_memdev_release,
+ .devnode = cxl_memdev_devnode,
+ .groups = cxl_memdev_attribute_groups,
+};
+
+static int cxl_mem_add_memdev(struct cxl_mem *cxlm)
+{
+ struct pci_dev *pdev = cxlm->pdev;
+ struct cxl_memdev *cxlmd;
+ struct device *dev;
+ int rc;
+
+ cxlmd = kzalloc(sizeof(*cxlmd), GFP_KERNEL);
+ if (!cxlmd)
+ return -ENOMEM;
+
+ cxlmd->cxlm = cxlm;
+
+ mutex_lock(&cxl_memdev_lock);
+ rc = idr_alloc(&cxl_mem_idr, cxlmd, 0, CXL_MEM_MAX_DEVS, GFP_KERNEL);
+ mutex_unlock(&cxl_memdev_lock);
+ if (rc < 0) {
+ kfree(cxlmd);
+ return rc;
+ }
+
+ cxlmd->id = rc;
+
+ dev = &cxlmd->dev;
+
+ /* sync with racing open */
+ mutex_lock(&cxl_memdev_lock);
+ device_initialize(dev);
+ dev->parent = &pdev->dev;
+ dev->devt = MKDEV(cxl_mem_major, cxlmd->id);
+ dev->type = &cxl_memdev_type;
+ dev_set_name(dev, "mem%d", cxlmd->id);
+
+ rc = cxl_register(dev);
+ mutex_unlock(&cxl_memdev_lock);
+
+ return rc;
+}
+
/**
* cxl_mem_identify() - Send the IDENTIFY command to the device.
* @cxlm: The device to identify.
@@ -534,6 +772,10 @@ static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id)
if (rc)
goto err;
+ rc = cxl_mem_add_memdev(cxlm);
+ if (rc)
+ goto err;
+
pci_set_drvdata(pdev, cxlm);
return 0;
@@ -571,6 +813,44 @@ static struct pci_driver cxl_mem_driver = {
.remove = cxl_mem_remove,
};
+static __init int cxl_mem_init(void)
+{
+ int rc;
+ dev_t devt;
+
+ rc = alloc_chrdev_region(&devt, 0, CXL_MEM_MAX_DEVS, "cxl");
+ if (rc)
+ return rc;
+
+ cxl_mem_major = MAJOR(devt);
+
+ cdev_init(&cxl_mem_cdev, &cxl_mem_fops);
+ rc = cdev_add(&cxl_mem_cdev, MKDEV(cxl_mem_major, 0), CXL_MEM_MAX_DEVS);
+ if (rc)
+ goto err_cdev;
+
+ rc = pci_register_driver(&cxl_mem_driver);
+ if (rc)
+ goto err_driver;
+
+ return 0;
+
+err_driver:
+ cdev_del(&cxl_mem_cdev);
+err_cdev:
+ unregister_chrdev_region(MKDEV(cxl_mem_major, 0), CXL_MEM_MAX_DEVS);
+
+ return rc;
+}
+
+static __exit void cxl_mem_exit(void)
+{
+ pci_unregister_driver(&cxl_mem_driver);
+ cdev_del(&cxl_mem_cdev);
+ unregister_chrdev_region(MKDEV(cxl_mem_major, 0), CXL_MEM_MAX_DEVS);
+}
+
MODULE_LICENSE("GPL v2");
-module_pci_driver(cxl_mem_driver);
+module_init(cxl_mem_init);
+module_exit(cxl_mem_exit);
MODULE_IMPORT_NS(CXL);
diff --git a/include/linux/device.h b/include/linux/device.h
index 89bb8b84173e..8659deee8ae6 100644
--- a/include/linux/device.h
+++ b/include/linux/device.h
@@ -895,6 +895,7 @@ extern int (*platform_notify_remove)(struct device *dev);
*
*/
struct device *get_device(struct device *dev);
+struct device *get_live_device(struct device *dev);
void put_device(struct device *dev);
bool kill_device(struct device *dev);
--
2.30.0
CXL devices must implement the Device Command Interface (described in
8.2.9 of the CXL 2.0 spec). While the driver already maintains a list of
commands it supports, there is still a need to be able to distinguish
between commands that the driver knows about from commands that may not
be supported by the hardware. No such commands currently are defined in
the driver.
The implementation leaves the statically defined table of commands and
supplements it with a bitmap to determine commands that are enabled.
---
There are multiple approaches that can be taken, but this is nice for a
few reasons.
Here are some of the other solutions:
Create a per instance table with only the supported commands.
1. Having a fixed command id -> command mapping is much easier to manage
for development and debugging.
2. Dealing with dynamic memory allocation for the table adds unnecessary
complexity.
3. Most tables for device types are likely to be quite similar.
4. Makes it difficult to implement helper macros like cxl_for_each_cmd()
If the per instance table did preserve ids, #1 above can be addressed.
However, as "enable" is currently the only mutable state for the
commands, it would yield a lot of overhead for not much gain.
Additionally, the other issues remain.
If "enable" remains the only mutable state, I believe this to be the
best solution. Once the number of mutable elements in a command grows,
it probably makes sense to move to per device instance state with a
fixed command ID mapping.
Signed-off-by: Ben Widawsky <[email protected]>
---
drivers/cxl/cxl.h | 4 ++++
drivers/cxl/mem.c | 38 +++++++++++++++++++++++++++++++++++++-
2 files changed, 41 insertions(+), 1 deletion(-)
diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
index 537ac4d8e6bd..963ba30cb200 100644
--- a/drivers/cxl/cxl.h
+++ b/drivers/cxl/cxl.h
@@ -17,6 +17,9 @@
#define CXL_GET_FIELD(word, field) FIELD_GET(field##_MASK, word)
+/* XXX: Arbitrary max */
+#define CXL_MAX_COMMANDS 32
+
/* Device (CXL 2.0 - 8.2.8.3) */
#define CXLDEV_CAP_ARRAY_REG 0x0
#define CXLDEV_CAP_ARRAY_CAP_ID 0
@@ -88,6 +91,7 @@ struct cxl_mem {
} ram;
char firmware_version[0x10];
+ DECLARE_BITMAP(enabled_cmds, CXL_MAX_COMMANDS);
/* Cap 0001h - CXL_CAP_CAP_ID_DEVICE_STATUS */
struct {
diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
index a824cfd4342a..20b26fa2c466 100644
--- a/drivers/cxl/mem.c
+++ b/drivers/cxl/mem.c
@@ -114,6 +114,8 @@ static struct {
* would typically be used for deprecated commands.
* * %CXL_CMD_FLAG_MANDATORY: Hardware must support this command. This flag is
* only used internally by the driver for sanity checking.
+ * * %CXL_CMD_INTERNAL_FLAG_PSEUDO: This is a pseudo command which doesn't have
+ * a direct mapping to hardware. They are implicitly always enabled.
*
* The cxl_mem_command is the driver's internal representation of commands that
* are supported by the driver. Some of these commands may not be supported by
@@ -129,6 +131,7 @@ struct cxl_mem_command {
#define CXL_CMD_INTERNAL_FLAG_NONE 0
#define CXL_CMD_INTERNAL_FLAG_HIDDEN BIT(0)
#define CXL_CMD_INTERNAL_FLAG_MANDATORY BIT(1)
+#define CXL_CMD_INTERNAL_FLAG_PSEUDO BIT(2)
};
/*
@@ -140,7 +143,7 @@ struct cxl_mem_command {
static struct cxl_mem_command mem_commands[] = {
CXL_CMD(INVALID, KERNEL, 0, 0, HIDDEN),
CXL_CMD(IDENTIFY, NONE, 0, 0x43, MANDATORY),
- CXL_CMD(RAW, NONE, ~0, ~0, MANDATORY),
+ CXL_CMD(RAW, NONE, ~0, ~0, PSEUDO),
};
#define cxl_for_each_cmd(cmd) \
@@ -618,6 +621,10 @@ static int cxl_validate_cmd_from_user(struct cxl_mem *cxlm,
c = &mem_commands[send_cmd->id];
info = &c->info;
+ /* Check that the command is enabled for hardware */
+ if (!test_bit(cxl_cmd_index(c), cxlm->enabled_cmds))
+ return -ENOTTY;
+
if (info->flags & CXL_MEM_COMMAND_FLAG_KERNEL)
return -EPERM;
@@ -1029,6 +1036,31 @@ static int cxl_mem_add_memdev(struct cxl_mem *cxlm)
return rc;
}
+/**
+ * cxl_mem_enumerate_cmds() - Enumerate commands for a device.
+ * @cxlm: The device.
+ *
+ * Returns 0 if enumerate completed successfully.
+ *
+ * CXL devices have optional support for certain commands. This function will
+ * determine the set of supported commands for the hardware and update the
+ * enabled_cmds bitmap in the @cxlm.
+ */
+static int cxl_mem_enumerate_cmds(struct cxl_mem *cxlm)
+{
+ struct cxl_mem_command *c;
+
+ /* All commands are considered enabled for now (except INVALID). */
+ cxl_for_each_cmd(c) {
+ if (c->flags & CXL_CMD_INTERNAL_FLAG_HIDDEN)
+ continue;
+
+ set_bit(cxl_cmd_index(c), cxlm->enabled_cmds);
+ }
+
+ return 0;
+}
+
/**
* cxl_mem_identify() - Send the IDENTIFY command to the device.
* @cxlm: The device to identify.
@@ -1147,6 +1179,10 @@ static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id)
if (rc)
goto err;
+ rc = cxl_mem_enumerate_cmds(cxlm);
+ if (rc)
+ return rc;
+
rc = cxl_mem_identify(cxlm);
if (rc)
goto err;
--
2.30.0
Sending this patch was a mistake. Please see the other 11/16 instead.
Thanks.
Ben
On 1/11/21 2:51 PM, Ben Widawsky wrote:
> diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig
> index 68da926ba5b1..0ac5080cd6e0 100644
> --- a/drivers/cxl/Kconfig
> +++ b/drivers/cxl/Kconfig
> @@ -33,4 +33,26 @@ config CXL_ACPI
> specification.
>
> If unsure say 'm'
> +
> +config CXL_MEM
> + tristate "CXL.mem: Endpoint Support"
> + depends on PCI && CXL_BUS_PROVIDER
> + default CXL_BUS_PROVIDER
> + help
> + The CXL.mem protocol allows a device to act as a provider of
> + "System RAM" and/or "Persistent Memory" that is fully coherent
> + as if the memory was attached to the typical CPU memory
> + controller.
All of those indented lines should use one tab (tristate, depends, default,
and help). The help text should be indented with one tab + 2 spaces,
like the lines below are.
> +
> + Say 'y/m' to enable a driver (named "cxl_mem.ko" when built as
> + a module) that will attach to CXL.mem devices for
> + configuration, provisioning, and health monitoring. This
> + driver is required for dynamic provisioning of CXL.mem
> + attached memory which is a pre-requisite for persistent memory
patch 2 spells this as: prerequisite
which is preferred IMO.
> + support. Typically volatile memory is mapped by platform
> + firmware and included in the platform memory map, but in some
> + cases the OS is responsible for mapping that memory. See
> + Chapter 2.3 Type 3 CXL Device in the CXL 2.0 specification.
> +
> + If unsure say 'm'.
That last line needs one tab + 2 spaces for indentation.
> endif
--
~Randy
On 1/11/21 2:51 PM, Ben Widawsky wrote:
> diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig
> new file mode 100644
> index 000000000000..68da926ba5b1
> --- /dev/null
> +++ b/drivers/cxl/Kconfig
> @@ -0,0 +1,36 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +menuconfig CXL_BUS
> + tristate "CXL (Compute Express Link) Devices Support"
> + help
> + CXL is a bus that is electrically compatible with PCI-E, but layers
> + three protocols on that signalling (CXL.io, CXL.cache, and CXL.mem).
> + The CXL.cache protocol allows devices to hold cachelines locally, the
> + CXL.mem protocol allows devices to be fully coherent memory targets,
> + the CXL.io protocol is equivalent to PCI-E. Say 'y' to enable support
> + for the configuration and management of devices supporting these
> + protocols.
> +
> +if CXL_BUS
> +
> +config CXL_BUS_PROVIDER
> + tristate
> +
> +config CXL_ACPI
> + tristate "CXL ACPI: Platform Support"
> + depends on ACPI
> + default CXL_BUS
> + select CXL_BUS_PROVIDER
> + help
> + Say 'y/m' to enable a driver (named "cxl_acpi.ko" when built
> + as a module) that will enable support for CXL.mem endpoint
> + devices. In general, CXL Platform Support is a prerequisite
> + for any CXL device driver that wants to claim ownership of a
> + component register space. By default platform firmware assumes
> + Linux is unaware of CXL capabilities and requires explicit
> + opt-in. This platform component also mediates resources
> + described by the CEDT (CXL Early Discovery Table). See
> + Chapter 9.14.1 CXL Early Discovery Table (CEDT) in the CXL 2.0
> + specification.
> +
> + If unsure say 'm'
End above sentence with a '.', please.
> +endif
--
~Randy
Something has gone wrong with CCing this to linux-acpi, but no worries.
On Tue, Jan 12, 2021 at 1:29 AM Ben Widawsky <[email protected]> wrote:
>
> From: Vishal Verma <[email protected]>
>
> Add support to advertise OS capabilities, and request OS control for CXL
> features using the ACPI _OSC mechanism. Advertise support for all
> possible CXL features, and attempt to request control for all possible
> features.
>
> Based on a patch by Sean Kelley.
>
> Signed-off-by: Vishal Verma <[email protected]>
> Signed-off-by: Ben Widawsky <[email protected]>
> ---
> Documentation/cxl/memory-devices.rst | 15 ++
> drivers/cxl/acpi.c | 260 ++++++++++++++++++++++++++-
> drivers/cxl/acpi.h | 20 +++
> 3 files changed, 292 insertions(+), 3 deletions(-)
>
> diff --git a/Documentation/cxl/memory-devices.rst b/Documentation/cxl/memory-devices.rst
> index aa4262280c67..6ce88f9d5f4f 100644
> --- a/Documentation/cxl/memory-devices.rst
> +++ b/Documentation/cxl/memory-devices.rst
> @@ -13,3 +13,18 @@ Driver Infrastructure
> =====================
>
> This sections covers the driver infrastructure for a CXL memory device.
> +
> +ACPI CXL
> +--------
> +
> +.. kernel-doc:: drivers/cxl/acpi.c
> + :doc: cxl acpi
> +
> +.. kernel-doc:: drivers/cxl/acpi.c
> + :internal:
> +
> +External Interfaces
> +===================
> +
> +.. kernel-doc:: drivers/cxl/acpi.c
> + :export:
> diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c
> index 0f1ba9b3f1ed..af9c0dfdee20 100644
> --- a/drivers/cxl/acpi.c
> +++ b/drivers/cxl/acpi.c
> @@ -11,7 +11,258 @@
> #include <linux/pci.h>
> #include "acpi.h"
>
> -/*
> +/**
> + * DOC: cxl acpi
> + *
> + * ACPI _OSC setup: The exported function cxl_bus_acquire() sets up ACPI
> + * Operating System Capabilities (_OSC) for the CXL bus. It declares support
> + * for all CXL capabilities, and attempts to request control for all possible
> + * capabilities. The resulting support and control sets are saved in global
> + * variables cxl_osc_support_set and cxl_osc_control_set. The internal
> + * functions cxl_osc_declare_support(), and cxl_osc_request_control() can be
> + * used to update the support and control sets in accordance with the ACPI
> + * rules for _OSC evaluation - most importantly, capabilities already granted
> + * should not be rescinded by either the OS or firmware.
> + */
> +
> +static u32 cxl_osc_support_set;
> +static u32 cxl_osc_control_set;
> +static DEFINE_MUTEX(acpi_desc_lock);
> +
> +struct pci_osc_bit_struct {
> + u32 bit;
> + char *desc;
> +};
> +
> +static struct pci_osc_bit_struct cxl_osc_support_bit[] = {
> + { CXL_OSC_PORT_REG_ACCESS_SUPPORT, "CXLPortRegAccess" },
> + { CXL_OSC_PORT_DEV_REG_ACCESS_SUPPORT, "CXLPortDevRegAccess" },
> + { CXL_OSC_PER_SUPPORT, "CXLProtocolErrorReporting" },
> + { CXL_OSC_NATIVE_HP_SUPPORT, "CXLNativeHotPlug" },
> +};
> +
> +static struct pci_osc_bit_struct cxl_osc_control_bit[] = {
> + { CXL_OSC_MEM_ERROR_CONTROL, "CXLMemErrorReporting" },
> +};
> +
> +static u8 cxl_osc_uuid_str[] = "68F2D50B-C469-4d8A-BD3D-941A103FD3FC";
> +
> +static void decode_osc_bits(struct device *dev, char *msg, u32 word,
> + struct pci_osc_bit_struct *table, int size)
> +{
> + char buf[80];
> + int i, len = 0;
> + struct pci_osc_bit_struct *entry;
> +
> + buf[0] = '\0';
> + for (i = 0, entry = table; i < size; i++, entry++)
> + if (word & entry->bit)
> + len += scnprintf(buf + len, sizeof(buf) - len, "%s%s",
> + len ? " " : "", entry->desc);
> +
> + dev_info(dev, "_OSC: %s [%s]\n", msg, buf);
> +}
> +
> +static void decode_cxl_osc_support(struct device *dev, char *msg, u32 word)
> +{
> + decode_osc_bits(dev, msg, word, cxl_osc_support_bit,
> + ARRAY_SIZE(cxl_osc_support_bit));
> +}
> +
> +static void decode_cxl_osc_control(struct device *dev, char *msg, u32 word)
> +{
> + decode_osc_bits(dev, msg, word, cxl_osc_control_bit,
> + ARRAY_SIZE(cxl_osc_control_bit));
> +}
> +
> +static acpi_status acpi_cap_run_osc(acpi_handle handle, const u32 *capbuf,
> + u8 *uuid_str, u32 *retval)
A nit: This is only called twice, every time with the same UUID, so it
would be good to avoid passing the UUID to this function.
> +{
> + struct acpi_osc_context context = {
> + .uuid_str = uuid_str,
> + .rev = 1,
> + .cap.length = 20,
> + .cap.pointer = (void *)capbuf,
> + };
> + acpi_status status;
> +
> + status = acpi_run_osc(handle, &context);
> + if (ACPI_SUCCESS(status)) {
> + /* pointer + offset to DWORD 5 */
> + *retval = *((u32 *)(context.ret.pointer + 16));
> + kfree(context.ret.pointer);
> + }
> + return status;
> +}
> +
The rest of the patch looks OK to me.
On 21-01-12 22:55:50, 刘乐乐(乐了) wrote:
> Ben,
>
> Thanks for your hard work. I have compiled this patch(aff2b059786d ,
> cxl-2.0v3) together qemu emulator v3, this is the first time I see a CXL
> device in linux.
>
> Still I have problems, I can saw the CXL device with `lspci -vvv` and a
> device /dev/cxl/mem0 . But I can't see the memory in system and lsmem
> command.
>
> Qemu command line is :
>
> sudo ./qemu-cxl/build/qemu-system-x86_64 -enable-kvm -smp 8 -drive
> file=/mnt/lele/vm/cxl-centos8-uefi.qcow2,format=qcow2,cache=none -drive if=pflash,format=raw,unit=0,file=/mnt/lele/edk2/usr/share/edk2-ovmf/x64/OVMF_CODE.fd,readonly=on
> -drive if=pflash,format=raw,readonly,unit=1,file=/mnt/lele/edk2/usr/share/edk2-ovmf/x64/OVMF_VARS.fd
> -m 32G -vnc :12 -machine type=pc-q35-4.0,hmat=on,accel=kvm -net nic -net
> tap,ifname=tap1,script=/mnt/lele/vm/qemu-ifup,downscript=no -object memory-backend-file,id=cxl-mem1,share,mem-path=/mnt/lele/cxl-mem.dat,size=512M
> -device pxb-cxl,id=cxl.0,bus=pcie.0,bus_nr=52,uid=0,len-window-base=1,window-base[0]=0x4c0000000,memdev[0]=cxl-mem1
> -device cxl-rp,id=rp0,bus=cxl.0,addr=0.0,chassis=0,slot=0 -device
> cxl-type3,bus=rp0,memdev=cxl-mem1,id=cxl-pmem0,size=256M
>
> So, what's the further step to show the memory in system memory?
The code to actually find the memory region and make the OS aware of it isn't
yet present. It's next on the list of things to work on in fact. You can look
for that in v4.
With QEMU patches in my gitlab, the memory isn't even made a region. There is a
#define you can set:
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index dee5a8884b..60c774840e 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -13,6 +13,8 @@
#include "sysemu/hostmem.h"
#include "hw/cxl/cxl.h"
+#define SET_PMEM_ADDR
+
typedef struct cxl_type3_dev {
/* Private */
PCIDevice parent_obj;
With that, even though you won't see the range in Linux, the memory device
should work if you send cycles to the address range at 0x4c0000000.
Thanks.
Ben
>
>
> Thanks,
> Daniel
>
> 在 2021/1/12 6:51, Ben Widawsky 写道:
> > Changes since v2 [1]
> >
> > - Rebased onto v5.11-rc2
> > - fixes for sparse (Ben)
> > - Add (and modify) IOCTL number to ioctl-number.rst. (Randy)
> > - Change @enable to @flags with enable field. (Dan)
> > - Move command "name" to UAPI enum. (Dan)
> > - Make command array use designated initializer. (Dan)
> > - Avoid copies to/from payload registers (Ben)
> > - Actually write the input payload length (Ben)
> > - Return ENOTTY instead of EINVAL for missing Send command. (Dan)
> > - Keep device locked and pinned during send commands. (Dan)
> > - Validate input payload size doesn't exceed hardware's limit. (Ben)
> > - Get rid of TAINT flag in UAPI. (Dan)
> > - Spelling fixes. (Dan)
> > - Document things that must be zero. (Dan)
> > - Use new TAINT (Dan)
> > - Replace cxl_debug with dev_debug (Dan)
> > - Squash debug info into relevant patches (Ben)
> > - Put CXL_CMD on a diet, #define the opcodes (Dan)
> > - Switch mailbox sync from spinlock to mutex (Ben)
> > - Store off (and debug print) hardware's payload size (Ben)
> > - Fix length of GENMASK for mailbox payload (Ben)
> > - Create concept of enabled commands. (Ben)
> > - Use CEL to enable commands based on hardware. (Ben)
> > - Move command struct definitions into mem.c (Ben)
> > - Create concept of hidden commands, and kernel only commands (Ben)
> > - Add payload min to sysfs (Ben)
> >
> > ---
> >
> > The patches can be found here:
> > https://gitlab.com/bwidawsk/linux/-/commits/cxl-2.0v3
> > I've also created #cxl on oftc for any discussion.
> >
> > ---
> >
> > Introduce support for “type-3” memory devices defined in the recently released
> > Compute Express Link (CXL) 2.0 specification[2]. Specifically, these are the
> > memory devices defined by section 8.2.8.5 of the CXL 2.0 spec. A reference
> > implementation emulating these devices has been submitted to the QEMU mailing
> > list [3] and is available on gitlab [4]. “Type-3” is a CXL device that acts as a
> > memory expander for RAM or PMEM. It might be interleaved with other CXL devices
> > in a given physical address range.
> >
> > These changes allow for foundational enumeration of CXL 2.0 memory devices as
> > well as basic userspace interaction. The functionality present is:
> > - Initial driver bring-up
> > - Device enumeration and an initial sysfs representation
> > - Submit a basic firmware command via ‘mailbox’ to an emulated memory device
> > with non-volatile capacity.
> > - Provide an interface to send commands to the hardware.
> >
> > Some of the functionality that is still missing includes:
> > - Memory interleaving at the host bridge, root port, or switch level
> > - CXL 1.1 Root Complex Integrated Endpoint Support
> > - CXL 2.0 Hot plug support
> > - A bevy of supported device commands
> >
> > In addition to the core functionality of discovering the spec defined registers
> > and resources, introduce a CXL device model that will be the foundation for
> > translating CXL capabilities into existing Linux infrastructure for Persistent
> > Memory and other memory devices. For now, this only includes support for the
> > management command mailbox that type-3 devices surface. These control devices
> > fill the role of “DIMMs” / nmemX memory-devices in LIBNVDIMM terms.
> >
> > Now, while implementing the driver some feedback for the specification was
> > generated to cover perceived gaps and address conflicts. The feedback is
> > presented as a reference implementation in the driver and QEMU emulation.
> > Specifically the following concepts are original to the Linux implementation and
> > feedback / collaboration is requested to develop these into specification
> > proposals:
> > 1. Top level ACPI object (ACPI0017)
> > 2. HW imposed address space and interleave constraints
> >
> > ACPI0017
> > --------
> > Introduce a new ACPI namespace device with an _HID of ACPI0017. The purpose of
> > this object is twofold, support a legacy OS with a set of out-of-tree CXL
> > modules, and establish an attach point for a driver that knows about
> > interleaving. Both of these boil down to the same point, to centralize Operating
> > System support for resources described by the CXL Early Discovery Table (CEDT).
> >
> > The legacy OS problem stems from the spec's description of a host bridge,
> > ACPI0016 is denoted as the _HID for host bridges, with a _CID of PNP0A08. In a
> > CXL unaware version of Linux, the core ACPI subsystem will bind a driver to
> > PNP0A08 and preclude a CXL-aware driver from binding to ACPI0016. An ACPI0017
> > device allows a standalone CXL-aware driver to register for handling /
> > coordinating CEDT and CXL-specific _OSC control.
> >
> > Similarly when managing interleaving there needs to be some management layer
> > above the ACPI0016 device that is capable of assembling leaf nodes into
> > interleave sets. As is the case with ACPI0012 that does this central
> > coordination for NFIT defined resources, ACPI0017 does the same for CEDT
> > described resources.
> >
> > Memory Windows
> > -------
> > For CXL.mem capable platforms, there is a need for a mechanism for platform
> > firmware to make the Operating System aware of any restrictions that hardware
> > might have in address space. For example, in a system with 4 host bridges all
> > participating in an interleave set, the firmware needs to provide some
> > description of this. That information is missing from the CXL 2.0 spec as of
> > today and it also is not implemented in the driver. A variety of ACPI based
> > mechanisms, for example _CRS fields on the ACPI0017 device, were considered.
> >
> > Next steps after this basic foundation is expanded command support and LIBNVDIMM
> > integration. This is the initial “release early / release often” version of the
> > Linux CXL enabling.
> >
> > [1]: https://lore.kernel.org/linux-cxl/[email protected]/
> > [2]: https://www.computeexpresslink.org/
> > [3]: https://lore.kernel.org/qemu-devel/[email protected]/T/#t
> > [4]: https://gitlab.com/bwidawsk/qemu/-/tree/cxl-2.0v2
> >
> > Ben Widawsky (12):
> > docs: cxl: Add basic documentation
> > cxl/mem: Map memory device registers
> > cxl/mem: Find device capabilities
> > cxl/mem: Implement polled mode mailbox
> > cxl/mem: Add basic IOCTL interface
> > cxl/mem: Add send command
> > taint: add taint for direct hardware access
> > cxl/mem: Add a "RAW" send command
> > cxl/mem: Create concept of enabled commands
> > cxl/mem: Use CEL for enabling commands
> > cxl/mem: Add limited Get Log command (0401h)
> > MAINTAINERS: Add maintainers of the CXL driver
> >
> > Dan Williams (2):
> > cxl/mem: Introduce a driver for CXL-2.0-Type-3 endpoints
> > cxl/mem: Register CXL memX devices
> >
> > Vishal Verma (2):
> > cxl/acpi: Add an acpi_cxl module for the CXL interconnect
> > cxl/acpi: add OSC support
> >
> > .clang-format | 1 +
> > Documentation/ABI/testing/sysfs-bus-cxl | 26 +
> > Documentation/admin-guide/sysctl/kernel.rst | 1 +
> > Documentation/admin-guide/tainted-kernels.rst | 6 +-
> > Documentation/cxl/index.rst | 12 +
> > Documentation/cxl/memory-devices.rst | 51 +
> > Documentation/index.rst | 1 +
> > .../userspace-api/ioctl/ioctl-number.rst | 1 +
> > MAINTAINERS | 10 +
> > drivers/Kconfig | 1 +
> > drivers/Makefile | 1 +
> > drivers/base/core.c | 14 +
> > drivers/cxl/Kconfig | 58 +
> > drivers/cxl/Makefile | 9 +
> > drivers/cxl/acpi.c | 351 ++++
> > drivers/cxl/acpi.h | 35 +
> > drivers/cxl/bus.c | 54 +
> > drivers/cxl/bus.h | 8 +
> > drivers/cxl/cxl.h | 147 ++
> > drivers/cxl/mem.c | 1475 +++++++++++++++++
> > drivers/cxl/pci.h | 34 +
> > include/acpi/actbl1.h | 50 +
> > include/linux/device.h | 1 +
> > include/linux/kernel.h | 3 +-
> > include/uapi/linux/cxl_mem.h | 168 ++
> > kernel/panic.c | 1 +
> > 26 files changed, 2517 insertions(+), 2 deletions(-)
> > create mode 100644 Documentation/ABI/testing/sysfs-bus-cxl
> > create mode 100644 Documentation/cxl/index.rst
> > create mode 100644 Documentation/cxl/memory-devices.rst
> > create mode 100644 drivers/cxl/Kconfig
> > create mode 100644 drivers/cxl/Makefile
> > create mode 100644 drivers/cxl/acpi.c
> > create mode 100644 drivers/cxl/acpi.h
> > create mode 100644 drivers/cxl/bus.c
> > create mode 100644 drivers/cxl/bus.h
> > create mode 100644 drivers/cxl/cxl.h
> > create mode 100644 drivers/cxl/mem.c
> > create mode 100644 drivers/cxl/pci.h
> > create mode 100644 include/uapi/linux/cxl_mem.h
> >
On Tue, Jan 12, 2021 at 10:55:50PM +0800, 刘乐乐(乐了) wrote:
> Ben,
>
> Thanks for your hard work. I have compiled this patch(aff2b059786d ,
> cxl-2.0v3) together qemu emulator v3, this is the first time I see a CXL
> device in linux.
>
> Still I have problems, I can saw the CXL device with `lspci -vvv` and a
> device /dev/cxl/mem0 . But I can't see the memory in system and lsmem
> command.
>
> Qemu command line is :
>
> sudo ./qemu-cxl/build/qemu-system-x86_64 -enable-kvm -smp 8 -drive
> file=/mnt/lele/vm/cxl-centos8-uefi.qcow2,format=qcow2,cache=none -drive if=pflash,format=raw,unit=0,file=/mnt/lele/edk2/usr/share/edk2-ovmf/x64/OVMF_CODE.fd,readonly=on
> -drive if=pflash,format=raw,readonly,unit=1,file=/mnt/lele/edk2/usr/share/edk2-ovmf/x64/OVMF_VARS.fd
> -m 32G -vnc :12 -machine type=pc-q35-4.0,hmat=on,accel=kvm -net nic -net
> tap,ifname=tap1,script=/mnt/lele/vm/qemu-ifup,downscript=no -object memory-backend-file,id=cxl-mem1,share,mem-path=/mnt/lele/cxl-mem.dat,size=512M
> -device pxb-cxl,id=cxl.0,bus=pcie.0,bus_nr=52,uid=0,len-window-base=1,window-base[0]=0x4c0000000,memdev[0]=cxl-mem1
> -device cxl-rp,id=rp0,bus=cxl.0,addr=0.0,chassis=0,slot=0 -device
> cxl-type3,bus=rp0,memdev=cxl-mem1,id=cxl-pmem0,size=256M
>
> So, what's the further step to show the memory in system memory?
Please use plain text email only. Your response is valuable, but I
think the linux-kernel and linux-pci mailing lists rejected it because
it was a multi-part message with images and other non-text elements.
If you look at https://lore.kernel.org/linux-cxl/[email protected]/T/#t,
you can see Ben's v3 00/16 email and his response to your response,
but your response is missing.
See http://vger.kernel.org/majordomo-info.html for details.
On Mon, 11 Jan 2021 14:51:06 -0800
Ben Widawsky <[email protected]> wrote:
> From: Vishal Verma <[email protected]>
>
> Add an acpi_cxl module to coordinate the ACPI portions of the CXL
> (Compute eXpress Link) interconnect. This driver binds to ACPI0017
> objects in the ACPI tree, and coordinates access to the resources
> provided by the ACPI CEDT (CXL Early Discovery Table).
>
> It also coordinates operations of the root port _OSC object to notify
> platform firmware that the OS has native support for the CXL
> capabilities of endpoints.
>
> Note: the actbl1.h changes are speculative. The expectation is that they
> will arrive through the ACPICA tree in due time.
I would pull the ACPICA changes out into a precursor patch.
>
> Cc: Ben Widawsky <[email protected]>
> Cc: Dan Williams <[email protected]>
> Signed-off-by: Vishal Verma <[email protected]>
> Signed-off-by: Ben Widawsky <[email protected]>
Hi,
I think it would be good to also add CEDT to the list in drivers/acpi/tables.c
so that we can dump it from /sys/firmware/acpi/tables/ and potentially
override it from an initrd.
https://elixir.bootlin.com/linux/v5.11-rc3/source/drivers/acpi/tables.c#L482
Can be very helpful whilst debugging. Related to that, anyone know if anyone
has acpica patches so we can have iasl -d work on the table? Would probably
be useful but I'd rather not duplicate work if it's already done.
A few minor things inline
Jonathan
> ---
> drivers/Kconfig | 1 +
> drivers/Makefile | 1 +
> drivers/cxl/Kconfig | 36 ++++++++++++++++
> drivers/cxl/Makefile | 5 +++
> drivers/cxl/acpi.c | 97 +++++++++++++++++++++++++++++++++++++++++++
> drivers/cxl/acpi.h | 15 +++++++
> include/acpi/actbl1.h | 50 ++++++++++++++++++++++
> 7 files changed, 205 insertions(+)
> create mode 100644 drivers/cxl/Kconfig
> create mode 100644 drivers/cxl/Makefile
> create mode 100644 drivers/cxl/acpi.c
> create mode 100644 drivers/cxl/acpi.h
>
> diff --git a/drivers/Kconfig b/drivers/Kconfig
> index dcecc9f6e33f..62c753a73651 100644
> --- a/drivers/Kconfig
> +++ b/drivers/Kconfig
> @@ -6,6 +6,7 @@ menu "Device Drivers"
> source "drivers/amba/Kconfig"
> source "drivers/eisa/Kconfig"
> source "drivers/pci/Kconfig"
> +source "drivers/cxl/Kconfig"
> source "drivers/pcmcia/Kconfig"
> source "drivers/rapidio/Kconfig"
>
> diff --git a/drivers/Makefile b/drivers/Makefile
> index fd11b9ac4cc3..678ea810410f 100644
> --- a/drivers/Makefile
> +++ b/drivers/Makefile
> @@ -73,6 +73,7 @@ obj-$(CONFIG_NVM) += lightnvm/
> obj-y += base/ block/ misc/ mfd/ nfc/
> obj-$(CONFIG_LIBNVDIMM) += nvdimm/
> obj-$(CONFIG_DAX) += dax/
> +obj-$(CONFIG_CXL_BUS) += cxl/
> obj-$(CONFIG_DMA_SHARED_BUFFER) += dma-buf/
> obj-$(CONFIG_NUBUS) += nubus/
> obj-y += macintosh/
> diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig
> new file mode 100644
> index 000000000000..68da926ba5b1
> --- /dev/null
> +++ b/drivers/cxl/Kconfig
> @@ -0,0 +1,36 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +menuconfig CXL_BUS
> + tristate "CXL (Compute Express Link) Devices Support"
> + help
> + CXL is a bus that is electrically compatible with PCI-E, but layers
For consistency with Kconfig in driver/pci probably better to spell PCI Express
throughout.
> + three protocols on that signalling (CXL.io, CXL.cache, and CXL.mem).
> + The CXL.cache protocol allows devices to hold cachelines locally, the
> + CXL.mem protocol allows devices to be fully coherent memory targets,
> + the CXL.io protocol is equivalent to PCI-E. Say 'y' to enable support
> + for the configuration and management of devices supporting these
> + protocols.
> +
> +if CXL_BUS
> +
> +config CXL_BUS_PROVIDER
> + tristate
> +
> +config CXL_ACPI
> + tristate "CXL ACPI: Platform Support"
> + depends on ACPI
> + default CXL_BUS
> + select CXL_BUS_PROVIDER
> + help
> + Say 'y/m' to enable a driver (named "cxl_acpi.ko" when built
> + as a module) that will enable support for CXL.mem endpoint
> + devices. In general, CXL Platform Support is a prerequisite
> + for any CXL device driver that wants to claim ownership of a
> + component register space. By default platform firmware assumes
> + Linux is unaware of CXL capabilities and requires explicit
> + opt-in. This platform component also mediates resources
> + described by the CEDT (CXL Early Discovery Table). See
> + Chapter 9.14.1 CXL Early Discovery Table (CEDT) in the CXL 2.0
> + specification.
> +
> + If unsure say 'm'
> +endif
> diff --git a/drivers/cxl/Makefile b/drivers/cxl/Makefile
> new file mode 100644
> index 000000000000..d38cd34a2582
> --- /dev/null
> +++ b/drivers/cxl/Makefile
> @@ -0,0 +1,5 @@
> +# SPDX-License-Identifier: GPL-2.0
> +obj-$(CONFIG_CXL_ACPI) += cxl_acpi.o
> +
> +ccflags-y += -DDEFAULT_SYMBOL_NAMESPACE=CXL
> +cxl_acpi-y := acpi.o
> diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c
> new file mode 100644
> index 000000000000..0f1ba9b3f1ed
> --- /dev/null
> +++ b/drivers/cxl/acpi.c
> @@ -0,0 +1,97 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/* Copyright(c) 2020 Intel Corporation. All rights reserved. */
> +#include <linux/platform_device.h>
> +#include <linux/list_sort.h>
> +#include <linux/module.h>
> +#include <linux/mutex.h>
> +#include <linux/sysfs.h>
> +#include <linux/list.h>
> +#include <linux/acpi.h>
> +#include <linux/sort.h>
> +#include <linux/pci.h>
Trivial: For includes I'm fairly sure general convention (as much as there
is one is to do alphabetical order, not reverse xmas tree).
> +#include "acpi.h"
> +
> +/*
> + * If/when CXL support is defined by other platform firmware the kernel
> + * will need a mechanism to select between the platform specific version
> + * of this routine, until then, hard-code ACPI assumptions
> + */
> +int cxl_bus_acquire(struct pci_dev *pdev)
> +{
> + struct acpi_device *adev;
> + struct pci_dev *root_port;
> + struct device *root;
> +
> + root_port = pcie_find_root_port(pdev);
> + if (!root_port)
> + return -ENXIO;
> +
> + root = root_port->dev.parent;
> + if (!root)
> + return -ENXIO;
> +
> + adev = ACPI_COMPANION(root);
> + if (!adev)
> + return -ENXIO;
> +
> + /* TODO: OSC enabling */
> +
> + return 0;
> +}
> +EXPORT_SYMBOL_GPL(cxl_bus_acquire);
> +
> +static void acpi_cedt_put_table(void *table)
> +{
> + acpi_put_table(table);
> +}
> +
> +static int cxl_acpi_probe(struct platform_device *pdev)
> +{
> + struct acpi_device *adev = ACPI_COMPANION(&pdev->dev);
> + struct cxl_acpi_desc *acpi_desc;
> + struct device *dev = &adev->dev;
> + struct acpi_table_header *tbl;
> + acpi_status status;
> + acpi_size sz;
> + int rc;
> +
> + status = acpi_get_table(ACPI_SIG_CEDT, 0, &tbl);
> + if (ACPI_FAILURE(status)) {
> + dev_err(dev, "failed to find CEDT at startup\n");
> + return 0;
> + }
> +
> + rc = devm_add_action_or_reset(dev, acpi_cedt_put_table, tbl);
> + if (rc)
> + return rc;
> +
> + sz = tbl->length;
> + dev_info(dev, "found CEDT at startup: %lld bytes\n", sz);
Is this useful? At least for my normal acpi boot I'll see CEDT in the list of
detected ACPI tables early in boot. This provides less info than that
print already has.
> +
> + acpi_desc = devm_kzalloc(dev, sizeof(*acpi_desc), GFP_KERNEL);
> + if (!acpi_desc)
> + return -ENOMEM;
> +
> + dev_set_drvdata(&adev->dev, acpi_desc);
> + acpi_desc->dev = &adev->dev;
> + acpi_desc->acpi_header = *tbl;
> +
> + return 0;
> +}
> +
> +static const struct acpi_device_id cxl_acpi_ids[] = {
> + { "ACPI0017", 0 },
> + { "", 0 },
> +};
> +MODULE_DEVICE_TABLE(acpi, cxl_acpi_ids);
> +
> +static struct platform_driver cxl_acpi_driver = {
> + .probe = cxl_acpi_probe,
> + .driver = {
> + .name = KBUILD_MODNAME,
> + .acpi_match_table = cxl_acpi_ids,
> + },
> +};
> +
> +module_platform_driver(cxl_acpi_driver);
> +MODULE_LICENSE("GPL v2");
> diff --git a/drivers/cxl/acpi.h b/drivers/cxl/acpi.h
> new file mode 100644
> index 000000000000..d638f8886ab7
> --- /dev/null
> +++ b/drivers/cxl/acpi.h
> @@ -0,0 +1,15 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/* Copyright(c) 2020 Intel Corporation. All rights reserved. */
> +
> +#ifndef __CXL_ACPI_H__
> +#define __CXL_ACPI_H__
> +#include <linux/acpi.h>
> +
> +struct cxl_acpi_desc {
> + struct acpi_table_header acpi_header;
> + struct device *dev;
> +};
> +
> +int cxl_bus_acquire(struct pci_dev *pci_dev);
> +
> +#endif /* __CXL_ACPI_H__ */
> diff --git a/include/acpi/actbl1.h b/include/acpi/actbl1.h
> index 43549547ed3e..f3a0303984c8 100644
> --- a/include/acpi/actbl1.h
> +++ b/include/acpi/actbl1.h
> @@ -28,6 +28,7 @@
> #define ACPI_SIG_BERT "BERT" /* Boot Error Record Table */
> #define ACPI_SIG_BGRT "BGRT" /* Boot Graphics Resource Table */
> #define ACPI_SIG_BOOT "BOOT" /* Simple Boot Flag Table */
> +#define ACPI_SIG_CEDT "CEDT" /* CXL Early Discovery Table */
> #define ACPI_SIG_CPEP "CPEP" /* Corrected Platform Error Polling table */
> #define ACPI_SIG_CSRT "CSRT" /* Core System Resource Table */
> #define ACPI_SIG_DBG2 "DBG2" /* Debug Port table type 2 */
> @@ -1624,6 +1625,55 @@ struct acpi_ibft_target {
> u16 reverse_chap_secret_offset;
> };
>
> +/*******************************************************************************
> + *
> + * CEDT - CXL Early Discovery Table (ACPI 6.4)
> + * Version 1
> + *
> + ******************************************************************************/
> +
> +struct acpi_table_cedt {
> + struct acpi_table_header header; /* Common ACPI table header */
> + u32 reserved;
> +};
> +
> +/* Values for CEDT structure types */
> +
> +enum acpi_cedt_type {
> + ACPI_CEDT_TYPE_HOST_BRIDGE = 0, /* CHBS - CXL Host Bridge Structure */
> +};
> +
> +struct acpi_cedt_structure {
> + u8 type;
> + u8 reserved;
> + u16 length;
> +};
> +
> +/*
> + * CEDT Structures, correspond to Type in struct acpi_cedt_structure
> + */
> +
> +/* 0: CXL Host Bridge Structure */
> +
> +struct acpi_cedt_chbs {
> + struct acpi_cedt_structure header;
> + u32 uid;
> + u32 version;
> + u32 reserved1;
> + u64 base;
> + u64 length;
> +};
> +
> +/* Values for version field above */
> +
> +#define ACPI_CEDT_CHBS_VERSION_CXL11 (0)
> +#define ACPI_CEDT_CHBS_VERSION_CXL20 (1)
> +
> +/* Values for length field above */
> +
> +#define ACPI_CEDT_CHBS_LENGTH_CXL11 (0x2000)
> +#define ACPI_CEDT_CHBS_LENGTH_CXL20 (0x10000)
> +
> /* Reset to default packing */
>
> #pragma pack()
On Mon, 11 Jan 2021 14:51:07 -0800
Ben Widawsky <[email protected]> wrote:
> From: Vishal Verma <[email protected]>
>
> Add support to advertise OS capabilities, and request OS control for CXL
> features using the ACPI _OSC mechanism. Advertise support for all
> possible CXL features, and attempt to request control for all possible
> features.
>
> Based on a patch by Sean Kelley.
>
> Signed-off-by: Vishal Verma <[email protected]>
> Signed-off-by: Ben Widawsky <[email protected]>
Whilst I understand the reasoning behind not just sharing this code with PCI,
it might be worth adding a comment to the code somewhere on that.
Otherwise, I suspect you'll very rapidly see patches cleaning up this duplication :)
Otherwise LGTM.
J
> ---
> Documentation/cxl/memory-devices.rst | 15 ++
> drivers/cxl/acpi.c | 260 ++++++++++++++++++++++++++-
> drivers/cxl/acpi.h | 20 +++
> 3 files changed, 292 insertions(+), 3 deletions(-)
>
> diff --git a/Documentation/cxl/memory-devices.rst b/Documentation/cxl/memory-devices.rst
> index aa4262280c67..6ce88f9d5f4f 100644
> --- a/Documentation/cxl/memory-devices.rst
> +++ b/Documentation/cxl/memory-devices.rst
> @@ -13,3 +13,18 @@ Driver Infrastructure
> =====================
>
> This sections covers the driver infrastructure for a CXL memory device.
> +
> +ACPI CXL
> +--------
> +
> +.. kernel-doc:: drivers/cxl/acpi.c
> + :doc: cxl acpi
> +
> +.. kernel-doc:: drivers/cxl/acpi.c
> + :internal:
> +
> +External Interfaces
> +===================
> +
> +.. kernel-doc:: drivers/cxl/acpi.c
> + :export:
> diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c
> index 0f1ba9b3f1ed..af9c0dfdee20 100644
> --- a/drivers/cxl/acpi.c
> +++ b/drivers/cxl/acpi.c
> @@ -11,7 +11,258 @@
> #include <linux/pci.h>
> #include "acpi.h"
>
> -/*
> +/**
> + * DOC: cxl acpi
> + *
> + * ACPI _OSC setup: The exported function cxl_bus_acquire() sets up ACPI
> + * Operating System Capabilities (_OSC) for the CXL bus. It declares support
> + * for all CXL capabilities, and attempts to request control for all possible
> + * capabilities. The resulting support and control sets are saved in global
> + * variables cxl_osc_support_set and cxl_osc_control_set. The internal
> + * functions cxl_osc_declare_support(), and cxl_osc_request_control() can be
> + * used to update the support and control sets in accordance with the ACPI
> + * rules for _OSC evaluation - most importantly, capabilities already granted
> + * should not be rescinded by either the OS or firmware.
> + */
> +
> +static u32 cxl_osc_support_set;
> +static u32 cxl_osc_control_set;
> +static DEFINE_MUTEX(acpi_desc_lock);
> +
> +struct pci_osc_bit_struct {
> + u32 bit;
> + char *desc;
> +};
> +
> +static struct pci_osc_bit_struct cxl_osc_support_bit[] = {
> + { CXL_OSC_PORT_REG_ACCESS_SUPPORT, "CXLPortRegAccess" },
> + { CXL_OSC_PORT_DEV_REG_ACCESS_SUPPORT, "CXLPortDevRegAccess" },
> + { CXL_OSC_PER_SUPPORT, "CXLProtocolErrorReporting" },
> + { CXL_OSC_NATIVE_HP_SUPPORT, "CXLNativeHotPlug" },
> +};
> +
> +static struct pci_osc_bit_struct cxl_osc_control_bit[] = {
> + { CXL_OSC_MEM_ERROR_CONTROL, "CXLMemErrorReporting" },
> +};
> +
> +static u8 cxl_osc_uuid_str[] = "68F2D50B-C469-4d8A-BD3D-941A103FD3FC";
> +
> +static void decode_osc_bits(struct device *dev, char *msg, u32 word,
> + struct pci_osc_bit_struct *table, int size)
> +{
> + char buf[80];
> + int i, len = 0;
> + struct pci_osc_bit_struct *entry;
> +
> + buf[0] = '\0';
> + for (i = 0, entry = table; i < size; i++, entry++)
> + if (word & entry->bit)
> + len += scnprintf(buf + len, sizeof(buf) - len, "%s%s",
> + len ? " " : "", entry->desc);
> +
> + dev_info(dev, "_OSC: %s [%s]\n", msg, buf);
> +}
> +
> +static void decode_cxl_osc_support(struct device *dev, char *msg, u32 word)
> +{
> + decode_osc_bits(dev, msg, word, cxl_osc_support_bit,
> + ARRAY_SIZE(cxl_osc_support_bit));
> +}
> +
> +static void decode_cxl_osc_control(struct device *dev, char *msg, u32 word)
> +{
> + decode_osc_bits(dev, msg, word, cxl_osc_control_bit,
> + ARRAY_SIZE(cxl_osc_control_bit));
> +}
> +
> +static acpi_status acpi_cap_run_osc(acpi_handle handle, const u32 *capbuf,
> + u8 *uuid_str, u32 *retval)
> +{
> + struct acpi_osc_context context = {
> + .uuid_str = uuid_str,
> + .rev = 1,
> + .cap.length = 20,
> + .cap.pointer = (void *)capbuf,
> + };
> + acpi_status status;
> +
> + status = acpi_run_osc(handle, &context);
> + if (ACPI_SUCCESS(status)) {
> + /* pointer + offset to DWORD 5 */
> + *retval = *((u32 *)(context.ret.pointer + 16));
> + kfree(context.ret.pointer);
> + }
> + return status;
> +}
> +
> +static acpi_status cxl_query_osc(acpi_handle handle, u32 support, u32 *control)
> +{
> + struct acpi_pci_root *root;
> + acpi_status status;
> + u32 result, capbuf[5];
> +
> + root = acpi_pci_find_root(handle);
> + if (!root)
> + return -ENXIO;
> +
> + support &= CXL_OSC_SUPPORT_VALID_MASK;
> + support |= cxl_osc_support_set;
> +
> + capbuf[OSC_QUERY_DWORD] = OSC_QUERY_ENABLE;
> + capbuf[PCI_OSC_SUPPORT_DWORD] = root->osc_support_set;
> + capbuf[PCI_OSC_CONTROL_DWORD] = root->osc_control_set;
> + capbuf[CXL_OSC_SUPPORT_DWORD] = support;
> + if (control) {
> + *control &= CXL_OSC_CONTROL_VALID_MASK;
> + capbuf[CXL_OSC_CONTROL_DWORD] = *control | cxl_osc_control_set;
> + } else {
> + /* Run _OSC query only with existing controls. */
> + capbuf[CXL_OSC_CONTROL_DWORD] = cxl_osc_control_set;
> + }
> +
> + status = acpi_cap_run_osc(handle, capbuf, cxl_osc_uuid_str, &result);
> + if (ACPI_SUCCESS(status)) {
> + cxl_osc_support_set = support;
> + if (control)
> + *control = result;
> + }
> + return status;
> +}
> +
> +/**
> + * cxl_osc_declare_support - Declare support for CXL root _OSC features.
> + * @handle: ACPI device handle for the PCI root bridge (or PCIe Root Complex).
> + * @flags: The _OSC bits to declare support for.
> + *
> + * The PCI specific DWORDS are obtained from the pci root port device, and
> + * used as-is. The resulting CXL support set is saved in cxl_osc_support_set.
> + * Any future calls to this are forced to be strictly incremental from the
> + * existing cxl_osc_support_set.
> + */
> +static acpi_status cxl_osc_declare_support(acpi_handle handle, u32 flags)
> +{
> + acpi_status status;
> +
> + mutex_lock(&acpi_desc_lock);
> + status = cxl_query_osc(handle, flags, NULL);
> + mutex_unlock(&acpi_desc_lock);
> + return status;
> +}
> +
> +/**
> + * cxl_osc_request_control - Request control of CXL root _OSC features.
> + * @adev: ACPI device for the PCI root bridge (or PCIe Root Complex).
> + * @mask: Mask of _OSC bits to request control of, place to store control mask.
> + * @req: Mask of _OSC bits the control of is essential to the caller.
> + *
> + * The PCI specific DWORDS are obtained from the pci root port device, and
> + * used as-is. The resulting CXL control set is saved in cxl_osc_control_set.
> + * Any future calls to this are forced to be strictly incremental from the
> + * existing cxl_osc_control_set.
> + */
> +static acpi_status cxl_osc_request_control(struct acpi_device *adev, u32 *mask,
> + u32 req)
> +{
> + acpi_handle handle = adev->handle;
> + struct device *dev = &adev->dev;
> + struct acpi_pci_root *root;
> + acpi_status status = AE_OK;
> + u32 ctrl, capbuf[5];
> +
> + if (!mask)
> + return AE_BAD_PARAMETER;
> +
> + ctrl = *mask & CXL_OSC_MEM_ERROR_CONTROL;
> + if ((ctrl & req) != req)
> + return AE_TYPE;
> +
> + root = acpi_pci_find_root(handle);
> + if (!root)
> + return -ENXIO;
> +
> + mutex_lock(&acpi_desc_lock);
> +
> + *mask = ctrl | cxl_osc_control_set;
> + /* No need to evaluate _OSC if the control was already granted. */
> + if ((cxl_osc_control_set & ctrl) == ctrl)
> + goto out;
> +
> + /* Need to check the available controls bits before requesting them. */
> + while (*mask) {
> + status = cxl_query_osc(handle, cxl_osc_support_set, mask);
> + if (ACPI_FAILURE(status))
> + goto out;
> + if (ctrl == *mask)
> + break;
> + decode_cxl_osc_control(dev, "platform does not support",
> + ctrl & ~(*mask));
> + ctrl = *mask;
> + }
> +
> + if ((ctrl & req) != req) {
> + decode_cxl_osc_control(dev,
> + "not requesting control; platform does not support",
> + req & ~(ctrl));
> + status = AE_SUPPORT;
> + goto out;
> + }
> +
> + capbuf[OSC_QUERY_DWORD] = 0;
> + capbuf[PCI_OSC_SUPPORT_DWORD] = root->osc_support_set;
> + capbuf[PCI_OSC_CONTROL_DWORD] = root->osc_control_set;
> + capbuf[CXL_OSC_SUPPORT_DWORD] = cxl_osc_support_set;
> + capbuf[CXL_OSC_CONTROL_DWORD] = ctrl;
> + status = acpi_cap_run_osc(handle, capbuf, cxl_osc_uuid_str, mask);
> + if (ACPI_SUCCESS(status))
> + cxl_osc_control_set = *mask;
> +out:
> + mutex_unlock(&acpi_desc_lock);
> + return status;
> +}
> +
> +static int cxl_negotiate_osc(struct acpi_device *adev)
> +{
> + u32 cxl_support, cxl_control, requested;
> + acpi_handle handle = adev->handle;
> + struct device *dev = &adev->dev;
> + acpi_status status;
> +
> + /* Declare support for everything */
> + cxl_support = CXL_OSC_SUPPORT_VALID_MASK;
> + decode_cxl_osc_support(dev, "OS supports", cxl_support);
> + status = cxl_osc_declare_support(handle, cxl_support);
> + if (ACPI_FAILURE(status)) {
> + dev_info(dev, "CXL_OSC failed (%s)\n",
> + acpi_format_exception(status));
> + return -ENXIO;
> + }
> +
> + /* Request control for everything */
> + cxl_control = CXL_OSC_CONTROL_VALID_MASK;
> + requested = cxl_control;
> + status = cxl_osc_request_control(adev, &cxl_control,
> + CXL_OSC_MEM_ERROR_CONTROL);
> + if (ACPI_SUCCESS(status)) {
> + decode_cxl_osc_control(dev, "OS now controls", cxl_control);
> + } else {
> + decode_cxl_osc_control(dev, "OS requested", requested);
> + decode_cxl_osc_control(dev, "platform willing to grant",
> + cxl_control);
> + dev_info(dev, "_OSC failed (%s)\n",
> + acpi_format_exception(status));
> + }
> + return 0;
> +}
> +
> +/**
> + * cxl_bus_acquire - Perform platform-specific bus operations
> + * @pdev: pci_dev associated with the CXL device
> + *
> + * This performs bus-specific operations such as ACPI _OSC to ensure that
> + * the bus is 'prepared'. Since the CXL definition of _OSC depends on the
> + * existing PCI _OSC DWORDS in 'Arg3', pull those in from the pci root port
> + * device, and merge those into the new CXL-augmented _OSC calls.
> + *
> * If/when CXL support is defined by other platform firmware the kernel
> * will need a mechanism to select between the platform specific version
> * of this routine, until then, hard-code ACPI assumptions
> @@ -21,6 +272,7 @@ int cxl_bus_acquire(struct pci_dev *pdev)
> struct acpi_device *adev;
> struct pci_dev *root_port;
> struct device *root;
> + int rc;
>
> root_port = pcie_find_root_port(pdev);
> if (!root_port)
> @@ -34,9 +286,11 @@ int cxl_bus_acquire(struct pci_dev *pdev)
> if (!adev)
> return -ENXIO;
>
> - /* TODO: OSC enabling */
> + rc = cxl_negotiate_osc(adev);
> + if (rc)
> + dev_err(&pdev->dev, "Failed to negotiate OSC\n");
>
> - return 0;
> + return rc;
> }
> EXPORT_SYMBOL_GPL(cxl_bus_acquire);
>
> diff --git a/drivers/cxl/acpi.h b/drivers/cxl/acpi.h
> index d638f8886ab7..6ef154021745 100644
> --- a/drivers/cxl/acpi.h
> +++ b/drivers/cxl/acpi.h
> @@ -12,4 +12,24 @@ struct cxl_acpi_desc {
>
> int cxl_bus_acquire(struct pci_dev *pci_dev);
>
> +/* Indexes into _OSC Capabilities Buffer */
> +#define PCI_OSC_SUPPORT_DWORD 1 /* DWORD 2 */
> +#define PCI_OSC_CONTROL_DWORD 2 /* DWORD 3 */
> +#define CXL_OSC_SUPPORT_DWORD 3 /* DWORD 4 */
> +#define CXL_OSC_CONTROL_DWORD 4 /* DWORD 5 */
> +
> +/* CXL Host Bridge _OSC: Capabilities DWORD 4: Support Field */
> +#define CXL_OSC_PORT_REG_ACCESS_SUPPORT 0x00000001
> +#define CXL_OSC_PORT_DEV_REG_ACCESS_SUPPORT 0x00000002
> +#define CXL_OSC_PER_SUPPORT 0x00000004
> +#define CXL_OSC_NATIVE_HP_SUPPORT 0x00000008
> +#define CXL_OSC_SUPPORT_VALID_MASK (CXL_OSC_PORT_REG_ACCESS_SUPPORT | \
> + CXL_OSC_PORT_DEV_REG_ACCESS_SUPPORT | \
> + CXL_OSC_PER_SUPPORT | \
> + CXL_OSC_NATIVE_HP_SUPPORT)
> +
> +/* CXL Host Bridge _OSC: Capabilities DWORD 5: Control Field */
> +#define CXL_OSC_MEM_ERROR_CONTROL 0x00000001
> +#define CXL_OSC_CONTROL_VALID_MASK (CXL_OSC_MEM_ERROR_CONTROL)
> +
> #endif /* __CXL_ACPI_H__ */
On Mon, 11 Jan 2021 14:51:08 -0800
Ben Widawsky <[email protected]> wrote:
> From: Dan Williams <[email protected]>
>
> The CXL.mem protocol allows a device to act as a provider of "System
> RAM" and/or "Persistent Memory" that is fully coherent as if the memory
> was attached to the typical CPU memory controller.
>
> With the CXL-2.0 specification a PCI endpoint can implement a "Type-3"
> device interface and give the operating system control over "Host
> Managed Device Memory". See section 2.3 Type 3 CXL Device.
>
> The memory range exported by the device may optionally be described by
> the platform firmware memory map, or by infrastructure like LIBNVDIMM to
> provision persistent memory capacity from one, or more, CXL.mem devices.
>
> A pre-requisite for Linux-managed memory-capacity provisioning is this
> cxl_mem driver that can speak the mailbox protocol defined in section
> 8.2.8.4 Mailbox Registers.
>
> For now just land the driver boiler-plate and fill it in with
> functionality in subsequent commits.
>
> Link: https://www.computeexpresslink.org/download-the-specification
> Signed-off-by: Dan Williams <[email protected]>
> Signed-off-by: Ben Widawsky <[email protected]>
Just one passing comment inline.
> diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
> new file mode 100644
> index 000000000000..005404888942
> --- /dev/null
> +++ b/drivers/cxl/mem.c
> @@ -0,0 +1,69 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/* Copyright(c) 2020 Intel Corporation. All rights reserved. */
> +#include <linux/module.h>
> +#include <linux/pci.h>
> +#include <linux/io.h>
> +#include "acpi.h"
> +#include "pci.h"
> +
> +static int cxl_mem_dvsec(struct pci_dev *pdev, int dvsec)
Is it worth pulling this out to a utility library now as we are going
to keep needing this for CXL devices?
Arguably, with a vendor_id parameter it might make sense to have
it as a utility function for pci rather than CXL alone.
> +{
> + int pos;
> +
> + pos = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_DVSEC);
> + if (!pos)
> + return 0;
> +
> + while (pos) {
> + u16 vendor, id;
> +
> + pci_read_config_word(pdev, pos + PCI_DVSEC_VENDOR_ID_OFFSET,
> + &vendor);
> + pci_read_config_word(pdev, pos + PCI_DVSEC_ID_OFFSET, &id);
> + if (vendor == PCI_DVSEC_VENDOR_ID_CXL && dvsec == id)
> + return pos;
> +
> + pos = pci_find_next_ext_capability(pdev, pos,
> + PCI_EXT_CAP_ID_DVSEC);
> + }
> +
> + return 0;
> +}
> +
> +static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> +{
> + struct device *dev = &pdev->dev;
> + int rc, regloc;
> +
> + rc = cxl_bus_acquire(pdev);
> + if (rc != 0) {
> + dev_err(dev, "failed to acquire interface\n");
> + return rc;
> + }
> +
> + regloc = cxl_mem_dvsec(pdev, PCI_DVSEC_ID_CXL_REGLOC);
> + if (!regloc) {
> + dev_err(dev, "register location dvsec not found\n");
> + return -ENXIO;
> + }
> +
> + return 0;
> +}
> +
> +static const struct pci_device_id cxl_mem_pci_tbl[] = {
> + /* PCI class code for CXL.mem Type-3 Devices */
> + { PCI_ANY_ID, PCI_ANY_ID, PCI_ANY_ID, PCI_ANY_ID,
> + PCI_CLASS_MEMORY_CXL, 0xffffff, 0 },
> + { /* terminate list */ },
> +};
> +MODULE_DEVICE_TABLE(pci, cxl_mem_pci_tbl);
> +
> +static struct pci_driver cxl_mem_driver = {
> + .name = KBUILD_MODNAME,
> + .id_table = cxl_mem_pci_tbl,
> + .probe = cxl_mem_probe,
> +};
> +
> +MODULE_LICENSE("GPL v2");
> +module_pci_driver(cxl_mem_driver);
> +MODULE_IMPORT_NS(CXL);
> diff --git a/drivers/cxl/pci.h b/drivers/cxl/pci.h
> new file mode 100644
> index 000000000000..a8a9935fa90b
> --- /dev/null
> +++ b/drivers/cxl/pci.h
> @@ -0,0 +1,20 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/* Copyright(c) 2020 Intel Corporation. All rights reserved. */
> +#ifndef __CXL_PCI_H__
> +#define __CXL_PCI_H__
> +
> +#define PCI_CLASS_MEMORY_CXL 0x050210
> +
> +/*
> + * See section 8.1 Configuration Space Registers in the CXL 2.0
> + * Specification
> + */
> +#define PCI_EXT_CAP_ID_DVSEC 0x23
> +#define PCI_DVSEC_VENDOR_ID_CXL 0x1E98
> +#define PCI_DVSEC_VENDOR_ID_OFFSET 0x4
> +#define PCI_DVSEC_ID_CXL 0x0
> +#define PCI_DVSEC_ID_OFFSET 0x8
> +
> +#define PCI_DVSEC_ID_CXL_REGLOC 0x8
> +
> +#endif /* __CXL_PCI_H__ */
On Tue, Jan 12, 2021 at 10:44 AM Jonathan Cameron
<[email protected]> wrote:
>
> On Mon, 11 Jan 2021 14:51:06 -0800
> Ben Widawsky <[email protected]> wrote:
>
> > From: Vishal Verma <[email protected]>
> >
> > Add an acpi_cxl module to coordinate the ACPI portions of the CXL
> > (Compute eXpress Link) interconnect. This driver binds to ACPI0017
> > objects in the ACPI tree, and coordinates access to the resources
> > provided by the ACPI CEDT (CXL Early Discovery Table).
> >
> > It also coordinates operations of the root port _OSC object to notify
> > platform firmware that the OS has native support for the CXL
> > capabilities of endpoints.
> >
> > Note: the actbl1.h changes are speculative. The expectation is that they
> > will arrive through the ACPICA tree in due time.
>
> I would pull the ACPICA changes out into a precursor patch.
>
> >
> > Cc: Ben Widawsky <[email protected]>
> > Cc: Dan Williams <[email protected]>
> > Signed-off-by: Vishal Verma <[email protected]>
> > Signed-off-by: Ben Widawsky <[email protected]>
>
> Hi,
>
> I think it would be good to also add CEDT to the list in drivers/acpi/tables.c
> so that we can dump it from /sys/firmware/acpi/tables/ and potentially
> override it from an initrd.
ACPICA changes will eventually come through the ACPI tree not this patch set.
>
> https://elixir.bootlin.com/linux/v5.11-rc3/source/drivers/acpi/tables.c#L482
> Can be very helpful whilst debugging. Related to that, anyone know if anyone
> has acpica patches so we can have iasl -d work on the table? Would probably
> be useful but I'd rather not duplicate work if it's already done.
>
The supplemental tables described here:
https://www.uefi.org/acpi
...do eventually make there way into ACPICA. Added Bob and Erik in
case they can comment on when CEDT and CDAT support will be picked up.
> A few minor things inline
>
> Jonathan
>
> > ---
> > drivers/Kconfig | 1 +
> > drivers/Makefile | 1 +
> > drivers/cxl/Kconfig | 36 ++++++++++++++++
> > drivers/cxl/Makefile | 5 +++
> > drivers/cxl/acpi.c | 97 +++++++++++++++++++++++++++++++++++++++++++
> > drivers/cxl/acpi.h | 15 +++++++
> > include/acpi/actbl1.h | 50 ++++++++++++++++++++++
> > 7 files changed, 205 insertions(+)
> > create mode 100644 drivers/cxl/Kconfig
> > create mode 100644 drivers/cxl/Makefile
> > create mode 100644 drivers/cxl/acpi.c
> > create mode 100644 drivers/cxl/acpi.h
> >
> > diff --git a/drivers/Kconfig b/drivers/Kconfig
> > index dcecc9f6e33f..62c753a73651 100644
> > --- a/drivers/Kconfig
> > +++ b/drivers/Kconfig
> > @@ -6,6 +6,7 @@ menu "Device Drivers"
> > source "drivers/amba/Kconfig"
> > source "drivers/eisa/Kconfig"
> > source "drivers/pci/Kconfig"
> > +source "drivers/cxl/Kconfig"
> > source "drivers/pcmcia/Kconfig"
> > source "drivers/rapidio/Kconfig"
> >
> > diff --git a/drivers/Makefile b/drivers/Makefile
> > index fd11b9ac4cc3..678ea810410f 100644
> > --- a/drivers/Makefile
> > +++ b/drivers/Makefile
> > @@ -73,6 +73,7 @@ obj-$(CONFIG_NVM) += lightnvm/
> > obj-y += base/ block/ misc/ mfd/ nfc/
> > obj-$(CONFIG_LIBNVDIMM) += nvdimm/
> > obj-$(CONFIG_DAX) += dax/
> > +obj-$(CONFIG_CXL_BUS) += cxl/
> > obj-$(CONFIG_DMA_SHARED_BUFFER) += dma-buf/
> > obj-$(CONFIG_NUBUS) += nubus/
> > obj-y += macintosh/
> > diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig
> > new file mode 100644
> > index 000000000000..68da926ba5b1
> > --- /dev/null
> > +++ b/drivers/cxl/Kconfig
> > @@ -0,0 +1,36 @@
> > +# SPDX-License-Identifier: GPL-2.0-only
> > +menuconfig CXL_BUS
> > + tristate "CXL (Compute Express Link) Devices Support"
> > + help
> > + CXL is a bus that is electrically compatible with PCI-E, but layers
>
> For consistency with Kconfig in driver/pci probably better to spell PCI Express
> throughout.
Sure.
>
> > + three protocols on that signalling (CXL.io, CXL.cache, and CXL.mem).
> > + The CXL.cache protocol allows devices to hold cachelines locally, the
> > + CXL.mem protocol allows devices to be fully coherent memory targets,
> > + the CXL.io protocol is equivalent to PCI-E. Say 'y' to enable support
> > + for the configuration and management of devices supporting these
> > + protocols.
> > +
> > +if CXL_BUS
> > +
> > +config CXL_BUS_PROVIDER
> > + tristate
> > +
> > +config CXL_ACPI
> > + tristate "CXL ACPI: Platform Support"
> > + depends on ACPI
> > + default CXL_BUS
> > + select CXL_BUS_PROVIDER
> > + help
> > + Say 'y/m' to enable a driver (named "cxl_acpi.ko" when built
> > + as a module) that will enable support for CXL.mem endpoint
> > + devices. In general, CXL Platform Support is a prerequisite
> > + for any CXL device driver that wants to claim ownership of a
> > + component register space. By default platform firmware assumes
> > + Linux is unaware of CXL capabilities and requires explicit
> > + opt-in. This platform component also mediates resources
> > + described by the CEDT (CXL Early Discovery Table). See
> > + Chapter 9.14.1 CXL Early Discovery Table (CEDT) in the CXL 2.0
> > + specification.
> > +
> > + If unsure say 'm'
> > +endif
> > diff --git a/drivers/cxl/Makefile b/drivers/cxl/Makefile
> > new file mode 100644
> > index 000000000000..d38cd34a2582
> > --- /dev/null
> > +++ b/drivers/cxl/Makefile
> > @@ -0,0 +1,5 @@
> > +# SPDX-License-Identifier: GPL-2.0
> > +obj-$(CONFIG_CXL_ACPI) += cxl_acpi.o
> > +
> > +ccflags-y += -DDEFAULT_SYMBOL_NAMESPACE=CXL
> > +cxl_acpi-y := acpi.o
> > diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c
> > new file mode 100644
> > index 000000000000..0f1ba9b3f1ed
> > --- /dev/null
> > +++ b/drivers/cxl/acpi.c
> > @@ -0,0 +1,97 @@
> > +// SPDX-License-Identifier: GPL-2.0-only
> > +/* Copyright(c) 2020 Intel Corporation. All rights reserved. */
> > +#include <linux/platform_device.h>
> > +#include <linux/list_sort.h>
> > +#include <linux/module.h>
> > +#include <linux/mutex.h>
> > +#include <linux/sysfs.h>
> > +#include <linux/list.h>
> > +#include <linux/acpi.h>
> > +#include <linux/sort.h>
> > +#include <linux/pci.h>
>
> Trivial: For includes I'm fairly sure general convention (as much as there
> is one is to do alphabetical order, not reverse xmas tree).
No, there's no guidance for this in coding-style so it's one of those
maintainer preference items.
>
> > +#include "acpi.h"
> > +
> > +/*
> > + * If/when CXL support is defined by other platform firmware the kernel
> > + * will need a mechanism to select between the platform specific version
> > + * of this routine, until then, hard-code ACPI assumptions
> > + */
> > +int cxl_bus_acquire(struct pci_dev *pdev)
> > +{
> > + struct acpi_device *adev;
> > + struct pci_dev *root_port;
> > + struct device *root;
> > +
> > + root_port = pcie_find_root_port(pdev);
> > + if (!root_port)
> > + return -ENXIO;
> > +
> > + root = root_port->dev.parent;
> > + if (!root)
> > + return -ENXIO;
> > +
> > + adev = ACPI_COMPANION(root);
> > + if (!adev)
> > + return -ENXIO;
> > +
> > + /* TODO: OSC enabling */
> > +
> > + return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(cxl_bus_acquire);
> > +
> > +static void acpi_cedt_put_table(void *table)
> > +{
> > + acpi_put_table(table);
> > +}
> > +
> > +static int cxl_acpi_probe(struct platform_device *pdev)
> > +{
> > + struct acpi_device *adev = ACPI_COMPANION(&pdev->dev);
> > + struct cxl_acpi_desc *acpi_desc;
> > + struct device *dev = &adev->dev;
> > + struct acpi_table_header *tbl;
> > + acpi_status status;
> > + acpi_size sz;
> > + int rc;
> > +
> > + status = acpi_get_table(ACPI_SIG_CEDT, 0, &tbl);
> > + if (ACPI_FAILURE(status)) {
> > + dev_err(dev, "failed to find CEDT at startup\n");
> > + return 0;
> > + }
> > +
> > + rc = devm_add_action_or_reset(dev, acpi_cedt_put_table, tbl);
> > + if (rc)
> > + return rc;
> > +
> > + sz = tbl->length;
> > + dev_info(dev, "found CEDT at startup: %lld bytes\n", sz);
>
> Is this useful? At least for my normal acpi boot I'll see CEDT in the list of
> detected ACPI tables early in boot. This provides less info than that
> print already has.
Agree.
CXL devices contain an array of capabilities that describe the
interactions software can have with the device or firmware running on
the device. A CXL compliant device must implement the device status and
the mailbox capability. A CXL compliant memory device must implement the
memory device capability.
Each of the capabilities can [will] provide an offset within the MMIO
region for interacting with the CXL device.
For more details see 8.2.8 of the CXL 2.0 specification.
Link: Link: https://www.computeexpresslink.org/download-the-specification
Signed-off-by: Ben Widawsky <[email protected]>
---
drivers/cxl/cxl.h | 82 ++++++++++++++++++++++++++++++++++++++-
drivers/cxl/mem.c | 97 +++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 178 insertions(+), 1 deletion(-)
diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
index d81d0ba4617c..a77286d04ce4 100644
--- a/drivers/cxl/cxl.h
+++ b/drivers/cxl/cxl.h
@@ -4,6 +4,38 @@
#ifndef __CXL_H__
#define __CXL_H__
+#include <linux/bitfield.h>
+#include <linux/bitops.h>
+#include <linux/io.h>
+
+#define CXL_SET_FIELD(value, field) \
+ ({ \
+ WARN_ON(!FIELD_FIT(field##_MASK, value)); \
+ FIELD_PREP(field##_MASK, value); \
+ })
+
+#define CXL_GET_FIELD(word, field) FIELD_GET(field##_MASK, word)
+
+/* Device (CXL 2.0 - 8.2.8.3) */
+#define CXLDEV_CAP_ARRAY_REG 0x0
+#define CXLDEV_CAP_ARRAY_CAP_ID 0
+#define CXLDEV_CAP_ARRAY_ID(x) ((x) & (0xffff))
+#define CXLDEV_CAP_ARRAY_COUNT(x) (((x) >> 32) & 0xffff)
+
+#define CXL_CAP_CAP_ID_DEVICE_STATUS 0x1
+#define CXL_CAP_CAP_ID_PRIMARY_MAILBOX 0x2
+#define CXL_CAP_CAP_ID_SECONDARY_MAILBOX 0x3
+#define CXL_CAP_CAP_ID_MEMDEV 0x4000
+
+/* Mailbox (CXL 2.0 - 8.2.8.4) */
+#define CXLDEV_MB_CAPS_OFFSET 0x00
+#define CXLDEV_MB_CAP_PAYLOAD_SIZE_MASK GENMASK(4, 0)
+#define CXLDEV_MB_CAP_PAYLOAD_SIZE_SHIFT 0
+#define CXLDEV_MB_CTRL_OFFSET 0x04
+#define CXLDEV_MB_CMD_OFFSET 0x08
+#define CXLDEV_MB_STATUS_OFFSET 0x10
+#define CXLDEV_MB_BG_CMD_STATUS_OFFSET 0x18
+
/**
* struct cxl_mem - A CXL memory device
* @pdev: The PCI device associated with this CXL device.
@@ -12,6 +44,54 @@
struct cxl_mem {
struct pci_dev *pdev;
void __iomem *regs;
+
+ /* Cap 0001h - CXL_CAP_CAP_ID_DEVICE_STATUS */
+ struct {
+ void __iomem *regs;
+ } status;
+
+ /* Cap 0002h - CXL_CAP_CAP_ID_PRIMARY_MAILBOX */
+ struct {
+ void __iomem *regs;
+ size_t payload_size;
+ } mbox;
+
+ /* Cap 4000h - CXL_CAP_CAP_ID_MEMDEV */
+ struct {
+ void __iomem *regs;
+ } mem;
};
-#endif
+#define cxl_reg(type) \
+ static inline void cxl_write_##type##_reg32(struct cxl_mem *cxlm, \
+ u32 reg, u32 value) \
+ { \
+ void __iomem *reg_addr = cxlm->type.regs; \
+ writel(value, reg_addr + reg); \
+ } \
+ static inline void cxl_write_##type##_reg64(struct cxl_mem *cxlm, \
+ u32 reg, u64 value) \
+ { \
+ void __iomem *reg_addr = cxlm->type.regs; \
+ writeq(value, reg_addr + reg); \
+ } \
+ static inline u32 cxl_read_##type##_reg32(struct cxl_mem *cxlm, \
+ u32 reg) \
+ { \
+ void __iomem *reg_addr = cxlm->type.regs; \
+ return readl(reg_addr + reg); \
+ } \
+ static inline u64 cxl_read_##type##_reg64(struct cxl_mem *cxlm, \
+ u32 reg) \
+ { \
+ void __iomem *reg_addr = cxlm->type.regs; \
+ return readq(reg_addr + reg); \
+ }
+
+cxl_reg(status);
+cxl_reg(mbox);
+
+#define cxl_payload_regs(cxlm) \
+ ((void __iomem *)(cxlm)->mbox.regs + CXLDEV_MB_PAYLOAD_OFFSET)
+
+#endif /* __CXL_H__ */
diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
index 8301db34d2ff..8da9f4a861ea 100644
--- a/drivers/cxl/mem.c
+++ b/drivers/cxl/mem.c
@@ -7,6 +7,91 @@
#include "pci.h"
#include "cxl.h"
+/**
+ * cxl_mem_setup_regs() - Setup necessary MMIO.
+ * @cxlm: The CXL memory device to communicate with.
+ *
+ * Return: 0 if all necessary registers mapped.
+ *
+ * A memory device is required by spec to implement a certain set of MMIO
+ * regions. The purpose of this function is to enumerate and map those
+ * registers.
+ *
+ * XXX: Register accessors need the mappings set up by this function, so
+ * any reads or writes must be read(b|w|l|q) or write(b|w|l|q)
+ */
+static int cxl_mem_setup_regs(struct cxl_mem *cxlm)
+{
+ u64 cap_array;
+ int cap;
+
+ /*
+ */
+ cap_array = readq(cxlm->regs + CXLDEV_CAP_ARRAY_REG);
+ if (CXLDEV_CAP_ARRAY_ID(cap_array) != CXLDEV_CAP_ARRAY_CAP_ID)
+ return -ENODEV;
+
+ for (cap = 1; cap <= CXLDEV_CAP_ARRAY_COUNT(cap_array); cap++) {
+ void __iomem *register_block;
+ u32 offset;
+ u16 cap_id;
+
+ cap_id = readl(cxlm->regs + cap * 0x10) & 0xffff;
+ offset = readl(cxlm->regs + cap * 0x10 + 0x4);
+ register_block = cxlm->regs + offset;
+
+ switch (cap_id) {
+ case CXL_CAP_CAP_ID_DEVICE_STATUS:
+ dev_dbg(&cxlm->pdev->dev,
+ "found Status capability (0x%x)\n", offset);
+ cxlm->status.regs = register_block;
+ break;
+ case CXL_CAP_CAP_ID_PRIMARY_MAILBOX:
+ dev_dbg(&cxlm->pdev->dev,
+ "found Mailbox capability (0x%x)\n", offset);
+ cxlm->mbox.regs = register_block;
+ break;
+ case CXL_CAP_CAP_ID_SECONDARY_MAILBOX:
+ dev_dbg(&cxlm->pdev->dev,
+ "found Secondary Mailbox capability (0x%x)\n",
+ offset);
+ break;
+ case CXL_CAP_CAP_ID_MEMDEV:
+ dev_dbg(&cxlm->pdev->dev,
+ "found Memory Device capability (0x%x)\n",
+ offset);
+ cxlm->mem.regs = register_block;
+ break;
+ default:
+ dev_warn(&cxlm->pdev->dev,
+ "Unknown cap ID: %d (0x%x)\n", cap_id, offset);
+ break;
+ }
+ }
+
+ if (!cxlm->status.regs || !cxlm->mbox.regs || !cxlm->mem.regs)
+ return -ENXIO;
+
+ return 0;
+}
+
+static int cxl_mem_setup_mailbox(struct cxl_mem *cxlm)
+{
+ const int cap = cxl_read_mbox_reg32(cxlm, CXLDEV_MB_CAPS_OFFSET);
+
+ cxlm->mbox.payload_size =
+ 1 << CXL_GET_FIELD(cap, CXLDEV_MB_CAP_PAYLOAD_SIZE);
+
+ /* 8.2.8.4.3 */
+ if (cxlm->mbox.payload_size < 256)
+ return -ENXIO;
+
+ dev_dbg(&cxlm->pdev->dev, "Mailbox payload sized %zu",
+ cxlm->mbox.payload_size);
+
+ return 0;
+}
+
/**
* cxl_mem_create() - Create a new &struct cxl_mem.
* @pdev: The pci device associated with the new &struct cxl_mem.
@@ -129,8 +214,20 @@ static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id)
if (rc)
return rc;
+ rc = cxl_mem_setup_regs(cxlm);
+ if (rc)
+ goto err;
+
+ rc = cxl_mem_setup_mailbox(cxlm);
+ if (rc)
+ goto err;
+
pci_set_drvdata(pdev, cxlm);
return 0;
+
+err:
+ kfree(cxlm);
+ return rc;
}
static void cxl_mem_remove(struct pci_dev *pdev)
--
2.30.0
All the necessary bits are initialized in order to find and map the
register space for CXL Memory Devices. This is accomplished by using the
Register Locator DVSEC (CXL 2.0 - 8.1.9.1) to determine which PCI BAR to
use, and how much of an offset from that BAR should be added.
If the memory device registers are found and mapped a new internal data
structure tracking device state is allocated.
Signed-off-by: Ben Widawsky <[email protected]>
---
drivers/cxl/cxl.h | 17 ++++++++
drivers/cxl/mem.c | 100 +++++++++++++++++++++++++++++++++++++++++++++-
drivers/cxl/pci.h | 14 +++++++
3 files changed, 130 insertions(+), 1 deletion(-)
create mode 100644 drivers/cxl/cxl.h
diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
new file mode 100644
index 000000000000..d81d0ba4617c
--- /dev/null
+++ b/drivers/cxl/cxl.h
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/* Copyright(c) 2020 Intel Corporation. */
+
+#ifndef __CXL_H__
+#define __CXL_H__
+
+/**
+ * struct cxl_mem - A CXL memory device
+ * @pdev: The PCI device associated with this CXL device.
+ * @regs: IO mappings to the device's MMIO
+ */
+struct cxl_mem {
+ struct pci_dev *pdev;
+ void __iomem *regs;
+};
+
+#endif
diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
index 005404888942..8301db34d2ff 100644
--- a/drivers/cxl/mem.c
+++ b/drivers/cxl/mem.c
@@ -5,6 +5,58 @@
#include <linux/io.h>
#include "acpi.h"
#include "pci.h"
+#include "cxl.h"
+
+/**
+ * cxl_mem_create() - Create a new &struct cxl_mem.
+ * @pdev: The pci device associated with the new &struct cxl_mem.
+ * @reg_lo: Lower 32b of the register locator
+ * @reg_hi: Upper 32b of the register locator.
+ *
+ * Return: The new &struct cxl_mem on success, NULL on failure.
+ *
+ * Map the BAR for a CXL memory device. This BAR has the memory device's
+ * registers for the device as specified in CXL specification.
+ */
+static struct cxl_mem *cxl_mem_create(struct pci_dev *pdev, u32 reg_lo,
+ u32 reg_hi)
+{
+ struct device *dev = &pdev->dev;
+ struct cxl_mem *cxlm;
+ void __iomem *regs;
+ u64 offset;
+ u8 bar;
+ int rc;
+
+ offset = ((u64)reg_hi << 32) | (reg_lo & CXL_REGLOC_ADDR_MASK);
+ bar = (reg_lo >> CXL_REGLOC_BIR_SHIFT) & CXL_REGLOC_BIR_MASK;
+
+ /* Basic sanity check that BAR is big enough */
+ if (pci_resource_len(pdev, bar) < offset) {
+ dev_err(dev, "BAR%d: %pr: too small (offset: %#llx)\n", bar,
+ &pdev->resource[bar], (unsigned long long)offset);
+ return NULL;
+ }
+
+ rc = pcim_iomap_regions(pdev, BIT(bar), pci_name(pdev));
+ if (rc != 0) {
+ dev_err(dev, "failed to map registers\n");
+ return NULL;
+ }
+
+ cxlm = devm_kzalloc(&pdev->dev, sizeof(*cxlm), GFP_KERNEL);
+ if (!cxlm) {
+ dev_err(dev, "No memory available\n");
+ return NULL;
+ }
+
+ regs = pcim_iomap_table(pdev)[bar];
+ cxlm->pdev = pdev;
+ cxlm->regs = regs + offset;
+
+ dev_dbg(dev, "Mapped CXL Memory Device resource\n");
+ return cxlm;
+}
static int cxl_mem_dvsec(struct pci_dev *pdev, int dvsec)
{
@@ -33,7 +85,8 @@ static int cxl_mem_dvsec(struct pci_dev *pdev, int dvsec)
static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id)
{
struct device *dev = &pdev->dev;
- int rc, regloc;
+ struct cxl_mem *cxlm;
+ int rc, regloc, i;
rc = cxl_bus_acquire(pdev);
if (rc != 0) {
@@ -41,15 +94,59 @@ static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id)
return rc;
}
+ rc = pcim_enable_device(pdev);
+ if (rc)
+ return rc;
+
regloc = cxl_mem_dvsec(pdev, PCI_DVSEC_ID_CXL_REGLOC);
if (!regloc) {
dev_err(dev, "register location dvsec not found\n");
return -ENXIO;
}
+ regloc += 0xc; /* Skip DVSEC + reserved fields */
+
+ rc = -ENXIO;
+ for (i = regloc; i < regloc + 0x24; i += 8) {
+ u32 reg_lo, reg_hi;
+ u8 reg_type;
+
+ /* "register low and high" contain other bits */
+ pci_read_config_dword(pdev, i, ®_lo);
+ pci_read_config_dword(pdev, i + 4, ®_hi);
+
+ reg_type =
+ (reg_lo >> CXL_REGLOC_RBI_SHIFT) & CXL_REGLOC_RBI_MASK;
+
+ if (reg_type == CXL_REGLOC_RBI_MEMDEV) {
+ rc = 0;
+ cxlm = cxl_mem_create(pdev, reg_lo, reg_hi);
+ if (!cxlm)
+ rc = -ENODEV;
+ break;
+ }
+ }
+
+ if (rc)
+ return rc;
+ pci_set_drvdata(pdev, cxlm);
return 0;
}
+static void cxl_mem_remove(struct pci_dev *pdev)
+{
+ struct cxl_mem *cxlm;
+
+ cxlm = pci_get_drvdata(pdev);
+ if (!cxlm)
+ return;
+
+ kfree(cxlm);
+
+ pcim_iounmap_regions(pdev, ~0);
+ pci_set_drvdata(pdev, NULL);
+}
+
static const struct pci_device_id cxl_mem_pci_tbl[] = {
/* PCI class code for CXL.mem Type-3 Devices */
{ PCI_ANY_ID, PCI_ANY_ID, PCI_ANY_ID, PCI_ANY_ID,
@@ -62,6 +159,7 @@ static struct pci_driver cxl_mem_driver = {
.name = KBUILD_MODNAME,
.id_table = cxl_mem_pci_tbl,
.probe = cxl_mem_probe,
+ .remove = cxl_mem_remove,
};
MODULE_LICENSE("GPL v2");
diff --git a/drivers/cxl/pci.h b/drivers/cxl/pci.h
index a8a9935fa90b..df222edb6ac3 100644
--- a/drivers/cxl/pci.h
+++ b/drivers/cxl/pci.h
@@ -17,4 +17,18 @@
#define PCI_DVSEC_ID_CXL_REGLOC 0x8
+/* BAR Indicator Register (BIR) */
+#define CXL_REGLOC_BIR_SHIFT 0
+#define CXL_REGLOC_BIR_MASK 0x7
+
+/* Register Block Identifier (RBI) */
+#define CXL_REGLOC_RBI_SHIFT 8
+#define CXL_REGLOC_RBI_MASK 0xff
+#define CXL_REGLOC_RBI_EMPTY 0
+#define CXL_REGLOC_RBI_COMPONENT 1
+#define CXL_REGLOC_RBI_VIRT 2
+#define CXL_REGLOC_RBI_MEMDEV 3
+
+#define CXL_REGLOC_ADDR_MASK 0xffff0000
+
#endif /* __CXL_PCI_H__ */
--
2.30.0
Add a straightforward IOCTL that provides a mechanism for userspace to
query the supported memory device commands. CXL commands as they appear
to userspace are described as part of the UAPI kerneldoc. The command
list returned via this IOCTL will contain the full set of commands that
the driver supports, however, some of those commands may not be
available for use by userspace.
Memory device commands are specified in 8.2.9 of the CXL 2.0
specification. They are submitted through a mailbox mechanism specified
in 8.2.8.4.
Signed-off-by: Ben Widawsky <[email protected]>
---
.clang-format | 1 +
Documentation/cxl/memory-devices.rst | 9 +
.../userspace-api/ioctl/ioctl-number.rst | 1 +
drivers/cxl/mem.c | 156 ++++++++++++++++++
include/uapi/linux/cxl_mem.h | 117 +++++++++++++
5 files changed, 284 insertions(+)
create mode 100644 include/uapi/linux/cxl_mem.h
diff --git a/.clang-format b/.clang-format
index 10dc5a9a61b3..3f11c8901b43 100644
--- a/.clang-format
+++ b/.clang-format
@@ -109,6 +109,7 @@ ForEachMacros:
- 'css_for_each_child'
- 'css_for_each_descendant_post'
- 'css_for_each_descendant_pre'
+ - 'cxl_for_each_cmd'
- 'device_for_each_child_node'
- 'dma_fence_chain_for_each'
- 'do_for_each_ftrace_op'
diff --git a/Documentation/cxl/memory-devices.rst b/Documentation/cxl/memory-devices.rst
index 5f723c25382b..ec54674b3822 100644
--- a/Documentation/cxl/memory-devices.rst
+++ b/Documentation/cxl/memory-devices.rst
@@ -32,6 +32,15 @@ CXL Memory Device
.. kernel-doc:: drivers/cxl/mem.c
:internal:
+CXL IOCTL Interface
+-------------------
+
+.. kernel-doc:: include/uapi/linux/cxl_mem.h
+ :doc: UAPI
+
+.. kernel-doc:: include/uapi/linux/cxl_mem.h
+ :internal:
+
External Interfaces
===================
diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst b/Documentation/userspace-api/ioctl/ioctl-number.rst
index a4c75a28c839..6eb8e634664d 100644
--- a/Documentation/userspace-api/ioctl/ioctl-number.rst
+++ b/Documentation/userspace-api/ioctl/ioctl-number.rst
@@ -352,6 +352,7 @@ Code Seq# Include File Comments
<mailto:[email protected]>
0xCC 00-0F drivers/misc/ibmvmc.h pseries VMC driver
0xCD 01 linux/reiserfs_fs.h
+0xCE 01-02 uapi/linux/cxl_mem.h Compute Express Link Memory Devices
0xCF 02 fs/cifs/ioctl.c
0xDB 00-0F drivers/char/mwave/mwavepub.h
0xDD 00-3F ZFCP device driver see drivers/s390/scsi/
diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
index da2bf941fe92..d4eb3f5b9469 100644
--- a/drivers/cxl/mem.c
+++ b/drivers/cxl/mem.c
@@ -7,6 +7,7 @@
#include <linux/idr.h>
#include <linux/pci.h>
#include <linux/io.h>
+#include <uapi/linux/cxl_mem.h>
#include "acpi.h"
#include "pci.h"
#include "cxl.h"
@@ -41,6 +42,7 @@
#define CXL_MAILBOX_TIMEOUT_US 2000
enum opcode {
+ CXL_MBOX_OP_INVALID = 0x0000,
CXL_MBOX_OP_IDENTIFY = 0x4000,
CXL_MBOX_OP_MAX = 0x10000
};
@@ -82,6 +84,90 @@ static DEFINE_IDR(cxl_mem_idr);
/* protect cxl_mem_idr allocations */
static DEFINE_MUTEX(cxl_memdev_lock);
+#define CXL_CMD(_id, _flags, sin, sout, f) \
+ [CXL_MEM_COMMAND_ID_##_id] = { \
+ { \
+ .id = CXL_MEM_COMMAND_ID_##_id, \
+ .flags = CXL_MEM_COMMAND_FLAG_##_flags, \
+ .size_in = sin, \
+ .size_out = sout, \
+ }, \
+ .flags = CXL_CMD_INTERNAL_FLAG_##f, \
+ .opcode = CXL_MBOX_OP_##_id, \
+ }
+
+/**
+ * struct cxl_mem_command - Driver representation of a memory device command
+ * @info: Command information as it exists for the UAPI
+ * @opcode: The actual bits used for the mailbox protocol
+ * @flags: Set of flags reflecting the state of the command.
+ *
+ * * %CXL_CMD_INTERNAL_FLAG_HIDDEN: Command is hidden from userspace. This
+ * would typically be used for deprecated commands.
+ * * %CXL_CMD_FLAG_MANDATORY: Hardware must support this command. This flag is
+ * only used internally by the driver for sanity checking.
+ *
+ * The cxl_mem_command is the driver's internal representation of commands that
+ * are supported by the driver. Some of these commands may not be supported by
+ * the hardware. The driver will use @info to validate the fields passed in by
+ * the user then submit the @opcode to the hardware.
+ *
+ * See struct cxl_command_info.
+ */
+struct cxl_mem_command {
+ const struct cxl_command_info info;
+ enum opcode opcode;
+ u32 flags;
+#define CXL_CMD_INTERNAL_FLAG_NONE 0
+#define CXL_CMD_INTERNAL_FLAG_HIDDEN BIT(0)
+#define CXL_CMD_INTERNAL_FLAG_MANDATORY BIT(1)
+};
+
+/*
+ * This table defines the supported mailbox commands for the driver. This table
+ * is made up of a UAPI structure. Non-negative values as parameters in the
+ * table will be validated against the user's input. For example, if size_in is
+ * 0, and the user passed in 1, it is an error.
+ */
+static struct cxl_mem_command mem_commands[] = {
+ CXL_CMD(INVALID, KERNEL, 0, 0, HIDDEN),
+ CXL_CMD(IDENTIFY, NONE, 0, 0x43, MANDATORY),
+};
+
+#define cxl_for_each_cmd(cmd) \
+ for ((cmd) = &mem_commands[0]; \
+ ((cmd) - mem_commands) < ARRAY_SIZE(mem_commands); (cmd)++)
+
+static inline int cxl_cmd_index(const struct cxl_mem_command *c)
+{
+ if (unlikely(c >= mem_commands &&
+ c < (mem_commands + sizeof(mem_commands)))) {
+ struct cxl_mem_command *p;
+
+ cxl_for_each_cmd(p)
+ if (p->opcode == c->opcode &&
+ p->flags == c->flags)
+ return p - mem_commands;
+ } else {
+ return c - mem_commands;
+ }
+
+ WARN_ON_ONCE(1);
+ return 0;
+}
+
+static inline struct cxl_mem_command *cxl_mem_find_command(u16 opcode)
+{
+ struct cxl_mem_command *c;
+
+ cxl_for_each_cmd(c) {
+ if (c->opcode == opcode)
+ return c;
+ }
+
+ return NULL;
+}
+
static int cxl_mem_wait_for_doorbell(struct cxl_mem *cxlm)
{
const int timeout = msecs_to_jiffies(CXL_MAILBOX_TIMEOUT_US);
@@ -140,6 +226,7 @@ static void cxl_mem_mbox_timeout(struct cxl_mem *cxlm,
static int cxl_mem_mbox_send_cmd(struct cxl_mem *cxlm,
struct mbox_cmd *mbox_cmd)
{
+ const struct cxl_mem_command *cmd;
u64 cmd_reg, status_reg;
size_t out_len;
int rc;
@@ -158,6 +245,13 @@ static int cxl_mem_mbox_send_cmd(struct cxl_mem *cxlm,
* 8. If output payload is non-empty, host reads Command Payload Registers
*/
+ cmd = cxl_mem_find_command(mbox_cmd->opcode);
+ if (!cmd) {
+ dev_info(&cxlm->pdev->dev,
+ "Unknown opcode 0x%04x being sent to hardware\n",
+ mbox_cmd->opcode);
+ }
+
/* #1 */
WARN_ON(cxl_doorbell_busy(cxlm));
@@ -200,6 +294,20 @@ static int cxl_mem_mbox_send_cmd(struct cxl_mem *cxlm,
out_len = CXL_GET_FIELD(cmd_reg, CXLDEV_MB_CMD_PAYLOAD_LENGTH);
mbox_cmd->size_out = out_len;
+ /*
+ * If the command had a fixed size output, but the hardware did
+ * something unexpected, just print an error and move on. It would be
+ * worth sending a bug report.
+ */
+ if (cmd && cmd->info.size_out >= 0 &&
+ mbox_cmd->size_out != cmd->info.size_out) {
+ bool too_big = mbox_cmd->size_out > cmd->info.size_out;
+
+ dev_err(&cxlm->pdev->dev,
+ "payload was %s than driver expectations\n",
+ too_big ? "larger" : "smaller");
+ }
+
/* #8 */
if (out_len && mbox_cmd->payload)
memcpy_fromio(mbox_cmd->payload, cxl_payload_regs(cxlm),
@@ -323,8 +431,56 @@ static int cxl_mem_release(struct inode *inode, struct file *file)
return 0;
}
+static int cxl_mem_count_commands(void)
+{
+ struct cxl_mem_command *c;
+ int n = 0;
+
+ cxl_for_each_cmd(c) {
+ if (c->flags & CXL_CMD_INTERNAL_FLAG_HIDDEN)
+ continue;
+ n++;
+ }
+
+ return n;
+}
+
static long cxl_mem_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
{
+ struct cxl_memdev *cxlmd = file->private_data;
+ struct device *dev = &cxlmd->dev;
+
+ if (cmd == CXL_MEM_QUERY_COMMANDS) {
+ struct cxl_mem_query_commands __user *q = (void __user *)arg;
+ struct cxl_mem_command *cmd;
+ u32 n_commands;
+ int j = 0;
+
+ dev_dbg(dev, "Query IOCTL\n");
+
+ if (get_user(n_commands, (u32 __user *)arg))
+ return -EFAULT;
+
+ /* returns the total number if 0 elements are requested. */
+ if (n_commands == 0)
+ return put_user(cxl_mem_count_commands(),
+ (u32 __user *)arg);
+
+ /* otherwise, return max(n_commands, total commands) */
+ cxl_for_each_cmd(cmd) {
+ const struct cxl_command_info *info = &cmd->info;
+
+ if (cmd->flags & CXL_CMD_INTERNAL_FLAG_HIDDEN)
+ continue;
+
+ if (copy_to_user(&q->commands[j++], info, sizeof(*info)))
+ return -EFAULT;
+
+ if (j == n_commands)
+ break;
+ }
+ }
+
return -ENOTTY;
}
diff --git a/include/uapi/linux/cxl_mem.h b/include/uapi/linux/cxl_mem.h
new file mode 100644
index 000000000000..847f825bbe18
--- /dev/null
+++ b/include/uapi/linux/cxl_mem.h
@@ -0,0 +1,117 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/*
+ * CXL IOCTLs for Memory Devices
+ */
+
+#ifndef _UAPI_CXL_MEM_H_
+#define _UAPI_CXL_MEM_H_
+
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
+/**
+ * DOC: UAPI
+ *
+ * CXL memory devices expose UAPI to have a standard user interface.
+ * Userspace can refer to these structure definitions and UAPI formats
+ * to communicate to driver. The commands themselves are somewhat obfuscated
+ * with macro magic. They have the form CXL_MEM_COMMAND_ID_<name>.
+ *
+ * For example "CXL_MEM_COMMAND_ID_INVALID"
+ *
+ * Not all of all commands that the driver supports are always available for use
+ * by userspace. Userspace must check the results from the QUERY command in
+ * order to determine the live set of commands.
+ */
+
+#define CXL_MEM_QUERY_COMMANDS _IOR(0xCE, 1, struct cxl_mem_query_commands)
+
+#undef CMDS
+#define CMDS \
+ C(INVALID, "Invalid Command"), \
+ C(IDENTIFY, "Identify Command"), \
+ C(MAX, "Last command")
+#undef C
+#define C(a, b) CXL_MEM_COMMAND_ID_##a
+
+enum { CMDS };
+
+/**
+ * struct cxl_command_info - Command information returned from a query.
+ * @id: ID number for the command.
+ * @flags: Flags that specify command behavior.
+ *
+ * * %CXL_MEM_COMMAND_FLAG_KERNEL: This command is reserved for exclusive
+ * kernel use.
+ * * %CXL_MEM_COMMAND_FLAG_MUTEX: This command may require coordination with
+ * the kernel in order to complete successfully.
+ *
+ * @size_in: Expected input size, or -1 if variable length.
+ * @size_out: Expected output size, or -1 if variable length.
+ *
+ * Represents a single command that is supported by both the driver and the
+ * hardware. The is returned as part of an array from the query ioctl. The
+ * following would be a command named "foobar" that takes a variable length
+ * input and returns 0 bytes of output.
+ *
+ * - @id = 10
+ * - @flags = CXL_MEM_COMMAND_FLAG_MUTEX
+ * - @size_in = -1
+ * - @size_out = 0
+ *
+ * See struct cxl_mem_query_commands.
+ */
+struct cxl_command_info {
+ __u32 id;
+
+ __u32 flags;
+#define CXL_MEM_COMMAND_FLAG_NONE 0
+#define CXL_MEM_COMMAND_FLAG_KERNEL BIT(0)
+#define CXL_MEM_COMMAND_FLAG_MUTEX BIT(1)
+
+ __s32 size_in;
+ __s32 size_out;
+};
+
+/**
+ * struct cxl_mem_query_commands - Query supported commands.
+ * @n_commands: In/out parameter. When @n_commands is > 0, the driver will
+ * return min(num_support_commands, n_commands). When @n_commands
+ * is 0, driver will return the number of total supported commands.
+ * @rsvd: Reserved for future use.
+ * @commands: Output array of supported commands. This array must be allocated
+ * by userspace to be at least min(num_support_commands, @n_commands)
+ *
+ * Allow userspace to query the available commands supported by both the driver,
+ * and the hardware. Commands that aren't supported by either the driver, or the
+ * hardware are not returned in the query.
+ *
+ * Examples:
+ *
+ * - { .n_commands = 0 } // Get number of supported commands
+ * - { .n_commands = 15, .commands = buf } // Return first 15 (or less)
+ * supported commands
+ *
+ * See struct cxl_command_info.
+ */
+struct cxl_mem_query_commands {
+ /*
+ * Input: Number of commands to return (space allocated by user)
+ * Output: Number of commands supported by the driver/hardware
+ *
+ * If n_commands is 0, kernel will only return number of commands and
+ * not try to populate commands[], thus allowing userspace to know how
+ * much space to allocate
+ */
+ __u32 n_commands;
+ __u32 rsvd;
+
+ struct cxl_command_info __user commands[]; /* out: supported commands */
+};
+
+#if defined(__cplusplus)
+}
+#endif
+
+#endif
--
2.30.0
The Command Effects Log (CEL) is specified in the CXL 2.0 specification.
The CEL is one of two types of logs, the other being vendor specific.
They are distinguished in hardware/spec via UUID. The CEL is immediately
useful for 2 things:
1. Determine which optional commands are supported by the CXL device.
2. Enumerate any vendor specific commands
The CEL can be used by the driver to determine which commands are
available in the hardware (though it isn't, yet). That set of commands
might itself be a subset of commands which are available to be used via
CXL_MEM_SEND_COMMAND IOCTL.
Prior to this, all commands that the driver exposed were explicitly
enabled. After this, only those commands that are found in the CEL are
enabled.
Signed-off-by: Ben Widawsky <[email protected]>
---
drivers/cxl/mem.c | 178 +++++++++++++++++++++++++++++++++--
include/uapi/linux/cxl_mem.h | 1 +
2 files changed, 172 insertions(+), 7 deletions(-)
diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
index 20b26fa2c466..6dfc8ff0aefb 100644
--- a/drivers/cxl/mem.c
+++ b/drivers/cxl/mem.c
@@ -44,8 +44,10 @@
enum opcode {
CXL_MBOX_OP_INVALID = 0x0000,
#define CXL_MBOX_OP_RAW CXL_MBOX_OP_INVALID
- CXL_MBOX_OP_IDENTIFY = 0x4000,
- CXL_MBOX_OP_MAX = 0x10000
+ CXL_MBOX_OP_GET_SUPPORTED_LOGS = 0x0400,
+ CXL_MBOX_OP_GET_LOG = 0x0401,
+ CXL_MBOX_OP_IDENTIFY = 0x4000,
+ CXL_MBOX_OP_MAX = 0x10000
};
/**
@@ -104,6 +106,16 @@ static struct {
.opcode = CXL_MBOX_OP_##_id, \
}
+enum {
+ CEL_UUID,
+ DEBUG_UUID
+};
+
+static const uuid_t log_uuid[] = {
+ UUID_INIT(0xda9c0b5, 0xbf41, 0x4b78, 0x8f, 0x79, 0x96, 0xb1, 0x62, 0x3b, 0x3f, 0x17),
+ UUID_INIT(0xe1819d9, 0x11a9, 0x400c, 0x81, 0x1f, 0xd6, 0x07, 0x19, 0x40, 0x3d, 0x86)
+};
+
/**
* struct cxl_mem_command - Driver representation of a memory device command
* @info: Command information as it exists for the UAPI
@@ -144,6 +156,7 @@ static struct cxl_mem_command mem_commands[] = {
CXL_CMD(INVALID, KERNEL, 0, 0, HIDDEN),
CXL_CMD(IDENTIFY, NONE, 0, 0x43, MANDATORY),
CXL_CMD(RAW, NONE, ~0, ~0, PSEUDO),
+ CXL_CMD(GET_SUPPORTED_LOGS, NONE, 0, ~0, MANDATORY),
};
#define cxl_for_each_cmd(cmd) \
@@ -1036,6 +1049,103 @@ static int cxl_mem_add_memdev(struct cxl_mem *cxlm)
return rc;
}
+struct cxl_mbox_get_supported_logs {
+ __le16 entries;
+ u8 rsvd[6];
+ struct gsl_entry {
+ uuid_t uuid;
+ __le32 size;
+ } __packed entry[2];
+} __packed;
+struct cxl_mbox_get_log {
+ uuid_t uuid;
+ __le32 offset;
+ __le32 length;
+} __packed;
+
+static int cxl_xfer_log(struct cxl_mem *cxlm, uuid_t *uuid, u32 size,
+ u8 *out)
+{
+ u32 remaining = size;
+ u32 offset = 0;
+
+ while (remaining) {
+ u32 xfer_size = min_t(u32, remaining, cxlm->mbox.payload_size);
+ struct mbox_cmd mbox_cmd;
+ int rc;
+ struct cxl_mbox_get_log log = {
+ .uuid = *uuid,
+ .offset = cpu_to_le32(offset),
+ .length = cpu_to_le32(xfer_size)
+ };
+
+ memcpy_toio(cxl_payload_regs(cxlm), &log, sizeof(log));
+ mbox_cmd = (struct mbox_cmd) {
+ .opcode = CXL_MBOX_OP_GET_LOG,
+ .payload = NULL,
+ .size_in = sizeof(log),
+ };
+
+ rc = cxl_mem_mbox_send_cmd(cxlm, &mbox_cmd);
+ if (rc)
+ return rc;
+
+ WARN_ON(mbox_cmd.size_out != xfer_size);
+
+ memcpy_fromio(out, cxl_payload_regs(cxlm), mbox_cmd.size_out);
+ out += xfer_size;
+ remaining -= xfer_size;
+ offset += xfer_size;
+ }
+
+ return 0;
+}
+
+static void cxl_enable_cmd(struct cxl_mem *cxlm,
+ const struct cxl_mem_command *cmd)
+{
+ if (test_and_set_bit(cxl_cmd_index(cmd), cxlm->enabled_cmds))
+ dev_warn(&cxlm->pdev->dev, "Command enabled twice\n");
+
+ dev_info(&cxlm->pdev->dev, "%s enabled",
+ command_names[cxl_cmd_index(cmd)].name);
+}
+
+/**
+ * cxl_walk_cel() - Walk through the Command Effects Log.
+ * @cxlm: Device.
+ * @size: Length of the Command Effects Log.
+ * @cel: CEL
+ *
+ * Iterate over each entry in the CEL and determine if the driver supports the
+ * command. If so, the command is enabled for the device and can be used later.
+ */
+static void cxl_walk_cel(struct cxl_mem *cxlm, size_t size, u8 *cel)
+{
+ struct cel_entry {
+ __le16 opcode;
+ __le16 effect;
+ } *cel_entry;
+ const int cel_entries = size / sizeof(*cel_entry);
+ int i;
+
+ cel_entry = (struct cel_entry *)cel;
+
+ for (i = 0; i < cel_entries; i++) {
+ const struct cel_entry *ce = &cel_entry[i];
+ const struct cxl_mem_command *cmd =
+ cxl_mem_find_command(le16_to_cpu(ce->opcode));
+
+ if (!cmd) {
+ dev_dbg(&cxlm->pdev->dev, "Unsupported opcode 0x%04x",
+ le16_to_cpu(ce->opcode));
+ continue;
+ }
+
+ cxl_enable_cmd(cxlm, cmd);
+ }
+}
+
/**
* cxl_mem_enumerate_cmds() - Enumerate commands for a device.
* @cxlm: The device.
@@ -1048,17 +1158,71 @@ static int cxl_mem_add_memdev(struct cxl_mem *cxlm)
*/
static int cxl_mem_enumerate_cmds(struct cxl_mem *cxlm)
{
- struct cxl_mem_command *c;
+ struct cxl_mbox_get_supported_logs gsl;
+ const struct cxl_mem_command *c;
+ struct mbox_cmd mbox_cmd;
+ int i, rc;
- /* All commands are considered enabled for now (except INVALID). */
+ /* Pseudo commands are always enabled */
cxl_for_each_cmd(c) {
- if (c->flags & CXL_CMD_INTERNAL_FLAG_HIDDEN)
+ if (c->flags & CXL_CMD_INTERNAL_FLAG_PSEUDO)
+ cxl_enable_cmd(cxlm, c);
+ }
+
+ rc = cxl_mem_mbox_get(cxlm, false);
+ if (rc)
+ return rc;
+
+ mbox_cmd = (struct mbox_cmd){
+ .opcode = CXL_MBOX_OP_GET_SUPPORTED_LOGS,
+ .payload = &gsl,
+ .size_in = 0,
+ };
+ rc = cxl_mem_mbox_send_cmd(cxlm, &mbox_cmd);
+ if (rc)
+ goto out;
+
+ for (i = 0; i < le16_to_cpu(gsl.entries); i++) {
+ u32 size = le32_to_cpu(gsl.entry[i].size);
+ uuid_t uuid = gsl.entry[i].uuid;
+ u8 *log;
+
+ dev_dbg(&cxlm->pdev->dev, "Found LOG type %pU of size %d",
+ &uuid, size);
+
+ if (!uuid_equal(&uuid, &log_uuid[CEL_UUID]))
continue;
- set_bit(cxl_cmd_index(c), cxlm->enabled_cmds);
+ /*
+ * It's a hardware bug if the log size is less than the input
+ * payload size because there are many mandatory commands.
+ */
+ if (sizeof(struct cxl_mbox_get_log) > size) {
+ dev_err(&cxlm->pdev->dev,
+ "CEL log size reported was too small (%d)",
+ size);
+ rc = -ENOMEM;
+ goto out;
+ }
+
+ log = kvmalloc(size, GFP_KERNEL);
+ if (!log) {
+ rc = -ENOMEM;
+ goto out;
+ }
+
+ rc = cxl_xfer_log(cxlm, &uuid, size, log);
+ if (rc)
+ goto out;
+
+ cxl_walk_cel(cxlm, size, log);
+
+ kvfree(log);
}
- return 0;
+out:
+ cxl_mem_mbox_put(cxlm);
+ return rc;
}
/**
diff --git a/include/uapi/linux/cxl_mem.h b/include/uapi/linux/cxl_mem.h
index 50acd6cc14d4..b504412d1db7 100644
--- a/include/uapi/linux/cxl_mem.h
+++ b/include/uapi/linux/cxl_mem.h
@@ -33,6 +33,7 @@ extern "C" {
C(INVALID, "Invalid Command"), \
C(IDENTIFY, "Identify Command"), \
C(RAW, "Raw device command"), \
+ C(GET_SUPPORTED_LOGS, "Get Supported Logs"), \
C(MAX, "Last command")
#undef C
#define C(a, b) CXL_MEM_COMMAND_ID_##a
--
2.30.0
The CXL memory device send interface will have a number of supported
commands. The raw command is not such a command. Raw commands allow
userspace to send a specified opcode to the underlying hardware and
bypass all driver checks on the command. This is useful for a couple of
usecases, mainly:
1. Undocumented vendor specific hardware commands
2. Prototyping new hardware commands not yet supported by the driver
While this all sounds very powerful it comes with a couple of caveats:
1. Bug reports using raw commands will not get the same level of
attention as bug reports using supported commands (via taint).
2. Supported commands will be rejected by the RAW command.
Signed-off-by: Ben Widawsky <[email protected]>
---
drivers/cxl/mem.c | 27 +++++++++++++++++++++++++++
include/uapi/linux/cxl_mem.h | 12 +++++++++++-
2 files changed, 38 insertions(+), 1 deletion(-)
diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
index f979788b4d9f..a824cfd4342a 100644
--- a/drivers/cxl/mem.c
+++ b/drivers/cxl/mem.c
@@ -43,6 +43,7 @@
enum opcode {
CXL_MBOX_OP_INVALID = 0x0000,
+#define CXL_MBOX_OP_RAW CXL_MBOX_OP_INVALID
CXL_MBOX_OP_IDENTIFY = 0x4000,
CXL_MBOX_OP_MAX = 0x10000
};
@@ -139,6 +140,7 @@ struct cxl_mem_command {
static struct cxl_mem_command mem_commands[] = {
CXL_CMD(INVALID, KERNEL, 0, 0, HIDDEN),
CXL_CMD(IDENTIFY, NONE, 0, 0x43, MANDATORY),
+ CXL_CMD(RAW, NONE, ~0, ~0, MANDATORY),
};
#define cxl_for_each_cmd(cmd) \
@@ -520,6 +522,9 @@ static int handle_mailbox_cmd_from_user(struct cxl_memdev *cxlmd,
command_names[cmd->info.id].name, mbox_cmd.opcode,
cmd->info.size_in);
+ if (cmd->info.id == CXL_MEM_COMMAND_ID_RAW)
+ add_taint(TAINT_RAW_PASSTHROUGH, LOCKDEP_STILL_OK);
+
rc = cxl_mem_mbox_send_cmd(cxlmd->cxlm, &mbox_cmd);
cxl_mem_mbox_put(cxlmd->cxlm);
if (rc)
@@ -581,6 +586,28 @@ static int cxl_validate_cmd_from_user(struct cxl_mem *cxlm,
if (send_cmd->size_in > cxlm->mbox.payload_size)
return -EINVAL;
+ /* Checks are bypassed for raw commands but along comes the taint! */
+ if (send_cmd->id == CXL_MEM_COMMAND_ID_RAW) {
+ const struct cxl_mem_command temp = {
+ {
+ .id = CXL_MEM_COMMAND_ID_RAW,
+ .flags = CXL_MEM_COMMAND_FLAG_NONE,
+ .size_in = send_cmd->size_in,
+ .size_out = send_cmd->size_out,
+ },
+ .flags = 0,
+ .opcode = send_cmd->raw.opcode
+ };
+
+ if (send_cmd->raw.rsvd)
+ return -EINVAL;
+
+ if (cxl_mem_find_command(send_cmd->raw.opcode))
+ return -EPERM;
+
+ return PTR_ERR_OR_ZERO(memcpy(out_cmd, &temp, sizeof(temp)));
+ }
+
if (send_cmd->flags & CXL_MEM_COMMAND_FLAG_MASK)
return -EINVAL;
diff --git a/include/uapi/linux/cxl_mem.h b/include/uapi/linux/cxl_mem.h
index cb4e2bee5228..50acd6cc14d4 100644
--- a/include/uapi/linux/cxl_mem.h
+++ b/include/uapi/linux/cxl_mem.h
@@ -32,6 +32,7 @@ extern "C" {
#define CMDS \
C(INVALID, "Invalid Command"), \
C(IDENTIFY, "Identify Command"), \
+ C(RAW, "Raw device command"), \
C(MAX, "Last command")
#undef C
#define C(a, b) CXL_MEM_COMMAND_ID_##a
@@ -117,6 +118,9 @@ struct cxl_mem_query_commands {
* @id: The command to send to the memory device. This must be one of the
* commands returned by the query command.
* @flags: Flags for the command (input).
+ * @raw: Special fields for raw commands
+ * @raw.opcode: Opcode passed to hardware when using the RAW command.
+ * @raw.rsvd: Must be zero.
* @rsvd: Must be zero.
* @retval: Return value from the memory device (output).
* @size_in: Size of the payload to provide to the device (input).
@@ -135,7 +139,13 @@ struct cxl_mem_query_commands {
struct cxl_send_command {
__u32 id;
__u32 flags;
- __u32 rsvd;
+ union {
+ struct {
+ __u16 opcode;
+ __u16 rsvd;
+ } raw;
+ __u32 rsvd;
+ };
__u32 retval;
struct {
--
2.30.0
Cc: Dan Williams <[email protected]>
Cc: Vishal Verma <[email protected]>
Cc: Ira Weiny <[email protected]>
Signed-off-by: Ben Widawsky <[email protected]>
---
MAINTAINERS | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index 6eff4f720c72..70c7ec4b2921 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -4459,6 +4459,16 @@ F: fs/configfs/
F: include/linux/configfs.h
F: samples/configfs/
+COMPUTE EXPRESS LINK (CXL)
+M: Vishal Verma <[email protected]>
+M: Ira Weiny <[email protected]>
+M: Ben Widawsky <[email protected]>
+M: Dan Williams <[email protected]>
+L: [email protected]
+S: Maintained
+F: drivers/cxl/
+F: include/uapi/linux/cxl_mem.h
+
CONSOLE SUBSYSTEM
M: Greg Kroah-Hartman <[email protected]>
S: Supported
--
2.30.0
The Get Log command returns the actual log entries that are advertised
via the Get Supported Logs command (0400h). CXL device logs are selected
by UUID which is part of the CXL spec. Because the driver tries to
sanitize what is sent to hardware, there becomes a need to restrict the
types of logs which can be accessed by userspace. For example, the
vendor specific log might only be consumable by proprietary, or offline
applications, and therefore a good candidate for userspace.
The current driver infrastructure does allow basic validation for all
commands, but doesn't inspect any of the payload data. Along with Get
Log support comes new infrastructure to add a hook for payload
validation. This infrastructure is used to filter out the CEL UUID,
which the userspace driver doesn't have business knowing, and taints on
invalid UUIDs being sent to hardware.
Signed-off-by: Ben Widawsky <[email protected]>
---
drivers/cxl/mem.c | 42 +++++++++++++++++++++++++++++++++++-
include/uapi/linux/cxl_mem.h | 1 +
2 files changed, 42 insertions(+), 1 deletion(-)
diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
index 6dfc8ff0aefb..593db737e7a4 100644
--- a/drivers/cxl/mem.c
+++ b/drivers/cxl/mem.c
@@ -94,7 +94,7 @@ static struct {
} command_names[] = { CMDS };
#undef C
-#define CXL_CMD(_id, _flags, sin, sout, f) \
+#define CXL_CMD_VALIDATE(_id, _flags, sin, sout, f, v) \
[CXL_MEM_COMMAND_ID_##_id] = { \
{ \
.id = CXL_MEM_COMMAND_ID_##_id, \
@@ -104,8 +104,12 @@ static struct {
}, \
.flags = CXL_CMD_INTERNAL_FLAG_##f, \
.opcode = CXL_MBOX_OP_##_id, \
+ .validate_payload = v, \
}
+#define CXL_CMD(_id, _flags, sin, sout, f) \
+ CXL_CMD_VALIDATE(_id, _flags, sin, sout, f, NULL)
+
enum {
CEL_UUID,
DEBUG_UUID
@@ -116,6 +120,8 @@ static const uuid_t log_uuid[] = {
UUID_INIT(0xe1819d9, 0x11a9, 0x400c, 0x81, 0x1f, 0xd6, 0x07, 0x19, 0x40, 0x3d, 0x86)
};
+static int validate_log_uuid(void __user *payload, size_t size);
+
/**
* struct cxl_mem_command - Driver representation of a memory device command
* @info: Command information as it exists for the UAPI
@@ -129,6 +135,10 @@ static const uuid_t log_uuid[] = {
* * %CXL_CMD_INTERNAL_FLAG_PSEUDO: This is a pseudo command which doesn't have
* a direct mapping to hardware. They are implicitly always enabled.
*
+ * @validate_payload: A function called after the command is validated but
+ * before it's sent to the hardware. The primary purpose is to validate, or
+ * fixup the actual payload.
+ *
* The cxl_mem_command is the driver's internal representation of commands that
* are supported by the driver. Some of these commands may not be supported by
* the hardware. The driver will use @info to validate the fields passed in by
@@ -144,6 +154,8 @@ struct cxl_mem_command {
#define CXL_CMD_INTERNAL_FLAG_HIDDEN BIT(0)
#define CXL_CMD_INTERNAL_FLAG_MANDATORY BIT(1)
#define CXL_CMD_INTERNAL_FLAG_PSEUDO BIT(2)
+
+ int (*validate_payload)(void __user *payload, size_t size);
};
/*
@@ -157,6 +169,8 @@ static struct cxl_mem_command mem_commands[] = {
CXL_CMD(IDENTIFY, NONE, 0, 0x43, MANDATORY),
CXL_CMD(RAW, NONE, ~0, ~0, PSEUDO),
CXL_CMD(GET_SUPPORTED_LOGS, NONE, 0, ~0, MANDATORY),
+ CXL_CMD_VALIDATE(GET_LOG, MUTEX, 0x18, ~0, MANDATORY,
+ validate_log_uuid),
};
#define cxl_for_each_cmd(cmd) \
@@ -515,6 +529,15 @@ static int handle_mailbox_cmd_from_user(struct cxl_memdev *cxlmd,
int rc;
if (cmd->info.size_in) {
+ if (cmd->validate_payload) {
+ rc = cmd->validate_payload(u64_to_user_ptr(in_payload),
+ cmd->info.size_in);
+ if (rc) {
+ cxl_mem_mbox_put(cxlmd->cxlm);
+ return -EFAULT;
+ }
+ }
+
/*
* Directly copy the userspace payload into the hardware. UAPI
* states that the buffer must already be little endian.
@@ -1063,6 +1086,23 @@ struct cxl_mbox_get_log {
__le32 length;
} __packed;
+static int validate_log_uuid(void __user *input, size_t size)
+{
+ struct cxl_mbox_get_log __user *get_log = input;
+ uuid_t payload_uuid;
+
+ if (copy_from_user(&payload_uuid, &get_log->uuid, sizeof(uuid_t)))
+ return -EFAULT;
+
+ /* All commands taint except debug vendor log */
+ if (uuid_equal(&payload_uuid, &log_uuid[DEBUG_UUID]))
+ return 0;
+
+ add_taint(TAINT_RAW_PASSTHROUGH, LOCKDEP_STILL_OK);
+
+ return 0;
+}
+
static int cxl_xfer_log(struct cxl_mem *cxlm, uuid_t *uuid, u32 size,
u8 *out)
{
diff --git a/include/uapi/linux/cxl_mem.h b/include/uapi/linux/cxl_mem.h
index b504412d1db7..4c4a12c7a4d5 100644
--- a/include/uapi/linux/cxl_mem.h
+++ b/include/uapi/linux/cxl_mem.h
@@ -34,6 +34,7 @@ extern "C" {
C(IDENTIFY, "Identify Command"), \
C(RAW, "Raw device command"), \
C(GET_SUPPORTED_LOGS, "Get Supported Logs"), \
+ C(GET_LOG, "Get Log"), \
C(MAX, "Last command")
#undef C
#define C(a, b) CXL_MEM_COMMAND_ID_##a
--
2.30.0
On Mon, 11 Jan 2021 14:51:10 -0800
Ben Widawsky <[email protected]> wrote:
> CXL devices contain an array of capabilities that describe the
> interactions software can have with the device or firmware running on
> the device. A CXL compliant device must implement the device status and
> the mailbox capability. A CXL compliant memory device must implement the
> memory device capability.
>
> Each of the capabilities can [will] provide an offset within the MMIO
> region for interacting with the CXL device.
>
> For more details see 8.2.8 of the CXL 2.0 specification.
>
> Link: Link: https://www.computeexpresslink.org/download-the-specification
> Signed-off-by: Ben Widawsky <[email protected]>
...
> /**
> * cxl_mem_create() - Create a new &struct cxl_mem.
> * @pdev: The pci device associated with the new &struct cxl_mem.
> @@ -129,8 +214,20 @@ static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> if (rc)
> return rc;
>
> + rc = cxl_mem_setup_regs(cxlm);
> + if (rc)
> + goto err;
> +
> + rc = cxl_mem_setup_mailbox(cxlm);
> + if (rc)
> + goto err;
> +
> pci_set_drvdata(pdev, cxlm);
> return 0;
> +
> +err:
> + kfree(cxlm);
From previous patch that was created using devm_kzalloc in which case
this free just introduced a double free if you hit this error path.
Having go rid of that you can do direct returns instead of goto.
> + return rc;
> }
>
> static void cxl_mem_remove(struct pci_dev *pdev)
On Tue, Jan 12, 2021 at 11:03 AM Jonathan Cameron
<[email protected]> wrote:
>
> On Mon, 11 Jan 2021 14:51:08 -0800
> Ben Widawsky <[email protected]> wrote:
>
> > From: Dan Williams <[email protected]>
> >
> > The CXL.mem protocol allows a device to act as a provider of "System
> > RAM" and/or "Persistent Memory" that is fully coherent as if the memory
> > was attached to the typical CPU memory controller.
> >
> > With the CXL-2.0 specification a PCI endpoint can implement a "Type-3"
> > device interface and give the operating system control over "Host
> > Managed Device Memory". See section 2.3 Type 3 CXL Device.
> >
> > The memory range exported by the device may optionally be described by
> > the platform firmware memory map, or by infrastructure like LIBNVDIMM to
> > provision persistent memory capacity from one, or more, CXL.mem devices.
> >
> > A pre-requisite for Linux-managed memory-capacity provisioning is this
> > cxl_mem driver that can speak the mailbox protocol defined in section
> > 8.2.8.4 Mailbox Registers.
> >
> > For now just land the driver boiler-plate and fill it in with
> > functionality in subsequent commits.
> >
> > Link: https://www.computeexpresslink.org/download-the-specification
> > Signed-off-by: Dan Williams <[email protected]>
> > Signed-off-by: Ben Widawsky <[email protected]>
>
> Just one passing comment inline.
>
> > diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
> > new file mode 100644
> > index 000000000000..005404888942
> > --- /dev/null
> > +++ b/drivers/cxl/mem.c
> > @@ -0,0 +1,69 @@
> > +// SPDX-License-Identifier: GPL-2.0-only
> > +/* Copyright(c) 2020 Intel Corporation. All rights reserved. */
> > +#include <linux/module.h>
> > +#include <linux/pci.h>
> > +#include <linux/io.h>
> > +#include "acpi.h"
> > +#include "pci.h"
> > +
> > +static int cxl_mem_dvsec(struct pci_dev *pdev, int dvsec)
>
> Is it worth pulling this out to a utility library now as we are going
> to keep needing this for CXL devices?
> Arguably, with a vendor_id parameter it might make sense to have
> it as a utility function for pci rather than CXL alone.
Sure, cxl_mem_dvsec() can move to a central location, but I'd wait for
the first incremental user to split it out.
On Tue, 12 Jan 2021 11:43:31 -0800
Dan Williams <[email protected]> wrote:
> On Tue, Jan 12, 2021 at 10:44 AM Jonathan Cameron
> <[email protected]> wrote:
> >
> > On Mon, 11 Jan 2021 14:51:06 -0800
> > Ben Widawsky <[email protected]> wrote:
> >
> > > From: Vishal Verma <[email protected]>
> > >
> > > Add an acpi_cxl module to coordinate the ACPI portions of the CXL
> > > (Compute eXpress Link) interconnect. This driver binds to ACPI0017
> > > objects in the ACPI tree, and coordinates access to the resources
> > > provided by the ACPI CEDT (CXL Early Discovery Table).
> > >
> > > It also coordinates operations of the root port _OSC object to notify
> > > platform firmware that the OS has native support for the CXL
> > > capabilities of endpoints.
> > >
> > > Note: the actbl1.h changes are speculative. The expectation is that they
> > > will arrive through the ACPICA tree in due time.
> >
> > I would pull the ACPICA changes out into a precursor patch.
>
>
> >
> > >
> > > Cc: Ben Widawsky <[email protected]>
> > > Cc: Dan Williams <[email protected]>
> > > Signed-off-by: Vishal Verma <[email protected]>
> > > Signed-off-by: Ben Widawsky <[email protected]>
> >
> > Hi,
> >
> > I think it would be good to also add CEDT to the list in drivers/acpi/tables.c
> > so that we can dump it from /sys/firmware/acpi/tables/ and potentially
> > override it from an initrd.
>
> ACPICA changes will eventually come through the ACPI tree not this patch set.
That particular file isn't from ACPICA though I value in the entry will be.
>
>
> >
> > https://elixir.bootlin.com/linux/v5.11-rc3/source/drivers/acpi/tables.c#L482
> > Can be very helpful whilst debugging. Related to that, anyone know if anyone
> > has acpica patches so we can have iasl -d work on the table? Would probably
> > be useful but I'd rather not duplicate work if it's already done.
> >
>
> The supplemental tables described here:
>
> https://www.uefi.org/acpi
>
> ...do eventually make there way into ACPICA. Added Bob and Erik in
> case they can comment on when CEDT and CDAT support will be picked up.
...
On Tue, Jan 12, 2021 at 1:29 AM Ben Widawsky <[email protected]> wrote:
>
> From: Vishal Verma <[email protected]>
>
> Add an acpi_cxl module to coordinate the ACPI portions of the CXL
> (Compute eXpress Link) interconnect. This driver binds to ACPI0017
> objects in the ACPI tree, and coordinates access to the resources
> provided by the ACPI CEDT (CXL Early Discovery Table).
>
> It also coordinates operations of the root port _OSC object to notify
> platform firmware that the OS has native support for the CXL
> capabilities of endpoints.
This doesn't happen here, but in the next patch.
> Note: the actbl1.h changes are speculative. The expectation is that they
> will arrive through the ACPICA tree in due time.
So why don't you put them into a separate patch and drop it from the
series when not necessary any more?
> Cc: Ben Widawsky <[email protected]>
> Cc: Dan Williams <[email protected]>
> Signed-off-by: Vishal Verma <[email protected]>
> Signed-off-by: Ben Widawsky <[email protected]>
> ---
> drivers/Kconfig | 1 +
> drivers/Makefile | 1 +
> drivers/cxl/Kconfig | 36 ++++++++++++++++
> drivers/cxl/Makefile | 5 +++
> drivers/cxl/acpi.c | 97 +++++++++++++++++++++++++++++++++++++++++++
> drivers/cxl/acpi.h | 15 +++++++
> include/acpi/actbl1.h | 50 ++++++++++++++++++++++
> 7 files changed, 205 insertions(+)
> create mode 100644 drivers/cxl/Kconfig
> create mode 100644 drivers/cxl/Makefile
> create mode 100644 drivers/cxl/acpi.c
> create mode 100644 drivers/cxl/acpi.h
>
> diff --git a/drivers/Kconfig b/drivers/Kconfig
> index dcecc9f6e33f..62c753a73651 100644
> --- a/drivers/Kconfig
> +++ b/drivers/Kconfig
> @@ -6,6 +6,7 @@ menu "Device Drivers"
> source "drivers/amba/Kconfig"
> source "drivers/eisa/Kconfig"
> source "drivers/pci/Kconfig"
> +source "drivers/cxl/Kconfig"
> source "drivers/pcmcia/Kconfig"
> source "drivers/rapidio/Kconfig"
>
> diff --git a/drivers/Makefile b/drivers/Makefile
> index fd11b9ac4cc3..678ea810410f 100644
> --- a/drivers/Makefile
> +++ b/drivers/Makefile
> @@ -73,6 +73,7 @@ obj-$(CONFIG_NVM) += lightnvm/
> obj-y += base/ block/ misc/ mfd/ nfc/
> obj-$(CONFIG_LIBNVDIMM) += nvdimm/
> obj-$(CONFIG_DAX) += dax/
> +obj-$(CONFIG_CXL_BUS) += cxl/
> obj-$(CONFIG_DMA_SHARED_BUFFER) += dma-buf/
> obj-$(CONFIG_NUBUS) += nubus/
> obj-y += macintosh/
> diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig
> new file mode 100644
> index 000000000000..68da926ba5b1
> --- /dev/null
> +++ b/drivers/cxl/Kconfig
> @@ -0,0 +1,36 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +menuconfig CXL_BUS
> + tristate "CXL (Compute Express Link) Devices Support"
> + help
> + CXL is a bus that is electrically compatible with PCI-E, but layers
> + three protocols on that signalling (CXL.io, CXL.cache, and CXL.mem).
> + The CXL.cache protocol allows devices to hold cachelines locally, the
> + CXL.mem protocol allows devices to be fully coherent memory targets,
> + the CXL.io protocol is equivalent to PCI-E. Say 'y' to enable support
> + for the configuration and management of devices supporting these
> + protocols.
> +
> +if CXL_BUS
> +
> +config CXL_BUS_PROVIDER
> + tristate
> +
> +config CXL_ACPI
> + tristate "CXL ACPI: Platform Support"
> + depends on ACPI
> + default CXL_BUS
> + select CXL_BUS_PROVIDER
> + help
> + Say 'y/m' to enable a driver (named "cxl_acpi.ko" when built
> + as a module) that will enable support for CXL.mem endpoint
> + devices. In general, CXL Platform Support is a prerequisite
> + for any CXL device driver that wants to claim ownership of a
> + component register space. By default platform firmware assumes
> + Linux is unaware of CXL capabilities and requires explicit
> + opt-in. This platform component also mediates resources
> + described by the CEDT (CXL Early Discovery Table). See
> + Chapter 9.14.1 CXL Early Discovery Table (CEDT) in the CXL 2.0
> + specification.
> +
> + If unsure say 'm'
Missing full stop.
> +endif
> diff --git a/drivers/cxl/Makefile b/drivers/cxl/Makefile
> new file mode 100644
> index 000000000000..d38cd34a2582
> --- /dev/null
> +++ b/drivers/cxl/Makefile
> @@ -0,0 +1,5 @@
> +# SPDX-License-Identifier: GPL-2.0
> +obj-$(CONFIG_CXL_ACPI) += cxl_acpi.o
> +
> +ccflags-y += -DDEFAULT_SYMBOL_NAMESPACE=CXL
> +cxl_acpi-y := acpi.o
> diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c
> new file mode 100644
> index 000000000000..0f1ba9b3f1ed
> --- /dev/null
> +++ b/drivers/cxl/acpi.c
> @@ -0,0 +1,97 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/* Copyright(c) 2020 Intel Corporation. All rights reserved. */
> +#include <linux/platform_device.h>
> +#include <linux/list_sort.h>
> +#include <linux/module.h>
> +#include <linux/mutex.h>
> +#include <linux/sysfs.h>
> +#include <linux/list.h>
> +#include <linux/acpi.h>
> +#include <linux/sort.h>
> +#include <linux/pci.h>
> +#include "acpi.h"
> +
> +/*
> + * If/when CXL support is defined by other platform firmware the kernel
> + * will need a mechanism to select between the platform specific version
> + * of this routine, until then, hard-code ACPI assumptions
> + */
> +int cxl_bus_acquire(struct pci_dev *pdev)
> +{
> + struct acpi_device *adev;
> + struct pci_dev *root_port;
> + struct device *root;
> +
> + root_port = pcie_find_root_port(pdev);
> + if (!root_port)
> + return -ENXIO;
> +
> + root = root_port->dev.parent;
> + if (!root)
> + return -ENXIO;
> +
> + adev = ACPI_COMPANION(root);
> + if (!adev)
> + return -ENXIO;
> +
> + /* TODO: OSC enabling */
> +
> + return 0;
> +}
> +EXPORT_SYMBOL_GPL(cxl_bus_acquire);
I would move the addition of cxl_bus_acquire() entirely to the next
patch, it looks quite confusing to me as is.
> +
> +static void acpi_cedt_put_table(void *table)
> +{
> + acpi_put_table(table);
> +}
> +
> +static int cxl_acpi_probe(struct platform_device *pdev)
> +{
> + struct acpi_device *adev = ACPI_COMPANION(&pdev->dev);
> + struct cxl_acpi_desc *acpi_desc;
> + struct device *dev = &adev->dev;
> + struct acpi_table_header *tbl;
> + acpi_status status;
> + acpi_size sz;
> + int rc;
> +
> + status = acpi_get_table(ACPI_SIG_CEDT, 0, &tbl);
> + if (ACPI_FAILURE(status)) {
> + dev_err(dev, "failed to find CEDT at startup\n");
> + return 0;
> + }
> +
> + rc = devm_add_action_or_reset(dev, acpi_cedt_put_table, tbl);
> + if (rc)
> + return rc;
> +
> + sz = tbl->length;
> + dev_info(dev, "found CEDT at startup: %lld bytes\n", sz);
> +
> + acpi_desc = devm_kzalloc(dev, sizeof(*acpi_desc), GFP_KERNEL);
> + if (!acpi_desc)
> + return -ENOMEM;
> +
> + dev_set_drvdata(&adev->dev, acpi_desc);
> + acpi_desc->dev = &adev->dev;
> + acpi_desc->acpi_header = *tbl;
> +
> + return 0;
> +}
> +
> +static const struct acpi_device_id cxl_acpi_ids[] = {
> + { "ACPI0017", 0 },
> + { "", 0 },
> +};
> +MODULE_DEVICE_TABLE(acpi, cxl_acpi_ids);
> +
> +static struct platform_driver cxl_acpi_driver = {
> + .probe = cxl_acpi_probe,
> + .driver = {
> + .name = KBUILD_MODNAME,
> + .acpi_match_table = cxl_acpi_ids,
> + },
> +};
> +
> +module_platform_driver(cxl_acpi_driver);
> +MODULE_LICENSE("GPL v2");
> diff --git a/drivers/cxl/acpi.h b/drivers/cxl/acpi.h
> new file mode 100644
> index 000000000000..d638f8886ab7
> --- /dev/null
> +++ b/drivers/cxl/acpi.h
> @@ -0,0 +1,15 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/* Copyright(c) 2020 Intel Corporation. All rights reserved. */
> +
> +#ifndef __CXL_ACPI_H__
> +#define __CXL_ACPI_H__
> +#include <linux/acpi.h>
> +
> +struct cxl_acpi_desc {
> + struct acpi_table_header acpi_header;
> + struct device *dev;
> +};
> +
> +int cxl_bus_acquire(struct pci_dev *pci_dev);
And so this should go to the next patch too IMV.
> +
> +#endif /* __CXL_ACPI_H__ */
> diff --git a/include/acpi/actbl1.h b/include/acpi/actbl1.h
> index 43549547ed3e..f3a0303984c8 100644
> --- a/include/acpi/actbl1.h
> +++ b/include/acpi/actbl1.h
> @@ -28,6 +28,7 @@
> #define ACPI_SIG_BERT "BERT" /* Boot Error Record Table */
> #define ACPI_SIG_BGRT "BGRT" /* Boot Graphics Resource Table */
> #define ACPI_SIG_BOOT "BOOT" /* Simple Boot Flag Table */
> +#define ACPI_SIG_CEDT "CEDT" /* CXL Early Discovery Table */
> #define ACPI_SIG_CPEP "CPEP" /* Corrected Platform Error Polling table */
> #define ACPI_SIG_CSRT "CSRT" /* Core System Resource Table */
> #define ACPI_SIG_DBG2 "DBG2" /* Debug Port table type 2 */
> @@ -1624,6 +1625,55 @@ struct acpi_ibft_target {
> u16 reverse_chap_secret_offset;
> };
>
> +/*******************************************************************************
> + *
> + * CEDT - CXL Early Discovery Table (ACPI 6.4)
> + * Version 1
> + *
> + ******************************************************************************/
> +
> +struct acpi_table_cedt {
> + struct acpi_table_header header; /* Common ACPI table header */
> + u32 reserved;
> +};
> +
> +/* Values for CEDT structure types */
> +
> +enum acpi_cedt_type {
> + ACPI_CEDT_TYPE_HOST_BRIDGE = 0, /* CHBS - CXL Host Bridge Structure */
> +};
> +
> +struct acpi_cedt_structure {
> + u8 type;
> + u8 reserved;
> + u16 length;
> +};
> +
> +/*
> + * CEDT Structures, correspond to Type in struct acpi_cedt_structure
> + */
> +
> +/* 0: CXL Host Bridge Structure */
> +
> +struct acpi_cedt_chbs {
> + struct acpi_cedt_structure header;
> + u32 uid;
> + u32 version;
> + u32 reserved1;
> + u64 base;
> + u64 length;
> +};
> +
> +/* Values for version field above */
> +
> +#define ACPI_CEDT_CHBS_VERSION_CXL11 (0)
> +#define ACPI_CEDT_CHBS_VERSION_CXL20 (1)
> +
> +/* Values for length field above */
> +
> +#define ACPI_CEDT_CHBS_LENGTH_CXL11 (0x2000)
> +#define ACPI_CEDT_CHBS_LENGTH_CXL20 (0x10000)
> +
> /* Reset to default packing */
>
> #pragma pack()
> --
> 2.30.0
>
> -----Original Message-----
> From: Dan Williams <[email protected]>
> Sent: Tuesday, January 12, 2021 11:44 AM
> To: Jonathan Cameron <[email protected]>
> Cc: Widawsky, Ben <[email protected]>; [email protected];
> Verma, Vishal L <[email protected]>; Linux Kernel Mailing List <linux-
> [email protected]>; Linux PCI <[email protected]>; Weiny, Ira
> <[email protected]>; Kelley, Sean V <[email protected]>; Wysocki,
> Rafael J <[email protected]>; Bjorn Helgaas <[email protected]>;
> Jon Masters <[email protected]>; Chris Browy <cbrowy@avery-
> design.com>; Randy Dunlap <[email protected]>; Christoph Hellwig
> <[email protected]>; [email protected]; Moore, Robert
> <[email protected]>; Kaneda, Erik <[email protected]>
> Subject: Re: [RFC PATCH v3 02/16] cxl/acpi: Add an acpi_cxl module for the
> CXL interconnect
>
> On Tue, Jan 12, 2021 at 10:44 AM Jonathan Cameron
> <[email protected]> wrote:
> >
> > On Mon, 11 Jan 2021 14:51:06 -0800
> > Ben Widawsky <[email protected]> wrote:
> >
> > > From: Vishal Verma <[email protected]>
> > >
> > > Add an acpi_cxl module to coordinate the ACPI portions of the CXL
> > > (Compute eXpress Link) interconnect. This driver binds to ACPI0017
> > > objects in the ACPI tree, and coordinates access to the resources
> > > provided by the ACPI CEDT (CXL Early Discovery Table).
> > >
> > > It also coordinates operations of the root port _OSC object to notify
> > > platform firmware that the OS has native support for the CXL
> > > capabilities of endpoints.
> > >
> > > Note: the actbl1.h changes are speculative. The expectation is that they
> > > will arrive through the ACPICA tree in due time.
> >
> > I would pull the ACPICA changes out into a precursor patch.
>
>
> >
> > >
> > > Cc: Ben Widawsky <[email protected]>
> > > Cc: Dan Williams <[email protected]>
> > > Signed-off-by: Vishal Verma <[email protected]>
> > > Signed-off-by: Ben Widawsky <[email protected]>
> >
> > Hi,
> >
> > I think it would be good to also add CEDT to the list in drivers/acpi/tables.c
> > so that we can dump it from /sys/firmware/acpi/tables/ and potentially
> > override it from an initrd.
>
> ACPICA changes will eventually come through the ACPI tree not this patch
> set.
>
>
> >
> > https://elixir.bootlin.com/linux/v5.11-
> rc3/source/drivers/acpi/tables.c#L482
> > Can be very helpful whilst debugging. Related to that, anyone know if
> anyone
> > has acpica patches so we can have iasl -d work on the table? Would
> probably
> > be useful but I'd rather not duplicate work if it's already done.
> >
>
> The supplemental tables described here:
>
> https://www.uefi.org/acpi
>
> ...do eventually make there way into ACPICA. Added Bob and Erik in
> case they can comment on when CEDT and CDAT support will be picked up.
We would be happy to add support. I think Ben has reached out to me earlier about something like this but I haven’t had a chance to implement... Sorry about the delay.. How soon is the iASL/ACPICA support needed for CDAT and CDET?
Erik
On Mon, 11 Jan 2021 14:51:11 -0800
Ben Widawsky <[email protected]> wrote:
> Provide enough functionality to utilize the mailbox of a memory device.
> The mailbox is used to interact with the firmware running on the memory
> device.
>
> The CXL specification defines separate capabilities for the mailbox and
> the memory device. While we can confirm the mailbox is ready, in order
> to actually interact with the memory device, you must also confirm the
> device's firmware is ready.
>
> Create a function to handle sending a command, optionally with a
> payload, to the memory device, polling on a result, and then optionally
> copying out the payload. The algorithm for doing this comes straight out
> of the CXL 2.0 specification.
>
> Primary mailboxes are capable of generating an interrupt when submitting
> a command in the background. That implementation is saved for a later
> time.
>
> Secondary mailboxes aren't implemented at this time.
>
> The flow is proven with one implemented command, "identify". Because the
> class code has already told the driver this is a memory device and the
> identify command is mandatory, it's safe to assume for sane devices that
> everything here will work.
>
> Signed-off-by: Ben Widawsky <[email protected]>
Hi Ben,
A few trivial things inline for this one but basically LGTM.
Jonathan
> ---
> drivers/cxl/cxl.h | 43 +++++++
> drivers/cxl/mem.c | 312 ++++++++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 355 insertions(+)
>
> diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
> index a77286d04ce4..ca3fa496e21c 100644
> --- a/drivers/cxl/cxl.h
> +++ b/drivers/cxl/cxl.h
> @@ -32,9 +32,40 @@
> #define CXLDEV_MB_CAP_PAYLOAD_SIZE_MASK GENMASK(4, 0)
> #define CXLDEV_MB_CAP_PAYLOAD_SIZE_SHIFT 0
> #define CXLDEV_MB_CTRL_OFFSET 0x04
> +#define CXLDEV_MB_CTRL_DOORBELL BIT(0)
> #define CXLDEV_MB_CMD_OFFSET 0x08
> +#define CXLDEV_MB_CMD_COMMAND_OPCODE_SHIFT 0
> +#define CXLDEV_MB_CMD_COMMAND_OPCODE_MASK GENMASK(15, 0)
> +#define CXLDEV_MB_CMD_PAYLOAD_LENGTH_SHIFT 16
> +#define CXLDEV_MB_CMD_PAYLOAD_LENGTH_MASK GENMASK(36, 16)
> #define CXLDEV_MB_STATUS_OFFSET 0x10
> +#define CXLDEV_MB_STATUS_RET_CODE_SHIFT 32
Given use of FIELD_GET etc we shouldn't need both shift and the mask
in these defines (hopefully) and you aren't using it in this patch.
> +#define CXLDEV_MB_STATUS_RET_CODE_MASK GENMASK(47, 32)
> #define CXLDEV_MB_BG_CMD_STATUS_OFFSET 0x18
> +#define CXLDEV_MB_PAYLOAD_OFFSET 0x20
> +
> +/* Memory Device (CXL 2.0 - 8.2.8.5.1.1) */
> +#define CXLMDEV_STATUS_OFFSET 0x0
> +#define CXLMDEV_DEV_FATAL BIT(0)
> +#define CXLMDEV_FW_HALT BIT(1)
> +#define CXLMDEV_STATUS_MEDIA_STATUS_SHIFT 2
> +#define CXLMDEV_STATUS_MEDIA_STATUS_MASK GENMASK(3, 2)
> +#define CXLMDEV_MS_NOT_READY 0
> +#define CXLMDEV_MS_READY 1
> +#define CXLMDEV_MS_ERROR 2
> +#define CXLMDEV_MS_DISABLED 3
> +#define CXLMDEV_READY(status) \
> + (CXL_GET_FIELD(status, CXLMDEV_STATUS_MEDIA_STATUS) == CXLMDEV_MS_READY)
> +#define CXLMDEV_MBOX_IF_READY BIT(4)
> +#define CXLMDEV_RESET_NEEDED_SHIFT 5
> +#define CXLMDEV_RESET_NEEDED_MASK GENMASK(7, 5)
> +#define CXLMDEV_RESET_NEEDED_NOT 0
> +#define CXLMDEV_RESET_NEEDED_COLD 1
> +#define CXLMDEV_RESET_NEEDED_WARM 2
> +#define CXLMDEV_RESET_NEEDED_HOT 3
> +#define CXLMDEV_RESET_NEEDED_CXL 4
> +#define CXLMDEV_RESET_NEEDED(status) \
> + (CXL_GET_FIELD(status, CXLMDEV_RESET_NEEDED) != CXLMDEV_RESET_NEEDED_NOT)
>
> /**
> * struct cxl_mem - A CXL memory device
> @@ -45,6 +76,16 @@ struct cxl_mem {
> struct pci_dev *pdev;
> void __iomem *regs;
>
> + struct {
> + struct range range;
> + } pmem;
> +
> + struct {
> + struct range range;
> + } ram;
> +
> + char firmware_version[0x10];
> +
> /* Cap 0001h - CXL_CAP_CAP_ID_DEVICE_STATUS */
> struct {
> void __iomem *regs;
> @@ -52,6 +93,7 @@ struct cxl_mem {
>
> /* Cap 0002h - CXL_CAP_CAP_ID_PRIMARY_MAILBOX */
> struct {
> + struct mutex mutex; /* Protects device mailbox and firmware */
> void __iomem *regs;
> size_t payload_size;
> } mbox;
> @@ -90,6 +132,7 @@ struct cxl_mem {
>
> cxl_reg(status);
> cxl_reg(mbox);
> +cxl_reg(mem);
>
> #define cxl_payload_regs(cxlm) \
> ((void __iomem *)(cxlm)->mbox.regs + CXLDEV_MB_PAYLOAD_OFFSET)
> diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
> index 8da9f4a861ea..e9ba97bbd7b9 100644
> --- a/drivers/cxl/mem.c
> +++ b/drivers/cxl/mem.c
> @@ -1,5 +1,6 @@
> // SPDX-License-Identifier: GPL-2.0-only
> /* Copyright(c) 2020 Intel Corporation. All rights reserved. */
> +#include <linux/sched/clock.h>
> #include <linux/module.h>
> #include <linux/pci.h>
> #include <linux/io.h>
> @@ -7,6 +8,248 @@
> #include "pci.h"
> #include "cxl.h"
>
> +#define cxl_doorbell_busy(cxlm) \
> + (cxl_read_mbox_reg32(cxlm, CXLDEV_MB_CTRL_OFFSET) & \
> + CXLDEV_MB_CTRL_DOORBELL)
> +
> +#define CXL_MAILBOX_TIMEOUT_US 2000
> +
> +enum opcode {
> + CXL_MBOX_OP_IDENTIFY = 0x4000,
> + CXL_MBOX_OP_MAX = 0x10000
> +};
> +
> +/**
> + * struct mbox_cmd - A command to be submitted to hardware.
> + * @opcode: (input) The command set and command submitted to hardware.
> + * @payload: (input/output) Pointer to the input and output payload.
> + * Payload can be NULL if the caller wants to populate the payload
> + * registers themselves (potentially avoiding a copy).
> + * @size_in: (input) Number of bytes to load from @payload.
> + * @size_out:
> + * - (input) Number of bytes allocated to load into @payload.
> + * - (output) Number of bytes loaded into @payload.
> + * @return_code: (output) Error code returned from hardware.
> + *
> + * This is the primary mechanism used to send commands to the hardware.
> + * All the fields except @payload correspond exactly to the fields described in
> + * Command Register section of the CXL 2.0 spec (8.2.8.4.5). @payload
> + * corresponds to the Command Payload Registers (8.2.8.4.8).
> + */
> +struct mbox_cmd {
> + u16 opcode;
> + void *payload;
> + size_t size_in;
> + size_t size_out;
> + u16 return_code;
> +};
> +
> +static int cxl_mem_wait_for_doorbell(struct cxl_mem *cxlm)
> +{
> + const int timeout = msecs_to_jiffies(CXL_MAILBOX_TIMEOUT_US);
> + const unsigned long start = jiffies;
> + unsigned long end = start;
> +
> + while (cxl_doorbell_busy(cxlm)) {
> + end = jiffies;
> +
> + if (time_after(end, start + timeout)) {
> + /* Check again in case preempted before timeout test */
> + if (!cxl_doorbell_busy(cxlm))
> + break;
> + return -ETIMEDOUT;
> + }
> + cpu_relax();
> + }
> +
> + dev_dbg(&cxlm->pdev->dev, "Doorbell wait took %dms",
> + jiffies_to_msecs(end) - jiffies_to_msecs(start));
> + return 0;
> +}
> +
> +static void cxl_mem_mbox_timeout(struct cxl_mem *cxlm,
> + struct mbox_cmd *mbox_cmd)
> +{
> + dev_warn(&cxlm->pdev->dev, "Mailbox command timed out\n");
> + dev_info(&cxlm->pdev->dev,
> + "\topcode: 0x%04x\n"
> + "\tpayload size: %zub\n",
> + mbox_cmd->opcode, mbox_cmd->size_in);
> + print_hex_dump_debug("Payload ", DUMP_PREFIX_OFFSET, 16, 1,
> + mbox_cmd->payload, mbox_cmd->size_in, true);
> +
> + /* Here's a good place to figure out if a device reset is needed */
> +}
> +
> +/**
> + * cxl_mem_mbox_send_cmd() - Send a mailbox command to a memory device.
> + * @cxlm: The CXL memory device to communicate with.
> + * @mbox_cmd: Command to send to the memory device.
> + *
> + * Context: Any context. Expects mbox_lock to be held.
> + * Return: -ETIMEDOUT if timeout occurred waiting for completion. 0 on success.
> + * Caller should check the return code in @mbox_cmd to make sure it
> + * succeeded.
> + *
> + * This is a generic form of the CXL mailbox send command, thus the only I/O
> + * operations used are cxl_read_mbox_reg(). Memory devices, and perhaps other
> + * types of CXL devices may have further information available upon error
> + * conditions.
> + *
> + * FIXME: As stated above, references to &struct cxl_mem should be changed to a
> + * more generic cxl structure when needed.
I'd flip that to a TODO. If this driver is ready to go before other usecases
perhaps need to clear this doesn't 'need' to be fixed.
> + */
> +static int cxl_mem_mbox_send_cmd(struct cxl_mem *cxlm,
> + struct mbox_cmd *mbox_cmd)
> +{
> + u64 cmd_reg, status_reg;
> + size_t out_len;
> + int rc;
> +
> + lockdep_assert_held(&cxlm->mbox.mutex);
> +
> + /*
> + * Here are the steps from 8.2.8.4 of the CXL 2.0 spec.
> + * 1. Caller reads MB Control Register to verify doorbell is clear
> + * 2. Caller writes Command Register
> + * 3. Caller writes Command Payload Registers if input payload is non-empty
> + * 4. Caller writes MB Control Register to set doorbell
> + * 5. Caller either polls for doorbell to be clear or waits for interrupt if configured
> + * 6. Caller reads MB Status Register to fetch Return code
> + * 7. If command successful, Caller reads Command Register to get Payload Length
> + * 8. If output payload is non-empty, host reads Command Payload Registers
> + */
> +
> + /* #1 */
> + WARN_ON(cxl_doorbell_busy(cxlm));
If it was busy, should we proceed anyway?
> +
> + cmd_reg = CXL_SET_FIELD(mbox_cmd->opcode, CXLDEV_MB_CMD_COMMAND_OPCODE);
> + if (mbox_cmd->size_in) {
> + cmd_reg |= CXL_SET_FIELD(mbox_cmd->size_in,
> + CXLDEV_MB_CMD_PAYLOAD_LENGTH);
> + if (mbox_cmd->payload)
> + memcpy_toio(cxl_payload_regs(cxlm), mbox_cmd->payload,
> + mbox_cmd->size_in);
I'm sure this is fine, but technically isn't this #3 then #2?
> + }
> +
> + /* #2, #3 */
> + cxl_write_mbox_reg64(cxlm, CXLDEV_MB_CMD_OFFSET, cmd_reg);
> +
> + /* #4 */
> + dev_dbg(&cxlm->pdev->dev, "Sending command\n");
> + cxl_write_mbox_reg32(cxlm, CXLDEV_MB_CTRL_OFFSET,
> + CXLDEV_MB_CTRL_DOORBELL);
> +
> + /* #5 */
> + rc = cxl_mem_wait_for_doorbell(cxlm);
> + if (rc == -ETIMEDOUT) {
> + cxl_mem_mbox_timeout(cxlm, mbox_cmd);
> + return rc;
> + }
> +
> + /* #6 */
> + status_reg = cxl_read_mbox_reg64(cxlm, CXLDEV_MB_STATUS_OFFSET);
> + mbox_cmd->return_code =
> + CXL_GET_FIELD(status_reg, CXLDEV_MB_STATUS_RET_CODE);
> +
> + if (mbox_cmd->return_code != 0) {
> + dev_dbg(&cxlm->pdev->dev, "Mailbox operation had an error\n");
> + return 0;
> + }
> +
> + /* #7 */
> + cmd_reg = cxl_read_mbox_reg64(cxlm, CXLDEV_MB_CMD_OFFSET);
> + out_len = CXL_GET_FIELD(cmd_reg, CXLDEV_MB_CMD_PAYLOAD_LENGTH);
> + mbox_cmd->size_out = out_len;
> +
> + /* #8 */
> + if (out_len && mbox_cmd->payload)
> + memcpy_fromio(mbox_cmd->payload, cxl_payload_regs(cxlm),
> + mbox_cmd->size_out);
> +
> + return 0;
> +}
> +
> +/**
> + * cxl_mem_mbox_get() - Acquire exclusive access to the mailbox.
> + * @cxlm: The memory device to gain access to.
> + *
> + * Context: Any context. Takes the mbox_lock.
> + * Return: 0 if exclusive access was acquired.
> + */
> +static int cxl_mem_mbox_get(struct cxl_mem *cxlm)
> +{
> + u64 md_status;
> + int rc = -EBUSY;
Trivial: Always set below so no need to init here.
> +
> + mutex_lock_io(&cxlm->mbox.mutex);
> +
> + /*
> + * XXX: There is some amount of ambiguity in the 2.0 version of the spec
> + * around the mailbox interface ready (8.2.8.5.1.1). The purpose of the
> + * bit is to allow firmware running on the device to notify us that it's
> + * ready to receive commands. It is unclear if the bit needs to be read
> + * every time one tries to use the mailbox, ie. the firmware can switch
> + * it on and off as needed. Second, there is no defined timeout for
> + * mailbox ready, like there is for the doorbell interface.
> + *
> + * As such, we make the following assumptions:
> + * 1. The firmware might toggle the Mailbox Interface Ready bit, and so
> + * we check it for every command.
> + * 2. If the doorbell is clear, the firmware should have first set the
> + * Mailbox Interface Ready bit. Therefore, waiting for the doorbell
> + * to be ready is a sufficient amount of time.
> + */
> + rc = cxl_mem_wait_for_doorbell(cxlm);
> + if (rc) {
> + dev_warn(&cxlm->pdev->dev, "Mailbox interface not ready\n");
> + goto out;
> + }
> +
> + md_status = cxl_read_mem_reg64(cxlm, CXLMDEV_STATUS_OFFSET);
> + if (md_status & CXLMDEV_MBOX_IF_READY && CXLMDEV_READY(md_status)) {
> + /*
> + * Hardware shouldn't allow a ready status but also have failure
> + * bits set. Spit out an error, this should be a bug report
> + */
> + if (md_status & CXLMDEV_DEV_FATAL) {
> + dev_err(&cxlm->pdev->dev,
> + "CXL device reporting ready and fatal\n");
> + rc = -EFAULT;
> + goto out;
> + }
> + if (md_status & CXLMDEV_FW_HALT) {
> + dev_err(&cxlm->pdev->dev,
> + "CXL device reporting ready and halted\n");
> + rc = -EFAULT;
> + goto out;
> + }
> + if (CXLMDEV_RESET_NEEDED(md_status)) {
> + dev_err(&cxlm->pdev->dev,
> + "CXL device reporting ready and reset needed\n");
> + rc = -EFAULT;
> + goto out;
> + }
> +
> + return 0;
> + }
> +
> +out:
> + mutex_unlock(&cxlm->mbox.mutex);
> + return rc;
> +}
> +
> +/**
> + * cxl_mem_mbox_put() - Release exclusive access to the mailbox.
> + * @cxlm: The CXL memory device to communicate with.
> + *
> + * Context: Any context. Expects mbox_lock to be held.
> + */
> +static void cxl_mem_mbox_put(struct cxl_mem *cxlm)
> +{
> + mutex_unlock(&cxlm->mbox.mutex);
> +}
> +
> /**
> * cxl_mem_setup_regs() - Setup necessary MMIO.
> * @cxlm: The CXL memory device to communicate with.
> @@ -135,6 +378,8 @@ static struct cxl_mem *cxl_mem_create(struct pci_dev *pdev, u32 reg_lo,
> return NULL;
> }
>
> + mutex_init(&cxlm->mbox.mutex);
> +
> regs = pcim_iomap_table(pdev)[bar];
> cxlm->pdev = pdev;
> cxlm->regs = regs + offset;
> @@ -167,6 +412,69 @@ static int cxl_mem_dvsec(struct pci_dev *pdev, int dvsec)
> return 0;
> }
>
> +/**
> + * cxl_mem_identify() - Send the IDENTIFY command to the device.
> + * @cxlm: The device to identify.
> + *
> + * Return: 0 if identify was executed successfully.
> + *
> + * This will dispatch the identify command to the device and on success populate
> + * structures to be exported to sysfs.
> + */
> +static int cxl_mem_identify(struct cxl_mem *cxlm)
> +{
> + struct cxl_mbox_identify {
> + char fw_revision[0x10];
> + __le64 total_capacity;
> + __le64 volatile_capacity;
> + __le64 persistent_capacity;
> + __le64 partition_align;
> + __le16 info_event_log_size;
> + __le16 warning_event_log_size;
> + __le16 failure_event_log_size;
> + __le16 fatal_event_log_size;
> + __le32 lsa_size;
> + u8 poison_list_max_mer[3];
> + __le16 inject_poison_limit;
> + u8 poison_caps;
> + u8 qos_telemetry_caps;
> + } __packed id;
> + struct mbox_cmd mbox_cmd;
> + int rc;
> +
> + /* Retrieve initial device memory map */
> + rc = cxl_mem_mbox_get(cxlm);
> + if (rc)
> + return rc;
> +
> + mbox_cmd = (struct mbox_cmd){
> + .opcode = CXL_MBOX_OP_IDENTIFY,
> + .payload = &id,
> + .size_in = 0,
> + };
> + rc = cxl_mem_mbox_send_cmd(cxlm, &mbox_cmd);
Description of this function has:
"Caller should check the return code in @mbox_cmd to make sure it succeeded."
I'm not seeing that here.
> + cxl_mem_mbox_put(cxlm);
> + if (rc)
> + return rc;
> +
> + if (mbox_cmd.size_out != sizeof(id))
> + return -ENXIO;
> +
> + /*
> + * TODO: enumerate DPA map, as 'ram' and 'pmem' do not alias.
> + * For now, only the capacity is exported in sysfs
> + */
> + cxlm->ram.range.start = 0;
> + cxlm->ram.range.end = le64_to_cpu(id.volatile_capacity) - 1;
> +
> + cxlm->pmem.range.start = 0;
> + cxlm->pmem.range.end = le64_to_cpu(id.persistent_capacity) - 1;
> +
> + memcpy(cxlm->firmware_version, id.fw_revision, sizeof(id.fw_revision));
> +
> + return rc;
> +}
> +
> static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> {
> struct device *dev = &pdev->dev;
> @@ -222,6 +530,10 @@ static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> if (rc)
> goto err;
>
> + rc = cxl_mem_identify(cxlm);
> + if (rc)
> + goto err;
> +
> pci_set_drvdata(pdev, cxlm);
> return 0;
>
On Mon, 11 Jan 2021 14:51:13 -0800
Ben Widawsky <[email protected]> wrote:
> Add a straightforward IOCTL that provides a mechanism for userspace to
> query the supported memory device commands. CXL commands as they appear
> to userspace are described as part of the UAPI kerneldoc. The command
> list returned via this IOCTL will contain the full set of commands that
> the driver supports, however, some of those commands may not be
> available for use by userspace.
>
> Memory device commands are specified in 8.2.9 of the CXL 2.0
> specification. They are submitted through a mailbox mechanism specified
> in 8.2.8.4.
>
> Signed-off-by: Ben Widawsky <[email protected]>
Hi Ben,
Maybe today I'll get a clear block to finish reading through these!
Anyhow, some comments inline.
Jonathan
> ---
> .clang-format | 1 +
> Documentation/cxl/memory-devices.rst | 9 +
> .../userspace-api/ioctl/ioctl-number.rst | 1 +
> drivers/cxl/mem.c | 156 ++++++++++++++++++
> include/uapi/linux/cxl_mem.h | 117 +++++++++++++
> 5 files changed, 284 insertions(+)
> create mode 100644 include/uapi/linux/cxl_mem.h
>
> diff --git a/.clang-format b/.clang-format
> index 10dc5a9a61b3..3f11c8901b43 100644
> --- a/.clang-format
> +++ b/.clang-format
> @@ -109,6 +109,7 @@ ForEachMacros:
> - 'css_for_each_child'
> - 'css_for_each_descendant_post'
> - 'css_for_each_descendant_pre'
> + - 'cxl_for_each_cmd'
> - 'device_for_each_child_node'
> - 'dma_fence_chain_for_each'
> - 'do_for_each_ftrace_op'
> diff --git a/Documentation/cxl/memory-devices.rst b/Documentation/cxl/memory-devices.rst
> index 5f723c25382b..ec54674b3822 100644
> --- a/Documentation/cxl/memory-devices.rst
> +++ b/Documentation/cxl/memory-devices.rst
> @@ -32,6 +32,15 @@ CXL Memory Device
> .. kernel-doc:: drivers/cxl/mem.c
> :internal:
>
> +CXL IOCTL Interface
> +-------------------
> +
> +.. kernel-doc:: include/uapi/linux/cxl_mem.h
> + :doc: UAPI
> +
> +.. kernel-doc:: include/uapi/linux/cxl_mem.h
> + :internal:
> +
> External Interfaces
> ===================
>
> diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst b/Documentation/userspace-api/ioctl/ioctl-number.rst
> index a4c75a28c839..6eb8e634664d 100644
> --- a/Documentation/userspace-api/ioctl/ioctl-number.rst
> +++ b/Documentation/userspace-api/ioctl/ioctl-number.rst
> @@ -352,6 +352,7 @@ Code Seq# Include File Comments
> <mailto:[email protected]>
> 0xCC 00-0F drivers/misc/ibmvmc.h pseries VMC driver
> 0xCD 01 linux/reiserfs_fs.h
> +0xCE 01-02 uapi/linux/cxl_mem.h Compute Express Link Memory Devices
> 0xCF 02 fs/cifs/ioctl.c
> 0xDB 00-0F drivers/char/mwave/mwavepub.h
> 0xDD 00-3F ZFCP device driver see drivers/s390/scsi/
> diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
> index da2bf941fe92..d4eb3f5b9469 100644
> --- a/drivers/cxl/mem.c
> +++ b/drivers/cxl/mem.c
> @@ -7,6 +7,7 @@
> #include <linux/idr.h>
> #include <linux/pci.h>
> #include <linux/io.h>
> +#include <uapi/linux/cxl_mem.h>
> #include "acpi.h"
> #include "pci.h"
> #include "cxl.h"
> @@ -41,6 +42,7 @@
> #define CXL_MAILBOX_TIMEOUT_US 2000
>
> enum opcode {
> + CXL_MBOX_OP_INVALID = 0x0000,
> CXL_MBOX_OP_IDENTIFY = 0x4000,
> CXL_MBOX_OP_MAX = 0x10000
> };
> @@ -82,6 +84,90 @@ static DEFINE_IDR(cxl_mem_idr);
> /* protect cxl_mem_idr allocations */
> static DEFINE_MUTEX(cxl_memdev_lock);
>
> +#define CXL_CMD(_id, _flags, sin, sout, f) \
Perhaps this would be better after the structure definition?
> + [CXL_MEM_COMMAND_ID_##_id] = { \
> + { \
I would suggest naming the sub structure in this initializer.
.info = { ...
> + .id = CXL_MEM_COMMAND_ID_##_id, \
> + .flags = CXL_MEM_COMMAND_FLAG_##_flags, \
> + .size_in = sin, \
> + .size_out = sout, \
> + }, \
> + .flags = CXL_CMD_INTERNAL_FLAG_##f, \
These look to be indented too far.
> + .opcode = CXL_MBOX_OP_##_id, \
> + }
> +
> +/**
> + * struct cxl_mem_command - Driver representation of a memory device command
> + * @info: Command information as it exists for the UAPI
> + * @opcode: The actual bits used for the mailbox protocol
> + * @flags: Set of flags reflecting the state of the command.
> + *
> + * * %CXL_CMD_INTERNAL_FLAG_HIDDEN: Command is hidden from userspace. This
> + * would typically be used for deprecated commands.
> + * * %CXL_CMD_FLAG_MANDATORY: Hardware must support this command. This flag is
> + * only used internally by the driver for sanity checking.
> + *
> + * The cxl_mem_command is the driver's internal representation of commands that
> + * are supported by the driver. Some of these commands may not be supported by
> + * the hardware. The driver will use @info to validate the fields passed in by
> + * the user then submit the @opcode to the hardware.
> + *
> + * See struct cxl_command_info.
> + */
> +struct cxl_mem_command {
> + const struct cxl_command_info info;
> + enum opcode opcode;
> + u32 flags;
> +#define CXL_CMD_INTERNAL_FLAG_NONE 0
> +#define CXL_CMD_INTERNAL_FLAG_HIDDEN BIT(0)
> +#define CXL_CMD_INTERNAL_FLAG_MANDATORY BIT(1)
> +};
> +
> +/*
> + * This table defines the supported mailbox commands for the driver. This table
> + * is made up of a UAPI structure. Non-negative values as parameters in the
> + * table will be validated against the user's input. For example, if size_in is
> + * 0, and the user passed in 1, it is an error.
> + */
> +static struct cxl_mem_command mem_commands[] = {
> + CXL_CMD(INVALID, KERNEL, 0, 0, HIDDEN),
> + CXL_CMD(IDENTIFY, NONE, 0, 0x43, MANDATORY),
> +};
> +
> +#define cxl_for_each_cmd(cmd) \
> + for ((cmd) = &mem_commands[0]; \
> + ((cmd) - mem_commands) < ARRAY_SIZE(mem_commands); (cmd)++)
> +
> +static inline int cxl_cmd_index(const struct cxl_mem_command *c)
> +{
> + if (unlikely(c >= mem_commands &&
> + c < (mem_commands + sizeof(mem_commands)))) {
This path could do with an explanatory comment.
> + struct cxl_mem_command *p;
> +
> + cxl_for_each_cmd(p)
> + if (p->opcode == c->opcode &&
> + p->flags == c->flags)
> + return p - mem_commands;
> + } else {
> + return c - mem_commands;
> + }
> +
> + WARN_ON_ONCE(1);
> + return 0;
> +}
> +
> +static inline struct cxl_mem_command *cxl_mem_find_command(u16 opcode)
> +{
> + struct cxl_mem_command *c;
> +
> + cxl_for_each_cmd(c) {
> + if (c->opcode == opcode)
> + return c;
> + }
> +
> + return NULL;
> +}
> +
> static int cxl_mem_wait_for_doorbell(struct cxl_mem *cxlm)
> {
> const int timeout = msecs_to_jiffies(CXL_MAILBOX_TIMEOUT_US);
> @@ -140,6 +226,7 @@ static void cxl_mem_mbox_timeout(struct cxl_mem *cxlm,
> static int cxl_mem_mbox_send_cmd(struct cxl_mem *cxlm,
> struct mbox_cmd *mbox_cmd)
> {
> + const struct cxl_mem_command *cmd;
> u64 cmd_reg, status_reg;
> size_t out_len;
> int rc;
> @@ -158,6 +245,13 @@ static int cxl_mem_mbox_send_cmd(struct cxl_mem *cxlm,
> * 8. If output payload is non-empty, host reads Command Payload Registers
> */
>
> + cmd = cxl_mem_find_command(mbox_cmd->opcode);
> + if (!cmd) {
> + dev_info(&cxlm->pdev->dev,
> + "Unknown opcode 0x%04x being sent to hardware\n",
> + mbox_cmd->opcode);
> + }
> +
> /* #1 */
> WARN_ON(cxl_doorbell_busy(cxlm));
>
> @@ -200,6 +294,20 @@ static int cxl_mem_mbox_send_cmd(struct cxl_mem *cxlm,
> out_len = CXL_GET_FIELD(cmd_reg, CXLDEV_MB_CMD_PAYLOAD_LENGTH);
> mbox_cmd->size_out = out_len;
>
> + /*
> + * If the command had a fixed size output, but the hardware did
> + * something unexpected, just print an error and move on. It would be
> + * worth sending a bug report.
> + */
> + if (cmd && cmd->info.size_out >= 0 &&
> + mbox_cmd->size_out != cmd->info.size_out) {
> + bool too_big = mbox_cmd->size_out > cmd->info.size_out;
> +
> + dev_err(&cxlm->pdev->dev,
> + "payload was %s than driver expectations\n",
> + too_big ? "larger" : "smaller");
> + }
> +
> /* #8 */
> if (out_len && mbox_cmd->payload)
> memcpy_fromio(mbox_cmd->payload, cxl_payload_regs(cxlm),
> @@ -323,8 +431,56 @@ static int cxl_mem_release(struct inode *inode, struct file *file)
> return 0;
> }
>
> +static int cxl_mem_count_commands(void)
> +{
> + struct cxl_mem_command *c;
> + int n = 0;
> +
> + cxl_for_each_cmd(c) {
> + if (c->flags & CXL_CMD_INTERNAL_FLAG_HIDDEN)
> + continue;
> + n++;
> + }
> +
> + return n;
> +}
> +
> static long cxl_mem_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
> {
> + struct cxl_memdev *cxlmd = file->private_data;
> + struct device *dev = &cxlmd->dev;
> +
> + if (cmd == CXL_MEM_QUERY_COMMANDS) {
> + struct cxl_mem_query_commands __user *q = (void __user *)arg;
> + struct cxl_mem_command *cmd;
> + u32 n_commands;
> + int j = 0;
> +
> + dev_dbg(dev, "Query IOCTL\n");
> +
> + if (get_user(n_commands, (u32 __user *)arg))
> + return -EFAULT;
> +
> + /* returns the total number if 0 elements are requested. */
> + if (n_commands == 0)
> + return put_user(cxl_mem_count_commands(),
> + (u32 __user *)arg);
Perhaps clearer to fill in the n_commands field of q?
> +
> + /* otherwise, return max(n_commands, total commands) */
Doesn't seem to be doing this...
> + cxl_for_each_cmd(cmd) {
> + const struct cxl_command_info *info = &cmd->info;
> +
> + if (cmd->flags & CXL_CMD_INTERNAL_FLAG_HIDDEN)
> + continue;
> +
> + if (copy_to_user(&q->commands[j++], info, sizeof(*info)))
> + return -EFAULT;
> +
> + if (j == n_commands)
> + break;
That doesn't look right. Shouldn't be returning -ENOTTY in either of the success
paths.
> + }
> + }
> +
> return -ENOTTY;
> }
>
> diff --git a/include/uapi/linux/cxl_mem.h b/include/uapi/linux/cxl_mem.h
> new file mode 100644
> index 000000000000..847f825bbe18
> --- /dev/null
> +++ b/include/uapi/linux/cxl_mem.h
> @@ -0,0 +1,117 @@
> +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
> +/*
> + * CXL IOCTLs for Memory Devices
> + */
> +
> +#ifndef _UAPI_CXL_MEM_H_
> +#define _UAPI_CXL_MEM_H_
> +
> +#if defined(__cplusplus)
> +extern "C" {
> +#endif
> +
> +/**
> + * DOC: UAPI
> + *
> + * CXL memory devices expose UAPI to have a standard user interface.
> + * Userspace can refer to these structure definitions and UAPI formats
> + * to communicate to driver. The commands themselves are somewhat obfuscated
> + * with macro magic. They have the form CXL_MEM_COMMAND_ID_<name>.
> + *
> + * For example "CXL_MEM_COMMAND_ID_INVALID"
> + *
> + * Not all of all commands that the driver supports are always available for use
> + * by userspace. Userspace must check the results from the QUERY command in
> + * order to determine the live set of commands.
> + */
> +
> +#define CXL_MEM_QUERY_COMMANDS _IOR(0xCE, 1, struct cxl_mem_query_commands)
> +
> +#undef CMDS
> +#define CMDS \
> + C(INVALID, "Invalid Command"), \
> + C(IDENTIFY, "Identify Command"), \
> + C(MAX, "Last command")
> +#undef C
> +#define C(a, b) CXL_MEM_COMMAND_ID_##a
> +
> +enum { CMDS };
> +
> +/**
> + * struct cxl_command_info - Command information returned from a query.
> + * @id: ID number for the command.
> + * @flags: Flags that specify command behavior.
> + *
> + * * %CXL_MEM_COMMAND_FLAG_KERNEL: This command is reserved for exclusive
> + * kernel use.
> + * * %CXL_MEM_COMMAND_FLAG_MUTEX: This command may require coordination with
> + * the kernel in order to complete successfully.
> + *
> + * @size_in: Expected input size, or -1 if variable length.
> + * @size_out: Expected output size, or -1 if variable length.
> + *
> + * Represents a single command that is supported by both the driver and the
> + * hardware. The is returned as part of an array from the query ioctl. The
This is?
> + * following would be a command named "foobar" that takes a variable length
> + * input and returns 0 bytes of output.
> + *
> + * - @id = 10
> + * - @flags = CXL_MEM_COMMAND_FLAG_MUTEX
> + * - @size_in = -1
> + * - @size_out = 0
> + *
> + * See struct cxl_mem_query_commands.
> + */
> +struct cxl_command_info {
> + __u32 id;
> +
> + __u32 flags;
> +#define CXL_MEM_COMMAND_FLAG_NONE 0
> +#define CXL_MEM_COMMAND_FLAG_KERNEL BIT(0)
> +#define CXL_MEM_COMMAND_FLAG_MUTEX BIT(1)
> +
> + __s32 size_in;
> + __s32 size_out;
> +};
> +
> +/**
> + * struct cxl_mem_query_commands - Query supported commands.
> + * @n_commands: In/out parameter. When @n_commands is > 0, the driver will
> + * return min(num_support_commands, n_commands). When @n_commands
> + * is 0, driver will return the number of total supported commands.
> + * @rsvd: Reserved for future use.
> + * @commands: Output array of supported commands. This array must be allocated
> + * by userspace to be at least min(num_support_commands, @n_commands)
> + *
> + * Allow userspace to query the available commands supported by both the driver,
> + * and the hardware. Commands that aren't supported by either the driver, or the
> + * hardware are not returned in the query.
> + *
> + * Examples:
> + *
> + * - { .n_commands = 0 } // Get number of supported commands
> + * - { .n_commands = 15, .commands = buf } // Return first 15 (or less)
> + * supported commands
> + *
> + * See struct cxl_command_info.
> + */
> +struct cxl_mem_query_commands {
> + /*
> + * Input: Number of commands to return (space allocated by user)
> + * Output: Number of commands supported by the driver/hardware
> + *
> + * If n_commands is 0, kernel will only return number of commands and
> + * not try to populate commands[], thus allowing userspace to know how
> + * much space to allocate
> + */
> + __u32 n_commands;
> + __u32 rsvd;
> +
> + struct cxl_command_info __user commands[]; /* out: supported commands */
> +};
> +
> +#if defined(__cplusplus)
> +}
> +#endif
> +
> +#endif
On Mon, 11 Jan 2021 14:51:12 -0800
Ben Widawsky <[email protected]> wrote:
> From: Dan Williams <[email protected]>
>
> Create the /sys/bus/cxl hierarchy to enumerate memory devices
> (per-endpoint control devices), memory address space devices (platform
> address ranges with interleaving, performance, and persistence
> attributes), and memory regions (active provisioned memory from an
> address space device that is in use as System RAM or delegated to
> libnvdimm as Persistent Memory regions).
>
> For now, only the per-endpoint control devices are registered on the
> 'cxl' bus.
>
> Signed-off-by: Dan Williams <[email protected]>
> Signed-off-by: Ben Widawsky <[email protected]>
Don't hide the driver core change inside a patch doing other stuff.
(it is a nice change though!)
Otherwise, just a request for units in the sysfs ABI docs.
Jonathan
> ---
> Documentation/ABI/testing/sysfs-bus-cxl | 26 +++
> Documentation/cxl/memory-devices.rst | 3 +
> drivers/base/core.c | 14 ++
> drivers/cxl/Makefile | 2 +
> drivers/cxl/bus.c | 54 +++++
> drivers/cxl/bus.h | 8 +
> drivers/cxl/cxl.h | 3 +
> drivers/cxl/mem.c | 282 +++++++++++++++++++++++-
> include/linux/device.h | 1 +
> 9 files changed, 392 insertions(+), 1 deletion(-)
> create mode 100644 Documentation/ABI/testing/sysfs-bus-cxl
> create mode 100644 drivers/cxl/bus.c
> create mode 100644 drivers/cxl/bus.h
>
> diff --git a/Documentation/ABI/testing/sysfs-bus-cxl b/Documentation/ABI/testing/sysfs-bus-cxl
> new file mode 100644
> index 000000000000..fe7b87eba988
> --- /dev/null
> +++ b/Documentation/ABI/testing/sysfs-bus-cxl
> @@ -0,0 +1,26 @@
> +What: /sys/bus/cxl/devices/memX/firmware_version
> +Date: December, 2020
> +KernelVersion: v5.12
> +Contact: [email protected]
> +Description:
> + (RO) "FW Revision" string as reported by the Identify
> + Memory Device Output Payload in the CXL-2.0
> + specification.
> +
> +What: /sys/bus/cxl/devices/memX/ram/size
> +Date: December, 2020
> +KernelVersion: v5.12
> +Contact: [email protected]
> +Description:
> + (RO) "Volatile Only Capacity" as reported by the
> + Identify Memory Device Output Payload in the CXL-2.0
> + specification.
Nice to see the format /units of these described in this doc as well as the spec.
Section number etc also good if you can add it to these docs.
> +
> +What: /sys/bus/cxl/devices/memX/pmem/size
> +Date: December, 2020
> +KernelVersion: v5.12
> +Contact: [email protected]
> +Description:
> + (RO) "Persistent Only Capacity" as reported by the
> + Identify Memory Device Output Payload in the CXL-2.0
> + specification.
> diff --git a/Documentation/cxl/memory-devices.rst b/Documentation/cxl/memory-devices.rst
> index 134c9b6b4ff4..5f723c25382b 100644
> --- a/Documentation/cxl/memory-devices.rst
> +++ b/Documentation/cxl/memory-devices.rst
> @@ -37,3 +37,6 @@ External Interfaces
>
> .. kernel-doc:: drivers/cxl/acpi.c
> :export:
> +
> +.. kernel-doc:: drivers/cxl/bus.c
> + :export:
> diff --git a/drivers/base/core.c b/drivers/base/core.c
> index 25e08e5f40bd..33432a4cbe23 100644
> --- a/drivers/base/core.c
> +++ b/drivers/base/core.c
> @@ -3179,6 +3179,20 @@ struct device *get_device(struct device *dev)
> }
> EXPORT_SYMBOL_GPL(get_device);
>
> +/**
> + * get_live_device() - increment reference count for device iff !dead
> + * @dev: device.
> + *
> + * Forward the call to get_device() if the device is still alive. If
> + * this is called with the device_lock() held then the device is
> + * guaranteed to not die until the device_lock() is dropped.
> + */
I like the idea of a nice generic wrapper for this but definitely not
deep inside a patch doing something else.
> +struct device *get_live_device(struct device *dev)
> +{
> + return dev && !dev->p->dead ? get_device(dev) : NULL;
> +}
> +EXPORT_SYMBOL_GPL(get_live_device);
> +
...
On Mon, 11 Jan 2021 14:51:14 -0800
Ben Widawsky <[email protected]> wrote:
> The send command allows userspace to issue mailbox commands directly to
> the hardware. The driver will verify basic properties of the command and
> possible inspect the input (or output) payload to determine whether or
> not the command is allowed (or might taint the kernel).
>
> The list of allowed commands and their properties can be determined by
> using the QUERY IOCTL for CXL memory devices.
>
> Signed-off-by: Ben Widawsky <[email protected]>
> ---
> drivers/cxl/mem.c | 204 ++++++++++++++++++++++++++++++++++-
> include/uapi/linux/cxl_mem.h | 39 +++++++
> 2 files changed, 239 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
> index d4eb3f5b9469..f979788b4d9f 100644
> --- a/drivers/cxl/mem.c
> +++ b/drivers/cxl/mem.c
> @@ -84,6 +84,13 @@ static DEFINE_IDR(cxl_mem_idr);
> /* protect cxl_mem_idr allocations */
> static DEFINE_MUTEX(cxl_memdev_lock);
>
> +#undef C
> +#define C(a, b) { b }
I'm not following why this is here?
> +static struct {
> + const char *name;
> +} command_names[] = { CMDS };
> +#undef C
> +
> #define CXL_CMD(_id, _flags, sin, sout, f) \
> [CXL_MEM_COMMAND_ID_##_id] = { \
> { \
...
> +
> +/**
> + * handle_mailbox_cmd_from_user() - Dispatch a mailbox command.
> + * @cxlmd: The CXL memory device to communicate with.
> + * @cmd: The validated command.
> + * @in_payload: Pointer to userspace's input payload.
> + * @out_payload: Pointer to userspace's output payload.
> + * @u: The command submitted by userspace. Has output fields.
> + *
> + * Return:
> + * * %0 - Mailbox transaction succeeded.
> + * * %-EFAULT - Something happened with copy_to/from_user.
> + * * %-EINTR - Mailbox acquisition interrupted.
> + * * %-E2BIG - Output payload would overrun buffer.
> + *
> + * Creates the appropriate mailbox command on behalf of a userspace request.
> + * Return value, size, and output payload are all copied out to @u. The
> + * parameters for the command must be validated before calling this function.
> + *
> + * A 0 return code indicates the command executed successfully, not that it was
> + * itself successful. IOW, the retval should always be checked if wanting to
cmd->retval perhaps to be more explicit?
> + * determine the actual result.
> + */
> +static int handle_mailbox_cmd_from_user(struct cxl_memdev *cxlmd,
> + const struct cxl_mem_command *cmd,
> + u64 in_payload,
> + u64 out_payload,
> + struct cxl_send_command __user *u)
> +{
> + struct mbox_cmd mbox_cmd = {
> + .opcode = cmd->opcode,
> + .size_in = cmd->info.size_in,
> + .payload = NULL, /* Copied by copy_to|from_user() */
> + };
> + int rc;
> +
> + if (cmd->info.size_in) {
> + /*
> + * Directly copy the userspace payload into the hardware. UAPI
> + * states that the buffer must already be little endian.
> + */
> + if (copy_from_user((__force void *)cxl_payload_regs(cxlmd->cxlm),
> + u64_to_user_ptr(in_payload),
> + cmd->info.size_in)) {
> + cxl_mem_mbox_put(cxlmd->cxlm);
mbox_get is after this point though it shouldn't be given we just
wrote into the mbox registers.
This seems unlikely to be a high performance path, so perhaps just
use a local buffer and let cxl_mem_mbox_send_cmd copy it into the registers.
> + return -EFAULT;
> + }
> + }
> +
> + rc = cxl_mem_mbox_get(cxlmd->cxlm, true);
> + if (rc)
> + return rc;
> +
> + dev_dbg(&cxlmd->dev,
> + "Submitting %s command for user\n"
> + "\topcode: %x\n"
> + "\tsize: %ub\n",
> + command_names[cmd->info.id].name, mbox_cmd.opcode,
> + cmd->info.size_in);
> +
> + rc = cxl_mem_mbox_send_cmd(cxlmd->cxlm, &mbox_cmd);
> + cxl_mem_mbox_put(cxlmd->cxlm);
> + if (rc)
> + return rc;
> +
> + if (mbox_cmd.size_out > cmd->info.size_out)
> + return -E2BIG;
> +
> + rc = put_user(mbox_cmd.return_code, &u->retval);
> + if (rc)
> + return rc;
> +
> + rc = put_user(mbox_cmd.size_out, &u->size_out);
> + if (rc)
> + return rc;
> +
> + if (mbox_cmd.size_out)
> + if (copy_to_user(u64_to_user_ptr(out_payload),
> + (__force void *)cxl_payload_regs(cxlmd->cxlm),
> + mbox_cmd.size_out))
> + return -EFAULT;
> +
> + return 0;
> +}
> +
...
>
> static long cxl_mem_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
> @@ -479,6 +644,37 @@ static long cxl_mem_ioctl(struct file *file, unsigned int cmd, unsigned long arg
> if (j == n_commands)
> break;
> }
> +
> + return 0;
Ah. That should have been in the earlier patch. Explains why the code works :)
> + } else if (cmd == CXL_MEM_SEND_COMMAND) {
> + struct cxl_send_command send, __user *u = (void __user *)arg;
> + struct cxl_mem_command c;
> + int rc;
> +
> + dev_dbg(dev, "Send IOCTL\n");
> +
> + if (copy_from_user(&send, u, sizeof(send)))
> + return -EFAULT;
> +
> + rc = device_lock_interruptible(dev);
> + if (rc)
> + return rc;
> +
> + if (!get_live_device(dev)) {
> + device_unlock(dev);
> + return -ENXIO;
> + }
> +
> + rc = cxl_validate_cmd_from_user(cxlmd->cxlm, &send, &c);
> + if (!rc)
> + rc = handle_mailbox_cmd_from_user(cxlmd, &c,
> + send.in_payload,
> + send.out_payload, u);
> +
> + put_device(dev);
> + device_unlock(dev);
> +
> + return rc;
> }
>
> return -ENOTTY;
> @@ -837,7 +1033,7 @@ static int cxl_mem_identify(struct cxl_mem *cxlm)
> int rc;
>
> /* Retrieve initial device memory map */
> - rc = cxl_mem_mbox_get(cxlm);
> + rc = cxl_mem_mbox_get(cxlm, false);
> if (rc)
> return rc;
>
> diff --git a/include/uapi/linux/cxl_mem.h b/include/uapi/linux/cxl_mem.h
> index 847f825bbe18..cb4e2bee5228 100644
> --- a/include/uapi/linux/cxl_mem.h
> +++ b/include/uapi/linux/cxl_mem.h
> @@ -26,6 +26,7 @@ extern "C" {
> */
>
> #define CXL_MEM_QUERY_COMMANDS _IOR(0xCE, 1, struct cxl_mem_query_commands)
> +#define CXL_MEM_SEND_COMMAND _IOWR(0xCE, 2, struct cxl_send_command)
>
> #undef CMDS
> #define CMDS \
> @@ -69,6 +70,7 @@ struct cxl_command_info {
> #define CXL_MEM_COMMAND_FLAG_NONE 0
> #define CXL_MEM_COMMAND_FLAG_KERNEL BIT(0)
> #define CXL_MEM_COMMAND_FLAG_MUTEX BIT(1)
> +#define CXL_MEM_COMMAND_FLAG_MASK GENMASK(31, 2)
Instinctively I'd expect FLAG_MASK to be GENMASK(1, 0)
and to be used as ~FLAG_MASK. As it's mask of flags, not
the mask to leave only valid flags.
>
> __s32 size_in;
> __s32 size_out;
> @@ -110,6 +112,43 @@ struct cxl_mem_query_commands {
> struct cxl_command_info __user commands[]; /* out: supported commands */
> };
>
> +/**
> + * struct cxl_send_command - Send a command to a memory device.
> + * @id: The command to send to the memory device. This must be one of the
> + * commands returned by the query command.
> + * @flags: Flags for the command (input).
> + * @rsvd: Must be zero.
> + * @retval: Return value from the memory device (output).
> + * @size_in: Size of the payload to provide to the device (input).
> + * @size_out: Size of the payload received from the device (input/output). This
> + * field is filled in by userspace to let the driver know how much
> + * space was allocated for output. It is populated by the driver to
> + * let userspace know how large the output payload actually was.
> + * @in_payload: Pointer to memory for payload input (little endian order).
> + * @out_payload: Pointer to memory for payload output (little endian order).
> + *
> + * Mechanism for userspace to send a command to the hardware for processing. The
> + * driver will do basic validation on the command sizes, but the payload input
> + * and output are not introspected. Userspace is required to allocate large
> + * enough buffers for max(size_in, size_out).
That sounds like both buffers must be the maximum between size_in and size_out.
Is intent that this is the maximum size_in for in_payload and max(size_out) for out_payload?
> + */
> +struct cxl_send_command {
> + __u32 id;
> + __u32 flags;
> + __u32 rsvd;
> + __u32 retval;
> +
> + struct {
> + __s32 size_in;
> + __u64 in_payload;
> + };
> +
> + struct {
> + __s32 size_out;
> + __u64 out_payload;
> + };
> +};
> +
> #if defined(__cplusplus)
> }
> #endif
On Mon, 11 Jan 2021 14:51:18 -0800
Ben Widawsky <[email protected]> wrote:
> CXL devices must implement the Device Command Interface (described in
> 8.2.9 of the CXL 2.0 spec). While the driver already maintains a list of
> commands it supports, there is still a need to be able to distinguish
> between commands that the driver knows about from commands that may not
> be supported by the hardware. No such commands currently are defined in
> the driver.
>
> The implementation leaves the statically defined table of commands and
> supplements it with a bitmap to determine commands that are enabled.
>
> ---
>
> There are multiple approaches that can be taken, but this is nice for a
> few reasons.
>
> Here are some of the other solutions:
>
> Create a per instance table with only the supported commands.
> 1. Having a fixed command id -> command mapping is much easier to manage
> for development and debugging.
> 2. Dealing with dynamic memory allocation for the table adds unnecessary
> complexity.
> 3. Most tables for device types are likely to be quite similar.
> 4. Makes it difficult to implement helper macros like cxl_for_each_cmd()
>
> If the per instance table did preserve ids, #1 above can be addressed.
> However, as "enable" is currently the only mutable state for the
> commands, it would yield a lot of overhead for not much gain.
> Additionally, the other issues remain.
>
> If "enable" remains the only mutable state, I believe this to be the
> best solution. Once the number of mutable elements in a command grows,
> it probably makes sense to move to per device instance state with a
> fixed command ID mapping.
Agreed with the logic.
However, patch wise, should either drop the --- above or move this below the
--- after your sign off. Otherwise you run the risk of git dropping your
sign off and resulting complaints from anyone run validation scripts
of the kernel tree that check for that.
>
> Signed-off-by: Ben Widawsky <[email protected]>
> ---
> drivers/cxl/cxl.h | 4 ++++
> drivers/cxl/mem.c | 38 +++++++++++++++++++++++++++++++++++++-
> 2 files changed, 41 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
> index 537ac4d8e6bd..963ba30cb200 100644
> --- a/drivers/cxl/cxl.h
> +++ b/drivers/cxl/cxl.h
> @@ -17,6 +17,9 @@
>
> #define CXL_GET_FIELD(word, field) FIELD_GET(field##_MASK, word)
>
> +/* XXX: Arbitrary max */
> +#define CXL_MAX_COMMANDS 32
If going this way, probably want a build time check that you don't
go beyond this value for a given command set. I haven't actually
thought about how to construct that but should be easy enough.
> +
> /* Device (CXL 2.0 - 8.2.8.3) */
> #define CXLDEV_CAP_ARRAY_REG 0x0
> #define CXLDEV_CAP_ARRAY_CAP_ID 0
> @@ -88,6 +91,7 @@ struct cxl_mem {
> } ram;
>
> char firmware_version[0x10];
> + DECLARE_BITMAP(enabled_cmds, CXL_MAX_COMMANDS);
>
> /* Cap 0001h - CXL_CAP_CAP_ID_DEVICE_STATUS */
> struct {
> diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
> index a824cfd4342a..20b26fa2c466 100644
> --- a/drivers/cxl/mem.c
> +++ b/drivers/cxl/mem.c
> @@ -114,6 +114,8 @@ static struct {
> * would typically be used for deprecated commands.
> * * %CXL_CMD_FLAG_MANDATORY: Hardware must support this command. This flag is
> * only used internally by the driver for sanity checking.
> + * * %CXL_CMD_INTERNAL_FLAG_PSEUDO: This is a pseudo command which doesn't have
> + * a direct mapping to hardware. They are implicitly always enabled.
> *
> * The cxl_mem_command is the driver's internal representation of commands that
> * are supported by the driver. Some of these commands may not be supported by
> @@ -129,6 +131,7 @@ struct cxl_mem_command {
> #define CXL_CMD_INTERNAL_FLAG_NONE 0
> #define CXL_CMD_INTERNAL_FLAG_HIDDEN BIT(0)
> #define CXL_CMD_INTERNAL_FLAG_MANDATORY BIT(1)
> +#define CXL_CMD_INTERNAL_FLAG_PSEUDO BIT(2)
> };
>
> /*
> @@ -140,7 +143,7 @@ struct cxl_mem_command {
> static struct cxl_mem_command mem_commands[] = {
> CXL_CMD(INVALID, KERNEL, 0, 0, HIDDEN),
> CXL_CMD(IDENTIFY, NONE, 0, 0x43, MANDATORY),
> - CXL_CMD(RAW, NONE, ~0, ~0, MANDATORY),
> + CXL_CMD(RAW, NONE, ~0, ~0, PSEUDO),
> };
>
> #define cxl_for_each_cmd(cmd) \
> @@ -618,6 +621,10 @@ static int cxl_validate_cmd_from_user(struct cxl_mem *cxlm,
> c = &mem_commands[send_cmd->id];
> info = &c->info;
>
> + /* Check that the command is enabled for hardware */
> + if (!test_bit(cxl_cmd_index(c), cxlm->enabled_cmds))
> + return -ENOTTY;
> +
> if (info->flags & CXL_MEM_COMMAND_FLAG_KERNEL)
> return -EPERM;
>
> @@ -1029,6 +1036,31 @@ static int cxl_mem_add_memdev(struct cxl_mem *cxlm)
> return rc;
> }
>
> +/**
> + * cxl_mem_enumerate_cmds() - Enumerate commands for a device.
> + * @cxlm: The device.
> + *
> + * Returns 0 if enumerate completed successfully.
> + *
> + * CXL devices have optional support for certain commands. This function will
> + * determine the set of supported commands for the hardware and update the
> + * enabled_cmds bitmap in the @cxlm.
> + */
> +static int cxl_mem_enumerate_cmds(struct cxl_mem *cxlm)
> +{
> + struct cxl_mem_command *c;
> +
> + /* All commands are considered enabled for now (except INVALID). */
> + cxl_for_each_cmd(c) {
> + if (c->flags & CXL_CMD_INTERNAL_FLAG_HIDDEN)
> + continue;
> +
> + set_bit(cxl_cmd_index(c), cxlm->enabled_cmds);
> + }
> +
> + return 0;
> +}
> +
> /**
> * cxl_mem_identify() - Send the IDENTIFY command to the device.
> * @cxlm: The device to identify.
> @@ -1147,6 +1179,10 @@ static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> if (rc)
> goto err;
>
> + rc = cxl_mem_enumerate_cmds(cxlm);
> + if (rc)
> + return rc;
> +
> rc = cxl_mem_identify(cxlm);
> if (rc)
> goto err;
On Mon, 11 Jan 2021 14:51:11 -0800
Ben Widawsky <[email protected]> wrote:
> Provide enough functionality to utilize the mailbox of a memory device.
> The mailbox is used to interact with the firmware running on the memory
> device.
>
> The CXL specification defines separate capabilities for the mailbox and
> the memory device. While we can confirm the mailbox is ready, in order
> to actually interact with the memory device, you must also confirm the
> device's firmware is ready.
>
> Create a function to handle sending a command, optionally with a
> payload, to the memory device, polling on a result, and then optionally
> copying out the payload. The algorithm for doing this comes straight out
> of the CXL 2.0 specification.
>
> Primary mailboxes are capable of generating an interrupt when submitting
> a command in the background. That implementation is saved for a later
> time.
>
> Secondary mailboxes aren't implemented at this time.
>
> The flow is proven with one implemented command, "identify". Because the
> class code has already told the driver this is a memory device and the
> identify command is mandatory, it's safe to assume for sane devices that
> everything here will work.
>
> Signed-off-by: Ben Widawsky <[email protected]>
One more thing that had me confused in a later patch (14)
J
> ---
> drivers/cxl/cxl.h | 43 +++++++
> drivers/cxl/mem.c | 312 ++++++++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 355 insertions(+)
>
> diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
> index a77286d04ce4..ca3fa496e21c 100644
> --- a/drivers/cxl/cxl.h
> +++ b/drivers/cxl/cxl.h
> @@ -32,9 +32,40 @@
> #define CXLDEV_MB_CAP_PAYLOAD_SIZE_MASK GENMASK(4, 0)
> #define CXLDEV_MB_CAP_PAYLOAD_SIZE_SHIFT 0
> #define CXLDEV_MB_CTRL_OFFSET 0x04
> +#define CXLDEV_MB_CTRL_DOORBELL BIT(0)
> #define CXLDEV_MB_CMD_OFFSET 0x08
> +#define CXLDEV_MB_CMD_COMMAND_OPCODE_SHIFT 0
> +#define CXLDEV_MB_CMD_COMMAND_OPCODE_MASK GENMASK(15, 0)
> +#define CXLDEV_MB_CMD_PAYLOAD_LENGTH_SHIFT 16
> +#define CXLDEV_MB_CMD_PAYLOAD_LENGTH_MASK GENMASK(36, 16)
> #define CXLDEV_MB_STATUS_OFFSET 0x10
> +#define CXLDEV_MB_STATUS_RET_CODE_SHIFT 32
> +#define CXLDEV_MB_STATUS_RET_CODE_MASK GENMASK(47, 32)
> #define CXLDEV_MB_BG_CMD_STATUS_OFFSET 0x18
> +#define CXLDEV_MB_PAYLOAD_OFFSET 0x20
> +
> +/* Memory Device (CXL 2.0 - 8.2.8.5.1.1) */
> +#define CXLMDEV_STATUS_OFFSET 0x0
> +#define CXLMDEV_DEV_FATAL BIT(0)
> +#define CXLMDEV_FW_HALT BIT(1)
> +#define CXLMDEV_STATUS_MEDIA_STATUS_SHIFT 2
> +#define CXLMDEV_STATUS_MEDIA_STATUS_MASK GENMASK(3, 2)
> +#define CXLMDEV_MS_NOT_READY 0
> +#define CXLMDEV_MS_READY 1
> +#define CXLMDEV_MS_ERROR 2
> +#define CXLMDEV_MS_DISABLED 3
> +#define CXLMDEV_READY(status) \
> + (CXL_GET_FIELD(status, CXLMDEV_STATUS_MEDIA_STATUS) == CXLMDEV_MS_READY)
> +#define CXLMDEV_MBOX_IF_READY BIT(4)
> +#define CXLMDEV_RESET_NEEDED_SHIFT 5
> +#define CXLMDEV_RESET_NEEDED_MASK GENMASK(7, 5)
> +#define CXLMDEV_RESET_NEEDED_NOT 0
> +#define CXLMDEV_RESET_NEEDED_COLD 1
> +#define CXLMDEV_RESET_NEEDED_WARM 2
> +#define CXLMDEV_RESET_NEEDED_HOT 3
> +#define CXLMDEV_RESET_NEEDED_CXL 4
> +#define CXLMDEV_RESET_NEEDED(status) \
> + (CXL_GET_FIELD(status, CXLMDEV_RESET_NEEDED) != CXLMDEV_RESET_NEEDED_NOT)
>
> /**
> * struct cxl_mem - A CXL memory device
> @@ -45,6 +76,16 @@ struct cxl_mem {
> struct pci_dev *pdev;
> void __iomem *regs;
>
> + struct {
> + struct range range;
> + } pmem;
> +
> + struct {
> + struct range range;
> + } ram;
> +
> + char firmware_version[0x10];
> +
> /* Cap 0001h - CXL_CAP_CAP_ID_DEVICE_STATUS */
> struct {
> void __iomem *regs;
> @@ -52,6 +93,7 @@ struct cxl_mem {
>
> /* Cap 0002h - CXL_CAP_CAP_ID_PRIMARY_MAILBOX */
> struct {
> + struct mutex mutex; /* Protects device mailbox and firmware */
> void __iomem *regs;
> size_t payload_size;
> } mbox;
> @@ -90,6 +132,7 @@ struct cxl_mem {
>
> cxl_reg(status);
> cxl_reg(mbox);
> +cxl_reg(mem);
>
> #define cxl_payload_regs(cxlm) \
> ((void __iomem *)(cxlm)->mbox.regs + CXLDEV_MB_PAYLOAD_OFFSET)
> diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
> index 8da9f4a861ea..e9ba97bbd7b9 100644
> --- a/drivers/cxl/mem.c
> +++ b/drivers/cxl/mem.c
> @@ -1,5 +1,6 @@
> // SPDX-License-Identifier: GPL-2.0-only
> /* Copyright(c) 2020 Intel Corporation. All rights reserved. */
> +#include <linux/sched/clock.h>
> #include <linux/module.h>
> #include <linux/pci.h>
> #include <linux/io.h>
> @@ -7,6 +8,248 @@
> #include "pci.h"
> #include "cxl.h"
>
> +#define cxl_doorbell_busy(cxlm) \
> + (cxl_read_mbox_reg32(cxlm, CXLDEV_MB_CTRL_OFFSET) & \
> + CXLDEV_MB_CTRL_DOORBELL)
> +
> +#define CXL_MAILBOX_TIMEOUT_US 2000
> +
> +enum opcode {
> + CXL_MBOX_OP_IDENTIFY = 0x4000,
> + CXL_MBOX_OP_MAX = 0x10000
> +};
> +
> +/**
> + * struct mbox_cmd - A command to be submitted to hardware.
> + * @opcode: (input) The command set and command submitted to hardware.
> + * @payload: (input/output) Pointer to the input and output payload.
> + * Payload can be NULL if the caller wants to populate the payload
> + * registers themselves (potentially avoiding a copy).
> + * @size_in: (input) Number of bytes to load from @payload.
> + * @size_out:
> + * - (input) Number of bytes allocated to load into @payload.
I'm not actually seeing where this is used as an input. I'd expect a min(input, output)
at the memcpy but there isn't one there.
> + * - (output) Number of bytes loaded into @payload.
> + * @return_code: (output) Error code returned from hardware.
> + *
> + * This is the primary mechanism used to send commands to the hardware.
> + * All the fields except @payload correspond exactly to the fields described in
> + * Command Register section of the CXL 2.0 spec (8.2.8.4.5). @payload
> + * corresponds to the Command Payload Registers (8.2.8.4.8).
> + */
> +struct mbox_cmd {
> + u16 opcode;
> + void *payload;
> + size_t size_in;
> + size_t size_out;
> + u16 return_code;
> +};
> +
> +static int cxl_mem_wait_for_doorbell(struct cxl_mem *cxlm)
> +{
> + const int timeout = msecs_to_jiffies(CXL_MAILBOX_TIMEOUT_US);
> + const unsigned long start = jiffies;
> + unsigned long end = start;
> +
> + while (cxl_doorbell_busy(cxlm)) {
> + end = jiffies;
> +
> + if (time_after(end, start + timeout)) {
> + /* Check again in case preempted before timeout test */
> + if (!cxl_doorbell_busy(cxlm))
> + break;
> + return -ETIMEDOUT;
> + }
> + cpu_relax();
> + }
> +
> + dev_dbg(&cxlm->pdev->dev, "Doorbell wait took %dms",
> + jiffies_to_msecs(end) - jiffies_to_msecs(start));
> + return 0;
> +}
> +
> +static void cxl_mem_mbox_timeout(struct cxl_mem *cxlm,
> + struct mbox_cmd *mbox_cmd)
> +{
> + dev_warn(&cxlm->pdev->dev, "Mailbox command timed out\n");
> + dev_info(&cxlm->pdev->dev,
> + "\topcode: 0x%04x\n"
> + "\tpayload size: %zub\n",
> + mbox_cmd->opcode, mbox_cmd->size_in);
> + print_hex_dump_debug("Payload ", DUMP_PREFIX_OFFSET, 16, 1,
> + mbox_cmd->payload, mbox_cmd->size_in, true);
> +
> + /* Here's a good place to figure out if a device reset is needed */
> +}
> +
> +/**
> + * cxl_mem_mbox_send_cmd() - Send a mailbox command to a memory device.
> + * @cxlm: The CXL memory device to communicate with.
> + * @mbox_cmd: Command to send to the memory device.
> + *
> + * Context: Any context. Expects mbox_lock to be held.
> + * Return: -ETIMEDOUT if timeout occurred waiting for completion. 0 on success.
> + * Caller should check the return code in @mbox_cmd to make sure it
> + * succeeded.
> + *
> + * This is a generic form of the CXL mailbox send command, thus the only I/O
> + * operations used are cxl_read_mbox_reg(). Memory devices, and perhaps other
> + * types of CXL devices may have further information available upon error
> + * conditions.
> + *
> + * FIXME: As stated above, references to &struct cxl_mem should be changed to a
> + * more generic cxl structure when needed.
> + */
> +static int cxl_mem_mbox_send_cmd(struct cxl_mem *cxlm,
> + struct mbox_cmd *mbox_cmd)
> +{
> + u64 cmd_reg, status_reg;
> + size_t out_len;
> + int rc;
> +
> + lockdep_assert_held(&cxlm->mbox.mutex);
> +
> + /*
> + * Here are the steps from 8.2.8.4 of the CXL 2.0 spec.
> + * 1. Caller reads MB Control Register to verify doorbell is clear
> + * 2. Caller writes Command Register
> + * 3. Caller writes Command Payload Registers if input payload is non-empty
> + * 4. Caller writes MB Control Register to set doorbell
> + * 5. Caller either polls for doorbell to be clear or waits for interrupt if configured
> + * 6. Caller reads MB Status Register to fetch Return code
> + * 7. If command successful, Caller reads Command Register to get Payload Length
> + * 8. If output payload is non-empty, host reads Command Payload Registers
> + */
> +
> + /* #1 */
> + WARN_ON(cxl_doorbell_busy(cxlm));
> +
> + cmd_reg = CXL_SET_FIELD(mbox_cmd->opcode, CXLDEV_MB_CMD_COMMAND_OPCODE);
> + if (mbox_cmd->size_in) {
> + cmd_reg |= CXL_SET_FIELD(mbox_cmd->size_in,
> + CXLDEV_MB_CMD_PAYLOAD_LENGTH);
> + if (mbox_cmd->payload)
> + memcpy_toio(cxl_payload_regs(cxlm), mbox_cmd->payload,
> + mbox_cmd->size_in);
> + }
> +
> + /* #2, #3 */
> + cxl_write_mbox_reg64(cxlm, CXLDEV_MB_CMD_OFFSET, cmd_reg);
> +
> + /* #4 */
> + dev_dbg(&cxlm->pdev->dev, "Sending command\n");
> + cxl_write_mbox_reg32(cxlm, CXLDEV_MB_CTRL_OFFSET,
> + CXLDEV_MB_CTRL_DOORBELL);
> +
> + /* #5 */
> + rc = cxl_mem_wait_for_doorbell(cxlm);
> + if (rc == -ETIMEDOUT) {
> + cxl_mem_mbox_timeout(cxlm, mbox_cmd);
> + return rc;
> + }
> +
> + /* #6 */
> + status_reg = cxl_read_mbox_reg64(cxlm, CXLDEV_MB_STATUS_OFFSET);
> + mbox_cmd->return_code =
> + CXL_GET_FIELD(status_reg, CXLDEV_MB_STATUS_RET_CODE);
> +
> + if (mbox_cmd->return_code != 0) {
> + dev_dbg(&cxlm->pdev->dev, "Mailbox operation had an error\n");
> + return 0;
> + }
> +
> + /* #7 */
> + cmd_reg = cxl_read_mbox_reg64(cxlm, CXLDEV_MB_CMD_OFFSET);
> + out_len = CXL_GET_FIELD(cmd_reg, CXLDEV_MB_CMD_PAYLOAD_LENGTH);
> + mbox_cmd->size_out = out_len;
> +
> + /* #8 */
> + if (out_len && mbox_cmd->payload)
> + memcpy_fromio(mbox_cmd->payload, cxl_payload_regs(cxlm),
> + mbox_cmd->size_out);
> +
> + return 0;
> +}
> +
> +/**
> + * cxl_mem_mbox_get() - Acquire exclusive access to the mailbox.
> + * @cxlm: The memory device to gain access to.
> + *
> + * Context: Any context. Takes the mbox_lock.
> + * Return: 0 if exclusive access was acquired.
> + */
> +static int cxl_mem_mbox_get(struct cxl_mem *cxlm)
> +{
> + u64 md_status;
> + int rc = -EBUSY;
> +
> + mutex_lock_io(&cxlm->mbox.mutex);
> +
> + /*
> + * XXX: There is some amount of ambiguity in the 2.0 version of the spec
> + * around the mailbox interface ready (8.2.8.5.1.1). The purpose of the
> + * bit is to allow firmware running on the device to notify us that it's
> + * ready to receive commands. It is unclear if the bit needs to be read
> + * every time one tries to use the mailbox, ie. the firmware can switch
> + * it on and off as needed. Second, there is no defined timeout for
> + * mailbox ready, like there is for the doorbell interface.
> + *
> + * As such, we make the following assumptions:
> + * 1. The firmware might toggle the Mailbox Interface Ready bit, and so
> + * we check it for every command.
> + * 2. If the doorbell is clear, the firmware should have first set the
> + * Mailbox Interface Ready bit. Therefore, waiting for the doorbell
> + * to be ready is a sufficient amount of time.
> + */
> + rc = cxl_mem_wait_for_doorbell(cxlm);
> + if (rc) {
> + dev_warn(&cxlm->pdev->dev, "Mailbox interface not ready\n");
> + goto out;
> + }
> +
> + md_status = cxl_read_mem_reg64(cxlm, CXLMDEV_STATUS_OFFSET);
> + if (md_status & CXLMDEV_MBOX_IF_READY && CXLMDEV_READY(md_status)) {
> + /*
> + * Hardware shouldn't allow a ready status but also have failure
> + * bits set. Spit out an error, this should be a bug report
> + */
> + if (md_status & CXLMDEV_DEV_FATAL) {
> + dev_err(&cxlm->pdev->dev,
> + "CXL device reporting ready and fatal\n");
> + rc = -EFAULT;
> + goto out;
> + }
> + if (md_status & CXLMDEV_FW_HALT) {
> + dev_err(&cxlm->pdev->dev,
> + "CXL device reporting ready and halted\n");
> + rc = -EFAULT;
> + goto out;
> + }
> + if (CXLMDEV_RESET_NEEDED(md_status)) {
> + dev_err(&cxlm->pdev->dev,
> + "CXL device reporting ready and reset needed\n");
> + rc = -EFAULT;
> + goto out;
> + }
> +
> + return 0;
> + }
> +
> +out:
> + mutex_unlock(&cxlm->mbox.mutex);
> + return rc;
> +}
> +
> +/**
> + * cxl_mem_mbox_put() - Release exclusive access to the mailbox.
> + * @cxlm: The CXL memory device to communicate with.
> + *
> + * Context: Any context. Expects mbox_lock to be held.
> + */
> +static void cxl_mem_mbox_put(struct cxl_mem *cxlm)
> +{
> + mutex_unlock(&cxlm->mbox.mutex);
> +}
> +
> /**
> * cxl_mem_setup_regs() - Setup necessary MMIO.
> * @cxlm: The CXL memory device to communicate with.
> @@ -135,6 +378,8 @@ static struct cxl_mem *cxl_mem_create(struct pci_dev *pdev, u32 reg_lo,
> return NULL;
> }
>
> + mutex_init(&cxlm->mbox.mutex);
> +
> regs = pcim_iomap_table(pdev)[bar];
> cxlm->pdev = pdev;
> cxlm->regs = regs + offset;
> @@ -167,6 +412,69 @@ static int cxl_mem_dvsec(struct pci_dev *pdev, int dvsec)
> return 0;
> }
>
> +/**
> + * cxl_mem_identify() - Send the IDENTIFY command to the device.
> + * @cxlm: The device to identify.
> + *
> + * Return: 0 if identify was executed successfully.
> + *
> + * This will dispatch the identify command to the device and on success populate
> + * structures to be exported to sysfs.
> + */
> +static int cxl_mem_identify(struct cxl_mem *cxlm)
> +{
> + struct cxl_mbox_identify {
> + char fw_revision[0x10];
> + __le64 total_capacity;
> + __le64 volatile_capacity;
> + __le64 persistent_capacity;
> + __le64 partition_align;
> + __le16 info_event_log_size;
> + __le16 warning_event_log_size;
> + __le16 failure_event_log_size;
> + __le16 fatal_event_log_size;
> + __le32 lsa_size;
> + u8 poison_list_max_mer[3];
> + __le16 inject_poison_limit;
> + u8 poison_caps;
> + u8 qos_telemetry_caps;
> + } __packed id;
> + struct mbox_cmd mbox_cmd;
> + int rc;
> +
> + /* Retrieve initial device memory map */
> + rc = cxl_mem_mbox_get(cxlm);
> + if (rc)
> + return rc;
> +
> + mbox_cmd = (struct mbox_cmd){
> + .opcode = CXL_MBOX_OP_IDENTIFY,
> + .payload = &id,
> + .size_in = 0,
> + };
> + rc = cxl_mem_mbox_send_cmd(cxlm, &mbox_cmd);
> + cxl_mem_mbox_put(cxlm);
> + if (rc)
> + return rc;
> +
> + if (mbox_cmd.size_out != sizeof(id))
> + return -ENXIO;
> +
> + /*
> + * TODO: enumerate DPA map, as 'ram' and 'pmem' do not alias.
> + * For now, only the capacity is exported in sysfs
> + */
> + cxlm->ram.range.start = 0;
> + cxlm->ram.range.end = le64_to_cpu(id.volatile_capacity) - 1;
> +
> + cxlm->pmem.range.start = 0;
> + cxlm->pmem.range.end = le64_to_cpu(id.persistent_capacity) - 1;
> +
> + memcpy(cxlm->firmware_version, id.fw_revision, sizeof(id.fw_revision));
> +
> + return rc;
> +}
> +
> static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> {
> struct device *dev = &pdev->dev;
> @@ -222,6 +530,10 @@ static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> if (rc)
> goto err;
>
> + rc = cxl_mem_identify(cxlm);
> + if (rc)
> + goto err;
> +
> pci_set_drvdata(pdev, cxlm);
> return 0;
>
On 21-01-14 17:40:12, Jonathan Cameron wrote:
> On Mon, 11 Jan 2021 14:51:11 -0800
> Ben Widawsky <[email protected]> wrote:
>
> > Provide enough functionality to utilize the mailbox of a memory device.
> > The mailbox is used to interact with the firmware running on the memory
> > device.
> >
> > The CXL specification defines separate capabilities for the mailbox and
> > the memory device. While we can confirm the mailbox is ready, in order
> > to actually interact with the memory device, you must also confirm the
> > device's firmware is ready.
> >
> > Create a function to handle sending a command, optionally with a
> > payload, to the memory device, polling on a result, and then optionally
> > copying out the payload. The algorithm for doing this comes straight out
> > of the CXL 2.0 specification.
> >
> > Primary mailboxes are capable of generating an interrupt when submitting
> > a command in the background. That implementation is saved for a later
> > time.
> >
> > Secondary mailboxes aren't implemented at this time.
> >
> > The flow is proven with one implemented command, "identify". Because the
> > class code has already told the driver this is a memory device and the
> > identify command is mandatory, it's safe to assume for sane devices that
> > everything here will work.
> >
> > Signed-off-by: Ben Widawsky <[email protected]>
> One more thing that had me confused in a later patch (14)
>
> J
> > ---
> > drivers/cxl/cxl.h | 43 +++++++
> > drivers/cxl/mem.c | 312 ++++++++++++++++++++++++++++++++++++++++++++++
> > 2 files changed, 355 insertions(+)
> >
> > diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
> > index a77286d04ce4..ca3fa496e21c 100644
> > --- a/drivers/cxl/cxl.h
> > +++ b/drivers/cxl/cxl.h
> > @@ -32,9 +32,40 @@
> > #define CXLDEV_MB_CAP_PAYLOAD_SIZE_MASK GENMASK(4, 0)
> > #define CXLDEV_MB_CAP_PAYLOAD_SIZE_SHIFT 0
> > #define CXLDEV_MB_CTRL_OFFSET 0x04
> > +#define CXLDEV_MB_CTRL_DOORBELL BIT(0)
> > #define CXLDEV_MB_CMD_OFFSET 0x08
> > +#define CXLDEV_MB_CMD_COMMAND_OPCODE_SHIFT 0
> > +#define CXLDEV_MB_CMD_COMMAND_OPCODE_MASK GENMASK(15, 0)
> > +#define CXLDEV_MB_CMD_PAYLOAD_LENGTH_SHIFT 16
> > +#define CXLDEV_MB_CMD_PAYLOAD_LENGTH_MASK GENMASK(36, 16)
> > #define CXLDEV_MB_STATUS_OFFSET 0x10
> > +#define CXLDEV_MB_STATUS_RET_CODE_SHIFT 32
> > +#define CXLDEV_MB_STATUS_RET_CODE_MASK GENMASK(47, 32)
> > #define CXLDEV_MB_BG_CMD_STATUS_OFFSET 0x18
> > +#define CXLDEV_MB_PAYLOAD_OFFSET 0x20
> > +
> > +/* Memory Device (CXL 2.0 - 8.2.8.5.1.1) */
> > +#define CXLMDEV_STATUS_OFFSET 0x0
> > +#define CXLMDEV_DEV_FATAL BIT(0)
> > +#define CXLMDEV_FW_HALT BIT(1)
> > +#define CXLMDEV_STATUS_MEDIA_STATUS_SHIFT 2
> > +#define CXLMDEV_STATUS_MEDIA_STATUS_MASK GENMASK(3, 2)
> > +#define CXLMDEV_MS_NOT_READY 0
> > +#define CXLMDEV_MS_READY 1
> > +#define CXLMDEV_MS_ERROR 2
> > +#define CXLMDEV_MS_DISABLED 3
> > +#define CXLMDEV_READY(status) \
> > + (CXL_GET_FIELD(status, CXLMDEV_STATUS_MEDIA_STATUS) == CXLMDEV_MS_READY)
> > +#define CXLMDEV_MBOX_IF_READY BIT(4)
> > +#define CXLMDEV_RESET_NEEDED_SHIFT 5
> > +#define CXLMDEV_RESET_NEEDED_MASK GENMASK(7, 5)
> > +#define CXLMDEV_RESET_NEEDED_NOT 0
> > +#define CXLMDEV_RESET_NEEDED_COLD 1
> > +#define CXLMDEV_RESET_NEEDED_WARM 2
> > +#define CXLMDEV_RESET_NEEDED_HOT 3
> > +#define CXLMDEV_RESET_NEEDED_CXL 4
> > +#define CXLMDEV_RESET_NEEDED(status) \
> > + (CXL_GET_FIELD(status, CXLMDEV_RESET_NEEDED) != CXLMDEV_RESET_NEEDED_NOT)
> >
> > /**
> > * struct cxl_mem - A CXL memory device
> > @@ -45,6 +76,16 @@ struct cxl_mem {
> > struct pci_dev *pdev;
> > void __iomem *regs;
> >
> > + struct {
> > + struct range range;
> > + } pmem;
> > +
> > + struct {
> > + struct range range;
> > + } ram;
> > +
> > + char firmware_version[0x10];
> > +
> > /* Cap 0001h - CXL_CAP_CAP_ID_DEVICE_STATUS */
> > struct {
> > void __iomem *regs;
> > @@ -52,6 +93,7 @@ struct cxl_mem {
> >
> > /* Cap 0002h - CXL_CAP_CAP_ID_PRIMARY_MAILBOX */
> > struct {
> > + struct mutex mutex; /* Protects device mailbox and firmware */
> > void __iomem *regs;
> > size_t payload_size;
> > } mbox;
> > @@ -90,6 +132,7 @@ struct cxl_mem {
> >
> > cxl_reg(status);
> > cxl_reg(mbox);
> > +cxl_reg(mem);
> >
> > #define cxl_payload_regs(cxlm) \
> > ((void __iomem *)(cxlm)->mbox.regs + CXLDEV_MB_PAYLOAD_OFFSET)
> > diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
> > index 8da9f4a861ea..e9ba97bbd7b9 100644
> > --- a/drivers/cxl/mem.c
> > +++ b/drivers/cxl/mem.c
> > @@ -1,5 +1,6 @@
> > // SPDX-License-Identifier: GPL-2.0-only
> > /* Copyright(c) 2020 Intel Corporation. All rights reserved. */
> > +#include <linux/sched/clock.h>
> > #include <linux/module.h>
> > #include <linux/pci.h>
> > #include <linux/io.h>
> > @@ -7,6 +8,248 @@
> > #include "pci.h"
> > #include "cxl.h"
> >
> > +#define cxl_doorbell_busy(cxlm) \
> > + (cxl_read_mbox_reg32(cxlm, CXLDEV_MB_CTRL_OFFSET) & \
> > + CXLDEV_MB_CTRL_DOORBELL)
> > +
> > +#define CXL_MAILBOX_TIMEOUT_US 2000
> > +
> > +enum opcode {
> > + CXL_MBOX_OP_IDENTIFY = 0x4000,
> > + CXL_MBOX_OP_MAX = 0x10000
> > +};
> > +
> > +/**
> > + * struct mbox_cmd - A command to be submitted to hardware.
> > + * @opcode: (input) The command set and command submitted to hardware.
> > + * @payload: (input/output) Pointer to the input and output payload.
> > + * Payload can be NULL if the caller wants to populate the payload
> > + * registers themselves (potentially avoiding a copy).
> > + * @size_in: (input) Number of bytes to load from @payload.
> > + * @size_out:
> > + * - (input) Number of bytes allocated to load into @payload.
>
> I'm not actually seeing where this is used as an input. I'd expect a min(input, output)
> at the memcpy but there isn't one there.
>
The functionality changed since v2 and this interface no longer deals with
restricting output length. This was because at some intermediate point I had
entirely removed the copying in/out of the payload registers from this command.
As you see, it came back.
For userspace submitted commands the lengths are all validated up front before
we get here.
For commands submitted directly from the kernel, I've gone back and forth about
how useful it is. If you feel strongly that it is useful, I can add it back. It
sounds like you want me to convert everything back to an intermediate buffer,
which I will respond in that patch, so I'd need to rework it anyway if that's
the path we end up taking.
> > + * - (output) Number of bytes loaded into @payload.
> > + * @return_code: (output) Error code returned from hardware.
> > + *
> > + * This is the primary mechanism used to send commands to the hardware.
> > + * All the fields except @payload correspond exactly to the fields described in
> > + * Command Register section of the CXL 2.0 spec (8.2.8.4.5). @payload
> > + * corresponds to the Command Payload Registers (8.2.8.4.8).
> > + */
> > +struct mbox_cmd {
> > + u16 opcode;
> > + void *payload;
> > + size_t size_in;
> > + size_t size_out;
> > + u16 return_code;
> > +};
> > +
> > +static int cxl_mem_wait_for_doorbell(struct cxl_mem *cxlm)
> > +{
> > + const int timeout = msecs_to_jiffies(CXL_MAILBOX_TIMEOUT_US);
> > + const unsigned long start = jiffies;
> > + unsigned long end = start;
> > +
> > + while (cxl_doorbell_busy(cxlm)) {
> > + end = jiffies;
> > +
> > + if (time_after(end, start + timeout)) {
> > + /* Check again in case preempted before timeout test */
> > + if (!cxl_doorbell_busy(cxlm))
> > + break;
> > + return -ETIMEDOUT;
> > + }
> > + cpu_relax();
> > + }
> > +
> > + dev_dbg(&cxlm->pdev->dev, "Doorbell wait took %dms",
> > + jiffies_to_msecs(end) - jiffies_to_msecs(start));
> > + return 0;
> > +}
> > +
> > +static void cxl_mem_mbox_timeout(struct cxl_mem *cxlm,
> > + struct mbox_cmd *mbox_cmd)
> > +{
> > + dev_warn(&cxlm->pdev->dev, "Mailbox command timed out\n");
> > + dev_info(&cxlm->pdev->dev,
> > + "\topcode: 0x%04x\n"
> > + "\tpayload size: %zub\n",
> > + mbox_cmd->opcode, mbox_cmd->size_in);
> > + print_hex_dump_debug("Payload ", DUMP_PREFIX_OFFSET, 16, 1,
> > + mbox_cmd->payload, mbox_cmd->size_in, true);
> > +
> > + /* Here's a good place to figure out if a device reset is needed */
> > +}
> > +
> > +/**
> > + * cxl_mem_mbox_send_cmd() - Send a mailbox command to a memory device.
> > + * @cxlm: The CXL memory device to communicate with.
> > + * @mbox_cmd: Command to send to the memory device.
> > + *
> > + * Context: Any context. Expects mbox_lock to be held.
> > + * Return: -ETIMEDOUT if timeout occurred waiting for completion. 0 on success.
> > + * Caller should check the return code in @mbox_cmd to make sure it
> > + * succeeded.
> > + *
> > + * This is a generic form of the CXL mailbox send command, thus the only I/O
> > + * operations used are cxl_read_mbox_reg(). Memory devices, and perhaps other
> > + * types of CXL devices may have further information available upon error
> > + * conditions.
> > + *
> > + * FIXME: As stated above, references to &struct cxl_mem should be changed to a
> > + * more generic cxl structure when needed.
> > + */
> > +static int cxl_mem_mbox_send_cmd(struct cxl_mem *cxlm,
> > + struct mbox_cmd *mbox_cmd)
> > +{
> > + u64 cmd_reg, status_reg;
> > + size_t out_len;
> > + int rc;
> > +
> > + lockdep_assert_held(&cxlm->mbox.mutex);
> > +
> > + /*
> > + * Here are the steps from 8.2.8.4 of the CXL 2.0 spec.
> > + * 1. Caller reads MB Control Register to verify doorbell is clear
> > + * 2. Caller writes Command Register
> > + * 3. Caller writes Command Payload Registers if input payload is non-empty
> > + * 4. Caller writes MB Control Register to set doorbell
> > + * 5. Caller either polls for doorbell to be clear or waits for interrupt if configured
> > + * 6. Caller reads MB Status Register to fetch Return code
> > + * 7. If command successful, Caller reads Command Register to get Payload Length
> > + * 8. If output payload is non-empty, host reads Command Payload Registers
> > + */
> > +
> > + /* #1 */
> > + WARN_ON(cxl_doorbell_busy(cxlm));
> > +
> > + cmd_reg = CXL_SET_FIELD(mbox_cmd->opcode, CXLDEV_MB_CMD_COMMAND_OPCODE);
> > + if (mbox_cmd->size_in) {
> > + cmd_reg |= CXL_SET_FIELD(mbox_cmd->size_in,
> > + CXLDEV_MB_CMD_PAYLOAD_LENGTH);
> > + if (mbox_cmd->payload)
> > + memcpy_toio(cxl_payload_regs(cxlm), mbox_cmd->payload,
> > + mbox_cmd->size_in);
> > + }
> > +
> > + /* #2, #3 */
> > + cxl_write_mbox_reg64(cxlm, CXLDEV_MB_CMD_OFFSET, cmd_reg);
> > +
> > + /* #4 */
> > + dev_dbg(&cxlm->pdev->dev, "Sending command\n");
> > + cxl_write_mbox_reg32(cxlm, CXLDEV_MB_CTRL_OFFSET,
> > + CXLDEV_MB_CTRL_DOORBELL);
> > +
> > + /* #5 */
> > + rc = cxl_mem_wait_for_doorbell(cxlm);
> > + if (rc == -ETIMEDOUT) {
> > + cxl_mem_mbox_timeout(cxlm, mbox_cmd);
> > + return rc;
> > + }
> > +
> > + /* #6 */
> > + status_reg = cxl_read_mbox_reg64(cxlm, CXLDEV_MB_STATUS_OFFSET);
> > + mbox_cmd->return_code =
> > + CXL_GET_FIELD(status_reg, CXLDEV_MB_STATUS_RET_CODE);
> > +
> > + if (mbox_cmd->return_code != 0) {
> > + dev_dbg(&cxlm->pdev->dev, "Mailbox operation had an error\n");
> > + return 0;
> > + }
> > +
> > + /* #7 */
> > + cmd_reg = cxl_read_mbox_reg64(cxlm, CXLDEV_MB_CMD_OFFSET);
> > + out_len = CXL_GET_FIELD(cmd_reg, CXLDEV_MB_CMD_PAYLOAD_LENGTH);
> > + mbox_cmd->size_out = out_len;
> > +
> > + /* #8 */
> > + if (out_len && mbox_cmd->payload)
> > + memcpy_fromio(mbox_cmd->payload, cxl_payload_regs(cxlm),
> > + mbox_cmd->size_out);
> > +
> > + return 0;
> > +}
> > +
> > +/**
> > + * cxl_mem_mbox_get() - Acquire exclusive access to the mailbox.
> > + * @cxlm: The memory device to gain access to.
> > + *
> > + * Context: Any context. Takes the mbox_lock.
> > + * Return: 0 if exclusive access was acquired.
> > + */
> > +static int cxl_mem_mbox_get(struct cxl_mem *cxlm)
> > +{
> > + u64 md_status;
> > + int rc = -EBUSY;
> > +
> > + mutex_lock_io(&cxlm->mbox.mutex);
> > +
> > + /*
> > + * XXX: There is some amount of ambiguity in the 2.0 version of the spec
> > + * around the mailbox interface ready (8.2.8.5.1.1). The purpose of the
> > + * bit is to allow firmware running on the device to notify us that it's
> > + * ready to receive commands. It is unclear if the bit needs to be read
> > + * every time one tries to use the mailbox, ie. the firmware can switch
> > + * it on and off as needed. Second, there is no defined timeout for
> > + * mailbox ready, like there is for the doorbell interface.
> > + *
> > + * As such, we make the following assumptions:
> > + * 1. The firmware might toggle the Mailbox Interface Ready bit, and so
> > + * we check it for every command.
> > + * 2. If the doorbell is clear, the firmware should have first set the
> > + * Mailbox Interface Ready bit. Therefore, waiting for the doorbell
> > + * to be ready is a sufficient amount of time.
> > + */
> > + rc = cxl_mem_wait_for_doorbell(cxlm);
> > + if (rc) {
> > + dev_warn(&cxlm->pdev->dev, "Mailbox interface not ready\n");
> > + goto out;
> > + }
> > +
> > + md_status = cxl_read_mem_reg64(cxlm, CXLMDEV_STATUS_OFFSET);
> > + if (md_status & CXLMDEV_MBOX_IF_READY && CXLMDEV_READY(md_status)) {
> > + /*
> > + * Hardware shouldn't allow a ready status but also have failure
> > + * bits set. Spit out an error, this should be a bug report
> > + */
> > + if (md_status & CXLMDEV_DEV_FATAL) {
> > + dev_err(&cxlm->pdev->dev,
> > + "CXL device reporting ready and fatal\n");
> > + rc = -EFAULT;
> > + goto out;
> > + }
> > + if (md_status & CXLMDEV_FW_HALT) {
> > + dev_err(&cxlm->pdev->dev,
> > + "CXL device reporting ready and halted\n");
> > + rc = -EFAULT;
> > + goto out;
> > + }
> > + if (CXLMDEV_RESET_NEEDED(md_status)) {
> > + dev_err(&cxlm->pdev->dev,
> > + "CXL device reporting ready and reset needed\n");
> > + rc = -EFAULT;
> > + goto out;
> > + }
> > +
> > + return 0;
> > + }
> > +
> > +out:
> > + mutex_unlock(&cxlm->mbox.mutex);
> > + return rc;
> > +}
> > +
> > +/**
> > + * cxl_mem_mbox_put() - Release exclusive access to the mailbox.
> > + * @cxlm: The CXL memory device to communicate with.
> > + *
> > + * Context: Any context. Expects mbox_lock to be held.
> > + */
> > +static void cxl_mem_mbox_put(struct cxl_mem *cxlm)
> > +{
> > + mutex_unlock(&cxlm->mbox.mutex);
> > +}
> > +
> > /**
> > * cxl_mem_setup_regs() - Setup necessary MMIO.
> > * @cxlm: The CXL memory device to communicate with.
> > @@ -135,6 +378,8 @@ static struct cxl_mem *cxl_mem_create(struct pci_dev *pdev, u32 reg_lo,
> > return NULL;
> > }
> >
> > + mutex_init(&cxlm->mbox.mutex);
> > +
> > regs = pcim_iomap_table(pdev)[bar];
> > cxlm->pdev = pdev;
> > cxlm->regs = regs + offset;
> > @@ -167,6 +412,69 @@ static int cxl_mem_dvsec(struct pci_dev *pdev, int dvsec)
> > return 0;
> > }
> >
> > +/**
> > + * cxl_mem_identify() - Send the IDENTIFY command to the device.
> > + * @cxlm: The device to identify.
> > + *
> > + * Return: 0 if identify was executed successfully.
> > + *
> > + * This will dispatch the identify command to the device and on success populate
> > + * structures to be exported to sysfs.
> > + */
> > +static int cxl_mem_identify(struct cxl_mem *cxlm)
> > +{
> > + struct cxl_mbox_identify {
> > + char fw_revision[0x10];
> > + __le64 total_capacity;
> > + __le64 volatile_capacity;
> > + __le64 persistent_capacity;
> > + __le64 partition_align;
> > + __le16 info_event_log_size;
> > + __le16 warning_event_log_size;
> > + __le16 failure_event_log_size;
> > + __le16 fatal_event_log_size;
> > + __le32 lsa_size;
> > + u8 poison_list_max_mer[3];
> > + __le16 inject_poison_limit;
> > + u8 poison_caps;
> > + u8 qos_telemetry_caps;
> > + } __packed id;
> > + struct mbox_cmd mbox_cmd;
> > + int rc;
> > +
> > + /* Retrieve initial device memory map */
> > + rc = cxl_mem_mbox_get(cxlm);
> > + if (rc)
> > + return rc;
> > +
> > + mbox_cmd = (struct mbox_cmd){
> > + .opcode = CXL_MBOX_OP_IDENTIFY,
> > + .payload = &id,
> > + .size_in = 0,
> > + };
> > + rc = cxl_mem_mbox_send_cmd(cxlm, &mbox_cmd);
> > + cxl_mem_mbox_put(cxlm);
> > + if (rc)
> > + return rc;
> > +
> > + if (mbox_cmd.size_out != sizeof(id))
> > + return -ENXIO;
> > +
> > + /*
> > + * TODO: enumerate DPA map, as 'ram' and 'pmem' do not alias.
> > + * For now, only the capacity is exported in sysfs
> > + */
> > + cxlm->ram.range.start = 0;
> > + cxlm->ram.range.end = le64_to_cpu(id.volatile_capacity) - 1;
> > +
> > + cxlm->pmem.range.start = 0;
> > + cxlm->pmem.range.end = le64_to_cpu(id.persistent_capacity) - 1;
> > +
> > + memcpy(cxlm->firmware_version, id.fw_revision, sizeof(id.fw_revision));
> > +
> > + return rc;
> > +}
> > +
> > static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> > {
> > struct device *dev = &pdev->dev;
> > @@ -222,6 +530,10 @@ static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> > if (rc)
> > goto err;
> >
> > + rc = cxl_mem_identify(cxlm);
> > + if (rc)
> > + goto err;
> > +
> > pci_set_drvdata(pdev, cxlm);
> > return 0;
> >
>
On Mon, 11 Jan 2021 14:51:19 -0800
Ben Widawsky <[email protected]> wrote:
> The Command Effects Log (CEL) is specified in the CXL 2.0 specification.
> The CEL is one of two types of logs, the other being vendor specific.
> They are distinguished in hardware/spec via UUID. The CEL is immediately
> useful for 2 things:
> 1. Determine which optional commands are supported by the CXL device.
> 2. Enumerate any vendor specific commands
>
> The CEL can be used by the driver to determine which commands are
> available in the hardware (though it isn't, yet). That set of commands
> might itself be a subset of commands which are available to be used via
> CXL_MEM_SEND_COMMAND IOCTL.
>
> Prior to this, all commands that the driver exposed were explicitly
> enabled. After this, only those commands that are found in the CEL are
> enabled.
>
> Signed-off-by: Ben Widawsky <[email protected]>
This patch made me wonder if the model for the command in quite right.
I think it would end up simpler with a pair of payload pointers for send
and receive (that can be equal when it makes sense).
A few other things inline.
Jonathan
> ---
> drivers/cxl/mem.c | 178 +++++++++++++++++++++++++++++++++--
> include/uapi/linux/cxl_mem.h | 1 +
> 2 files changed, 172 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
> index 20b26fa2c466..6dfc8ff0aefb 100644
> --- a/drivers/cxl/mem.c
> +++ b/drivers/cxl/mem.c
> @@ -44,8 +44,10 @@
> enum opcode {
> CXL_MBOX_OP_INVALID = 0x0000,
> #define CXL_MBOX_OP_RAW CXL_MBOX_OP_INVALID
> - CXL_MBOX_OP_IDENTIFY = 0x4000,
Fix the space vs tabs in earlier patch so this diff doesn't end up looking
weird and make people wonder what changed about that line.
> - CXL_MBOX_OP_MAX = 0x10000
> + CXL_MBOX_OP_GET_SUPPORTED_LOGS = 0x0400,
> + CXL_MBOX_OP_GET_LOG = 0x0401,
> + CXL_MBOX_OP_IDENTIFY = 0x4000,
> + CXL_MBOX_OP_MAX = 0x10000
> };
>
> /**
> @@ -104,6 +106,16 @@ static struct {
> .opcode = CXL_MBOX_OP_##_id, \
> }
>
> +enum {
> + CEL_UUID,
> + DEBUG_UUID
> +};
> +
> +static const uuid_t log_uuid[] = {
> + UUID_INIT(0xda9c0b5, 0xbf41, 0x4b78, 0x8f, 0x79, 0x96, 0xb1, 0x62, 0x3b, 0x3f, 0x17),
Whilst it will make some long lines, probably good to make the association of these with the enum
explicit.
[CEL_UUID] = UUID_INIT(...),
[DEBUG_UUID] = UUID_INIT(...),
Trailing comma for both this and the enum probably wise as I'm sure we'll get more of these
at some point.
> + UUID_INIT(0xe1819d9, 0x11a9, 0x400c, 0x81, 0x1f, 0xd6, 0x07, 0x19, 0x40, 0x3d, 0x86)
> +};
> +
> /**
> * struct cxl_mem_command - Driver representation of a memory device command
> * @info: Command information as it exists for the UAPI
> @@ -144,6 +156,7 @@ static struct cxl_mem_command mem_commands[] = {
> CXL_CMD(INVALID, KERNEL, 0, 0, HIDDEN),
> CXL_CMD(IDENTIFY, NONE, 0, 0x43, MANDATORY),
> CXL_CMD(RAW, NONE, ~0, ~0, PSEUDO),
> + CXL_CMD(GET_SUPPORTED_LOGS, NONE, 0, ~0, MANDATORY),
> };
>
> #define cxl_for_each_cmd(cmd) \
> @@ -1036,6 +1049,103 @@ static int cxl_mem_add_memdev(struct cxl_mem *cxlm)
> return rc;
> }
>
> +struct cxl_mbox_get_supported_logs {
> + __le16 entries;
> + u8 rsvd[6];
> + struct gsl_entry {
> + uuid_t uuid;
> + __le32 size;
> + } __packed entry[2];
> +} __packed;
blank line here. Help my eyes parse the code if nothing else.
> +struct cxl_mbox_get_log {
> + uuid_t uuid;
> + __le32 offset;
> + __le32 length;
> +} __packed;
> +
> +static int cxl_xfer_log(struct cxl_mem *cxlm, uuid_t *uuid, u32 size,
> + u8 *out)
> +{
> + u32 remaining = size;
> + u32 offset = 0;
> +
> + while (remaining) {
> + u32 xfer_size = min_t(u32, remaining, cxlm->mbox.payload_size);
> + struct mbox_cmd mbox_cmd;
> + int rc;
> + struct cxl_mbox_get_log log = {
> + .uuid = *uuid,
> + .offset = cpu_to_le32(offset),
> + .length = cpu_to_le32(xfer_size)
> + };
> +
> + memcpy_toio(cxl_payload_regs(cxlm), &log, sizeof(log));
> + mbox_cmd = (struct mbox_cmd) {
> + .opcode = CXL_MBOX_OP_GET_LOG,
> + .payload = NULL,
> + .size_in = sizeof(log),
> + };
The fact that you end up bypassing the payload transfer stuff in mbox_cmd
rather suggests it's not a particularly good model. + it keeps confusing
me.
While the hardware uses a single region for the payload, there is nothing
saying the code has to work that way. Why not have separate payload_in and
payload_out pointers? Occasionally you might set them to the same buffer, but
elsewhere you could avoid the direct memcpy()s you are doing around the
send_cmd().
> +
> + rc = cxl_mem_mbox_send_cmd(cxlm, &mbox_cmd);
> + if (rc)
> + return rc;
> +
> + WARN_ON(mbox_cmd.size_out != xfer_size);
> +
> + memcpy_fromio(out, cxl_payload_regs(cxlm), mbox_cmd.size_out);
> + out += xfer_size;
> + remaining -= xfer_size;
> + offset += xfer_size;
> + }
> +
> + return 0;
> +}
> +
> +static void cxl_enable_cmd(struct cxl_mem *cxlm,
> + const struct cxl_mem_command *cmd)
> +{
> + if (test_and_set_bit(cxl_cmd_index(cmd), cxlm->enabled_cmds))
> + dev_warn(&cxlm->pdev->dev, "Command enabled twice\n");
> +
> + dev_info(&cxlm->pdev->dev, "%s enabled",
> + command_names[cxl_cmd_index(cmd)].name);
Is there anything stopping this being in the previous patch?
I think it would make more sense there if it can be.
> +}
> +
> +/**
> + * cxl_walk_cel() - Walk through the Command Effects Log.
> + * @cxlm: Device.
> + * @size: Length of the Command Effects Log.
> + * @cel: CEL
> + *
> + * Iterate over each entry in the CEL and determine if the driver supports the
> + * command. If so, the command is enabled for the device and can be used later.
> + */
> +static void cxl_walk_cel(struct cxl_mem *cxlm, size_t size, u8 *cel)
> +{
> + struct cel_entry {
> + __le16 opcode;
> + __le16 effect;
> + } *cel_entry;
> + const int cel_entries = size / sizeof(*cel_entry);
> + int i;
> +
> + cel_entry = (struct cel_entry *)cel;
> +
> + for (i = 0; i < cel_entries; i++) {
> + const struct cel_entry *ce = &cel_entry[i];
> + const struct cxl_mem_command *cmd =
> + cxl_mem_find_command(le16_to_cpu(ce->opcode));
> +
> + if (!cmd) {
> + dev_dbg(&cxlm->pdev->dev, "Unsupported opcode 0x%04x",
> + le16_to_cpu(ce->opcode));
> + continue;
> + }
> +
> + cxl_enable_cmd(cxlm, cmd);
> + }
> +}
> +
> /**
> * cxl_mem_enumerate_cmds() - Enumerate commands for a device.
> * @cxlm: The device.
> @@ -1048,17 +1158,71 @@ static int cxl_mem_add_memdev(struct cxl_mem *cxlm)
> */
> static int cxl_mem_enumerate_cmds(struct cxl_mem *cxlm)
> {
> - struct cxl_mem_command *c;
> + struct cxl_mbox_get_supported_logs gsl;
> + const struct cxl_mem_command *c;
Make this const in previous patch to reduce the diff a tiny bit.
> + struct mbox_cmd mbox_cmd;
> + int i, rc;
>
> - /* All commands are considered enabled for now (except INVALID). */
> + /* Pseudo commands are always enabled */
> cxl_for_each_cmd(c) {
> - if (c->flags & CXL_CMD_INTERNAL_FLAG_HIDDEN)
> + if (c->flags & CXL_CMD_INTERNAL_FLAG_PSEUDO)
> + cxl_enable_cmd(cxlm, c);
> + }
> +
> + rc = cxl_mem_mbox_get(cxlm, false);
> + if (rc)
> + return rc;
> +
> + mbox_cmd = (struct mbox_cmd){
> + .opcode = CXL_MBOX_OP_GET_SUPPORTED_LOGS,
> + .payload = &gsl,
> + .size_in = 0,
I raised a question on an earlier patch on why we aren't setting
size_out. Your docs say you should and it would make sense to
me to always do so, but right now it's unused.
> + };
> + rc = cxl_mem_mbox_send_cmd(cxlm, &mbox_cmd);
> + if (rc)
> + goto out;
> +
> + for (i = 0; i < le16_to_cpu(gsl.entries); i++) {
> + u32 size = le32_to_cpu(gsl.entry[i].size);
> + uuid_t uuid = gsl.entry[i].uuid;
> + u8 *log;
> +
> + dev_dbg(&cxlm->pdev->dev, "Found LOG type %pU of size %d",
> + &uuid, size);
> +
> + if (!uuid_equal(&uuid, &log_uuid[CEL_UUID]))
> continue;
>
> - set_bit(cxl_cmd_index(c), cxlm->enabled_cmds);
> + /*
> + * It's a hardware bug if the log size is less than the input
> + * payload size because there are many mandatory commands.
> + */
> + if (sizeof(struct cxl_mbox_get_log) > size) {
> + dev_err(&cxlm->pdev->dev,
> + "CEL log size reported was too small (%d)",
> + size);
> + rc = -ENOMEM;
> + goto out;
> + }
> +
> + log = kvmalloc(size, GFP_KERNEL);
> + if (!log) {
> + rc = -ENOMEM;
> + goto out;
> + }
> +
> + rc = cxl_xfer_log(cxlm, &uuid, size, log);
> + if (rc)
> + goto out;
> +
> + cxl_walk_cel(cxlm, size, log);
> +
> + kvfree(log);
> }
>
> - return 0;
> +out:
> + cxl_mem_mbox_put(cxlm);
> + return rc;
> }
>
> /**
> diff --git a/include/uapi/linux/cxl_mem.h b/include/uapi/linux/cxl_mem.h
> index 50acd6cc14d4..b504412d1db7 100644
> --- a/include/uapi/linux/cxl_mem.h
> +++ b/include/uapi/linux/cxl_mem.h
> @@ -33,6 +33,7 @@ extern "C" {
> C(INVALID, "Invalid Command"), \
> C(IDENTIFY, "Identify Command"), \
> C(RAW, "Raw device command"), \
> + C(GET_SUPPORTED_LOGS, "Get Supported Logs"), \
> C(MAX, "Last command")
> #undef C
> #define C(a, b) CXL_MEM_COMMAND_ID_##a
On Mon, 11 Jan 2021 14:51:20 -0800
Ben Widawsky <[email protected]> wrote:
> The Get Log command returns the actual log entries that are advertised
> via the Get Supported Logs command (0400h). CXL device logs are selected
> by UUID which is part of the CXL spec. Because the driver tries to
> sanitize what is sent to hardware, there becomes a need to restrict the
> types of logs which can be accessed by userspace. For example, the
> vendor specific log might only be consumable by proprietary, or offline
> applications, and therefore a good candidate for userspace.
>
> The current driver infrastructure does allow basic validation for all
> commands, but doesn't inspect any of the payload data. Along with Get
> Log support comes new infrastructure to add a hook for payload
> validation. This infrastructure is used to filter out the CEL UUID,
> which the userspace driver doesn't have business knowing, and taints on
> invalid UUIDs being sent to hardware.
>
> Signed-off-by: Ben Widawsky <[email protected]>
Just a minor question for this one.
Thanks, J
... \
> @@ -515,6 +529,15 @@ static int handle_mailbox_cmd_from_user(struct cxl_memdev *cxlmd,
> int rc;
>
> if (cmd->info.size_in) {
> + if (cmd->validate_payload) {
> + rc = cmd->validate_payload(u64_to_user_ptr(in_payload),
> + cmd->info.size_in);
Is it worth moving this out of the region in which we hold the mbox?
(after fixing the bug that I think means we don't actually hold it at this point)
Perhaps not, but it does feel odd to do validation under the lock.
> + if (rc) {
> + cxl_mem_mbox_put(cxlmd->cxlm);
> + return -EFAULT;
> + }
> + }
> +
> /*
> * Directly copy the userspace payload into the hardware. UAPI
> * states that the buffer must already be little endian.
On 21-01-14 18:02:11, Jonathan Cameron wrote:
> On Mon, 11 Jan 2021 14:51:19 -0800
> Ben Widawsky <[email protected]> wrote:
>
> > The Command Effects Log (CEL) is specified in the CXL 2.0 specification.
> > The CEL is one of two types of logs, the other being vendor specific.
> > They are distinguished in hardware/spec via UUID. The CEL is immediately
> > useful for 2 things:
> > 1. Determine which optional commands are supported by the CXL device.
> > 2. Enumerate any vendor specific commands
> >
> > The CEL can be used by the driver to determine which commands are
> > available in the hardware (though it isn't, yet). That set of commands
> > might itself be a subset of commands which are available to be used via
> > CXL_MEM_SEND_COMMAND IOCTL.
> >
> > Prior to this, all commands that the driver exposed were explicitly
> > enabled. After this, only those commands that are found in the CEL are
> > enabled.
> >
> > Signed-off-by: Ben Widawsky <[email protected]>
>
> This patch made me wonder if the model for the command in quite right.
> I think it would end up simpler with a pair of payload pointers for send
> and receive (that can be equal when it makes sense).
>
> A few other things inline.
>
> Jonathan
I'll address the others separately, but could you elaborate on this? I'm not
sure I follow your meaning.
[snip]
On Thu, 14 Jan 2021 09:50:44 -0800
Ben Widawsky <[email protected]> wrote:
> On 21-01-14 17:40:12, Jonathan Cameron wrote:
> > On Mon, 11 Jan 2021 14:51:11 -0800
> > Ben Widawsky <[email protected]> wrote:
> >
> > > Provide enough functionality to utilize the mailbox of a memory device.
> > > The mailbox is used to interact with the firmware running on the memory
> > > device.
> > >
> > > The CXL specification defines separate capabilities for the mailbox and
> > > the memory device. While we can confirm the mailbox is ready, in order
> > > to actually interact with the memory device, you must also confirm the
> > > device's firmware is ready.
> > >
> > > Create a function to handle sending a command, optionally with a
> > > payload, to the memory device, polling on a result, and then optionally
> > > copying out the payload. The algorithm for doing this comes straight out
> > > of the CXL 2.0 specification.
> > >
> > > Primary mailboxes are capable of generating an interrupt when submitting
> > > a command in the background. That implementation is saved for a later
> > > time.
> > >
> > > Secondary mailboxes aren't implemented at this time.
> > >
> > > The flow is proven with one implemented command, "identify". Because the
> > > class code has already told the driver this is a memory device and the
> > > identify command is mandatory, it's safe to assume for sane devices that
> > > everything here will work.
> > >
> > > Signed-off-by: Ben Widawsky <[email protected]>
> > One more thing that had me confused in a later patch (14)
> >
> > J
> > > ---
> > > drivers/cxl/cxl.h | 43 +++++++
> > > drivers/cxl/mem.c | 312 ++++++++++++++++++++++++++++++++++++++++++++++
> > > 2 files changed, 355 insertions(+)
> > >
> > > diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
> > > index a77286d04ce4..ca3fa496e21c 100644
> > > --- a/drivers/cxl/cxl.h
> > > +++ b/drivers/cxl/cxl.h
> > > @@ -32,9 +32,40 @@
> > > #define CXLDEV_MB_CAP_PAYLOAD_SIZE_MASK GENMASK(4, 0)
> > > #define CXLDEV_MB_CAP_PAYLOAD_SIZE_SHIFT 0
> > > #define CXLDEV_MB_CTRL_OFFSET 0x04
> > > +#define CXLDEV_MB_CTRL_DOORBELL BIT(0)
> > > #define CXLDEV_MB_CMD_OFFSET 0x08
> > > +#define CXLDEV_MB_CMD_COMMAND_OPCODE_SHIFT 0
> > > +#define CXLDEV_MB_CMD_COMMAND_OPCODE_MASK GENMASK(15, 0)
> > > +#define CXLDEV_MB_CMD_PAYLOAD_LENGTH_SHIFT 16
> > > +#define CXLDEV_MB_CMD_PAYLOAD_LENGTH_MASK GENMASK(36, 16)
> > > #define CXLDEV_MB_STATUS_OFFSET 0x10
> > > +#define CXLDEV_MB_STATUS_RET_CODE_SHIFT 32
> > > +#define CXLDEV_MB_STATUS_RET_CODE_MASK GENMASK(47, 32)
> > > #define CXLDEV_MB_BG_CMD_STATUS_OFFSET 0x18
> > > +#define CXLDEV_MB_PAYLOAD_OFFSET 0x20
> > > +
> > > +/* Memory Device (CXL 2.0 - 8.2.8.5.1.1) */
> > > +#define CXLMDEV_STATUS_OFFSET 0x0
> > > +#define CXLMDEV_DEV_FATAL BIT(0)
> > > +#define CXLMDEV_FW_HALT BIT(1)
> > > +#define CXLMDEV_STATUS_MEDIA_STATUS_SHIFT 2
> > > +#define CXLMDEV_STATUS_MEDIA_STATUS_MASK GENMASK(3, 2)
> > > +#define CXLMDEV_MS_NOT_READY 0
> > > +#define CXLMDEV_MS_READY 1
> > > +#define CXLMDEV_MS_ERROR 2
> > > +#define CXLMDEV_MS_DISABLED 3
> > > +#define CXLMDEV_READY(status) \
> > > + (CXL_GET_FIELD(status, CXLMDEV_STATUS_MEDIA_STATUS) == CXLMDEV_MS_READY)
> > > +#define CXLMDEV_MBOX_IF_READY BIT(4)
> > > +#define CXLMDEV_RESET_NEEDED_SHIFT 5
> > > +#define CXLMDEV_RESET_NEEDED_MASK GENMASK(7, 5)
> > > +#define CXLMDEV_RESET_NEEDED_NOT 0
> > > +#define CXLMDEV_RESET_NEEDED_COLD 1
> > > +#define CXLMDEV_RESET_NEEDED_WARM 2
> > > +#define CXLMDEV_RESET_NEEDED_HOT 3
> > > +#define CXLMDEV_RESET_NEEDED_CXL 4
> > > +#define CXLMDEV_RESET_NEEDED(status) \
> > > + (CXL_GET_FIELD(status, CXLMDEV_RESET_NEEDED) != CXLMDEV_RESET_NEEDED_NOT)
> > >
> > > /**
> > > * struct cxl_mem - A CXL memory device
> > > @@ -45,6 +76,16 @@ struct cxl_mem {
> > > struct pci_dev *pdev;
> > > void __iomem *regs;
> > >
> > > + struct {
> > > + struct range range;
> > > + } pmem;
> > > +
> > > + struct {
> > > + struct range range;
> > > + } ram;
> > > +
> > > + char firmware_version[0x10];
> > > +
> > > /* Cap 0001h - CXL_CAP_CAP_ID_DEVICE_STATUS */
> > > struct {
> > > void __iomem *regs;
> > > @@ -52,6 +93,7 @@ struct cxl_mem {
> > >
> > > /* Cap 0002h - CXL_CAP_CAP_ID_PRIMARY_MAILBOX */
> > > struct {
> > > + struct mutex mutex; /* Protects device mailbox and firmware */
> > > void __iomem *regs;
> > > size_t payload_size;
> > > } mbox;
> > > @@ -90,6 +132,7 @@ struct cxl_mem {
> > >
> > > cxl_reg(status);
> > > cxl_reg(mbox);
> > > +cxl_reg(mem);
> > >
> > > #define cxl_payload_regs(cxlm) \
> > > ((void __iomem *)(cxlm)->mbox.regs + CXLDEV_MB_PAYLOAD_OFFSET)
> > > diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
> > > index 8da9f4a861ea..e9ba97bbd7b9 100644
> > > --- a/drivers/cxl/mem.c
> > > +++ b/drivers/cxl/mem.c
> > > @@ -1,5 +1,6 @@
> > > // SPDX-License-Identifier: GPL-2.0-only
> > > /* Copyright(c) 2020 Intel Corporation. All rights reserved. */
> > > +#include <linux/sched/clock.h>
> > > #include <linux/module.h>
> > > #include <linux/pci.h>
> > > #include <linux/io.h>
> > > @@ -7,6 +8,248 @@
> > > #include "pci.h"
> > > #include "cxl.h"
> > >
> > > +#define cxl_doorbell_busy(cxlm) \
> > > + (cxl_read_mbox_reg32(cxlm, CXLDEV_MB_CTRL_OFFSET) & \
> > > + CXLDEV_MB_CTRL_DOORBELL)
> > > +
> > > +#define CXL_MAILBOX_TIMEOUT_US 2000
> > > +
> > > +enum opcode {
> > > + CXL_MBOX_OP_IDENTIFY = 0x4000,
> > > + CXL_MBOX_OP_MAX = 0x10000
> > > +};
> > > +
> > > +/**
> > > + * struct mbox_cmd - A command to be submitted to hardware.
> > > + * @opcode: (input) The command set and command submitted to hardware.
> > > + * @payload: (input/output) Pointer to the input and output payload.
> > > + * Payload can be NULL if the caller wants to populate the payload
> > > + * registers themselves (potentially avoiding a copy).
> > > + * @size_in: (input) Number of bytes to load from @payload.
> > > + * @size_out:
> > > + * - (input) Number of bytes allocated to load into @payload.
> >
> > I'm not actually seeing where this is used as an input. I'd expect a min(input, output)
> > at the memcpy but there isn't one there.
> >
>
> The functionality changed since v2 and this interface no longer deals with
> restricting output length. This was because at some intermediate point I had
> entirely removed the copying in/out of the payload registers from this command.
> As you see, it came back.
>
> For userspace submitted commands the lengths are all validated up front before
> we get here.
>
> For commands submitted directly from the kernel, I've gone back and forth about
> how useful it is. If you feel strongly that it is useful, I can add it back. It
> sounds like you want me to convert everything back to an intermediate buffer,
> which I will respond in that patch, so I'd need to rework it anyway if that's
> the path we end up taking.
Fix the docs to make that (output) only works for me. Perhaps also add a line saying that
output payload length must be large enough.
>
> > > + * - (output) Number of bytes loaded into @payload.
> > > + * @return_code: (output) Error code returned from hardware.
> > > + *
> > > + * This is the primary mechanism used to send commands to the hardware.
> > > + * All the fields except @payload correspond exactly to the fields described in
> > > + * Command Register section of the CXL 2.0 spec (8.2.8.4.5). @payload
> > > + * corresponds to the Command Payload Registers (8.2.8.4.8).
> > > + */
> > > +struct mbox_cmd {
> > > + u16 opcode;
> > > + void *payload;
> > > + size_t size_in;
> > > + size_t size_out;
> > > + u16 return_code;
> > > +};
> > > +
> > > +static int cxl_mem_wait_for_doorbell(struct cxl_mem *cxlm)
> > > +{
> > > + const int timeout = msecs_to_jiffies(CXL_MAILBOX_TIMEOUT_US);
> > > + const unsigned long start = jiffies;
> > > + unsigned long end = start;
> > > +
> > > + while (cxl_doorbell_busy(cxlm)) {
> > > + end = jiffies;
> > > +
> > > + if (time_after(end, start + timeout)) {
> > > + /* Check again in case preempted before timeout test */
> > > + if (!cxl_doorbell_busy(cxlm))
> > > + break;
> > > + return -ETIMEDOUT;
> > > + }
> > > + cpu_relax();
> > > + }
> > > +
> > > + dev_dbg(&cxlm->pdev->dev, "Doorbell wait took %dms",
> > > + jiffies_to_msecs(end) - jiffies_to_msecs(start));
> > > + return 0;
> > > +}
> > > +
> > > +static void cxl_mem_mbox_timeout(struct cxl_mem *cxlm,
> > > + struct mbox_cmd *mbox_cmd)
> > > +{
> > > + dev_warn(&cxlm->pdev->dev, "Mailbox command timed out\n");
> > > + dev_info(&cxlm->pdev->dev,
> > > + "\topcode: 0x%04x\n"
> > > + "\tpayload size: %zub\n",
> > > + mbox_cmd->opcode, mbox_cmd->size_in);
> > > + print_hex_dump_debug("Payload ", DUMP_PREFIX_OFFSET, 16, 1,
> > > + mbox_cmd->payload, mbox_cmd->size_in, true);
> > > +
> > > + /* Here's a good place to figure out if a device reset is needed */
> > > +}
> > > +
> > > +/**
> > > + * cxl_mem_mbox_send_cmd() - Send a mailbox command to a memory device.
> > > + * @cxlm: The CXL memory device to communicate with.
> > > + * @mbox_cmd: Command to send to the memory device.
> > > + *
> > > + * Context: Any context. Expects mbox_lock to be held.
> > > + * Return: -ETIMEDOUT if timeout occurred waiting for completion. 0 on success.
> > > + * Caller should check the return code in @mbox_cmd to make sure it
> > > + * succeeded.
> > > + *
> > > + * This is a generic form of the CXL mailbox send command, thus the only I/O
> > > + * operations used are cxl_read_mbox_reg(). Memory devices, and perhaps other
> > > + * types of CXL devices may have further information available upon error
> > > + * conditions.
> > > + *
> > > + * FIXME: As stated above, references to &struct cxl_mem should be changed to a
> > > + * more generic cxl structure when needed.
> > > + */
> > > +static int cxl_mem_mbox_send_cmd(struct cxl_mem *cxlm,
> > > + struct mbox_cmd *mbox_cmd)
> > > +{
> > > + u64 cmd_reg, status_reg;
> > > + size_t out_len;
> > > + int rc;
> > > +
> > > + lockdep_assert_held(&cxlm->mbox.mutex);
> > > +
> > > + /*
> > > + * Here are the steps from 8.2.8.4 of the CXL 2.0 spec.
> > > + * 1. Caller reads MB Control Register to verify doorbell is clear
> > > + * 2. Caller writes Command Register
> > > + * 3. Caller writes Command Payload Registers if input payload is non-empty
> > > + * 4. Caller writes MB Control Register to set doorbell
> > > + * 5. Caller either polls for doorbell to be clear or waits for interrupt if configured
> > > + * 6. Caller reads MB Status Register to fetch Return code
> > > + * 7. If command successful, Caller reads Command Register to get Payload Length
> > > + * 8. If output payload is non-empty, host reads Command Payload Registers
> > > + */
> > > +
> > > + /* #1 */
> > > + WARN_ON(cxl_doorbell_busy(cxlm));
> > > +
> > > + cmd_reg = CXL_SET_FIELD(mbox_cmd->opcode, CXLDEV_MB_CMD_COMMAND_OPCODE);
> > > + if (mbox_cmd->size_in) {
> > > + cmd_reg |= CXL_SET_FIELD(mbox_cmd->size_in,
> > > + CXLDEV_MB_CMD_PAYLOAD_LENGTH);
> > > + if (mbox_cmd->payload)
> > > + memcpy_toio(cxl_payload_regs(cxlm), mbox_cmd->payload,
> > > + mbox_cmd->size_in);
> > > + }
> > > +
> > > + /* #2, #3 */
> > > + cxl_write_mbox_reg64(cxlm, CXLDEV_MB_CMD_OFFSET, cmd_reg);
> > > +
> > > + /* #4 */
> > > + dev_dbg(&cxlm->pdev->dev, "Sending command\n");
> > > + cxl_write_mbox_reg32(cxlm, CXLDEV_MB_CTRL_OFFSET,
> > > + CXLDEV_MB_CTRL_DOORBELL);
> > > +
> > > + /* #5 */
> > > + rc = cxl_mem_wait_for_doorbell(cxlm);
> > > + if (rc == -ETIMEDOUT) {
> > > + cxl_mem_mbox_timeout(cxlm, mbox_cmd);
> > > + return rc;
> > > + }
> > > +
> > > + /* #6 */
> > > + status_reg = cxl_read_mbox_reg64(cxlm, CXLDEV_MB_STATUS_OFFSET);
> > > + mbox_cmd->return_code =
> > > + CXL_GET_FIELD(status_reg, CXLDEV_MB_STATUS_RET_CODE);
> > > +
> > > + if (mbox_cmd->return_code != 0) {
> > > + dev_dbg(&cxlm->pdev->dev, "Mailbox operation had an error\n");
> > > + return 0;
> > > + }
> > > +
> > > + /* #7 */
> > > + cmd_reg = cxl_read_mbox_reg64(cxlm, CXLDEV_MB_CMD_OFFSET);
> > > + out_len = CXL_GET_FIELD(cmd_reg, CXLDEV_MB_CMD_PAYLOAD_LENGTH);
> > > + mbox_cmd->size_out = out_len;
> > > +
> > > + /* #8 */
> > > + if (out_len && mbox_cmd->payload)
> > > + memcpy_fromio(mbox_cmd->payload, cxl_payload_regs(cxlm),
> > > + mbox_cmd->size_out);
> > > +
> > > + return 0;
> > > +}
> > > +
> > > +/**
> > > + * cxl_mem_mbox_get() - Acquire exclusive access to the mailbox.
> > > + * @cxlm: The memory device to gain access to.
> > > + *
> > > + * Context: Any context. Takes the mbox_lock.
> > > + * Return: 0 if exclusive access was acquired.
> > > + */
> > > +static int cxl_mem_mbox_get(struct cxl_mem *cxlm)
> > > +{
> > > + u64 md_status;
> > > + int rc = -EBUSY;
> > > +
> > > + mutex_lock_io(&cxlm->mbox.mutex);
> > > +
> > > + /*
> > > + * XXX: There is some amount of ambiguity in the 2.0 version of the spec
> > > + * around the mailbox interface ready (8.2.8.5.1.1). The purpose of the
> > > + * bit is to allow firmware running on the device to notify us that it's
> > > + * ready to receive commands. It is unclear if the bit needs to be read
> > > + * every time one tries to use the mailbox, ie. the firmware can switch
> > > + * it on and off as needed. Second, there is no defined timeout for
> > > + * mailbox ready, like there is for the doorbell interface.
> > > + *
> > > + * As such, we make the following assumptions:
> > > + * 1. The firmware might toggle the Mailbox Interface Ready bit, and so
> > > + * we check it for every command.
> > > + * 2. If the doorbell is clear, the firmware should have first set the
> > > + * Mailbox Interface Ready bit. Therefore, waiting for the doorbell
> > > + * to be ready is a sufficient amount of time.
> > > + */
> > > + rc = cxl_mem_wait_for_doorbell(cxlm);
> > > + if (rc) {
> > > + dev_warn(&cxlm->pdev->dev, "Mailbox interface not ready\n");
> > > + goto out;
> > > + }
> > > +
> > > + md_status = cxl_read_mem_reg64(cxlm, CXLMDEV_STATUS_OFFSET);
> > > + if (md_status & CXLMDEV_MBOX_IF_READY && CXLMDEV_READY(md_status)) {
> > > + /*
> > > + * Hardware shouldn't allow a ready status but also have failure
> > > + * bits set. Spit out an error, this should be a bug report
> > > + */
> > > + if (md_status & CXLMDEV_DEV_FATAL) {
> > > + dev_err(&cxlm->pdev->dev,
> > > + "CXL device reporting ready and fatal\n");
> > > + rc = -EFAULT;
> > > + goto out;
> > > + }
> > > + if (md_status & CXLMDEV_FW_HALT) {
> > > + dev_err(&cxlm->pdev->dev,
> > > + "CXL device reporting ready and halted\n");
> > > + rc = -EFAULT;
> > > + goto out;
> > > + }
> > > + if (CXLMDEV_RESET_NEEDED(md_status)) {
> > > + dev_err(&cxlm->pdev->dev,
> > > + "CXL device reporting ready and reset needed\n");
> > > + rc = -EFAULT;
> > > + goto out;
> > > + }
> > > +
> > > + return 0;
> > > + }
> > > +
> > > +out:
> > > + mutex_unlock(&cxlm->mbox.mutex);
> > > + return rc;
> > > +}
> > > +
> > > +/**
> > > + * cxl_mem_mbox_put() - Release exclusive access to the mailbox.
> > > + * @cxlm: The CXL memory device to communicate with.
> > > + *
> > > + * Context: Any context. Expects mbox_lock to be held.
> > > + */
> > > +static void cxl_mem_mbox_put(struct cxl_mem *cxlm)
> > > +{
> > > + mutex_unlock(&cxlm->mbox.mutex);
> > > +}
> > > +
> > > /**
> > > * cxl_mem_setup_regs() - Setup necessary MMIO.
> > > * @cxlm: The CXL memory device to communicate with.
> > > @@ -135,6 +378,8 @@ static struct cxl_mem *cxl_mem_create(struct pci_dev *pdev, u32 reg_lo,
> > > return NULL;
> > > }
> > >
> > > + mutex_init(&cxlm->mbox.mutex);
> > > +
> > > regs = pcim_iomap_table(pdev)[bar];
> > > cxlm->pdev = pdev;
> > > cxlm->regs = regs + offset;
> > > @@ -167,6 +412,69 @@ static int cxl_mem_dvsec(struct pci_dev *pdev, int dvsec)
> > > return 0;
> > > }
> > >
> > > +/**
> > > + * cxl_mem_identify() - Send the IDENTIFY command to the device.
> > > + * @cxlm: The device to identify.
> > > + *
> > > + * Return: 0 if identify was executed successfully.
> > > + *
> > > + * This will dispatch the identify command to the device and on success populate
> > > + * structures to be exported to sysfs.
> > > + */
> > > +static int cxl_mem_identify(struct cxl_mem *cxlm)
> > > +{
> > > + struct cxl_mbox_identify {
> > > + char fw_revision[0x10];
> > > + __le64 total_capacity;
> > > + __le64 volatile_capacity;
> > > + __le64 persistent_capacity;
> > > + __le64 partition_align;
> > > + __le16 info_event_log_size;
> > > + __le16 warning_event_log_size;
> > > + __le16 failure_event_log_size;
> > > + __le16 fatal_event_log_size;
> > > + __le32 lsa_size;
> > > + u8 poison_list_max_mer[3];
> > > + __le16 inject_poison_limit;
> > > + u8 poison_caps;
> > > + u8 qos_telemetry_caps;
> > > + } __packed id;
> > > + struct mbox_cmd mbox_cmd;
> > > + int rc;
> > > +
> > > + /* Retrieve initial device memory map */
> > > + rc = cxl_mem_mbox_get(cxlm);
> > > + if (rc)
> > > + return rc;
> > > +
> > > + mbox_cmd = (struct mbox_cmd){
> > > + .opcode = CXL_MBOX_OP_IDENTIFY,
> > > + .payload = &id,
> > > + .size_in = 0,
> > > + };
> > > + rc = cxl_mem_mbox_send_cmd(cxlm, &mbox_cmd);
> > > + cxl_mem_mbox_put(cxlm);
> > > + if (rc)
> > > + return rc;
> > > +
> > > + if (mbox_cmd.size_out != sizeof(id))
> > > + return -ENXIO;
> > > +
> > > + /*
> > > + * TODO: enumerate DPA map, as 'ram' and 'pmem' do not alias.
> > > + * For now, only the capacity is exported in sysfs
> > > + */
> > > + cxlm->ram.range.start = 0;
> > > + cxlm->ram.range.end = le64_to_cpu(id.volatile_capacity) - 1;
> > > +
> > > + cxlm->pmem.range.start = 0;
> > > + cxlm->pmem.range.end = le64_to_cpu(id.persistent_capacity) - 1;
> > > +
> > > + memcpy(cxlm->firmware_version, id.fw_revision, sizeof(id.fw_revision));
> > > +
> > > + return rc;
> > > +}
> > > +
> > > static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> > > {
> > > struct device *dev = &pdev->dev;
> > > @@ -222,6 +530,10 @@ static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> > > if (rc)
> > > goto err;
> > >
> > > + rc = cxl_mem_identify(cxlm);
> > > + if (rc)
> > > + goto err;
> > > +
> > > pci_set_drvdata(pdev, cxlm);
> > > return 0;
> > >
> >
On Thu, 14 Jan 2021 10:13:40 -0800
Ben Widawsky <[email protected]> wrote:
> On 21-01-14 18:02:11, Jonathan Cameron wrote:
> > On Mon, 11 Jan 2021 14:51:19 -0800
> > Ben Widawsky <[email protected]> wrote:
> >
> > > The Command Effects Log (CEL) is specified in the CXL 2.0 specification.
> > > The CEL is one of two types of logs, the other being vendor specific.
> > > They are distinguished in hardware/spec via UUID. The CEL is immediately
> > > useful for 2 things:
> > > 1. Determine which optional commands are supported by the CXL device.
> > > 2. Enumerate any vendor specific commands
> > >
> > > The CEL can be used by the driver to determine which commands are
> > > available in the hardware (though it isn't, yet). That set of commands
> > > might itself be a subset of commands which are available to be used via
> > > CXL_MEM_SEND_COMMAND IOCTL.
> > >
> > > Prior to this, all commands that the driver exposed were explicitly
> > > enabled. After this, only those commands that are found in the CEL are
> > > enabled.
> > >
> > > Signed-off-by: Ben Widawsky <[email protected]>
> >
> > This patch made me wonder if the model for the command in quite right.
> > I think it would end up simpler with a pair of payload pointers for send
> > and receive (that can be equal when it makes sense).
> >
> > A few other things inline.
> >
> > Jonathan
>
> I'll address the others separately, but could you elaborate on this? I'm not
> sure I follow your meaning.
Further down in the review..
"
The fact that you end up bypassing the payload transfer stuff in mbox_cmd
rather suggests it's not a particularly good model. + it keeps confusing
me.
While the hardware uses a single region for the payload, there is nothing
saying the code has to work that way. Why not have separate payload_in and
payload_out pointers? Occasionally you might set them to the same buffer, but
elsewhere you could avoid the direct memcpy()s you are doing around the
send_cmd().
"
Jonathan
>
> [snip]
>
>
On 21-01-14 18:32:17, Jonathan Cameron wrote:
> On Thu, 14 Jan 2021 10:13:40 -0800
> Ben Widawsky <[email protected]> wrote:
>
> > On 21-01-14 18:02:11, Jonathan Cameron wrote:
> > > On Mon, 11 Jan 2021 14:51:19 -0800
> > > Ben Widawsky <[email protected]> wrote:
> > >
> > > > The Command Effects Log (CEL) is specified in the CXL 2.0 specification.
> > > > The CEL is one of two types of logs, the other being vendor specific.
> > > > They are distinguished in hardware/spec via UUID. The CEL is immediately
> > > > useful for 2 things:
> > > > 1. Determine which optional commands are supported by the CXL device.
> > > > 2. Enumerate any vendor specific commands
> > > >
> > > > The CEL can be used by the driver to determine which commands are
> > > > available in the hardware (though it isn't, yet). That set of commands
> > > > might itself be a subset of commands which are available to be used via
> > > > CXL_MEM_SEND_COMMAND IOCTL.
> > > >
> > > > Prior to this, all commands that the driver exposed were explicitly
> > > > enabled. After this, only those commands that are found in the CEL are
> > > > enabled.
> > > >
> > > > Signed-off-by: Ben Widawsky <[email protected]>
> > >
> > > This patch made me wonder if the model for the command in quite right.
> > > I think it would end up simpler with a pair of payload pointers for send
> > > and receive (that can be equal when it makes sense).
> > >
> > > A few other things inline.
> > >
> > > Jonathan
> >
> > I'll address the others separately, but could you elaborate on this? I'm not
> > sure I follow your meaning.
>
> Further down in the review..
> "
> The fact that you end up bypassing the payload transfer stuff in mbox_cmd
> rather suggests it's not a particularly good model. + it keeps confusing
> me.
>
> While the hardware uses a single region for the payload, there is nothing
> saying the code has to work that way. Why not have separate payload_in and
> payload_out pointers? Occasionally you might set them to the same buffer, but
> elsewhere you could avoid the direct memcpy()s you are doing around the
> send_cmd().
>
> "
>
> Jonathan
>
>
Ah I was confused if that was a separate statement.
Can you specify the function prototype you're hoping for (or modification to the
structure)?
I really like the lowest level function to simply model the hardware. I get to
write the 8 steps out and clearly implement them.
I personally don't think it's so awkward, but again, give me something more
specific and I'll consider it.
On Thu, 14 Jan 2021 11:04:25 -0800
Ben Widawsky <[email protected]> wrote:
> On 21-01-14 18:32:17, Jonathan Cameron wrote:
> > On Thu, 14 Jan 2021 10:13:40 -0800
> > Ben Widawsky <[email protected]> wrote:
> >
> > > On 21-01-14 18:02:11, Jonathan Cameron wrote:
> > > > On Mon, 11 Jan 2021 14:51:19 -0800
> > > > Ben Widawsky <[email protected]> wrote:
> > > >
> > > > > The Command Effects Log (CEL) is specified in the CXL 2.0 specification.
> > > > > The CEL is one of two types of logs, the other being vendor specific.
> > > > > They are distinguished in hardware/spec via UUID. The CEL is immediately
> > > > > useful for 2 things:
> > > > > 1. Determine which optional commands are supported by the CXL device.
> > > > > 2. Enumerate any vendor specific commands
> > > > >
> > > > > The CEL can be used by the driver to determine which commands are
> > > > > available in the hardware (though it isn't, yet). That set of commands
> > > > > might itself be a subset of commands which are available to be used via
> > > > > CXL_MEM_SEND_COMMAND IOCTL.
> > > > >
> > > > > Prior to this, all commands that the driver exposed were explicitly
> > > > > enabled. After this, only those commands that are found in the CEL are
> > > > > enabled.
> > > > >
> > > > > Signed-off-by: Ben Widawsky <[email protected]>
> > > >
> > > > This patch made me wonder if the model for the command in quite right.
> > > > I think it would end up simpler with a pair of payload pointers for send
> > > > and receive (that can be equal when it makes sense).
> > > >
> > > > A few other things inline.
> > > >
> > > > Jonathan
> > >
> > > I'll address the others separately, but could you elaborate on this? I'm not
> > > sure I follow your meaning.
> >
> > Further down in the review..
> > "
> > The fact that you end up bypassing the payload transfer stuff in mbox_cmd
> > rather suggests it's not a particularly good model. + it keeps confusing
> > me.
> >
> > While the hardware uses a single region for the payload, there is nothing
> > saying the code has to work that way. Why not have separate payload_in and
> > payload_out pointers? Occasionally you might set them to the same buffer, but
> > elsewhere you could avoid the direct memcpy()s you are doing around the
> > send_cmd().
> >
> > "
> >
> > Jonathan
> >
> >
>
> Ah I was confused if that was a separate statement.
>
> Can you specify the function prototype you're hoping for (or modification to the
> structure)?
>
> I really like the lowest level function to simply model the hardware. I get to
> write the 8 steps out and clearly implement them.
>
> I personally don't think it's so awkward, but again, give me something more
> specific and I'll consider it.
Looking at the case in this patch as an example
Something like (completely untested)...
Note only real change here is having an extra pointer in mbox_cmd so we can
have in and out buffers provided. Doesn't change the flow at all means we can
use the internal memcpy_from_io memcpy_to_io in more cases.
Zero copy is still fine if we are in a case where it makes sense.
struct mbox_cmd {
u16 opcode;
//separate in and out pointers.
void *payload_in;
void *payload_out;
size_t size_in;
size_t size_out;
u16 return_code;
};
static int cxl_mem_mbox_send_cmd(struct cxl_mem *cxlm,
struct mbox_cmd *mbox_cmd)
{
u64 cmd_reg, status_reg;
size_t out_len;
int rc;
lockdep_assert_held(&cxlm->mbox.mutex);
/*
* Here are the steps from 8.2.8.4 of the CXL 2.0 spec.
* 1. Caller reads MB Control Register to verify doorbell is clear
* 2. Caller writes Command Register
* 3. Caller writes Command Payload Registers if input payload is non-empty
* 4. Caller writes MB Control Register to set doorbell
* 5. Caller either polls for doorbell to be clear or waits for interrupt if configured
* 6. Caller reads MB Status Register to fetch Return code
* 7. If command successful, Caller reads Command Register to get Payload Length
* 8. If output payload is non-empty, host reads Command Payload Registers
*/
/* #1 */
WARN_ON(cxl_doorbell_busy(cxlm));
cmd_reg = CXL_SET_FIELD(mbox_cmd->opcode, CXLDEV_MB_CMD_COMMAND_OPCODE);
if (mbox_cmd->size_in) {
cmd_reg |= CXL_SET_FIELD(mbox_cmd->size_in,
CXLDEV_MB_CMD_PAYLOAD_LENGTH);
//leave this here for your userspace zero copy path...
if (mbox_cmd->payload_in)
memcpy_toio(cxl_payload_regs(cxlm), mbox_cmd->payload_in,
mbox_cmd->size_in);
}
/* #2, #3 */
cxl_write_mbox_reg64(cxlm, CXLDEV_MB_CMD_OFFSET, cmd_reg);
/* #4 */
+ dev_dbg(&cxlm->pdev->dev, "Sending command\n");
cxl_write_mbox_reg32(cxlm, CXLDEV_MB_CTRL_OFFSET,
CXLDEV_MB_CTRL_DOORBELL);
/* #5 */
rc = cxl_mem_wait_for_doorbell(cxlm);
if (rc == -ETIMEDOUT) {
cxl_mem_mbox_timeout(cxlm, mbox_cmd);
return rc;
}
/* #6 */
status_reg = cxl_read_mbox_reg64(cxlm, CXLDEV_MB_STATUS_OFFSET);
mbox_cmd->return_code =
CXL_GET_FIELD(status_reg, CXLDEV_MB_STATUS_RET_CODE);
if (mbox_cmd->return_code != 0) {
dev_dbg(&cxlm->pdev->dev, "Mailbox operation had an error\n");
return 0;
}
/* #7 */
cmd_reg = cxl_read_mbox_reg64(cxlm, CXLDEV_MB_CMD_OFFSET);
out_len = CXL_GET_FIELD(cmd_reg, CXLDEV_MB_CMD_PAYLOAD_LENGTH);
/* #8 */
if (out_len && mbox_cmd->payload_out)
memcpy_fromio(mbox_cmd->payload_out, cxl_payload_regs(cxlm),
min(mbox_cmd->size_out, out_len));
//move this down here - because they the output buffer might theory not
//bit big enough.
mbox_cmd->size_out = out_len;
return 0;
}
....
static int cxl_xfer_log(struct cxl_mem *cxlm, uuid_t *uuid, u32 size,
u8 *out)
{
u32 remaining = size;
u32 offset = 0;
while (remaining) {
u32 xfer_size = min_t(u32, remaining, cxlm->mbox.payload_size);
struct mbox_cmd mbox_cmd;
int rc;
struct cxl_mbox_get_log log = {
.uuid = *uuid,
.offset = cpu_to_le32(offset),
.length = cpu_to_le32(xfer_size),
};
mbox_cmd = (struct mbox_cmd) {
.opcode = CXL_MBOX_OP_GET_LOG,
.payload_in = &log,
.size_in = sizeof(log),
.payload_out = out,
.size_out = min(xfer_size, remaining),
};
rc = cxl_mem_mbox_send_cmd(cxlm, &mbox_cmd);
if (rc)
return rc;
WARN_ON(mbox_cmd.size_out != xfer_size);
out += xfer_size;
remaining -= xfer_size;
offset += xfer_size;
}
return 0;
}
On Tue, 2021-01-12 at 18:43 +0000, Jonathan Cameron wrote:
> On Mon, 11 Jan 2021 14:51:06 -0800
> Ben Widawsky <[email protected]> wrote:
>
> > From: Vishal Verma <[email protected]>
> >
> > Add an acpi_cxl module to coordinate the ACPI portions of the CXL
> > (Compute eXpress Link) interconnect. This driver binds to ACPI0017
> > objects in the ACPI tree, and coordinates access to the resources
> > provided by the ACPI CEDT (CXL Early Discovery Table).
> >
> > It also coordinates operations of the root port _OSC object to notify
> > platform firmware that the OS has native support for the CXL
> > capabilities of endpoints.
> >
> > Note: the actbl1.h changes are speculative. The expectation is that they
> > will arrive through the ACPICA tree in due time.
>
> I would pull the ACPICA changes out into a precursor patch.
Done.
>
> >
> > Cc: Ben Widawsky <[email protected]>
> > Cc: Dan Williams <[email protected]>
> > Signed-off-by: Vishal Verma <[email protected]>
> > Signed-off-by: Ben Widawsky <[email protected]>
>
> Hi,
>
> I think it would be good to also add CEDT to the list in drivers/acpi/tables.c
> so that we can dump it from /sys/firmware/acpi/tables/ and potentially
> override it from an initrd.
Dumping it from /sys/firmware/acpi/tables already works, but I did add
it to tables.c so it can also be overridden. I did this in a separate
patch, in case ACPI folks want to do this differently.
>
> https://elixir.bootlin.com/linux/v5.11-rc3/source/drivers/acpi/tables.c#L482
> Can be very helpful whilst debugging. Related to that, anyone know if anyone
> has acpica patches so we can have iasl -d work on the table? Would probably
> be useful but I'd rather not duplicate work if it's already done.
>
> A few minor things inline
Agreed with all of your other comments below, and I've addresed them for
the next revision. Thanks for the review Jonathan!
>
> Jonathan
>
>
On Wed, 2021-01-13 at 13:40 +0100, Rafael J. Wysocki wrote:
> On Tue, Jan 12, 2021 at 1:29 AM Ben Widawsky <[email protected]> wrote:
> >
> > From: Vishal Verma <[email protected]>
> >
> > Add an acpi_cxl module to coordinate the ACPI portions of the CXL
> > (Compute eXpress Link) interconnect. This driver binds to ACPI0017
> > objects in the ACPI tree, and coordinates access to the resources
> > provided by the ACPI CEDT (CXL Early Discovery Table).
> >
> > It also coordinates operations of the root port _OSC object to notify
> > platform firmware that the OS has native support for the CXL
> > capabilities of endpoints.
>
> This doesn't happen here, but in the next patch.
>
> > Note: the actbl1.h changes are speculative. The expectation is that they
> > will arrive through the ACPICA tree in due time.
>
> So why don't you put them into a separate patch and drop it from the
> series when not necessary any more?
[snip]
> > +/*
> > + * If/when CXL support is defined by other platform firmware the kernel
> > + * will need a mechanism to select between the platform specific version
> > + * of this routine, until then, hard-code ACPI assumptions
> > + */
> > +int cxl_bus_acquire(struct pci_dev *pdev)
> > +{
> > + struct acpi_device *adev;
> > + struct pci_dev *root_port;
> > + struct device *root;
> > +
> > + root_port = pcie_find_root_port(pdev);
> > + if (!root_port)
> > + return -ENXIO;
> > +
> > + root = root_port->dev.parent;
> > + if (!root)
> > + return -ENXIO;
> > +
> > + adev = ACPI_COMPANION(root);
> > + if (!adev)
> > + return -ENXIO;
> > +
> > + /* TODO: OSC enabling */
> > +
> > + return 0;
> > +}
> > +EXPORT_SYMBOL_GPL(cxl_bus_acquire);
>
> I would move the addition of cxl_bus_acquire() entirely to the next
> patch, it looks quite confusing to me as is.
Makes sense - and also agreed with all of your other comments. I've
cleaned this up for the next revision. Thanks Rafael!
>
On Wed, Jan 13, 2021 at 9:55 AM Kaneda, Erik <[email protected]> wrote:
[..]
> > The supplemental tables described here:
> >
> > https://www.uefi.org/acpi
> >
> > ...do eventually make there way into ACPICA. Added Bob and Erik in
> > case they can comment on when CEDT and CDAT support will be picked up.
>
> We would be happy to add support. I think Ben has reached out to me earlier about something like this but I haven’t had a chance to implement... Sorry about the delay.. How soon is the iASL/ACPICA support needed for CDAT and CDET?
CDAT and CEDT definitions in include/acpi/actbl* would be useful as
soon as v5.12. If that's too soon I don't think we're blocked, but the
driver is fast approaching the point where it needs to consult these
tables to enable higher order functionality.
On 21-01-14 17:10:38, Jonathan Cameron wrote:
> On Mon, 11 Jan 2021 14:51:14 -0800
> Ben Widawsky <[email protected]> wrote:
>
> > The send command allows userspace to issue mailbox commands directly to
> > the hardware. The driver will verify basic properties of the command and
> > possible inspect the input (or output) payload to determine whether or
> > not the command is allowed (or might taint the kernel).
> >
> > The list of allowed commands and their properties can be determined by
> > using the QUERY IOCTL for CXL memory devices.
> >
> > Signed-off-by: Ben Widawsky <[email protected]>
> > ---
> > drivers/cxl/mem.c | 204 ++++++++++++++++++++++++++++++++++-
> > include/uapi/linux/cxl_mem.h | 39 +++++++
> > 2 files changed, 239 insertions(+), 4 deletions(-)
> >
> > diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
> > index d4eb3f5b9469..f979788b4d9f 100644
> > --- a/drivers/cxl/mem.c
> > +++ b/drivers/cxl/mem.c
> > @@ -84,6 +84,13 @@ static DEFINE_IDR(cxl_mem_idr);
> > /* protect cxl_mem_idr allocations */
> > static DEFINE_MUTEX(cxl_memdev_lock);
> >
> > +#undef C
> > +#define C(a, b) { b }
>
> I'm not following why this is here?
>
It's used for a debug message in handle_mailbox_cmd_from_user(). This is all the
macro magic stolen from ftrace. Or, did I miss the question?
> > +static struct {
> > + const char *name;
> > +} command_names[] = { CMDS };
> > +#undef C
> > +
> > #define CXL_CMD(_id, _flags, sin, sout, f) \
> > [CXL_MEM_COMMAND_ID_##_id] = { \
> > { \
> ...
>
> > +
> > +/**
> > + * handle_mailbox_cmd_from_user() - Dispatch a mailbox command.
> > + * @cxlmd: The CXL memory device to communicate with.
> > + * @cmd: The validated command.
> > + * @in_payload: Pointer to userspace's input payload.
> > + * @out_payload: Pointer to userspace's output payload.
> > + * @u: The command submitted by userspace. Has output fields.
> > + *
> > + * Return:
> > + * * %0 - Mailbox transaction succeeded.
> > + * * %-EFAULT - Something happened with copy_to/from_user.
> > + * * %-EINTR - Mailbox acquisition interrupted.
> > + * * %-E2BIG - Output payload would overrun buffer.
> > + *
> > + * Creates the appropriate mailbox command on behalf of a userspace request.
> > + * Return value, size, and output payload are all copied out to @u. The
> > + * parameters for the command must be validated before calling this function.
> > + *
> > + * A 0 return code indicates the command executed successfully, not that it was
> > + * itself successful. IOW, the retval should always be checked if wanting to
>
> cmd->retval perhaps to be more explicit?
>
> > + * determine the actual result.
> > + */
> > +static int handle_mailbox_cmd_from_user(struct cxl_memdev *cxlmd,
> > + const struct cxl_mem_command *cmd,
> > + u64 in_payload,
> > + u64 out_payload,
> > + struct cxl_send_command __user *u)
> > +{
> > + struct mbox_cmd mbox_cmd = {
> > + .opcode = cmd->opcode,
> > + .size_in = cmd->info.size_in,
> > + .payload = NULL, /* Copied by copy_to|from_user() */
> > + };
> > + int rc;
> > +
> > + if (cmd->info.size_in) {
> > + /*
> > + * Directly copy the userspace payload into the hardware. UAPI
> > + * states that the buffer must already be little endian.
> > + */
> > + if (copy_from_user((__force void *)cxl_payload_regs(cxlmd->cxlm),
> > + u64_to_user_ptr(in_payload),
> > + cmd->info.size_in)) {
> > + cxl_mem_mbox_put(cxlmd->cxlm);
>
> mbox_get is after this point though it shouldn't be given we just
> wrote into the mbox registers.
>
> This seems unlikely to be a high performance path, so perhaps just
> use a local buffer and let cxl_mem_mbox_send_cmd copy it into the registers.
>
You're correct about the get() needing to be first. I will fix it. As for
performance path - so while this does potentially help with performance, it
actually ends up being I think a little cleaner to not have to deal with a local
buffer.
How strongly do you feel about it? I'd say if you don't care so much, let's keep
it as is and find a reason to undo later.
> > + return -EFAULT;
> > + }
> > + }
> > +
> > + rc = cxl_mem_mbox_get(cxlmd->cxlm, true);
> > + if (rc)
> > + return rc;
> > +
> > + dev_dbg(&cxlmd->dev,
> > + "Submitting %s command for user\n"
> > + "\topcode: %x\n"
> > + "\tsize: %ub\n",
> > + command_names[cmd->info.id].name, mbox_cmd.opcode,
> > + cmd->info.size_in);
> > +
> > + rc = cxl_mem_mbox_send_cmd(cxlmd->cxlm, &mbox_cmd);
> > + cxl_mem_mbox_put(cxlmd->cxlm);
> > + if (rc)
> > + return rc;
> > +
> > + if (mbox_cmd.size_out > cmd->info.size_out)
> > + return -E2BIG;
> > +
> > + rc = put_user(mbox_cmd.return_code, &u->retval);
> > + if (rc)
> > + return rc;
> > +
> > + rc = put_user(mbox_cmd.size_out, &u->size_out);
> > + if (rc)
> > + return rc;
> > +
> > + if (mbox_cmd.size_out)
> > + if (copy_to_user(u64_to_user_ptr(out_payload),
> > + (__force void *)cxl_payload_regs(cxlmd->cxlm),
> > + mbox_cmd.size_out))
> > + return -EFAULT;
> > +
> > + return 0;
> > +}
> > +
>
> ...
Yeah...
>
> >
> > static long cxl_mem_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
> > @@ -479,6 +644,37 @@ static long cxl_mem_ioctl(struct file *file, unsigned int cmd, unsigned long arg
> > if (j == n_commands)
> > break;
> > }
> > +
> > + return 0;
>
> Ah. That should have been in the earlier patch. Explains why the code works :)
>
>
> > + } else if (cmd == CXL_MEM_SEND_COMMAND) {
> > + struct cxl_send_command send, __user *u = (void __user *)arg;
> > + struct cxl_mem_command c;
> > + int rc;
> > +
> > + dev_dbg(dev, "Send IOCTL\n");
> > +
> > + if (copy_from_user(&send, u, sizeof(send)))
> > + return -EFAULT;
> > +
> > + rc = device_lock_interruptible(dev);
> > + if (rc)
> > + return rc;
> > +
> > + if (!get_live_device(dev)) {
> > + device_unlock(dev);
> > + return -ENXIO;
> > + }
> > +
> > + rc = cxl_validate_cmd_from_user(cxlmd->cxlm, &send, &c);
> > + if (!rc)
> > + rc = handle_mailbox_cmd_from_user(cxlmd, &c,
> > + send.in_payload,
> > + send.out_payload, u);
> > +
> > + put_device(dev);
> > + device_unlock(dev);
> > +
> > + return rc;
> > }
> >
> > return -ENOTTY;
> > @@ -837,7 +1033,7 @@ static int cxl_mem_identify(struct cxl_mem *cxlm)
> > int rc;
> >
> > /* Retrieve initial device memory map */
> > - rc = cxl_mem_mbox_get(cxlm);
> > + rc = cxl_mem_mbox_get(cxlm, false);
> > if (rc)
> > return rc;
> >
> > diff --git a/include/uapi/linux/cxl_mem.h b/include/uapi/linux/cxl_mem.h
> > index 847f825bbe18..cb4e2bee5228 100644
> > --- a/include/uapi/linux/cxl_mem.h
> > +++ b/include/uapi/linux/cxl_mem.h
> > @@ -26,6 +26,7 @@ extern "C" {
> > */
> >
> > #define CXL_MEM_QUERY_COMMANDS _IOR(0xCE, 1, struct cxl_mem_query_commands)
> > +#define CXL_MEM_SEND_COMMAND _IOWR(0xCE, 2, struct cxl_send_command)
> >
> > #undef CMDS
> > #define CMDS \
> > @@ -69,6 +70,7 @@ struct cxl_command_info {
> > #define CXL_MEM_COMMAND_FLAG_NONE 0
> > #define CXL_MEM_COMMAND_FLAG_KERNEL BIT(0)
> > #define CXL_MEM_COMMAND_FLAG_MUTEX BIT(1)
> > +#define CXL_MEM_COMMAND_FLAG_MASK GENMASK(31, 2)
>
> Instinctively I'd expect FLAG_MASK to be GENMASK(1, 0)
> and to be used as ~FLAG_MASK. As it's mask of flags, not
> the mask to leave only valid flags.
>
Fine with me.
> >
> > __s32 size_in;
> > __s32 size_out;
> > @@ -110,6 +112,43 @@ struct cxl_mem_query_commands {
> > struct cxl_command_info __user commands[]; /* out: supported commands */
> > };
> >
> > +/**
> > + * struct cxl_send_command - Send a command to a memory device.
> > + * @id: The command to send to the memory device. This must be one of the
> > + * commands returned by the query command.
> > + * @flags: Flags for the command (input).
> > + * @rsvd: Must be zero.
> > + * @retval: Return value from the memory device (output).
> > + * @size_in: Size of the payload to provide to the device (input).
> > + * @size_out: Size of the payload received from the device (input/output). This
> > + * field is filled in by userspace to let the driver know how much
> > + * space was allocated for output. It is populated by the driver to
> > + * let userspace know how large the output payload actually was.
> > + * @in_payload: Pointer to memory for payload input (little endian order).
> > + * @out_payload: Pointer to memory for payload output (little endian order).
> > + *
> > + * Mechanism for userspace to send a command to the hardware for processing. The
> > + * driver will do basic validation on the command sizes, but the payload input
> > + * and output are not introspected. Userspace is required to allocate large
> > + * enough buffers for max(size_in, size_out).
>
> That sounds like both buffers must be the maximum between size_in and size_out.
> Is intent that this is the maximum size_in for in_payload and max(size_out) for out_payload?
This comment reflects the way the interface was in v2. It needs fixing.
>
> > + */
> > +struct cxl_send_command {
> > + __u32 id;
> > + __u32 flags;
> > + __u32 rsvd;
> > + __u32 retval;
> > +
> > + struct {
> > + __s32 size_in;
> > + __u64 in_payload;
> > + };
> > +
> > + struct {
> > + __s32 size_out;
> > + __u64 out_payload;
> > + };
> > +};
> > +
> > #if defined(__cplusplus)
> > }
> > #endif
>
On 21-01-14 17:25:31, Jonathan Cameron wrote:
> On Mon, 11 Jan 2021 14:51:18 -0800
> Ben Widawsky <[email protected]> wrote:
>
> > CXL devices must implement the Device Command Interface (described in
> > 8.2.9 of the CXL 2.0 spec). While the driver already maintains a list of
> > commands it supports, there is still a need to be able to distinguish
> > between commands that the driver knows about from commands that may not
> > be supported by the hardware. No such commands currently are defined in
> > the driver.
> >
> > The implementation leaves the statically defined table of commands and
> > supplements it with a bitmap to determine commands that are enabled.
> >
> > ---
> >
> > There are multiple approaches that can be taken, but this is nice for a
> > few reasons.
> >
> > Here are some of the other solutions:
> >
> > Create a per instance table with only the supported commands.
> > 1. Having a fixed command id -> command mapping is much easier to manage
> > for development and debugging.
> > 2. Dealing with dynamic memory allocation for the table adds unnecessary
> > complexity.
> > 3. Most tables for device types are likely to be quite similar.
> > 4. Makes it difficult to implement helper macros like cxl_for_each_cmd()
> >
> > If the per instance table did preserve ids, #1 above can be addressed.
> > However, as "enable" is currently the only mutable state for the
> > commands, it would yield a lot of overhead for not much gain.
> > Additionally, the other issues remain.
> >
> > If "enable" remains the only mutable state, I believe this to be the
> > best solution. Once the number of mutable elements in a command grows,
> > it probably makes sense to move to per device instance state with a
> > fixed command ID mapping.
> Agreed with the logic.
>
> However, patch wise, should either drop the --- above or move this below the
> --- after your sign off. Otherwise you run the risk of git dropping your
> sign off and resulting complaints from anyone run validation scripts
> of the kernel tree that check for that.
>
> >
> > Signed-off-by: Ben Widawsky <[email protected]>
> > ---
> > drivers/cxl/cxl.h | 4 ++++
> > drivers/cxl/mem.c | 38 +++++++++++++++++++++++++++++++++++++-
> > 2 files changed, 41 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
> > index 537ac4d8e6bd..963ba30cb200 100644
> > --- a/drivers/cxl/cxl.h
> > +++ b/drivers/cxl/cxl.h
> > @@ -17,6 +17,9 @@
> >
> > #define CXL_GET_FIELD(word, field) FIELD_GET(field##_MASK, word)
> >
> > +/* XXX: Arbitrary max */
> > +#define CXL_MAX_COMMANDS 32
>
> If going this way, probably want a build time check that you don't
> go beyond this value for a given command set. I haven't actually
> thought about how to construct that but should be easy enough.
>
I think BUILD_BUG_ON of ARRAY_SIZE mem_commands is sufficient. Let me know if
you want anything more.
> > +
> > /* Device (CXL 2.0 - 8.2.8.3) */
> > #define CXLDEV_CAP_ARRAY_REG 0x0
> > #define CXLDEV_CAP_ARRAY_CAP_ID 0
> > @@ -88,6 +91,7 @@ struct cxl_mem {
> > } ram;
> >
> > char firmware_version[0x10];
> > + DECLARE_BITMAP(enabled_cmds, CXL_MAX_COMMANDS);
> >
> > /* Cap 0001h - CXL_CAP_CAP_ID_DEVICE_STATUS */
> > struct {
> > diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
> > index a824cfd4342a..20b26fa2c466 100644
> > --- a/drivers/cxl/mem.c
> > +++ b/drivers/cxl/mem.c
> > @@ -114,6 +114,8 @@ static struct {
> > * would typically be used for deprecated commands.
> > * * %CXL_CMD_FLAG_MANDATORY: Hardware must support this command. This flag is
> > * only used internally by the driver for sanity checking.
> > + * * %CXL_CMD_INTERNAL_FLAG_PSEUDO: This is a pseudo command which doesn't have
> > + * a direct mapping to hardware. They are implicitly always enabled.
> > *
> > * The cxl_mem_command is the driver's internal representation of commands that
> > * are supported by the driver. Some of these commands may not be supported by
> > @@ -129,6 +131,7 @@ struct cxl_mem_command {
> > #define CXL_CMD_INTERNAL_FLAG_NONE 0
> > #define CXL_CMD_INTERNAL_FLAG_HIDDEN BIT(0)
> > #define CXL_CMD_INTERNAL_FLAG_MANDATORY BIT(1)
> > +#define CXL_CMD_INTERNAL_FLAG_PSEUDO BIT(2)
> > };
> >
> > /*
> > @@ -140,7 +143,7 @@ struct cxl_mem_command {
> > static struct cxl_mem_command mem_commands[] = {
> > CXL_CMD(INVALID, KERNEL, 0, 0, HIDDEN),
> > CXL_CMD(IDENTIFY, NONE, 0, 0x43, MANDATORY),
> > - CXL_CMD(RAW, NONE, ~0, ~0, MANDATORY),
> > + CXL_CMD(RAW, NONE, ~0, ~0, PSEUDO),
> > };
> >
> > #define cxl_for_each_cmd(cmd) \
> > @@ -618,6 +621,10 @@ static int cxl_validate_cmd_from_user(struct cxl_mem *cxlm,
> > c = &mem_commands[send_cmd->id];
> > info = &c->info;
> >
> > + /* Check that the command is enabled for hardware */
> > + if (!test_bit(cxl_cmd_index(c), cxlm->enabled_cmds))
> > + return -ENOTTY;
> > +
> > if (info->flags & CXL_MEM_COMMAND_FLAG_KERNEL)
> > return -EPERM;
> >
> > @@ -1029,6 +1036,31 @@ static int cxl_mem_add_memdev(struct cxl_mem *cxlm)
> > return rc;
> > }
> >
> > +/**
> > + * cxl_mem_enumerate_cmds() - Enumerate commands for a device.
> > + * @cxlm: The device.
> > + *
> > + * Returns 0 if enumerate completed successfully.
> > + *
> > + * CXL devices have optional support for certain commands. This function will
> > + * determine the set of supported commands for the hardware and update the
> > + * enabled_cmds bitmap in the @cxlm.
> > + */
> > +static int cxl_mem_enumerate_cmds(struct cxl_mem *cxlm)
> > +{
> > + struct cxl_mem_command *c;
> > +
> > + /* All commands are considered enabled for now (except INVALID). */
> > + cxl_for_each_cmd(c) {
> > + if (c->flags & CXL_CMD_INTERNAL_FLAG_HIDDEN)
> > + continue;
> > +
> > + set_bit(cxl_cmd_index(c), cxlm->enabled_cmds);
> > + }
> > +
> > + return 0;
> > +}
> > +
> > /**
> > * cxl_mem_identify() - Send the IDENTIFY command to the device.
> > * @cxlm: The device to identify.
> > @@ -1147,6 +1179,10 @@ static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> > if (rc)
> > goto err;
> >
> > + rc = cxl_mem_enumerate_cmds(cxlm);
> > + if (rc)
> > + return rc;
> > +
> > rc = cxl_mem_identify(cxlm);
> > if (rc)
> > goto err;
>
On Thu, 21 Jan 2021 10:40:41 -0800
Ben Widawsky <[email protected]> wrote:
> On 21-01-14 17:25:31, Jonathan Cameron wrote:
> > On Mon, 11 Jan 2021 14:51:18 -0800
> > Ben Widawsky <[email protected]> wrote:
> >
> > > CXL devices must implement the Device Command Interface (described in
> > > 8.2.9 of the CXL 2.0 spec). While the driver already maintains a list of
> > > commands it supports, there is still a need to be able to distinguish
> > > between commands that the driver knows about from commands that may not
> > > be supported by the hardware. No such commands currently are defined in
> > > the driver.
> > >
> > > The implementation leaves the statically defined table of commands and
> > > supplements it with a bitmap to determine commands that are enabled.
> > >
> > > ---
> > >
> > > There are multiple approaches that can be taken, but this is nice for a
> > > few reasons.
> > >
> > > Here are some of the other solutions:
> > >
> > > Create a per instance table with only the supported commands.
> > > 1. Having a fixed command id -> command mapping is much easier to manage
> > > for development and debugging.
> > > 2. Dealing with dynamic memory allocation for the table adds unnecessary
> > > complexity.
> > > 3. Most tables for device types are likely to be quite similar.
> > > 4. Makes it difficult to implement helper macros like cxl_for_each_cmd()
> > >
> > > If the per instance table did preserve ids, #1 above can be addressed.
> > > However, as "enable" is currently the only mutable state for the
> > > commands, it would yield a lot of overhead for not much gain.
> > > Additionally, the other issues remain.
> > >
> > > If "enable" remains the only mutable state, I believe this to be the
> > > best solution. Once the number of mutable elements in a command grows,
> > > it probably makes sense to move to per device instance state with a
> > > fixed command ID mapping.
> > Agreed with the logic.
> >
> > However, patch wise, should either drop the --- above or move this below the
> > --- after your sign off. Otherwise you run the risk of git dropping your
> > sign off and resulting complaints from anyone run validation scripts
> > of the kernel tree that check for that.
> >
> > >
> > > Signed-off-by: Ben Widawsky <[email protected]>
> > > ---
> > > drivers/cxl/cxl.h | 4 ++++
> > > drivers/cxl/mem.c | 38 +++++++++++++++++++++++++++++++++++++-
> > > 2 files changed, 41 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h
> > > index 537ac4d8e6bd..963ba30cb200 100644
> > > --- a/drivers/cxl/cxl.h
> > > +++ b/drivers/cxl/cxl.h
> > > @@ -17,6 +17,9 @@
> > >
> > > #define CXL_GET_FIELD(word, field) FIELD_GET(field##_MASK, word)
> > >
> > > +/* XXX: Arbitrary max */
> > > +#define CXL_MAX_COMMANDS 32
> >
> > If going this way, probably want a build time check that you don't
> > go beyond this value for a given command set. I haven't actually
> > thought about how to construct that but should be easy enough.
> >
>
> I think BUILD_BUG_ON of ARRAY_SIZE mem_commands is sufficient. Let me know if
> you want anything more.
That is fine.
>
> > > +
> > > /* Device (CXL 2.0 - 8.2.8.3) */
> > > #define CXLDEV_CAP_ARRAY_REG 0x0
> > > #define CXLDEV_CAP_ARRAY_CAP_ID 0
> > > @@ -88,6 +91,7 @@ struct cxl_mem {
> > > } ram;
> > >
> > > char firmware_version[0x10];
> > > + DECLARE_BITMAP(enabled_cmds, CXL_MAX_COMMANDS);
> > >
> > > /* Cap 0001h - CXL_CAP_CAP_ID_DEVICE_STATUS */
> > > struct {
> > > diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
> > > index a824cfd4342a..20b26fa2c466 100644
> > > --- a/drivers/cxl/mem.c
> > > +++ b/drivers/cxl/mem.c
> > > @@ -114,6 +114,8 @@ static struct {
> > > * would typically be used for deprecated commands.
> > > * * %CXL_CMD_FLAG_MANDATORY: Hardware must support this command. This flag is
> > > * only used internally by the driver for sanity checking.
> > > + * * %CXL_CMD_INTERNAL_FLAG_PSEUDO: This is a pseudo command which doesn't have
> > > + * a direct mapping to hardware. They are implicitly always enabled.
> > > *
> > > * The cxl_mem_command is the driver's internal representation of commands that
> > > * are supported by the driver. Some of these commands may not be supported by
> > > @@ -129,6 +131,7 @@ struct cxl_mem_command {
> > > #define CXL_CMD_INTERNAL_FLAG_NONE 0
> > > #define CXL_CMD_INTERNAL_FLAG_HIDDEN BIT(0)
> > > #define CXL_CMD_INTERNAL_FLAG_MANDATORY BIT(1)
> > > +#define CXL_CMD_INTERNAL_FLAG_PSEUDO BIT(2)
> > > };
> > >
> > > /*
> > > @@ -140,7 +143,7 @@ struct cxl_mem_command {
> > > static struct cxl_mem_command mem_commands[] = {
> > > CXL_CMD(INVALID, KERNEL, 0, 0, HIDDEN),
> > > CXL_CMD(IDENTIFY, NONE, 0, 0x43, MANDATORY),
> > > - CXL_CMD(RAW, NONE, ~0, ~0, MANDATORY),
> > > + CXL_CMD(RAW, NONE, ~0, ~0, PSEUDO),
> > > };
> > >
> > > #define cxl_for_each_cmd(cmd) \
> > > @@ -618,6 +621,10 @@ static int cxl_validate_cmd_from_user(struct cxl_mem *cxlm,
> > > c = &mem_commands[send_cmd->id];
> > > info = &c->info;
> > >
> > > + /* Check that the command is enabled for hardware */
> > > + if (!test_bit(cxl_cmd_index(c), cxlm->enabled_cmds))
> > > + return -ENOTTY;
> > > +
> > > if (info->flags & CXL_MEM_COMMAND_FLAG_KERNEL)
> > > return -EPERM;
> > >
> > > @@ -1029,6 +1036,31 @@ static int cxl_mem_add_memdev(struct cxl_mem *cxlm)
> > > return rc;
> > > }
> > >
> > > +/**
> > > + * cxl_mem_enumerate_cmds() - Enumerate commands for a device.
> > > + * @cxlm: The device.
> > > + *
> > > + * Returns 0 if enumerate completed successfully.
> > > + *
> > > + * CXL devices have optional support for certain commands. This function will
> > > + * determine the set of supported commands for the hardware and update the
> > > + * enabled_cmds bitmap in the @cxlm.
> > > + */
> > > +static int cxl_mem_enumerate_cmds(struct cxl_mem *cxlm)
> > > +{
> > > + struct cxl_mem_command *c;
> > > +
> > > + /* All commands are considered enabled for now (except INVALID). */
> > > + cxl_for_each_cmd(c) {
> > > + if (c->flags & CXL_CMD_INTERNAL_FLAG_HIDDEN)
> > > + continue;
> > > +
> > > + set_bit(cxl_cmd_index(c), cxlm->enabled_cmds);
> > > + }
> > > +
> > > + return 0;
> > > +}
> > > +
> > > /**
> > > * cxl_mem_identify() - Send the IDENTIFY command to the device.
> > > * @cxlm: The device to identify.
> > > @@ -1147,6 +1179,10 @@ static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> > > if (rc)
> > > goto err;
> > >
> > > + rc = cxl_mem_enumerate_cmds(cxlm);
> > > + if (rc)
> > > + return rc;
> > > +
> > > rc = cxl_mem_identify(cxlm);
> > > if (rc)
> > > goto err;
> >
On Thu, 21 Jan 2021 10:15:46 -0800
Ben Widawsky <[email protected]> wrote:
> On 21-01-14 17:10:38, Jonathan Cameron wrote:
> > On Mon, 11 Jan 2021 14:51:14 -0800
> > Ben Widawsky <[email protected]> wrote:
> >
> > > The send command allows userspace to issue mailbox commands directly to
> > > the hardware. The driver will verify basic properties of the command and
> > > possible inspect the input (or output) payload to determine whether or
> > > not the command is allowed (or might taint the kernel).
> > >
> > > The list of allowed commands and their properties can be determined by
> > > using the QUERY IOCTL for CXL memory devices.
> > >
> > > Signed-off-by: Ben Widawsky <[email protected]>
> > > ---
> > > drivers/cxl/mem.c | 204 ++++++++++++++++++++++++++++++++++-
> > > include/uapi/linux/cxl_mem.h | 39 +++++++
> > > 2 files changed, 239 insertions(+), 4 deletions(-)
> > >
> > > diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
> > > index d4eb3f5b9469..f979788b4d9f 100644
> > > --- a/drivers/cxl/mem.c
> > > +++ b/drivers/cxl/mem.c
> > > @@ -84,6 +84,13 @@ static DEFINE_IDR(cxl_mem_idr);
> > > /* protect cxl_mem_idr allocations */
> > > static DEFINE_MUTEX(cxl_memdev_lock);
> > >
> > > +#undef C
> > > +#define C(a, b) { b }
> >
> > I'm not following why this is here?
> >
>
> It's used for a debug message in handle_mailbox_cmd_from_user(). This is all the
> macro magic stolen from ftrace. Or, did I miss the question?
>
> > > +static struct {
> > > + const char *name;
> > > +} command_names[] = { CMDS };
> > > +#undef C
Mostly that you define it then undef it without use that I can see.
> > > +
> > > #define CXL_CMD(_id, _flags, sin, sout, f) \
> > > [CXL_MEM_COMMAND_ID_##_id] = { \
> > > { \
> > ...
> >
> > > +
> > > +/**
> > > + * handle_mailbox_cmd_from_user() - Dispatch a mailbox command.
> > > + * @cxlmd: The CXL memory device to communicate with.
> > > + * @cmd: The validated command.
> > > + * @in_payload: Pointer to userspace's input payload.
> > > + * @out_payload: Pointer to userspace's output payload.
> > > + * @u: The command submitted by userspace. Has output fields.
> > > + *
> > > + * Return:
> > > + * * %0 - Mailbox transaction succeeded.
> > > + * * %-EFAULT - Something happened with copy_to/from_user.
> > > + * * %-EINTR - Mailbox acquisition interrupted.
> > > + * * %-E2BIG - Output payload would overrun buffer.
> > > + *
> > > + * Creates the appropriate mailbox command on behalf of a userspace request.
> > > + * Return value, size, and output payload are all copied out to @u. The
> > > + * parameters for the command must be validated before calling this function.
> > > + *
> > > + * A 0 return code indicates the command executed successfully, not that it was
> > > + * itself successful. IOW, the retval should always be checked if wanting to
> >
> > cmd->retval perhaps to be more explicit?
> >
> > > + * determine the actual result.
> > > + */
> > > +static int handle_mailbox_cmd_from_user(struct cxl_memdev *cxlmd,
> > > + const struct cxl_mem_command *cmd,
> > > + u64 in_payload,
> > > + u64 out_payload,
> > > + struct cxl_send_command __user *u)
> > > +{
> > > + struct mbox_cmd mbox_cmd = {
> > > + .opcode = cmd->opcode,
> > > + .size_in = cmd->info.size_in,
> > > + .payload = NULL, /* Copied by copy_to|from_user() */
> > > + };
> > > + int rc;
> > > +
> > > + if (cmd->info.size_in) {
> > > + /*
> > > + * Directly copy the userspace payload into the hardware. UAPI
> > > + * states that the buffer must already be little endian.
> > > + */
> > > + if (copy_from_user((__force void *)cxl_payload_regs(cxlmd->cxlm),
> > > + u64_to_user_ptr(in_payload),
> > > + cmd->info.size_in)) {
> > > + cxl_mem_mbox_put(cxlmd->cxlm);
> >
> > mbox_get is after this point though it shouldn't be given we just
> > wrote into the mbox registers.
> >
> > This seems unlikely to be a high performance path, so perhaps just
> > use a local buffer and let cxl_mem_mbox_send_cmd copy it into the registers.
> >
>
> You're correct about the get() needing to be first. I will fix it. As for
> performance path - so while this does potentially help with performance, it
> actually ends up being I think a little cleaner to not have to deal with a local
> buffer.
>
> How strongly do you feel about it? I'd say if you don't care so much, let's keep
> it as is and find a reason to undo later.
A slightly interesting corner. The fact that there are no other cases of this
particular sequence in kernel bothered me... It's more than possible I've
missed something in the following.
So with a bounce buffered we'd have
copy_from_user()
then
memcpy_toio()
here we end loosing the fact that memcpy_to_io() might not be a 'simple' memcpy().
In the generic asm form it's just a (__force void *) like you have here done using
__io_virt() (which might make sense here if you keep this, to make it clear
what's going on)
However, not all architectures are using the generic form of memcpy_toio()
and even if the ones we care about are safe today using the above construct,
it's more than possible some future architecture might be more 'exciting'.
So basically I'm doubtful that this construct is safe.
Jonathan
On 21-01-22 11:43:57, Jonathan Cameron wrote:
> On Thu, 21 Jan 2021 10:15:46 -0800
> Ben Widawsky <[email protected]> wrote:
>
> > On 21-01-14 17:10:38, Jonathan Cameron wrote:
> > > On Mon, 11 Jan 2021 14:51:14 -0800
> > > Ben Widawsky <[email protected]> wrote:
> > >
> > > > The send command allows userspace to issue mailbox commands directly to
> > > > the hardware. The driver will verify basic properties of the command and
> > > > possible inspect the input (or output) payload to determine whether or
> > > > not the command is allowed (or might taint the kernel).
> > > >
> > > > The list of allowed commands and their properties can be determined by
> > > > using the QUERY IOCTL for CXL memory devices.
> > > >
> > > > Signed-off-by: Ben Widawsky <[email protected]>
> > > > ---
> > > > drivers/cxl/mem.c | 204 ++++++++++++++++++++++++++++++++++-
> > > > include/uapi/linux/cxl_mem.h | 39 +++++++
> > > > 2 files changed, 239 insertions(+), 4 deletions(-)
> > > >
> > > > diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
> > > > index d4eb3f5b9469..f979788b4d9f 100644
> > > > --- a/drivers/cxl/mem.c
> > > > +++ b/drivers/cxl/mem.c
> > > > @@ -84,6 +84,13 @@ static DEFINE_IDR(cxl_mem_idr);
> > > > /* protect cxl_mem_idr allocations */
> > > > static DEFINE_MUTEX(cxl_memdev_lock);
> > > >
> > > > +#undef C
> > > > +#define C(a, b) { b }
> > >
> > > I'm not following why this is here?
> > >
> >
> > It's used for a debug message in handle_mailbox_cmd_from_user(). This is all the
> > macro magic stolen from ftrace. Or, did I miss the question?
> >
> > > > +static struct {
> > > > + const char *name;
> > > > +} command_names[] = { CMDS };
> > > > +#undef C
>
> Mostly that you define it then undef it without use that I can see.
>
> > > > +
> > > > #define CXL_CMD(_id, _flags, sin, sout, f) \
> > > > [CXL_MEM_COMMAND_ID_##_id] = { \
> > > > { \
> > > ...
> > >
> > > > +
> > > > +/**
> > > > + * handle_mailbox_cmd_from_user() - Dispatch a mailbox command.
> > > > + * @cxlmd: The CXL memory device to communicate with.
> > > > + * @cmd: The validated command.
> > > > + * @in_payload: Pointer to userspace's input payload.
> > > > + * @out_payload: Pointer to userspace's output payload.
> > > > + * @u: The command submitted by userspace. Has output fields.
> > > > + *
> > > > + * Return:
> > > > + * * %0 - Mailbox transaction succeeded.
> > > > + * * %-EFAULT - Something happened with copy_to/from_user.
> > > > + * * %-EINTR - Mailbox acquisition interrupted.
> > > > + * * %-E2BIG - Output payload would overrun buffer.
> > > > + *
> > > > + * Creates the appropriate mailbox command on behalf of a userspace request.
> > > > + * Return value, size, and output payload are all copied out to @u. The
> > > > + * parameters for the command must be validated before calling this function.
> > > > + *
> > > > + * A 0 return code indicates the command executed successfully, not that it was
> > > > + * itself successful. IOW, the retval should always be checked if wanting to
> > >
> > > cmd->retval perhaps to be more explicit?
> > >
> > > > + * determine the actual result.
> > > > + */
> > > > +static int handle_mailbox_cmd_from_user(struct cxl_memdev *cxlmd,
> > > > + const struct cxl_mem_command *cmd,
> > > > + u64 in_payload,
> > > > + u64 out_payload,
> > > > + struct cxl_send_command __user *u)
> > > > +{
> > > > + struct mbox_cmd mbox_cmd = {
> > > > + .opcode = cmd->opcode,
> > > > + .size_in = cmd->info.size_in,
> > > > + .payload = NULL, /* Copied by copy_to|from_user() */
> > > > + };
> > > > + int rc;
> > > > +
> > > > + if (cmd->info.size_in) {
> > > > + /*
> > > > + * Directly copy the userspace payload into the hardware. UAPI
> > > > + * states that the buffer must already be little endian.
> > > > + */
> > > > + if (copy_from_user((__force void *)cxl_payload_regs(cxlmd->cxlm),
> > > > + u64_to_user_ptr(in_payload),
> > > > + cmd->info.size_in)) {
> > > > + cxl_mem_mbox_put(cxlmd->cxlm);
> > >
> > > mbox_get is after this point though it shouldn't be given we just
> > > wrote into the mbox registers.
> > >
> > > This seems unlikely to be a high performance path, so perhaps just
> > > use a local buffer and let cxl_mem_mbox_send_cmd copy it into the registers.
> > >
> >
> > You're correct about the get() needing to be first. I will fix it. As for
> > performance path - so while this does potentially help with performance, it
> > actually ends up being I think a little cleaner to not have to deal with a local
> > buffer.
> >
> > How strongly do you feel about it? I'd say if you don't care so much, let's keep
> > it as is and find a reason to undo later.
>
> A slightly interesting corner. The fact that there are no other cases of this
> particular sequence in kernel bothered me... It's more than possible I've
> missed something in the following.
>
> So with a bounce buffered we'd have
> copy_from_user()
> then
> memcpy_toio()
>
> here we end loosing the fact that memcpy_to_io() might not be a 'simple' memcpy().
> In the generic asm form it's just a (__force void *) like you have here done using
> __io_virt() (which might make sense here if you keep this, to make it clear
> what's going on)
>
> However, not all architectures are using the generic form of memcpy_toio()
> and even if the ones we care about are safe today using the above construct,
> it's more than possible some future architecture might be more 'exciting'.
>
> So basically I'm doubtful that this construct is safe.
>
> Jonathan
>
Sounds reasonable.
Thanks for digging. I'll go back to the bounce buffer in v4.
On 21-01-14 18:08:26, Jonathan Cameron wrote:
> On Mon, 11 Jan 2021 14:51:20 -0800
> Ben Widawsky <[email protected]> wrote:
>
> > The Get Log command returns the actual log entries that are advertised
> > via the Get Supported Logs command (0400h). CXL device logs are selected
> > by UUID which is part of the CXL spec. Because the driver tries to
> > sanitize what is sent to hardware, there becomes a need to restrict the
> > types of logs which can be accessed by userspace. For example, the
> > vendor specific log might only be consumable by proprietary, or offline
> > applications, and therefore a good candidate for userspace.
> >
> > The current driver infrastructure does allow basic validation for all
> > commands, but doesn't inspect any of the payload data. Along with Get
> > Log support comes new infrastructure to add a hook for payload
> > validation. This infrastructure is used to filter out the CEL UUID,
> > which the userspace driver doesn't have business knowing, and taints on
> > invalid UUIDs being sent to hardware.
> >
> > Signed-off-by: Ben Widawsky <[email protected]>
>
> Just a minor question for this one.
>
> Thanks, J
> ... \
> > @@ -515,6 +529,15 @@ static int handle_mailbox_cmd_from_user(struct cxl_memdev *cxlmd,
> > int rc;
> >
> > if (cmd->info.size_in) {
> > + if (cmd->validate_payload) {
> > + rc = cmd->validate_payload(u64_to_user_ptr(in_payload),
> > + cmd->info.size_in);
>
> Is it worth moving this out of the region in which we hold the mbox?
> (after fixing the bug that I think means we don't actually hold it at this point)
>
> Perhaps not, but it does feel odd to do validation under the lock.
>
>
When moving to a bounce buffer the locking resolves itself and ultimately this
doesn't happen under lock anymore.
if (cmd->info.size_in) {
if (cmd->validate_payload) {
rc = cmd->validate_payload(u64_to_user_ptr(in_payload),
cmd->info.size_in);
if (rc)
goto out;
}
mbox_cmd.payload_in = kvzalloc(cmd->info.size_in, GFP_KERNEL);
if (!mbox_cmd.payload_in) {
rc = -ENOMEM;
goto out;
}
if (copy_from_user(mbox_cmd.payload_in,
u64_to_user_ptr(in_payload),
cmd->info.size_in)) {
rc = -EFAULT;
goto out;
}
}
rc = cxl_mem_mbox_get(cxlm, true);
> > + if (rc) {
> > + cxl_mem_mbox_put(cxlmd->cxlm);
> > + return -EFAULT;
> > + }
> > + }
> > +
> > /*
> > * Directly copy the userspace payload into the hardware. UAPI
> > * states that the buffer must already be little endian.