Add firmware update capability to the CXL memdev driver, and emulation
in cxl_test. Since the 'Transfer FW' mailbox command is a background
command, this series depends on background command support [1].
Since Transfer FW can be a long-running background command, it is
desirable to retain kernel control over it, so that one command doesn't
monopolize the mailbox interface. The sysfs based firmware loader
mechanism that was developed for FPGAs is a suitable candidate to help
accomplish this. Patch 1 goes into more detail on this.
The poll interval for the Transfer FW command is arbitrarily set at 1
second, and a poll count of 30, giving us a total wait time of thirty
seconds before which each slice of the transfer times out. This seems
like a good mix of responsiveness and a total wait - the spec doesn't
have any guidance on any upper or lower bounds for this. This likely
does not need to be user-configurable, so for now it is just hard-coded
in the driver.
Patch 2 is a fix for the Poison commands effect population found while
developing these.
Patch 3 is a cleanup to give names to command effects.
Patch 4 implements the emulation of firmware update related commands in
the cxl_test environment to enable unit testing.
[1]: https://git.kernel.org/pub/scm/linux/kernel/git/cxl/cxl.git/log/?h=for-6.5/cxl-background
Signed-off-by: Vishal Verma <[email protected]>
---
Changes in v4:
- Link to v3: https://lore.kernel.org/r/[email protected]
- Collect review tags (Dave, Jonathan)
- Remove a double free (Jonathan)
- Remove stray whitespace (Jonathan)
Changes in v3:
- Link to v2: https://lore.kernel.org/r/[email protected]
- Collect review tags (Alison, Jonathan)
- Remove unused leftover fw_name variable from the previous rev (Jonathan)
- Add a comment to clarify cur_size calculation and alignment (Jonathan)
- Clean up error return using dev_err_probe() (Jonathan)
- Remove unintended error handling change leftover from v1 (Jonathan)
- Refactor returning from the switch() in mock_activate_fw() (Jonathan)
Changes in v2:
- Link to v1: https://lore.kernel.org/r/[email protected]
- Make command field names more consistent with the spec (Jonathan)
- Remove pointers to info/activate/transfer from fw state (Jonathan)
- Use struct_size() insted of open coding it (Jonathan)
- Fix a memory leak with 'transfer' in cxl_fw_write() (Jonathan)
- Move fw setup/teardown into a devm action (Jonathan, Dan)
- Use decimals in command struct definitions to match the spec (Alison)
- Move fw loader registration up into cxl_pci_probe() (Dan)
- Remove the fw_name cargo cult (Dan)
- Change timeout for each transfer chunk to 30s (Dan)
- Add a timeout in cxl_fw_write() as well (not just in abort)
- Use IS_ALIGNED() instead of open coding it (Dan)
- Clean up cur_size calculation in ->write() (Dan)
- Remove the lock and 'clear_to_send (Dan)
- Use atomic bitops for the cancel flag (Dan)
- Gate fw loader setup on command availability (Dan)
- Clean up the effects list in cxl_test using defines (Jonathan)
- Use the sha256 routine in include/crypto/sha2.h (Jonathan)
---
Vishal Verma (4):
cxl: add a firmware update mechanism using the sysfs firmware loader
tools/testing/cxl: Fix command effects for inject/clear poison
tools/testing/cxl: Use named effects for the Command Effect Log
tools/testing/cxl: add firmware update emulation to CXL memdevs
drivers/cxl/cxlmem.h | 82 +++++++++
drivers/cxl/core/memdev.c | 308 ++++++++++++++++++++++++++++++++
drivers/cxl/pci.c | 4 +
tools/testing/cxl/test/mem.c | 192 +++++++++++++++++++-
Documentation/ABI/testing/sysfs-bus-cxl | 11 ++
drivers/cxl/Kconfig | 1 +
6 files changed, 589 insertions(+), 9 deletions(-)
---
base-commit: ccadf1310fb0bc8d2cbcd14f94a6279c12ea9bee
change-id: 20230602-vv-fw_update-e1c96a60c687
Best regards,
--
Vishal Verma <[email protected]>
Add emulation for the 'Get FW Info', 'Transfer FW', and 'Activate FW'
CXL mailbox commands to the cxl_test emulated memdevs to enable
end-to-end unit testing of a firmware update flow. For now, only
advertise an 'offline activation' capability as that is all the CXL
memdev driver currently implements.
Add some canned values for the serial number fields, and create a
platform device sysfs knob to calculate the sha256sum of the firmware
image that was received, so a unit test can compare it with the original
file that was uploaded.
Cc: Davidlohr Bueso <[email protected]>
Cc: Jonathan Cameron <[email protected]>
Cc: Russ Weight <[email protected]>
Cc: Alison Schofield <[email protected]>
Cc: Ira Weiny <[email protected]>
Cc: Dave Jiang <[email protected]>
Cc: Ben Widawsky <[email protected]>
Cc: Dan Williams <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Reviewed-by: Dave Jiang <[email protected]>
Signed-off-by: Vishal Verma <[email protected]>
---
tools/testing/cxl/test/mem.c | 160 +++++++++++++++++++++++++++++++++++++++++++
1 file changed, 160 insertions(+)
diff --git a/tools/testing/cxl/test/mem.c b/tools/testing/cxl/test/mem.c
index 68668d8df1cd..1166f470e0c7 100644
--- a/tools/testing/cxl/test/mem.c
+++ b/tools/testing/cxl/test/mem.c
@@ -8,11 +8,14 @@
#include <linux/sizes.h>
#include <linux/bits.h>
#include <asm/unaligned.h>
+#include <crypto/sha2.h>
#include <cxlmem.h>
#include "trace.h"
#define LSA_SIZE SZ_128K
+#define FW_SIZE SZ_64M
+#define FW_SLOTS 3
#define DEV_SIZE SZ_2G
#define EFFECT(x) (1U << x)
@@ -72,6 +75,20 @@ static struct cxl_cel_entry mock_cel[] = {
.opcode = cpu_to_le16(CXL_MBOX_OP_CLEAR_POISON),
.effect = cpu_to_le16(EFFECT(DATA_CHANGE_IMMEDIATE)),
},
+ {
+ .opcode = cpu_to_le16(CXL_MBOX_OP_GET_FW_INFO),
+ .effect = CXL_CMD_EFFECT_NONE,
+ },
+ {
+ .opcode = cpu_to_le16(CXL_MBOX_OP_TRANSFER_FW),
+ .effect = cpu_to_le16(EFFECT(CONF_CHANGE_COLD_RESET) |
+ EFFECT(BACKGROUND_OP)),
+ },
+ {
+ .opcode = cpu_to_le16(CXL_MBOX_OP_ACTIVATE_FW),
+ .effect = cpu_to_le16(EFFECT(CONF_CHANGE_COLD_RESET) |
+ EFFECT(CONF_CHANGE_IMMEDIATE)),
+ },
};
/* See CXL 2.0 Table 181 Get Health Info Output Payload */
@@ -123,6 +140,10 @@ struct mock_event_store {
struct cxl_mockmem_data {
void *lsa;
+ void *fw;
+ int fw_slot;
+ int fw_staged;
+ size_t fw_size;
u32 security_state;
u8 user_pass[NVDIMM_PASSPHRASE_LEN];
u8 master_pass[NVDIMM_PASSPHRASE_LEN];
@@ -1128,6 +1149,87 @@ static struct attribute *cxl_mock_mem_core_attrs[] = {
};
ATTRIBUTE_GROUPS(cxl_mock_mem_core);
+static int mock_fw_info(struct cxl_dev_state *cxlds,
+ struct cxl_mbox_cmd *cmd)
+{
+ struct cxl_mockmem_data *mdata = dev_get_drvdata(cxlds->dev);
+ struct cxl_mbox_get_fw_info fw_info = {
+ .num_slots = FW_SLOTS,
+ .slot_info = (mdata->fw_slot & 0x7) |
+ ((mdata->fw_staged & 0x7) << 3),
+ .activation_cap = 0,
+ };
+
+ strcpy(fw_info.slot_1_revision, "cxl_test_fw_001");
+ strcpy(fw_info.slot_2_revision, "cxl_test_fw_002");
+ strcpy(fw_info.slot_3_revision, "cxl_test_fw_003");
+ strcpy(fw_info.slot_4_revision, "");
+
+ if (cmd->size_out < sizeof(fw_info))
+ return -EINVAL;
+
+ memcpy(cmd->payload_out, &fw_info, sizeof(fw_info));
+ return 0;
+}
+
+static int mock_transfer_fw(struct cxl_dev_state *cxlds,
+ struct cxl_mbox_cmd *cmd)
+{
+ struct cxl_mbox_transfer_fw *transfer = cmd->payload_in;
+ struct cxl_mockmem_data *mdata = dev_get_drvdata(cxlds->dev);
+ void *fw = mdata->fw;
+ size_t offset, length;
+
+ offset = le32_to_cpu(transfer->offset) * CXL_FW_TRANSFER_ALIGNMENT;
+ length = cmd->size_in - sizeof(*transfer);
+ if (offset + length > FW_SIZE)
+ return -EINVAL;
+
+ switch (transfer->action) {
+ case CXL_FW_TRANSFER_ACTION_FULL:
+ if (offset != 0)
+ return -EINVAL;
+ fallthrough;
+ case CXL_FW_TRANSFER_ACTION_END:
+ if (transfer->slot == 0 || transfer->slot > FW_SLOTS)
+ return -EINVAL;
+ mdata->fw_size = offset + length;
+ break;
+ case CXL_FW_TRANSFER_ACTION_INITIATE:
+ case CXL_FW_TRANSFER_ACTION_CONTINUE:
+ break;
+ case CXL_FW_TRANSFER_ACTION_ABORT:
+ return 0;
+ default:
+ return -EINVAL;
+ }
+
+ memcpy(fw + offset, transfer->data, length);
+ return 0;
+}
+
+static int mock_activate_fw(struct cxl_dev_state *cxlds,
+ struct cxl_mbox_cmd *cmd)
+{
+ struct cxl_mbox_activate_fw *activate = cmd->payload_in;
+ struct cxl_mockmem_data *mdata = dev_get_drvdata(cxlds->dev);
+
+ if (activate->slot == 0 || activate->slot > FW_SLOTS)
+ return -EINVAL;
+
+ switch (activate->action) {
+ case CXL_FW_ACTIVATE_ONLINE:
+ mdata->fw_slot = activate->slot;
+ mdata->fw_staged = 0;
+ return 0;
+ case CXL_FW_ACTIVATE_OFFLINE:
+ mdata->fw_staged = activate->slot;
+ return 0;
+ }
+
+ return -EINVAL;
+}
+
static int cxl_mock_mbox_send(struct cxl_dev_state *cxlds, struct cxl_mbox_cmd *cmd)
{
struct device *dev = cxlds->dev;
@@ -1194,6 +1296,15 @@ static int cxl_mock_mbox_send(struct cxl_dev_state *cxlds, struct cxl_mbox_cmd *
case CXL_MBOX_OP_CLEAR_POISON:
rc = mock_clear_poison(cxlds, cmd);
break;
+ case CXL_MBOX_OP_GET_FW_INFO:
+ rc = mock_fw_info(cxlds, cmd);
+ break;
+ case CXL_MBOX_OP_TRANSFER_FW:
+ rc = mock_transfer_fw(cxlds, cmd);
+ break;
+ case CXL_MBOX_OP_ACTIVATE_FW:
+ rc = mock_activate_fw(cxlds, cmd);
+ break;
default:
break;
}
@@ -1209,6 +1320,11 @@ static void label_area_release(void *lsa)
vfree(lsa);
}
+static void fw_buf_release(void *buf)
+{
+ vfree(buf);
+}
+
static bool is_rcd(struct platform_device *pdev)
{
const struct platform_device_id *id = platform_get_device_id(pdev);
@@ -1241,10 +1357,19 @@ static int cxl_mock_mem_probe(struct platform_device *pdev)
mdata->lsa = vmalloc(LSA_SIZE);
if (!mdata->lsa)
return -ENOMEM;
+ mdata->fw = vmalloc(FW_SIZE);
+ if (!mdata->fw)
+ return -ENOMEM;
+ mdata->fw_slot = 2;
+
rc = devm_add_action_or_reset(dev, label_area_release, mdata->lsa);
if (rc)
return rc;
+ rc = devm_add_action_or_reset(dev, fw_buf_release, mdata->fw);
+ if (rc)
+ return rc;
+
cxlds = cxl_dev_state_create(dev);
if (IS_ERR(cxlds))
return PTR_ERR(cxlds);
@@ -1286,6 +1411,10 @@ static int cxl_mock_mem_probe(struct platform_device *pdev)
if (IS_ERR(cxlmd))
return PTR_ERR(cxlmd);
+ rc = cxl_memdev_setup_fw_upload(cxlds);
+ if (rc)
+ return rc;
+
cxl_mem_get_event_records(cxlds, CXLDEV_EVENT_STATUS_ALL);
return 0;
@@ -1324,9 +1453,40 @@ static ssize_t security_lock_store(struct device *dev, struct device_attribute *
static DEVICE_ATTR_RW(security_lock);
+static ssize_t fw_buf_checksum_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct cxl_mockmem_data *mdata = dev_get_drvdata(dev);
+ u8 hash[SHA256_DIGEST_SIZE];
+ unsigned char *hstr, *hptr;
+ struct sha256_state sctx;
+ ssize_t written = 0;
+ int i;
+
+ sha256_init(&sctx);
+ sha256_update(&sctx, mdata->fw, mdata->fw_size);
+ sha256_final(&sctx, hash);
+
+ hstr = kzalloc((SHA256_DIGEST_SIZE * 2) + 1, GFP_KERNEL);
+ if (!hstr)
+ return -ENOMEM;
+
+ hptr = hstr;
+ for (i = 0; i < SHA256_DIGEST_SIZE; i++)
+ hptr += sprintf(hptr, "%02x", hash[i]);
+
+ written = sysfs_emit(buf, "%s\n", hstr);
+
+ kfree(hstr);
+ return written;
+}
+
+static DEVICE_ATTR_RO(fw_buf_checksum);
+
static struct attribute *cxl_mock_mem_attrs[] = {
&dev_attr_security_lock.attr,
&dev_attr_event_trigger.attr,
+ &dev_attr_fw_buf_checksum.attr,
NULL
};
ATTRIBUTE_GROUPS(cxl_mock_mem);
--
2.40.1
The CXL spec (3.0, section 8.2.9.8.4) Lists Inject Poison and Clear
Poison as having the effects of "Immediate Data Change". Fix this in the
mock driver so that the command effect log is populated correctly.
Fixes: 371c16101ee8 ("tools/testing/cxl: Mock the Inject Poison mailbox command")
Cc: Alison Schofield <[email protected]>
Cc: Dan Williams <[email protected]>
Reviewed-by: Alison Schofield <[email protected]>
Reviewed-by: Jonathan Cameron <[email protected]>
Reviewed-by: Dave Jiang <[email protected]>
Signed-off-by: Vishal Verma <[email protected]>
---
tools/testing/cxl/test/mem.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tools/testing/cxl/test/mem.c b/tools/testing/cxl/test/mem.c
index 34b48027b3de..403cd3608772 100644
--- a/tools/testing/cxl/test/mem.c
+++ b/tools/testing/cxl/test/mem.c
@@ -52,11 +52,11 @@ static struct cxl_cel_entry mock_cel[] = {
},
{
.opcode = cpu_to_le16(CXL_MBOX_OP_INJECT_POISON),
- .effect = cpu_to_le16(0),
+ .effect = cpu_to_le16(EFFECT(2)),
},
{
.opcode = cpu_to_le16(CXL_MBOX_OP_CLEAR_POISON),
- .effect = cpu_to_le16(0),
+ .effect = cpu_to_le16(EFFECT(2)),
},
};
--
2.40.1