by Jonathan Cameron

[permalink] [raw]

Subject: Re: [PATCH V16 3/6] PCI/DOE: Add DOE mailbox support functions

On Tue, 19 Jul 2022 13:52:46 -0700
[email protected] wrote:

> From: Jonathan Cameron <[email protected]>
>
> Introduced in a PCIe r6.0, sec 6.30, DOE provides a config space based
> mailbox with standard protocol discovery. Each mailbox is accessed
> through a DOE Extended Capability.
>
> Each DOE mailbox must support the DOE discovery protocol in addition to
> any number of additional protocols.
>
> Define core PCIe functionality to manage a single PCIe DOE mailbox at a
> defined config space offset. Functionality includes iterating,
> creating, query of supported protocol, and task submission. Destruction
> of the mailboxes is device managed.
>
> Cc: "Li, Ming" <[email protected]>
> Cc: Bjorn Helgaas <[email protected]>
> Cc: Matthew Wilcox <[email protected]>
> Acked-by: Bjorn Helgaas <[email protected]>
> Signed-off-by: Jonathan Cameron <[email protected]>
> Co-developed-by: Ira Weiny <[email protected]>
> Signed-off-by: Ira Weiny <[email protected]>

FYI. Gregory Price reported an an issue that I think
is related to calling INIT_WORK() rather than INIT_WORK_ONSTACK()
and associated debug options in his build.

https://lore.kernel.org/linux-cxl/[email protected]/T/#m88a7f50dcce52f30c8bf5c3dcc06fa9843b54a2d

I've highlighted one path to this below.

> diff --git a/drivers/pci/doe.c b/drivers/pci/doe.c
> new file mode 100644
> index 000000000000..e402f05068a5
> --- /dev/null
> +++ b/drivers/pci/doe.c
> @@ -0,0 +1,536 @@

> +static int pci_doe_discovery(struct pci_doe_mb *doe_mb, u8 *index, u16 *vid,
> + u8 *protocol)
> +{
> + u32 request_pl = FIELD_PREP(PCI_DOE_DATA_OBJECT_DISC_REQ_3_INDEX,
> + *index);
> + u32 response_pl;
> + DECLARE_COMPLETION_ONSTACK(c);
> + struct pci_doe_task task = {
> + .prot.vid = PCI_VENDOR_ID_PCI_SIG,
> + .prot.type = PCI_DOE_PROTOCOL_DISCOVERY,
> + .request_pl = &request_pl,
> + .request_pl_sz = sizeof(request_pl),
> + .response_pl = &response_pl,
> + .response_pl_sz = sizeof(response_pl),
> + .complete = pci_doe_task_complete,
> + .private = &c,
> + };
This structure contains a work_struct and is on the stack. However...

> + int rc;
> +
> + rc = pci_doe_submit_task(doe_mb, &task);
> + if (rc < 0)
> + return rc;
> +
> + wait_for_completion(&c);
> +
> + if (task.rv != sizeof(response_pl))
> + return -EIO;
> +
> + *vid = FIELD_GET(PCI_DOE_DATA_OBJECT_DISC_RSP_3_VID, response_pl);
> + *protocol = FIELD_GET(PCI_DOE_DATA_OBJECT_DISC_RSP_3_PROTOCOL,
> + response_pl);
> + *index = FIELD_GET(PCI_DOE_DATA_OBJECT_DISC_RSP_3_NEXT_INDEX,
> + response_pl);
> +
> + return 0;
> +}

...

> +int pci_doe_submit_task(struct pci_doe_mb *doe_mb, struct pci_doe_task *task)
> +{
> + if (!pci_doe_supports_prot(doe_mb, task->prot.vid, task->prot.type))
> + return -EINVAL;
> +
> + /*
> + * DOE requests must be a whole number of DW and the response needs to
> + * be big enough for at least 1 DW
> + */
> + if (task->request_pl_sz % sizeof(u32) ||
> + task->response_pl_sz < sizeof(u32))
> + return -EINVAL;
> +
> + if (test_bit(PCI_DOE_FLAG_DEAD, &doe_mb->flags))
> + return -EIO;
> +
> + task->doe_mb = doe_mb;
> + INIT_WORK(&task->work, doe_statemachine_work);

Here we don't call the INIT_WORK_ONSTACK() Variant.

> + queue_work(doe_mb->work_queue, &task->work);
> + return 0;
> +}
> +EXPORT_SYMBOL_GPL(pci_doe_submit_task);