2022-06-30 18:01:32

by Cristian Marussi

[permalink] [raw]
Subject: [PATCH v2 0/2] Add SCMI full message tracing

Hi,

after a few recent troubles handling too strictly out-of-spec replies from
SCMI servers deployed in the wild, I though it could have been useful to
have a basic way to dump at will the effective full payloads of
successfully transmitted/received SCMI messages.

The existing SCMI traces already collect a bunch of information about SCMI
message exchanges but they do NOT keep any payload information: this is
certainly preferable most of the time since dumping full SCMI messages to
the trace buffer involves a full copy of the payload.

For this reason I added a new distinct trace_scmi_msg_dump with this series
in order to be able to selectively enable at will message dumping only when
required.

Only successfully transmitted and received (valid) xfers are dumped.

Following the advice from Jim, I added some parsing/interpretation of the
header, so that the final result is something like:

root@deb-guest:~# echo 1 > /sys/kernel/debug/tracing/events/scmi/scmi_msg_dump/enable
root@deb-guest:~# cat /sys/kernel/debug/tracing/trace_pipe

++ cmd/reply
sugov:0-257 [000] ..... 401.954788: scmi_msg_dump: pt=13 t=CMND msg_id=07 seq=0082 s=0 pyld=0000000000000000
<idle>-0 [000] d.h2. 401.955085: scmi_msg_dump: pt=13 t=RESP msg_id=07 seq=0082 s=0 pyld=


++ cmd/reply/delayed
cat-263 [001] ..... 471.533806: scmi_msg_dump: pt=15 t=CMND msg_id=06 seq=008A s=0 pyld=0100000001000000
<idle>-0 [000] d.h2. 471.554001: scmi_msg_dump: pt=15 t=RESP msg_id=06 seq=008A s=0 pyld=
<idle>-0 [000] d.h2. 471.574102: scmi_msg_dump: pt=15 t=DLYD msg_id=06 seq=008A s=0 pyld=01000000a05a320000000000efbeaddefecafeca


++ enable notif/notif
iio_generic_buf-282 [000] ..... 535.327307: scmi_msg_dump: pt=15 t=CMND msg_id=0A seq=00AB s=0 pyld=0800000003000000
<idle>-0 [000] d.h2. 535.327561: scmi_msg_dump: pt=15 t=RESP msg_id=0A seq=00AB s=0 pyld=00000000
<idle>-0 [000] d.h2. 535.334421: scmi_msg_dump: pt=15 t=NOTI msg_id=01 seq=0000 s=0 pyld=000000000800000008daffffffffffff008268d4c075fd1610daffffffffffff008268d4c075fd1618daffffffffffff008268d4c075fd16
<idle>-0 [000] d.h2. 535.434649: scmi_msg_dump: pt=15 t=NOTI msg_id=01 seq=0000 s=0 pyld=000000000800000009daffffffffffff008268d4c075fd1611daffffffffffff008268d4c075fd1619daffffffffffff008268d4c075fd16

Payload is dumped as it comes through byte-by-byte without any endianity
conversion to avoid further load on the system.

Thanks,
Cristian

---
V1 --> V2
- changed dumping format

Cristian Marussi (2):
include: trace: Add SCMI full message tracing
firmware: arm_scmi: Use new SCMI full message tracing

drivers/firmware/arm_scmi/driver.c | 21 ++++++++++++++++++++
include/trace/events/scmi.h | 31 ++++++++++++++++++++++++++++++
2 files changed, 52 insertions(+)

--
2.32.0


2022-07-01 14:25:30

by Sudeep Holla

[permalink] [raw]
Subject: Re: [PATCH v2 0/2] Add SCMI full message tracing

On Thu, Jun 30, 2022 at 06:31:33PM +0100, Cristian Marussi wrote:
> Hi,
>
> after a few recent troubles handling too strictly out-of-spec replies from
> SCMI servers deployed in the wild, I though it could have been useful to
> have a basic way to dump at will the effective full payloads of
> successfully transmitted/received SCMI messages.
>
> The existing SCMI traces already collect a bunch of information about SCMI
> message exchanges but they do NOT keep any payload information: this is
> certainly preferable most of the time since dumping full SCMI messages to
> the trace buffer involves a full copy of the payload.
>
> For this reason I added a new distinct trace_scmi_msg_dump with this series
> in order to be able to selectively enable at will message dumping only when
> required.
>
> Only successfully transmitted and received (valid) xfers are dumped.

Looks good to me. I would like to hear from Jim if possible. I plan to
merge this ASAP.

--
Regards,
Sudeep

2022-07-01 15:06:50

by Jim Quinlan

[permalink] [raw]
Subject: Re: [PATCH v2 0/2] Add SCMI full message tracing

Hi,
LGTM, I appreciate this change. For testing, I fed your sample output to this:

perl -MData::Dumper -nae 's/^.+scmi_msg_dump:// && print Dumper
{ (map { split /=/; } split" ") }' < scmi.txt

which gives this

$VAR1 = {
'pt' => '15',
'seq' => '0000',
'pyld' =>
'000000000800000009daffffffffffff008268d4c075fd1611daffffffffffff008268d4c075fd1619daffffffffffff008268d4c075fd16',
'msg_id' => '01',
's' => '0',
't' => 'NOTI'
};
which is what I wanted. BTW, if I ever write a script that would
take this SCMI trace text and
decode it to nice cmd names and arguments -- where would I submit such a script?

Thanks and Regards,
Jim Quinlan
Broadcom STB


On Fri, Jul 1, 2022 at 9:54 AM Sudeep Holla <[email protected]> wrote:
>
> On Thu, Jun 30, 2022 at 06:31:33PM +0100, Cristian Marussi wrote:
> > Hi,
> >
> > after a few recent troubles handling too strictly out-of-spec replies from
> > SCMI servers deployed in the wild, I though it could have been useful to
> > have a basic way to dump at will the effective full payloads of
> > successfully transmitted/received SCMI messages.
> >
> > The existing SCMI traces already collect a bunch of information about SCMI
> > message exchanges but they do NOT keep any payload information: this is
> > certainly preferable most of the time since dumping full SCMI messages to
> > the trace buffer involves a full copy of the payload.
> >
> > For this reason I added a new distinct trace_scmi_msg_dump with this series
> > in order to be able to selectively enable at will message dumping only when
> > required.
> >
> > Only successfully transmitted and received (valid) xfers are dumped.
>
> Looks good to me. I would like to hear from Jim if possible. I plan to
> merge this ASAP.
>
> --
> Regards,
> Sudeep


Attachments:
smime.p7s (4.11 kB)
S/MIME Cryptographic Signature

2022-07-01 15:29:12

by Sudeep Holla

[permalink] [raw]
Subject: Re: [PATCH v2 0/2] Add SCMI full message tracing

On Fri, Jul 01, 2022 at 11:03:38AM -0400, Jim Quinlan wrote:
> Hi,
> LGTM, I appreciate this change. For testing, I fed your sample output to this:
>

Thanks!

> perl -MData::Dumper -nae 's/^.+scmi_msg_dump:// && print Dumper
> { (map { split /=/; } split" ") }' < scmi.txt
>
> which gives this
>
> $VAR1 = {
> 'pt' => '15',
> 'seq' => '0000',
> 'pyld' =>
> '000000000800000009daffffffffffff008268d4c075fd1611daffffffffffff008268d4c075fd1619daffffffffffff008268d4c075fd16',
> 'msg_id' => '01',
> 's' => '0',
> 't' => 'NOTI'
> };
> which is what I wanted. BTW, if I ever write a script that would
> take this SCMI trace text and
> decode it to nice cmd names and arguments -- where would I submit such a script?
>

I am not 100% sure. After reading this, the first thoughts were scripts/
or tools/scripts but looking at the content I am less sure about latter.
Worth posting and then check what people have to say once this lands upstream.
Or even once this is merged into -next officially.

--
Regards,
Sudeep

2022-07-06 09:43:22

by Sudeep Holla

[permalink] [raw]
Subject: Re: [PATCH v2 0/2] Add SCMI full message tracing

On Thu, 30 Jun 2022 18:31:33 +0100, Cristian Marussi wrote:
> after a few recent troubles handling too strictly out-of-spec replies from
> SCMI servers deployed in the wild, I though it could have been useful to
> have a basic way to dump at will the effective full payloads of
> successfully transmitted/received SCMI messages.
>
> The existing SCMI traces already collect a bunch of information about SCMI
> message exchanges but they do NOT keep any payload information: this is
> certainly preferable most of the time since dumping full SCMI messages to
> the trace buffer involves a full copy of the payload.
>
> [...]

Applied to sudeep.holla/linux (for-next/scmi), thanks!

[1/2] include: trace: Add SCMI full message tracing
https://git.kernel.org/sudeep.holla/c/2bd0467074
[2/2] firmware: arm_scmi: Use new SCMI full message tracing
https://git.kernel.org/sudeep.holla/c/b60e088682

--
Regards,
Sudeep