Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <2cb26bc2-704b-86df-15ed-8c18f81addaf@quicinc.com>
Date:   Thu, 28 Apr 2022 08:46:15 +0530
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.6.1
Subject: Re: [PATCHv10 5/6] lib: Add register read/write tracing support
Content-Language: en-US
To:     Arnd Bergmann <arnd@arndb.de>
CC:     Will Deacon <will@kernel.org>,
        Catalin Marinas <catalin.marinas@arm.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Marc Zyngier <maz@kernel.org>,
        "Trilok Soni" <quic_tsoni@quicinc.com>,
        <quic_psodagud@quicinc.com>, gregkh <gregkh@linuxfoundation.org>,
        Linux ARM <linux-arm-kernel@lists.infradead.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        linux-arm-msm <linux-arm-msm@vger.kernel.org>,
        Prasad Sodagudi <psodagud@codeaurora.org>
References: <cover.1644824638.git.quic_saipraka@quicinc.com>
 <8cf9304d9941c25d920c4835cbc624ff5c2ac2cb.1644824638.git.quic_saipraka@quicinc.com>
 <CAK8P3a1WwghVToFBNZMpG7Wcji_k3CebK3--LL8YNJs_Wu3rBQ@mail.gmail.com>
From:   Sai Prakash Ranjan <quic_saipraka@quicinc.com>
In-Reply-To: <CAK8P3a1WwghVToFBNZMpG7Wcji_k3CebK3--LL8YNJs_Wu3rBQ@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk

Hi Arnd,

On 4/27/2022 9:44 PM, Arnd Bergmann wrote:
> On Thu, Feb 24, 2022 at 7:07 AM Sai Prakash Ranjan
> <quic_saipraka@quicinc.com> wrote:
>> From: Prasad Sodagudi <psodagud@codeaurora.org>
>>
>> Generic MMIO read/write i.e., __raw_{read,write}{b,l,w,q} accessors
>> are typically used to read/write from/to memory mapped registers
>> and can cause hangs or some undefined behaviour in following few
>> cases,
>>
>> * If the access to the register space is unclocked, for example: if
>>    there is an access to multimedia(MM) block registers without MM
>>    clocks.
>>
>> * If the register space is protected and not set to be accessible from
>>    non-secure world, for example: only EL3 (EL: Exception level) access
>>    is allowed and any EL2/EL1 access is forbidden.
>>
>> * If xPU(memory/register protection units) is controlling access to
>>    certain memory/register space for specific clients.
>>
>> and more...
>>
>> Such cases usually results in instant reboot/SErrors/NOC or interconnect
>> hangs and tracing these register accesses can be very helpful to debug
>> such issues during initial development stages and also in later stages.
>>
>> So use ftrace trace events to log such MMIO register accesses which
>> provides rich feature set such as early enablement of trace events,
>> filtering capability, dumping ftrace logs on console and many more.
>>
>> Sample output:
>>
>> rwmmio_write: __qcom_geni_serial_console_write+0x160/0x1e0 width=32 val=0xa0d5d addr=0xfffffbfffdbff700
>> rwmmio_post_write: __qcom_geni_serial_console_write+0x160/0x1e0 width=32 val=0xa0d5d addr=0xfffffbfffdbff700
>> rwmmio_read: qcom_geni_serial_poll_bit+0x94/0x138 width=32 addr=0xfffffbfffdbff610
>> rwmmio_post_read: qcom_geni_serial_poll_bit+0x94/0x138 width=32 val=0x0 addr=0xfffffbfffdbff610
>>
>> Signed-off-by: Prasad Sodagudi <psodagud@codeaurora.org>
>> Co-developed-by: Sai Prakash Ranjan <quic_saipraka@quicinc.com>
>> Signed-off-by: Sai Prakash Ranjan <quic_saipraka@quicinc.com>
> I think this is ok in general. I saw that Steve had a minor comment, and
> I suppose you could have just resent the same patches with a fixup in order
> to have me pick it up into the asm-generic tree for 5.19.

I had it ready the same day Steve gave the comment :) but given there was no further
reviews on other patches, I thought of slowing down the posting of new versions.
I will send v11 now.

> There is one more thing that I saw looking through this patch again: the
> address you print is the virtual __iomem token, but it might be more
> valuable to have the physical address instead, which can be looked up
> in the devicetree to know which register is affected.
>
> There is a small extra cost to walk the page table, and I'm not sure
> if we actually have an interface for it (vmalloc_to_page is almost
> what we want, but it returns an invalid page pointer). Any suggestions
> on this?
>
>

Right, it would be useful but currently we rely on the caller information (the function name and
the offset) to identify who writes to the location and then post process to identify the register
written based on it. I am also not aware of the interface for page table walk and the walk on this
hot trace path (note that we are capturing every register read/write) would slow down the system
further.

Even in the internal versions, we get the physical address postmortem from the parser tool [1].

[1] https://source.codeaurora.org/quic/la/platform/vendor/qcom-opensource/tools/tree/linux-ramdump-parser-v2/parsers/rtb.py#n72

Given that we can add fields to this tracepoint without breaking ABI, we can probably add this addon
at a later point.

Thanks,
Sai