Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <85a1cca1-f59b-6a0c-dee3-9d9ed5d6b6d1@gmail.com>
Date:   Fri, 14 Apr 2023 05:27:01 -0700
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Thunderbird/102.10.0
Subject: Re: [PATCH v2 2/3] PCI: brcmstb: CLKREQ# accomodations of downstream
 device
Content-Language: en-US
To:     Jim Quinlan <jim2101024@gmail.com>,
        Cyril Brulebois <kibi@debian.org>
Cc:     linux-pci@vger.kernel.org,
        Nicolas Saenz Julienne <nsaenz@kernel.org>,
        Bjorn Helgaas <bhelgaas@google.com>,
        Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
        Phil Elwell <phil@raspberrypi.com>,
        bcm-kernel-feedback-list@broadcom.com, james.quinlan@broadcom.com,
        Florian Fainelli <f.fainelli@gmail.com>,
        Lorenzo Pieralisi <lpieralisi@kernel.org>,
        =?UTF-8?Q?Krzysztof_Wilczy=c5=84ski?= <kw@linux.com>,
        Rob Herring <robh@kernel.org>,
        "moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE" 
        <linux-rpi-kernel@lists.infradead.org>,
        "moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE" 
        <linux-arm-kernel@lists.infradead.org>,
        open list <linux-kernel@vger.kernel.org>
References: <20230411165919.23955-1-jim2101024@gmail.com>
 <20230411165919.23955-3-jim2101024@gmail.com>
 <20230413143935.pmbyjk2boxl3rwne@mraw.org>
 <CANCKTBtXKAYf1LxR4qN+dVyxsWgyDztUVB4EdG=xhHbuhNCq5w@mail.gmail.com>
 <20230413200646.ddgsoqgmaae343nl@mraw.org>
 <CANCKTBuZ=Hxy9WgnjbauhHqXGx4QU_t8pgX=3che2K89=2BT9A@mail.gmail.com>
From:   Florian Fainelli <f.fainelli@gmail.com>
In-Reply-To: <CANCKTBuZ=Hxy9WgnjbauhHqXGx4QU_t8pgX=3che2K89=2BT9A@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk


On 4/14/2023 5:14 AM, Jim Quinlan wrote:
> On Thu, Apr 13, 2023 at 4:06 PM Cyril Brulebois <kibi@debian.org> wrote:
>>
>> Hi Jim,
>>
>> Jim Quinlan <jim2101024@gmail.com> (2023-04-13):
>>> Can you provide (a) the full boot log prior to applying the patch
>>> series and (b) full boot log after applying the series, using an
>>> IDENTICAL setup. If it fails on both then it has little to do with my
>>> patch series.
>>
>> Just to be clear, the issue I reported was with:
>>   - Raspberry Pi Compute Module 4 (Rev 1.1, 4G RAM, 32G storage)
>>   - Raspberry Pi Compute Module 4 IO Board
>>   - SupaHub PCIe-to-multiple-USB adapter, reference PCE6U1C-R02, VER 006S
>>
>> This was my minimal reproducer for the kernel panic at boot-up, which
>> goes away with either v1 or v2. When I realized I didn't actually check
>> whether the SupaHub board was working correctly, I plugged 2 devices to
>> obtain this setup:
>>   - Raspberry Pi Compute Module 4 (Rev 1.1, 4G RAM, 32G storage)
>>   - Raspberry Pi Compute Module 4 IO Board
>>   - SupaHub PCIe-to-multiple-USB adapter, reference PCE6U1C-R02, VER 006S
>>   - Kingston DataTraveler G4 32GB on USB-A port #1 of the SupaHub board.
>>   - Logitech K120 keyboard on USB-A port #2 of the SupaHub board.
>>
>> It turns out that this particular revision of the SupaHub board isn't
>> supported by xhci_hcd directly (failing to probe with error -110) and
>> one needs to enable CONFIG_USB_XHCI_PCI_RENESAS=m and also ship its
>> accompanying firmware (/lib/firmware/renesas_usb_fw.mem). With this
>> updated kernel config, I'm able to use the keyboard and to read data
>> from the memory stick without problems (70 MB/s).
>>
>>> In my last series your testing somehow conflated the effect of an
>>> unrelated MMC interrupt issue so please be precise.
>>
>> I wish things would be simpler and didn't involve combinatorics, let
>> alone other bugs/regressions at times, but I'm really trying my best to
>> navigate and report issues and test patches when I can spare some time…
> 
> Hi Cyril,
> 
> I want to encourage you and others doing testing and bug reporting:
> everyone wins when a bug or issue is reported, fixed, and tested.
> I'm just asking that when you have negative results, that you provide
> information on the "before" and "after" test results of
> the patch series, and run both on the same test environment.

Cyril, based upon the table and logs you provided whereby you have used 
the following:

- Raspberry Pi Compute Module 4 (Rev 1.0, 8G RAM, 32G storage)
- Raspberry Pi Compute Module 4 IO Board
- SupaHub PCIe-to-multiple-USB adapter, reference PCE6U1C-R02, VER 006S

in the before/unpatched case we have a PCIe link down and in the 
after/patched we have a PCIe link up but a kernel panic. Neither are 
great nor resulting in a fully functional PCIe device.

Looking at:

https://www.amazon.co.uk/SupaHub-Express-BandWidth-Capable-Expanding/dp/B092ZQWG5B

it would appear that it can accept an external power supply, do you have 
one connected to that USB expansion card by any chance? Are you able to 
boot the kernel before/after if you disconnect any USB peripheral?

This looks like a broader electrical problem than the scope of this 
patch, though it would be neat if we could find a combination that 
works. At least with Jim's patch we have a PCIe link with 
uni-directional CLKREQ# so we could try a variety of things.

Does that SupaHub board plugged to the CM4 1.0 system work fine in the 
Raspberry Pi kernel tree?
-- 
Florian