Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 21 Oct 2022 21:32:48 +0200
From:   =?utf-8?Q?Ond=C5=99ej?= Jirman <megi@xff.cz>
To:     Peter Geis <pgwipeout@gmail.com>
Cc:     Heiko Stuebner <heiko@sntech.de>,
        linux-rockchip@lists.infradead.org,
        Rob Herring <robh+dt@kernel.org>,
        Krzysztof Kozlowski <krzysztof.kozlowski+dt@linaro.org>,
        Michael Riesch <michael.riesch@wolfvision.net>,
        Nicolas Frattaroli <frattaroli.nicolas@gmail.com>,
        Sascha Hauer <s.hauer@pengutronix.de>,
        Frank Wunderlich <frank-w@public-files.de>,
        Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>,
        Yifeng Zhao <yifeng.zhao@rock-chips.com>,
        Johan Jonker <jbx6244@gmail.com>,
        "open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS" 
        <devicetree@vger.kernel.org>,
        "moderated list:ARM/Rockchip SoC support" 
        <linux-arm-kernel@lists.infradead.org>,
        open list <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v2] arm64: dts: rockchip: rk356x: Fix PCIe register map
 and ranges
Message-ID: <20221021193248.2he6amnj7knk4biu@core>
Mail-Followup-To: =?utf-8?Q?Ond=C5=99ej?= Jirman <megi@xff.cz>,
        Peter Geis <pgwipeout@gmail.com>, Heiko Stuebner <heiko@sntech.de>,
        linux-rockchip@lists.infradead.org,
        Rob Herring <robh+dt@kernel.org>,
        Krzysztof Kozlowski <krzysztof.kozlowski+dt@linaro.org>,
        Michael Riesch <michael.riesch@wolfvision.net>,
        Nicolas Frattaroli <frattaroli.nicolas@gmail.com>,
        Sascha Hauer <s.hauer@pengutronix.de>,
        Frank Wunderlich <frank-w@public-files.de>,
        Ezequiel Garcia <ezequiel@vanguardiasur.com.ar>,
        Yifeng Zhao <yifeng.zhao@rock-chips.com>,
        Johan Jonker <jbx6244@gmail.com>,
        "open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS" <devicetree@vger.kernel.org>,
        "moderated list:ARM/Rockchip SoC support" <linux-arm-kernel@lists.infradead.org>,
        open list <linux-kernel@vger.kernel.org>
References: <20221005085439.740992-1-megi@xff.cz>
 <CAMdYzYrEXEqOmMeozGBbAAvrujZcOxLh4VYOmu5DSjPWTS-5zQ@mail.gmail.com>
 <20221005220812.4psu6kckej63yo2z@core>
 <4679102.Wku2Vz74k6@phil>
 <CAMdYzYq3S2rR3Kb61irpV9xHYijNiJY0mkVnJwPrpXzxg_Zh9g@mail.gmail.com>
 <20221021153913.l5ry6v4mcnzcmj2v@core>
 <CAMdYzYpYC6ME_ZYE65UWq__i+rit6_os-+do+JLmEL7y-jKr9g@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <CAMdYzYpYC6ME_ZYE65UWq__i+rit6_os-+do+JLmEL7y-jKr9g@mail.gmail.com>
Precedence: bulk

On Fri, Oct 21, 2022 at 12:48:15PM -0400, Peter Geis wrote:
> On Fri, Oct 21, 2022 at 11:39 AM Ondřej Jirman <megi@xff.cz> wrote:
> >
> > On Fri, Oct 21, 2022 at 09:07:50AM -0400, Peter Geis wrote:
> > > Good Morning Heiko,
> > >
> > > Apologies for just getting to this, I'm still in the middle of moving
> > > and just got my lab set back up.
> > >
> > > I've tested this patch series and it leads to the same regression with
> > > NVMe drives. A loop of md5sum on two identical 4GB random files
> > > produces the following results:
> > > d11cf0caa541b72551ca22dc5bef2de0  test-rand.img
> > > fad97e91da8d4fd554c895cafa89809b  test-rand2.img
> > > 2d56a7baa05c38535f4c19a2b371f90a  test-rand.img
> > > 74e8e6f93d7c3dc3ad250e91176f5901  test-rand2.img
> > > 25cfcfecf4dd529e4e9fbbe2be482053  test-rand.img
> > > 74e8e6f93d7c3dc3ad250e91176f5901  test-rand2.img
> > > b9637505bf88ed725f6d03deb7065dab  test-rand.img
> > > f7437e88d524ea92e097db51dce1c60d  test-rand2.img
> > >
> > > Before this patch series:
> > > d11cf0caa541b72551ca22dc5bef2de0  test-rand.img
> > > d11cf0caa541b72551ca22dc5bef2de0  test-rand2.img
> > > d11cf0caa541b72551ca22dc5bef2de0  test-rand.img
> > > d11cf0caa541b72551ca22dc5bef2de0  test-rand2.img
> > > d11cf0caa541b72551ca22dc5bef2de0  test-rand.img
> > > d11cf0caa541b72551ca22dc5bef2de0  test-rand2.img
> > > d11cf0caa541b72551ca22dc5bef2de0  test-rand.img
> > > d11cf0caa541b72551ca22dc5bef2de0  test-rand2.img
> > >
> > > Though I do love where this patch is going and would like to see if it
> > > can be made to work, in its current form it does not.
> >
> > Thanks for the test. Can you please also test v1? Also please share lspci -vvv
> > of your nvme drive, so that we can see allocated address ranges, etc.
> 
> Good catch, with your patch as is, the following issue crops up:
> Region 0: Memory at 300000000 (64-bit, non-prefetchable) [size=16K]
> Region 2: I/O ports at 1000 [disabled] [size=256]
> 
> However, with a simple fix, we can get this:
> Region 0: Memory at 300000000 (64-bit, non-prefetchable) [virtual] [size=16K]
> Region 2: I/O ports at 1000 [virtual] [size=256]
> 
> and with it a working NVMe drive.
> 
> Change the following range:
> 0x02000000 0x0 0x40000000 0x3 0x00000000 0x0 0x40000000>;
> to
> 0x02000000 0x0 0x00000000 0x3 0x00000000 0x0 0x40000000>;

I've already tried this, but this unfrotunately breaks the wifi cards.
(those only use the I/O space) Maybe because I/O and memory address spaces
now overlap, I don't know. That's why I used the 1GiB offset for memory
space.

kind regards,
	o.

> I still haven't tested this with other cards yet, and another patch
> that does similar work I've tested successfully as well with NVMe
> drives. I'll have to get back to you on the results of greater
> testing.
> 
> Very Respectfully,
> Peter Geis
> 
> >
> > kind regards,
> >         o.
> >
> > > Very Respectfully,
> > > Peter Geis