Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34;
Date:   Wed, 13 Sep 2023 12:14:34 +0200
From:   Stephan Gerhold <stephan@gerhold.net>
To:     Konrad Dybcio <konrad.dybcio@linaro.org>,
        Bryan O'Donoghue <bryan.odonoghue@linaro.org>
Cc:     Bjorn Andersson <andersson@kernel.org>,
        Andy Gross <agross@kernel.org>, linux-arm-msm@vger.kernel.org,
        devicetree@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH 4/9] arm64: dts: qcom: msm8916: Reserve firmware memory
 dynamically
Message-ID: <ZQGLioVhjCK3TRva@gerhold.net>
References: <20230911-msm8916-rmem-v1-0-b7089ec3e3a1@gerhold.net>
 <20230911-msm8916-rmem-v1-4-b7089ec3e3a1@gerhold.net>
 <912f90ee-0816-43ae-bc6f-a9a9a3e33d8a@linaro.org>
 <9b1beb38-9ec2-4bdb-97f5-fccf98d3b0c3@linaro.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <9b1beb38-9ec2-4bdb-97f5-fccf98d3b0c3@linaro.org>
Content-Transfer-Encoding: 7bit
Precedence: bulk

On Wed, Sep 13, 2023 at 10:12:12AM +0100, Bryan O'Donoghue wrote:
> On 13/09/2023 10:06, Konrad Dybcio wrote:
> > On 11.09.2023 19:41, Stephan Gerhold wrote:
> > > Most of the reserved firmware memory on MSM8916 can be relocated when
> > > respecting the required alignment. To avoid having to precompute the
> > > reserved memory regions in every board DT, describe the actual
> > > requirements (size, alignment, alloc-ranges) using the dynamic reserved
> > > memory allocation.
> > > 
> > > This approach has several advantages:
> > > 
> > >   1. We can define "templates" for the reserved memory regions in
> > >      msm8916.dtsi and keep only device-specific details in the board DT.
> > >      This is useful for the "mpss" region size for example, which varies
> > >      from device to device. It is no longer necessary to redefine all
> > >      firmware regions to shift their addresses.
> > > 
> > >   2. When some of the functionality (e.g. WCNSS, Modem, Venus) is not
> > >      enabled or needed for a device, the reserved memory can stay
> > >      disabled, freeing up the unused reservation for Linux.
> > > 
> > >   3. Devices with special requirements for one of the firmware regions
> > >      are handled automatically. For example, msm8916-longcheer-l8150
> > >      has non-relocatable "wcnss" firmware that must be loaded exactly
> > >      at address 0x8b600000. When this is defined as a static region,
> > >      the other dynamic allocations automatically adjust to a different
> > >      place with suitable alignment.
> > > 
> > > All in all this approach significantly reduces the boilerplate necessary
> > > to define the different firmware regions, and makes it easier to enable
> > > functionality on the different devices.
> > > 
> > > Signed-off-by: Stephan Gerhold <stephan@gerhold.net>
> > > ---
> > [...]
> > 
> > >   		mpss_mem: mpss@86800000 {
> > > +			/*
> > > +			 * The memory region for the mpss firmware is generally
> > > +			 * relocatable and could be allocated dynamically.
> > > +			 * However, many firmware versions tend to fail when
> > > +			 * loaded to some special addresses, so it is hard to
> > > +			 * define reliable alloc-ranges.
> > > +			 *
> > > +			 * alignment = <0x0 0x400000>;
> > > +			 * alloc-ranges = <0x0 0x86800000 0x0 0x8000000>;
> > > +			 */
> > Do we know of any devices that this would actually work on?

Yes, the "modem" firmware on DB410c seems to be fine with literally all
correctly aligned addresses I've tested so far. But when I manually
experimented with other addresses on actual smartphones it exploded on
certain addresses, specific to the firmware version / device.

> I have the same question here.
> 
> How was this tested ?
> 

Bryan, your question sounds more general, about the dynamic allocation
of the firmware regions in general (also for wcnss etc).

I'll just explain my tests for both with more detail :D

For wcnss/venus a different memory region has been used in postmarketOS
for years because I shifted it early on to make room for the larger
modem firmware. I've never heard of any problems with this on the 50+
different MSM8916-related devices we support there, with many different
firmware versions. When preparing this patch set I experimented some
more and noticed there is a limit (I think around 0x90000000) where TZ
refuses to load/verify the firmware. The alloc-ranges is chosen
conservatively to avoid potential trouble. Also because I didn't see
any use case to have a huge range specified there...

For mpss it's more tricky. First, if you'd comment out the alignment and
alloc-ranges it would most likely work correctly because Linux just
allocates it at the start of the range, on 0x86800000 like it was
before. However, the DT spec doesn't strictly specify where inside the
range the region is allocated, so I manually experimented with some
other addresses within the range:

 - On DB410c it works just fine. All addresses I tried work without any
   problems.

 - On longcheer-l8150 the modem firmare works fine when the memory
   region starts somewhere between 0x86800000 and 0x8a800000. It also
   works again after 0x8e800000. But on anything between 0x8a800000 and
   0x8e800000 it's broken for who knows what reason.

 - On some Samsung devices only 0x86800000 and maybe one or two other
   addresses worked, again for who knows what reason. Most other
   addresses were broken.

The symptom is usually some timeout while starting, the actual signature
verification seems to work fine. I believe that in theory it's supposed
to be relocatable, but in practice they hardcoded some addresses or did
some other funny stuff in some of the firmware versions.

To be safe my conclusion was to keep mpss at a fixed address and only
allocate the others dynamically. This is how the patch implements it.

Thanks,
Stephan