Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Thu, 9 Apr 2020 13:06:31 +0800
From:   Jiaxun Yang <jiaxun.yang@flygoat.com>
To:     Chris Packham <Chris.Packham@alliedtelesis.co.nz>
Cc:     "f.fainelli@gmail.com" <f.fainelli@gmail.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "Hamish Martin" <Hamish.Martin@alliedtelesis.co.nz>,
        "linux-mips@vger.kernel.org" <linux-mips@vger.kernel.org>,
        "devicetree@vger.kernel.org" <devicetree@vger.kernel.org>
Subject: Re: Dealing with holes in CPU address space
Message-ID: <20200409130631.69413b76@flygoat-x1e>
In-Reply-To: <f5c7a3387dde5667be4fc462838edfaefae31e16.camel@alliedtelesis.co.nz>
References: <fcb8f2655452f60a7c734e2ce54ac4d47eec7e92.camel@alliedtelesis.co.nz>
        <20200408152922.14f90ff3@flygoat-x1e>
        <2e10a19b6608a6c3413b52180c69500aa255a701.camel@alliedtelesis.co.nz>
        <dd91aa7f-ed7d-44de-6887-ad25c7e4d4ff@gmail.com>
        <f5c7a3387dde5667be4fc462838edfaefae31e16.camel@alliedtelesis.co.nz>
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Thu, 9 Apr 2020 04:50:23 +0000
Chris Packham <Chris.Packham@alliedtelesis.co.nz> wrote:

> On Wed, 2020-04-08 at 21:03 -0700, Florian Fainelli wrote:
> > 
> > On 4/8/2020 2:33 PM, Chris Packham wrote:  
> > > On Wed, 2020-04-08 at 15:29 +0800, Jiaxun Yang wrote:  
> > > > On Wed, 8 Apr 2020 05:14:22 +0000
> > > > Chris Packham <Chris.Packham@alliedtelesis.co.nz> wrote:
> > > >   
> > > > > Hi All,
> > > > > 
> > > > > I'm trying to port an old Broadcom MIPS CPU (BCM53003) to a
> > > > > shiny
> > > > > new
> > > > > kernel. I have some old historic source from a long forgotten
> > > > > Broadcom
> > > > > LDK but I'd prefer to do things the modern way with device-
> > > > > trees.
> > > > > 
> > > > > The problem I've been grappling with is trying to open up
> > > > > access to
> > > > > all of the RAM on the board. It has 512MB of DDR2. The CPU has
> > > > > two
> > > > > areas where this appears. The first 128MB is from 0 to
> > > > > 0x07ffffff
> > > > > the
> > > > > second area is from 0x88000000 to 0x9fffffff.
> > > > > 
> > > > > SoC peripherals are at 0x18000000 and there is an IO window
> > > > > for flash
> > > > > at 0x20000000.
> > > > > 
> > > > > The old code has some custom tlb initialisation to deal with
> > > > > this
> > > > > but
> > > > > I figured it should be possible with the following dts
> > > > > snippet.
> > > > > 
> > > > >         memory@0 {
> > > > >                 device_type = "memory";
> > > > >                 reg = <0x00000000 0x08000000
> > > > >                        0x88000000 0x18000000>;
> > > > >         };
> > > > > 
> > > > > I end up with only 128MB available. This appears to be
> > > > > because the default HIGHMEM_START of 0x20000000 stops the rest
> > > > > from
> > > > > being made available. If I add an override of HIGHMEM_START to
> > > > > 0xffffffff I seem to have the full 512MB avaiable but then I
> > > > > get a
> > > > > kernel panic  
> > > > 
> > > > Hi,
> > > > 
> > > > Have you tried to enable CONFIG_HIGHMEM?
> > > >   
> > > 
> > > I have but that didn't seem to help. As I understand it HIGHMEM is
> > > intended for situations when you have more physical RAM that can
> > > be addressed (e.g. >4GB on a 32-bit system).  
> > 
> > On MIPS you may have to enable HIGHMEM as soon as you run out of
> > virtual
> > kernel address space to map the entire amount of memory that is
> > populated AFAICT. The kernel has a little under 1GB of virtual
> > address
> > space that can be mapped via the TLB since the first 512MB are
> > occupied
> > by KSEG0/1.
> >   
> 
> My adventures thus far with HIGHMEM have got as far as
> 
>   This processor doesn't support highmem. 2490368k highmem ignored
> 
> Which I think has something to do with the max_low_pfn and highend_pfn
> being different.
> 

You might have cpu_has_dc_aliases defined.
HIGHMEM is unsafe on these systems due to Cache Alias issue.

Here is a comment in mips/kernel/cpu-probe.c:

/*
 * Early versions of the 74K do not update the cache tags on a
 * vtag miss/ptag hit which can occur in the case of KSEG0/KUSEG
 * aliases.  In this case it is better to treat the cache as always
 * having aliases.  Also disable the synonym tag update feature
 * where available.  In this case no opportunistic tag update will
 * happen where a load causes a virtual address miss but a physical
 * address hit during a D-cache look-up.
 */

Probably you system have this kind of issue?

You can determine waysize of D-Cache and set a larger pagesize.

--
Jiaxun Yang