Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Tue, 12 Jul 2022 18:55:48 +0800
From:   Li Chen <me@linux.beauty>
To:     "Arnd Bergmann" <arnd@arndb.de>
Cc:     "Catalin Marinas" <catalin.marinas@arm.com>,
        "Will Deacon" <will@kernel.org>,
        "Rob Herring" <robh+dt@kernel.org>,
        "Frank Rowand" <frowand.list@gmail.com>,
        "Andrew Morton" <akpm@linux-foundation.org>,
        "Li Chen" <lchen@ambarella.com>,
        "Linux ARM" <linux-arm-kernel@lists.infradead.org>,
        "Linux Kernel Mailing List" <linux-kernel@vger.kernel.org>,
        "DTML" <devicetree@vger.kernel.org>,
        "Linux-MM" <linux-mm@kvack.org>
Message-ID: <181f20d0403.121f433c8600165.2068876337784123868@linux.beauty>
In-Reply-To: <CAK8P3a3gX-JMh6E2X3rH+U37zhkA6b0+AJDtXCJfdZiMocxLjg@mail.gmail.com>
References: <20220711122459.13773-1-me@linux.beauty> <20220711122459.13773-5-me@linux.beauty>
 <CAK8P3a2Mr0ZMXGDx6htYEbBBtm4mubk-meSASJjPRK1j1O-hEA@mail.gmail.com>
 <181efcca6ae.de84203d522625.7740936811073442334@linux.beauty>
 <CAK8P3a30o1RLifV1TMqDJ26vLhVdOzz3wP6yPrayLV2GPxUtwQ@mail.gmail.com> <181f1d88b64.e2eb2601586551.453778983551010212@linux.beauty> <CAK8P3a3gX-JMh6E2X3rH+U37zhkA6b0+AJDtXCJfdZiMocxLjg@mail.gmail.com>
Subject: Re: [PATCH 4/4] sample/reserved_mem: Introduce a sample of struct
 page and dio support to no-map rmem
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Importance: Medium
User-Agent: Zoho Mail
Precedence: bulk

Hi Arnd,
 ---- On Tue, 12 Jul 2022 18:08:10 +0800  Arnd Bergmann <arnd@arndb.de> wrote --- 
 > On Tue, Jul 12, 2022 at 11:58 AM Li Chen <me@linux.beauty> wrote:
 > >  > On Tue, Jul 12, 2022 at 2:26 AM Li Chen <me@linux.beauty> wrote:
 > >  > >  ---- On Mon, 11 Jul 2022 21:28:10 +0800  Arnd Bergmann <arnd@arndb.de> wrote ---
 > >  > >  > On Mon, Jul 11, 2022 at 2:24 PM Li Chen <me@linux.beauty> wrote:
 > >  > >  > The problem here is that the DT is meant to describe the platform in an OS
 > >  > >  > independent way, so having a binding that just corresponds to a user space
 > >  > >  > interface is not a good abstraction.
 > >  > >
 > >  > > Gotcha, but IMO dts + rmem is the only choice for our use case. In our real
 > >  > > case, we use reg instead of size to specify the physical address, so
 > >  > > memremap cannot be used.
 > >  >
 > >  > Does your hardware require a fixed address for the buffer? If it can be
 > >  > anywhere in memory (or at least within a certain range) but just has to
 > >  > be physically contiguous, the normal way would be to use a CMA area
 > >  > to allocate from, which gives you 'struct page' backed pages.
 > >
 > > The limitation is our DSP can only access 32bit memory, but total dram is > 4G, so I cannot use
 > > "size = <...>" in our real case (it might get memory above 4G). I'm not sure if other vendors' DSP also has
 > > this limitation, if so, how do they deal with it if throughput matters.
 > 
 > This is a common limitation that gets handled automatically by setting
 > the dma_mask of the device through the dma-ranges property in DT.
 > When the driver does dma_alloc_coherent() or similar to gets its buffer,
 > it will then allocate pages below this boundary.
 
Thanks for the tip! I wasn't aware that dma-ranges can be used by devices other than PCIe controllers.

 > If you need a large contiguous memory area, then using CMA allows
 > you to specify a region of memory that is kept reserved for DMA
 > allocations, so a call to dma_alloc_coherent() on your device will
 > get contiguous pages from that area, and move other data in those
 > pages elsewhere if necessary. non-movable data is allocated from
 > pages outside of the CMA reserved area in this case.

We need a large memory pool, around 2G. I will try CMA and dma-ranges later!

Regards,
Li