Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Thu, 20 Jul 2023 11:31:56 -0700
From:   Bjorn Andersson <quic_bjorande@quicinc.com>
To:     Akhil P Oommen <quic_akhilpo@quicinc.com>
CC:     Dmitry Baryshkov <dmitry.baryshkov@linaro.org>,
        Jordan Crouse <jorcrous@amazon.com>,
        <freedreno@lists.freedesktop.org>, Sean Paul <sean@poorly.run>,
        <linux-arm-msm@vger.kernel.org>,
        Konrad Dybcio <konrad.dybcio@somainline.org>,
        Abhinav Kumar <quic_abhinavk@quicinc.com>,
        <dri-devel@lists.freedesktop.org>, <linux-kernel@vger.kernel.org>,
        "Nathan Chancellor" <nathan@kernel.org>,
        Rob Clark <robdclark@gmail.com>,
        "Daniel Vetter" <daniel@ffwll.ch>,
        Ricardo Ribalda <ribalda@chromium.org>,
        "Joel Fernandes (Google)" <joel@joelfernandes.org>,
        David Airlie <airlied@gmail.com>
Subject: Re: [Freedreno] [PATCH] drm/msm: Check for the GPU IOMMU during bind
Message-ID: <20230720183156.GA2667611@hu-bjorande-lv.qualcomm.com>
References: <20230309222049.4180579-1-jorcrous@amazon.com>
 <d73f6733-e605-0cf8-7909-8cced6e3b70d@linaro.org>
 <20230707150307.vb4otu5e6hwoadyf@amazon.com>
 <cc153fa9-b9e0-f714-ce5b-1a4a0cb55cc7@linaro.org>
 <2xnvyjlwuxft2uk2pirlbvbrg7krcb4alz7yyna72g4t2qrrfm@qtawbelv3n4l>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <2xnvyjlwuxft2uk2pirlbvbrg7krcb4alz7yyna72g4t2qrrfm@qtawbelv3n4l>
Precedence: bulk

On Mon, Jul 10, 2023 at 03:20:44AM +0530, Akhil P Oommen wrote:
> On Fri, Jul 07, 2023 at 08:27:18PM +0300, Dmitry Baryshkov wrote:
> > 
> > On 07/07/2023 18:03, Jordan Crouse wrote:
> > > On Thu, Jul 06, 2023 at 09:55:13PM +0300, Dmitry Baryshkov wrote:
> > > > 
> > > > On 10/03/2023 00:20, Jordan Crouse wrote:
> > > > > While booting with amd,imageon on a headless target the GPU probe was
> > > > > failing with -ENOSPC in get_pages() from msm_gem.c.
> > > > > 
> > > > > Investigation showed that the driver was using the default 16MB VRAM
> > > > > carveout because msm_use_mmu() was returning false since headless devices
> > > > > use a dummy parent device. Avoid this by extending the existing is_a2xx
> > > > > priv member to check the GPU IOMMU state on all platforms and use that
> > > > > check in msm_use_mmu().
> > > > > 
> > > > > This works for memory allocations but it doesn't prevent the VRAM carveout
> > > > > from being created because that happens before we have a chance to check
> > > > > the GPU IOMMU state in adreno_bind.
> > > > > 
> > > > > There are a number of possible options to resolve this but none of them are
> > > > > very clean. The easiest way is to likely specify vram=0 as module parameter
> > > > > on headless devices so that the memory doesn't get wasted.
> > > > 
> > > > This patch was on my plate for quite a while, please excuse me for
> > > > taking it so long.
> > > 
> > > No worries. I'm also chasing a bunch of other stuff too.
> > > 
> > > > I see the following problem with the current code. We have two different
> > > > instances than can access memory: MDP/DPU and GPU. And each of them can
> > > > either have or miss the MMU.
> > > > 
> > > > For some time I toyed with the idea of determining whether the allocated
> > > > BO is going to be used by display or by GPU, but then I abandoned it. We
> > > > can have display BOs being filled by GPU, so handling it this way would
> > > > complicate things a lot.
> > > > 
> > > > This actually rings a tiny bell in my head with the idea of splitting
> > > > the display and GPU parts to two different drivers, but I'm not sure
> > > > what would be the overall impact.
> > > 
> > > As I now exclusively work on headless devices I would be 100% for this,
> > > but I'm sure that our laptop friends might not agree :)
> > 
> > I do not know here. This is probably a question to Rob, as he better
> > understands the interaction between GPU and display parts of the userspace.
> 
> I fully support this if it is feasible.
> 

I second this.

> In our architecture, display and GPU are completely independent subsystems.
> Like Jordan mentioned, there are IOT products without display. And I wouldn't
> be surprised if there is a product with just display and uses software rendering.
> 

And we have SA8295P/SA8540P with two MDSS instances and one GPU.

Regards,
Bjorn