Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <e67392e43bfc037db6969297b65b0b6945df8b27.camel@pengutronix.de>
Subject: Re: DMA-buf and uncached system memory
From:   Lucas Stach <l.stach@pengutronix.de>
To:     Nicolas Dufresne <nicolas@ndufresne.ca>,
        Christian =?ISO-8859-1?Q?K=F6nig?= <christian.koenig@amd.com>,
        Pekka Paalanen <ppaalanen@gmail.com>
Cc:     "Sharma, Shashank" <Shashank.Sharma@amd.com>,
        lkml <linux-kernel@vger.kernel.org>,
        dri-devel <dri-devel@lists.freedesktop.org>,
        linaro-mm-sig@lists.linaro.org,
        Sumit Semwal <sumit.semwal@linaro.org>,
        linux-media <linux-media@vger.kernel.org>
Date:   Mon, 27 Jun 2022 16:06:31 +0200
In-Reply-To: <0abe475a7b4b04758c03a9d19b228e86d95ac1dd.camel@ndufresne.ca>
References: <91ff0bbb-ea3a-2663-3453-dea96ccd6dd8@amd.com>
         <YCuPhOT4GhY3RR/6@phenom.ffwll.local>
         <9178e19f5c0e141772b61b759abaa0d176f902b6.camel@ndufresne.ca>
         <CAPj87rPYQNkgVEdHECQcHcYe2nCpgF3RYQKk_=wwhvJSxwHXCg@mail.gmail.com>
         <c6e65ee1-531e-d72c-a6a6-da7149e34f18@amd.com>
         <20220623101326.18beeab3@eldfell>
         <954d0a9b-29ef-52ef-f6ca-22d7e6aa3f4d@amd.com>
         <4b69f9f542d6efde2190b73c87096e87fa24d8ef.camel@pengutronix.de>
         <adc626ec-ff5a-5c06-44ce-09111be450cd@amd.com>
         <fbb228cd78e9bebd7e7921c19e0c4c09d0891f23.camel@pengutronix.de>
         <e691bccc-171d-f674-2817-13a945970f4a@amd.com>
         <95cca943bbfda6af07339fb8d2dc7f4da3aa0280.camel@pengutronix.de>
         <0abe475a7b4b04758c03a9d19b228e86d95ac1dd.camel@ndufresne.ca>
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.40.4 (3.40.4-1.fc34) 
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Precedence: bulk

Am Montag, dem 27.06.2022 um 09:54 -0400 schrieb Nicolas Dufresne:
> Le jeudi 23 juin 2022 à 11:33 +0200, Lucas Stach a écrit :
> > > 
> > > See for example on AMD/Intel hardware most of the engines can perfectly 
> > > deal with cache coherent memory accesses. Only the display engines can't.
> > > 
> > > So on import time we can't even say if the access can be coherent and 
> > > snoop the CPU cache or not because we don't know how the imported 
> > > DMA-buf will be used later on.
> > > 
> > So for those mixed use cases, wouldn't it help to have something
> > similar to the dma_sync in the DMA-buf API, so your scanout usage can
> > tell the exporter that it's going to do non-snoop access and any dirty
> > cache lines must be cleaned? Signaling this to the exporter would allow
> > to skip the cache maintenance if the buffer is in CPU uncached memory,
> > which again is a default case for the ARM SoC world.
> 
> Telling the exporter for every scan is unneeded overhead. If that information is
> made available "properly", then tracking it in attach/detach is sufficient and
> lightweight.

That isn't sufficient. The AMD GPU is a single device, but internally
has different engines that have different capabilities with regard to
snooping the caches. So you will likely end up with needing the cache
clean if the V4L2 buffer is going directly to scanout, which doesn't
snoop, but if the usage changes to sampling you don't need any cache
flushes.

Also I don't see a big overhead when comparing a kernel internal call
that tells the exporter that the importer is going to access the buffer
without snooping and thus needs the cache clean once every frame and
the need to always clean the cache before DQBUF when a potentially non-
snooping importer is attached.

Regards,
Lucas