Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <8a2b68f1e8da19a8d81535e0ff4391e99691e141.camel@ndufresne.ca>
Subject: Re: [PATCH 2/2] media: v4l2-mem2mem: add a list for buf used by hw
From:   Nicolas Dufresne <nicolas@ndufresne.ca>
To:     Randy Li <ayaka@soulik.info>
Cc:     linux-media@vger.kernel.org, Tomasz Figa <tfiga@chromium.org>,
        Hsia-Jun Li <Randy.Li@synaptics.com>, mchehab@kernel.org,
        laurent.pinchart@ideasonboard.com, hiroh@chromium.org,
        hans.verkuil@cisco.com, hverkuil@xs4all.nl,
        linux-kernel@vger.kernel.org,
        Benjamin Gaignard <benjamin.gaignard@collabora.com>,
        p.zabel@pengutronix.de, ezequiel@vanguardiasur.com.ar
Date:   Fri, 04 Aug 2023 16:42:05 -0400
In-Reply-To: <585fdf97-4ed9-7d9a-78a8-e271d5b05301@soulik.info>
References: <20230704040044.681850-1-randy.li@synaptics.com>
         <20230704040044.681850-3-randy.li@synaptics.com>
         <20230712093301.nkj2vok2x7esdhb3@chromium.org>
         <f8f766c0166c502e29b06cda71f6531e44a91a17.camel@ndufresne.ca>
         <CAAFQd5CO4TS6wMsnaL7ob4CXogj5KT52x85YUUN1ZwDkOxW0oQ@mail.gmail.com>
         <583e22718b80cc5e1ae631528c83c95e97de5cae.camel@ndufresne.ca>
         <CAAFQd5CAJ7GxiY5=bBAa+L=1WJth6QZ3+PG83=GX+eEx1S4uhg@mail.gmail.com>
         <7d340df3-e14c-24de-4fc2-b7dca619447c@synaptics.com>
         <CAAFQd5BKHQPNVpDvpQaFn-q721BJknJCUB72urc2=EKsAH=OCg@mail.gmail.com>
         <10ad26e6-b2d0-d0e6-40c8-2cc70613188b@synaptics.com>
         <70ed7b3197f34c9f3dce9c83c527884c89df5449.camel@ndufresne.ca>
         <585fdf97-4ed9-7d9a-78a8-e271d5b05301@soulik.info>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.48.4 (3.48.4-1.fc38) 
MIME-Version: 1.0
Precedence: bulk

Hi Randy,

Le vendredi 04 ao=C3=BBt 2023 =C3=A0 00:16 +0800, Randy Li a =C3=A9crit=C2=
=A0:
> On 2023/7/29 00:19, Nicolas Dufresne wrote:
> > Le vendredi 28 juillet 2023 =C3=A0 15:37 +0800, Hsia-Jun Li a =C3=A9cri=
t=C2=A0:
> > > > I think this is one reason to migrate to the stateless decoder desi=
gn.
> > > >=20
> > > I didn't know such plan here. I don't think the current stateless API
> > > could export the reconstruction buffers for encoder or post-processin=
g
> > > buffer for decoder to us.
> > Someone suggested introduce auxiliary queues in our meeting in Lyon a w=
hile ago,
> > but I bet everyone got too busy with finalizing APIs, fixing fluster te=
sts etc.
> > The suggestion felt like it would be possible to add it after the fact.=
 This was
> > also being discussed in the context of supporting multi-scalers (standa=
lone our
> > inline with the codec, like VC8000D+). It could also cover for primary =
and
> > secondary buffers, along with encoder primary, and reconstruction buffe=
rs, but
> > also auxiliary reference data. This would also be needed to properly su=
pport
> > Vulkan Video fwiw, and could also help with a transition to DMABuf Heap=
s and
> > memory accounting.
> >=20
> > I've also had corridor discussion around having multi-instance media co=
nstroller
> > devices. It wasn't clear how to bind the media instance to the video no=
de
> > instances, but assuming there is a way, it would be a tad more flexible=
 (but
> > massively more complex).
>=20
> I think we should answer to those questions before we decided what we wan=
t:
>=20
> A. Should a queue only has the buffers for the same format and sizes?

I initially started answering these, but ended up concluding this is more s=
ome
sort of personal notes. I believe the discussion is diverging. Remember tha=
t in
an existing API, one cannot fix all theoretical issues in one go. In order =
to
move forward, you have to tackle very specific use case and find a way forw=
ard.
If you are to break compatibility as much as you suggest, you'd rather look=
 into
writing a new API.

[...]