Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37;
MIME-Version: 1.0
References: <SEYPR03MB704641091854162959578D7E9AFFA@SEYPR03MB7046.apcprd03.prod.outlook.com>
 <CA+VMnFyhp9D8cjtvLVzdKGETouOuH=MKgjOu1pn00WDRB=5oUg@mail.gmail.com>
In-Reply-To: <CA+VMnFyhp9D8cjtvLVzdKGETouOuH=MKgjOu1pn00WDRB=5oUg@mail.gmail.com>
From:   Oded Gabbay <ogabbay@kernel.org>
Date:   Fri, 22 Sep 2023 18:08:38 +0300
Message-ID: <CAFCwf12sUL5bcXhYKwRkMxLtSDtLfTK003oxkRDVmThx1ARV-A@mail.gmail.com>
Subject: Re: kernel.org 6.5.4 , NPU driver, --not support (RFC)
To:     Jagan Teki <jagan@edgeble.ai>
Cc:     Cancan Chang <Cancan.Chang@amlogic.com>,
        linux-media <linux-media@vger.kernel.org>,
        linux-kernel <linux-kernel@vger.kernel.org>,
        Dave Airlie <airlied@redhat.com>,
        Daniel Vetter <daniel.vetter@ffwll.ch>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Fri, Sep 22, 2023 at 12:38=E2=80=AFPM Jagan Teki <jagan@edgeble.ai> wrot=
e:
>
> On Fri, 22 Sept 2023 at 15:04, Cancan Chang <Cancan.Chang@amlogic.com> wr=
ote:
> >
> > Dear Media Maintainers:
> >      Thanks for your attention. Before describing my problem=EF=BC=8Cle=
t me introduce to you what I  mean by NPU.
> >      NPU is Neural Processing Unit, It is designed for deep learning ac=
celeration, It is also called TPU, APU ..
> >
> >      The real problems:
> >       When I was about to upstream my NPU driver codes to linux mainlin=
e, i meet two problems:
> >         1.  According to my research, There is no NPU module path in th=
e linux (base on linux 6.5.4) , I have searched all linux projects and foun=
d no organization or comany that has submitted NPU code. Is there a path pr=
epared for NPU driver currently?
> >         2.   If there is no NPU driver path currently, I am going to pu=
t my NPU driver code in the drivers/media/platform/amlogic/ =EF=BB=BF, beca=
use my NPU driver belongs to amlogic. and amlogic NPU is mainly used for AI=
 vision applications. Is this plan suitabe for you?
>
> If I'm correct about the discussion with Oded Gabby before. I think
> the drivers/accel/ is proper for AI Accelerators including NPU.
>
> + Oded in case he can comment.
>
> Thanks,
> Jagan.
Thanks Jagan for adding me to this thread. Adding Dave & Daniel as well.

Indeed, the drivers/accel is the place for Accelerators, mainly for
AI/Deep-Learning accelerators.
We currently have 3 drivers there already.

The accel subsystem is part of the larger drm subsystem. Basically, to
get into accel, you need to integrate your driver with the drm at the
basic level (registering a device, hooking up with the proper
callbacks). ofc the more you use code from drm, the better.
You can take a look at the drivers under accel for some examples on
how to do that.

Could you please describe in a couple of sentences what your
accelerator does, which engines it contains, how you program it. i.e.
Is it a fixed-function device where you write to a couple of registers
to execute workloads, or is it a fully programmable device where you
load compiled code into it (GPU style) ?

For better background on the accel subsystem, please read the following:
https://docs.kernel.org/accel/introduction.html
This introduction also contains links to other important email threads
and to Dave Airlie's BOF summary in LPC2022.

Thanks,
Oded