Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20221022214622.18042-1-ogabbay@kernel.org> <af4c71cb-be60-e354-ca4f-23e834aca6e1@suse.de>
 <CAFCwf12HDZvsr1TrRFQH9Vi26S-Xf9ULgxtBazme90Sj5AzhQQ@mail.gmail.com>
 <c22bd93e-8bd2-6865-711a-37aeadbca7f9@nvidia.com> <CAPM=9tyEqzQ09WcRtE1Zd3XjOaR9a2ms-vL-O5x2ong7iPF=4w@mail.gmail.com>
 <Y1fFRIxRq+enLWyd@nvidia.com>
In-Reply-To: <Y1fFRIxRq+enLWyd@nvidia.com>
From:   Alex Deucher <alexdeucher@gmail.com>
Date:   Tue, 25 Oct 2022 10:21:34 -0400
Message-ID: <CADnq5_PH3xR_nnN2wQ2NjydUoT+X0z_9OvHtaabwcPbwP-o+9A@mail.gmail.com>
Subject: Re: [RFC PATCH 0/3] new subsystem for compute accelerator devices
To:     Jason Gunthorpe <jgg@nvidia.com>
Cc:     Dave Airlie <airlied@gmail.com>,
        Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>,
        Jiho Chu <jiho.chu@samsung.com>,
        Jeffrey Hugo <quic_jhugo@quicinc.com>,
        Thomas Zimmermann <tzimmermann@suse.de>,
        Arnd Bergmann <arnd@arndb.de>,
        John Hubbard <jhubbard@nvidia.com>,
        Oded Gabbay <ogabbay@kernel.org>, linux-kernel@vger.kernel.org,
        dri-devel@lists.freedesktop.org,
        Christoph Hellwig <hch@infradead.org>,
        Jacek Lawrynowicz <jacek.lawrynowicz@linux.intel.com>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        Alex Deucher <alexander.deucher@amd.com>,
        Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>,
        Kevin Hilman <khilman@baylibre.com>,
        Maciej Kwapulinski <maciej.kwapulinski@linux.intel.com>,
        Jagan Teki <jagan@amarulasolutions.com>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Tue, Oct 25, 2022 at 7:15 AM Jason Gunthorpe <jgg@nvidia.com> wrote:
>
> On Tue, Oct 25, 2022 at 12:27:11PM +1000, Dave Airlie wrote:
>
> > The userspace for those is normally bespoke like ROCm, which uses
> > amdkfd, and amdkfd doesn't operate like most device files from what I
> > know, so I'm not sure we'd want it to operate as an accel device.
>
> I intensely dislike this direction that drivers will create their own
> char devs buried inside their device driver with no support or
> supervision.
>
> We've been here before with RDMA and it is just a complete mess.
>
> Whatever special non-drm stuff amdkfd need to do should be supported
> through the new subsystem, in a proper maintainable way.

We plan to eventually move ROCm over the drm interfaces once we get
user mode queues working on non-compute queues which is already in
progress.  ROCm already uses the existing drm nodes and libdrm for a
number of things today (buffer sharing, media and compute command
submission in certain cases, etc.).  I don't see much value in the
accel nodes for AMD products at this time.  Even when we transition,
there are still a bunch of things that we'd need to think about, so
the current kfd node may stick around until we figure out a plan for
those areas.  E.g., the kfd node provides platform level compute
topology information; e.g., the NUMA details for connected GPUs and
CPUs, non-GPU compute node information, cache level topologies, etc.

Alex

>
> Jason