Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Feedback-ID: i56a14606:Fastmail
User-Agent: Cyrus-JMAP/3.9.0-alpha0-527-gee7b8d90aa-fm-20230629.001-gee7b8d90
Mime-Version: 1.0
Message-Id: <2dff4d81-9b45-4d69-9e30-972f2c9318d9@app.fastmail.com>
In-Reply-To: <7b55040f-f2d2-372e-cf8a-5ac4a456fdb1@amd.com>
References: <20230703123557.3355657-1-arnd@kernel.org>
 <f4bc3739-3ff1-6fa7-5d7d-890f451315bf@amd.com>
 <788147c6-defd-496a-8174-1571b73d1a71@app.fastmail.com>
 <f1b190cb-3af7-178b-baeb-b59363868779@amd.com>
 <9b03246d-b46e-4b91-968a-e9ffc2fc26db@app.fastmail.com>
 <7b55040f-f2d2-372e-cf8a-5ac4a456fdb1@amd.com>
Date:   Tue, 04 Jul 2023 17:24:49 +0200
From:   "Arnd Bergmann" <arnd@arndb.de>
To:     =?UTF-8?Q?Christian_K=C3=B6nig?= <christian.koenig@amd.com>,
        "Arnd Bergmann" <arnd@kernel.org>,
        "Alex Deucher" <alexander.deucher@amd.com>,
        "Pan, Xinhui" <Xinhui.Pan@amd.com>,
        "Dave Airlie" <airlied@gmail.com>,
        "Daniel Vetter" <daniel@ffwll.ch>
Cc:     "Hawking Zhang" <Hawking.Zhang@amd.com>,
        "Lijo Lazar" <lijo.lazar@amd.com>,
        "Mario Limonciello" <mario.limonciello@amd.com>,
        "YiPeng Chai" <YiPeng.Chai@amd.com>, "Le Ma" <le.ma@amd.com>,
        "Bokun Zhang" <Bokun.Zhang@amd.com>,
        "Srinivasan Shanmugam" <srinivasan.shanmugam@amd.com>,
        "Shiwu Zhang" <shiwu.zhang@amd.com>, amd-gfx@lists.freedesktop.org,
        dri-devel@lists.freedesktop.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH] drm/amdgpu: avoid integer overflow warning in
 amdgpu_device_resize_fb_bar()
Content-Type: text/plain;charset=utf-8
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Tue, Jul 4, 2023, at 16:51, Christian K=C3=B6nig wrote:
> Am 04.07.23 um 16:31 schrieb Arnd Bergmann:
>> On Tue, Jul 4, 2023, at 14:33, Christian K=C3=B6nig wrote:
>>>
>>> Modern AMD GPUs have 16GiB of local memory (VRAM), making those
>>> accessible to a CPU which can only handle 32bit addresses by resizing
>>> the BAR is impossible to begin with.
>>>
>>> But going a step further even without resizing it is pretty hard to =
get
>>> that hardware working on such an architecture.
>> I'd still like to understand this part better, as we have a lot of
>> arm64 chips with somewhat flawed PCIe implementations, often with
>> a tiny 64-bit memory space, but otherwise probably capable of
>> using a GPU.
>
> Yeah, those are unfortunately very well known to us :(
>
>> What exactly do you expect to happen here?
>>
>> a) Use only part of the VRAM but otherwise work as expected
>> b) Access all of the VRAM, but at a performance cost for
>>     bank switching?
>
> We have tons of x86 systems where we can't resize the BAR (because of=20
> lack of BIOS setup of the root PCIe windows). So bank switching is sti=
ll=20
> perfectly supported.

Ok, good.

> After investigating (which sometimes even includes involving engineers=20
> from ARM) we usually find that those boards doesn't even remotely comp=
ly=20
> to the PCIe specification, both regarding power as well as functional=20
> things like DMA coherency.

Makes sense, the power usage is clearly going to make this
impossible on a lot of boards. I would have expected noncoherent
DMA to be a solvable problem, since that generally works with
all drivers that use the dma-mapping interfaces correctly,
but I understand that drivers/gpu/* often does its own thing
here, which may make that harder.

     Arnd