Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) client-ip=23.128.96.35;
Message-ID: <1333e15b-f229-460a-8965-01ff3e778a4d@amd.com>
Date:   Thu, 12 Oct 2023 14:35:15 +0200
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH drm-misc-next 2/3] drm/gpuva_mgr: generalize
 dma_resv/extobj handling and GEM validation
Content-Language: en-US
To:     Dave Airlie <airlied@gmail.com>
Cc:     =?UTF-8?Q?Thomas_Hellstr=C3=B6m_=28Intel=29?= 
        <thomas_os@shipmail.org>, Danilo Krummrich <dakr@redhat.com>,
        daniel@ffwll.ch, matthew.brost@intel.com,
        thomas.hellstrom@linux.intel.com, sarah.walker@imgtec.com,
        donald.robson@imgtec.com, boris.brezillon@collabora.com,
        faith.ekstrand@collabora.com, bskeggs@redhat.com,
        Liam.Howlett@oracle.com, nouveau@lists.freedesktop.org,
        linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org
References: <20230820215320.4187-1-dakr@redhat.com>
 <20230820215320.4187-3-dakr@redhat.com>
 <0c50ff22-0f11-1e27-c32e-694ce2b1e6c5@shipmail.org> <ZO864yp3UyVEfEjz@pollux>
 <88c45fe6-0942-707c-9ea7-8486c177fcd7@shipmail.org> <ZO9Zq2RhbX8EeHrn@pollux>
 <736b6b6d-9e04-a27d-7d60-0c45d696b304@shipmail.org>
 <ZPB26A0/oLHTmyqk@cassiopeiae>
 <a8f28d62-daec-927a-a33d-5be3eec6a1ed@shipmail.org>
 <ZPDk/lao1JlBNGoJ@cassiopeiae>
 <8a8253ae-0b85-df90-b480-64eeebfafc6d@shipmail.org>
 <CAPM=9tz3o-m+8VJJ6hxWhykat0kpp1UE7dBJE3X91aHHo1Y2VA@mail.gmail.com>
 <76963abd-77a1-4bbd-9537-7b230e648a90@amd.com>
 <CAPM=9twSHGRoSoXxG+hz1T8iBX2VgPFvFsNCDnK_nHW9WJYBtw@mail.gmail.com>
From:   =?UTF-8?Q?Christian_K=C3=B6nig?= <christian.koenig@amd.com>
In-Reply-To: <CAPM=9twSHGRoSoXxG+hz1T8iBX2VgPFvFsNCDnK_nHW9WJYBtw@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?UEFtZmdjSGxsM25wNlFVdk1ReWtwUGJFditKZThDdFRSL0hJQU5mNVpQY016?=
 =?utf-8?B?TGQxYXNlRitBNEZ0dklZZjVraTNHR0xMZWxIVWt1MWxEQm5jeXJTZnFaSDRE?=
 =?utf-8?B?Yk90dWQwVTZ2NjIwSVB5blpPaDJQaTZjOG1yVjdEYitaakZYd3loUzFwNHBJ?=
 =?utf-8?B?aGdtNFBwYnlLWjE3Wmd5NksyMlBhbXhjbGw5K1JMVEZtN2g4bGhYOTJPR2hU?=
 =?utf-8?B?cWlYbklUK0lNVnB0VEtjNlowQnlUM3lCeUNPNG0yYkcvcm9ITmVJYkM0bE1R?=
 =?utf-8?B?K2p2QytPb050RThjUVF2SlZJOHZPb3Y3cjZvbktORkZUTEt2ZVFzaHd2MjFO?=
 =?utf-8?B?ZHg5d3N2U05DMHpodk9FMmtrTlVrTTNhWjNFNUhQTTFzMzkrOVhCbVh0Qm1J?=
 =?utf-8?B?ZUhwbXdJZFA0NnhuWXQ0cFFseEpMOTJVb2ZWa2hVZ0ZnWER5VXFERXRYUnEv?=
 =?utf-8?B?ZiszYm15aVl5bzhDVE9CTVdwR2JHNFFHMXlqRXdURUY4RHZWRmozZjBxemZZ?=
 =?utf-8?B?L21nMGxGWmJKcjdZSFNKdncwa3NGYWtsL1Rncko4STRDcWNVNzN0SW5Pemti?=
 =?utf-8?B?QlNMN0VOUk05cFp1dWRiOFREb1IzUWlqUVUyeWVrZlE1QzV2WUUwaHRKdjYw?=
 =?utf-8?B?djJ1QjN0M0pkQTNiYVNNcTNzb3Q4QUs5WG1aQ3JySVREMTNrdmxzVnZGVDND?=
 =?utf-8?B?VFhuQy85MlY0RGdIVU1iaUZSZzlibnJWTzJwMTZrTjBPQmtUbExpSGRiOVZa?=
 =?utf-8?B?ZXdUVW9tZmZpSDJvWC9GU0xpQzI5VEI2Vkk3TUZOKytXaU12RDRkaVZKazls?=
 =?utf-8?B?KzJ3b3dobERVSTBoRC9MRThCSDF0QkgwV1dZSkc2RnJhc292Q00zelU3RzRJ?=
 =?utf-8?B?cmhtS1RYNDdwb3RhSm5qWlhGMDZRUFIxSWdWcUdGOTA4VHJMMEhpQW5xWm5z?=
 =?utf-8?B?ZWI5d2d5aFFPcUVNN0R1R3F5c3U1eUpONzRoMUVnOVZhaHJlQ2ZYcThtb3Jz?=
 =?utf-8?B?LzNpb1VRcW1Id2NoUm1HNHJjZy9iN05reld3TkxQdG5OS0wzN2FPNVRCclFa?=
 =?utf-8?B?SEhRaHdPMnJjTjBJeS9EREpIUk1MYmJGL3ZhQmJaRW02d0wvTEdYTzRKUFdI?=
 =?utf-8?B?WTEzeWJieS9ua0w3TkxuWWRRbU9kYk1yOXZWQ3A4UktWbDhBdm4vS3hqa2o2?=
 =?utf-8?B?WnpOMU9nbGlpQ2JvZmtyMytUMitpM1ZBZDV1MXMxMC9TWDRZZmQyNTQ3OU1h?=
 =?utf-8?B?NUZYdW9YZ2Q0Z3VnNmpySUNZeUpzeitaTlZwSkNFeThSeC9rWE5halVXekJN?=
 =?utf-8?B?WVhRaldpdlUzQU1lWXFERGxnZHJPQjJXR0NIVnVwNm04NE5rdGMxZk9UTE53?=
 =?utf-8?B?WTljSE5FRnZ5TzNXdFNuMTFVRkpjaWxlaGhBcS9Ta3N6UTVZU0tzbVpTMEg1?=
 =?utf-8?B?RkxEWkZkYW5EcHB5Z2s1Wkl0VGovTTFGMkxrSGNhNnJjUWVlMVNoclZHelhx?=
 =?utf-8?B?OHpMOVVHbE9jY0RVTVJjaXhkOU9jbnpzSWNjUWlZek8vQTlweUhWcDNoZTE0?=
 =?utf-8?B?WHFvVlBEOUFVOHJ5QmpwZEZiVklWQlowcktLUHh5Z3FzRlMrQnpOTUNHeUhZ?=
 =?utf-8?B?S0RoVVpVNkovUyt1UU9PN1paSWs0RHZ0RlNidkt0SWFwcGNZQ05VcmZKSjFv?=
 =?utf-8?B?enRlUU45ck9qSkRRQkVwWkVuZ2cyS0ZYOHlFcFdMQy9HK3BCY3ZnbHphT3NE?=
 =?utf-8?B?cnpmbi9JNE9BQWhrc3ZCU1dLZVc2ZWtVVHNSTWJyR0NLZU1GazQrajlVUGxt?=
 =?utf-8?B?YnVLM21rWTR5eTJhckZUOGhhUGxtVWthenhOQXNxcW43YWNIS1gwaisxLzB4?=
 =?utf-8?B?YkZRVlZva0lNa25zby9aYVpIbjdqam5DaE01bWhLd3plSmJzVGRuNEN2RHNG?=
 =?utf-8?B?M2pVM3ZCblBCdnZDT3czYXVpOGNrUXMzQUJmU1dIOGtEZXBVNFh6M25IU3JF?=
 =?utf-8?B?ZHhBbmZlclQydWdzS3o4Wi9SYS93K0hmcmdjT3NYZDUvbW40dUFHbm9SeWVH?=
 =?utf-8?B?aWY3eWhSdEtKSlg5bC9SR2JmUHk2MTJyVzlkMW5XMW1UUEYzd1J5dDY4U3Ey?=
 =?utf-8?Q?Ur9I=3D?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 96370a12-b4ca-4826-9e0f-08dbcb1fb3fd
X-MS-Exchange-CrossTenant-AuthSource: BN8PR12MB3587.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 Oct 2023 12:35:23.0084
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: EgHHwSzxgMW3YrU/YGnrvOnDOncICu37EJGAU9uTVTVAugFwn6fMmgASDJKPlhpQ
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7865
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Thu, 12 Oct 2023 05:35:54 -0700 (PDT)

Am 12.10.23 um 12:33 schrieb Dave Airlie:
> On Wed, 11 Oct 2023 at 17:07, Christian König <christian.koenig@amd.com> wrote:
>> Am 10.10.23 um 22:23 schrieb Dave Airlie:
>>>> I think we're then optimizing for different scenarios. Our compute
>>>> driver will use mostly external objects only, and if shared, I don't
>>>> forsee them bound to many VMs. What saves us currently here is that in
>>>> compute mode we only really traverse the extobj list after a preempt
>>>> fence wait, or when a vm is using a new context for the first time. So
>>>> vm's extobj list is pretty large. Each bo's vma list will typically be
>>>> pretty small.
>>> Can I ask why we are optimising for this userspace, this seems
>>> incredibly broken.
>>>
>>> We've has this sort of problem in the past with Intel letting the tail
>>> wag the horse, does anyone remember optimising relocations for a
>>> userspace that didn't actually need to use relocations?
>>>
>>> We need to ask why this userspace is doing this, can we get some
>>> pointers to it? compute driver should have no reason to use mostly
>>> external objects, the OpenCL and level0 APIs should be good enough to
>>> figure this out.
>> Well that is pretty normal use case, AMD works the same way.
>>
>> In a multi GPU compute stack you have mostly all the data shared between
>> different hardware devices.
>>
>> As I said before looking at just the Vulcan use case is not a good idea
>> at all.
>>
> It's okay, I don't think anyone is doing that, some of the these
> use-cases are buried in server land and you guys don't communicate
> them very well.

Yeah, well everybody is trying very hard to get away from those 
approaches :)

But so far there hasn't been any breakthrough.

>
> multi-gpu compute would I'd hope be moving towards HMM/SVM type
> solutions though?

Unfortunately not in the foreseeable future. HMM seems more and more 
like a dead end, at least for AMD.

AMD still has hardware support in all of their MI* products, but for 
Navi the features necessary for implementing HMM have been dropped. And 
it looks more and more like their are not going to come back.

Additional to that from the software side Felix summarized it in the HMM 
peer2peer discussion thread recently quite well. A buffer object based 
approach is not only simpler to handle, but also performant vise 
multiple magnitudes faster.

> I'm also not into looking at use-cases that used to be important but
> might not as important going forward.

Well multimedia applications and OpenGL are still around, but it's not 
the main focus any more.

Christian.

>
> Dave.
>
>
>> Christian.
>>
>>> Dave.