Received-SPF: pass (google.com: domain of linux-ext4-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) client-ip=2620:137:e000::1:18;
From:   "Sierra Guiza, Alejandro (Alex)" <Alex.Sierra@amd.com>
To:     "Sierra Guiza, Alejandro (Alex)" <Alex.Sierra@amd.com>,
        "jgg@nvidia.com" <jgg@nvidia.com>
CC:     "rcampbell@nvidia.com" <rcampbell@nvidia.com>,
        "willy@infradead.org" <willy@infradead.org>,
        "david@redhat.com" <david@redhat.com>,
        "Kuehling, Felix" <Felix.Kuehling@amd.com>,
        "apopple@nvidia.com" <apopple@nvidia.com>,
        "amd-gfx@lists.freedesktop.org" <amd-gfx@lists.freedesktop.org>,
        "linux-xfs@vger.kernel.org" <linux-xfs@vger.kernel.org>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "jglisse@redhat.com" <jglisse@redhat.com>,
        "dri-devel@lists.freedesktop.org" <dri-devel@lists.freedesktop.org>,
        "akpm@linux-foundation.org" <akpm@linux-foundation.org>,
        "linux-ext4@vger.kernel.org" <linux-ext4@vger.kernel.org>,
        "hch@lst.de" <hch@lst.de>
Subject: RE: [PATCH] drm/amdkfd: Add SVM API support capability bits
Thread-Topic: [PATCH] drm/amdkfd: Add SVM API support capability bits
Thread-Index: AQHYRHyQ/YsZujGeV0SWTSHGpB4J2KzYcKpw
Date:   Wed, 30 Mar 2022 21:26:52 +0000
Message-ID: <SN6PR12MB2717CBD5A2A27E1C0026D614FD1F9@SN6PR12MB2717.namprd12.prod.outlook.com>
References: <20220330212420.12003-1-alex.sierra@amd.com>
In-Reply-To: <20220330212420.12003-1-alex.sierra@amd.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?MqO+qt+6C9u67/YFFpMFzVEW3N13q8mj2eTBsXTfXo9E14lqofepeEVpzv/8?=
 =?us-ascii?Q?3EiI5dp5oYNT1KMRhjXxoqY8GJWei3ME0lNXcluE4TOpoUV4y7GHKa2Q8LmB?=
 =?us-ascii?Q?D6k50Jdu0E7VXVcW/UPY1Upp2Rr+9xa6hLoEWTmzcjdw9fEkrQBzrZQ/r4wY?=
 =?us-ascii?Q?3scrExSr3+IRSx2Wtz5i4kKh+A5Ng+RFXyZJcFAwd6PXJWKOt3FCenH3OKKC?=
 =?us-ascii?Q?h+zYQP2Uh5Y48Lw7YrPpIKDFrND15YZ6qXyNrzYrVa8rNN5lm6s240eBlYcz?=
 =?us-ascii?Q?NVr+iBTHccqyhSeWFVggpI1S7cP2AwkgsOSJUpM8PSpwnf1WCLvHfk0aXAEV?=
 =?us-ascii?Q?Atg7FxCJB2VOf1ECKQWWWYO4e5r5Af3meO+Hq5mSUvPsnGwqsBjf+Q14ICJL?=
 =?us-ascii?Q?v25HRmNwhcpo1eEok6Pob/tzNOit20x0W/tPLTRO5+AUGTGv+tseuwFo9qi+?=
 =?us-ascii?Q?+hSya3IZb3OyAo8FBLiDC5dBr3u3cLw2r4xztCONDZHVwk3VvJXagV2UcX86?=
 =?us-ascii?Q?2N1D5rVi4qlrndr3N+MJerKwZTqM4RiPzc4YUgrjQxHpfmvWEwZp6yzKLOpq?=
 =?us-ascii?Q?GY8aaHHvPl7VimOkhVckUDFWdY4KEohPs7kLltzgR/Hn7npGL/b+FrsMSHwV?=
 =?us-ascii?Q?15lYRyslnw9fs28LahlCU8r36Fp/jXe8UNeyzQpJBWCwmnSaOGXmtLCke9rb?=
 =?us-ascii?Q?kHpN+qVlytwhituKx+HWYVex1KbXpo0l2Elsa9iFH9VkwIlN4ZN4xTpiH1fc?=
 =?us-ascii?Q?GOcXu5jWm29yjDk0HDJeSW0sOIgSzYRp3VM3n7Gnow8MlGM6gSRuJD4kSDyQ?=
 =?us-ascii?Q?5o+G67FW022uq2AsDtrAd/RZ0aQRIAEBpcLGQLNFbj9b8AETIpptTzUIfK12?=
 =?us-ascii?Q?96gsegKOKdKgWVEl8dkt9yYkDUK0o6m9xiwH3yQGFS3HfG8TcF6MCvRBnjAY?=
 =?us-ascii?Q?QFTg+0g+p8Gge66JO1XtzIQITMiqQurLqtP0nELtCISbC/BWPgoOc1WNSo8g?=
 =?us-ascii?Q?TTJDF7G5YkzcaCOkbueGjHwBC1HrXbuITzf4MLanxLW8GKSdSb8wUsOFaJ1O?=
 =?us-ascii?Q?szqpllB6fHvSjpJ50JeospiLT+8kQkEjmWyuzOoO/obctf3P7p1oSErUBnA5?=
 =?us-ascii?Q?7bvfFdhpCg2msGX7fHo0s0mHnIyJG++Qver8NvBopNvPw+1lf4Z/RSqQUbuE?=
 =?us-ascii?Q?LhWHUHpo3nI+56vfeI+GeklVgXhANUUVZJe/V6NW6dlgG89DEyzfBomPJgE5?=
 =?us-ascii?Q?CPkseveW4HbykRhUkB4hq9Dku3/bC3Cx1u14qWLiSa0wjCxHX73O8Y8E8f2r?=
 =?us-ascii?Q?X9zJhoIVGXcQYniSOIE5Xlrzo2j93erLAYQnXtw8xHNuUWRpiHSiGriJs5K0?=
 =?us-ascii?Q?effNNeBIc7nysS4YdVyZXfRXIK1odK8pVDx+f251N03rVG0kP5J8n+6P1RCj?=
 =?us-ascii?Q?eWZSsUr6XNyLrFICO8FYTea+FLQc3L+YjTGZj6bH2I5ql8r8RM6gKyzFvqUs?=
 =?us-ascii?Q?t6pBcBq5E6tBjni9eX6VC/ec7gNktr1Kb+Kn771nzOH402Kfk7IjUAxZ7VyT?=
 =?us-ascii?Q?53kuz6F1URanC7OYZUiv88ByLWPq+QAY2LLUmHNyRnXPIXv0f67dqSyTpYOp?=
 =?us-ascii?Q?gpmcM8YUWpJmXb5o0ASVFxjmNOhECb79Ke7AX6QrYKVqgcRzw8LZ62/e2XyK?=
 =?us-ascii?Q?sdn0eoe18z1M3DdjfV3kBvkpDf2M1+SewEthpEVR1UerdnLt?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB2717.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 04679cc0-b089-4767-0fa9-08da1294022b
X-MS-Exchange-CrossTenant-originalarrivaltime: 30 Mar 2022 21:26:52.6089
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: ilrCU2O5ANg/HosGcSnG8/kXVcPpUl3u934neE0KZtY2XWcHze19DW6alF35D8X6VqKyi8xjAK5NkQMdCL5eVw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR12MB2468
X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,RDNS_NONE,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE
	autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
	lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-ext4.vger.kernel.org>
X-Mailing-List: linux-ext4@vger.kernel.org

Please ignore this patch.

> -----Original Message-----
> From: amd-gfx <amd-gfx-bounces@lists.freedesktop.org> On Behalf Of Alex
> Sierra
> Sent: Wednesday, March 30, 2022 4:24 PM
> To: jgg@nvidia.com
> Cc: rcampbell@nvidia.com; willy@infradead.org; david@redhat.com;
> Kuehling, Felix <Felix.Kuehling@amd.com>; apopple@nvidia.com; amd-
> gfx@lists.freedesktop.org; linux-xfs@vger.kernel.org; linux-mm@kvack.org;
> jglisse@redhat.com; dri-devel@lists.freedesktop.org; akpm@linux-
> foundation.org; linux-ext4@vger.kernel.org; hch@lst.de
> Subject: [PATCH] drm/amdkfd: Add SVM API support capability bits
>=20
> From: Philip Yang <Philip.Yang@amd.com>
>=20
> SVMAPISupported property added to HSA_CAPABILITY, the value match
> HSA_CAPABILITY defined in Thunk spec:
>=20
> SVMAPISupported: it will not be supported on older kernels that don't hav=
e
> HMM or on systems with GFXv8 or older GPUs without support for 48-bit
> virtual addresses.
>=20
> CoherentHostAccess property added to HSA_MEMORYPROPERTY, the value
> match HSA_MEMORYPROPERTY defined in Thunk spec:
>=20
> CoherentHostAccess: whether or not device memory can be coherently
> accessed by the host CPU.
>=20
> Signed-off-by: Philip Yang <Philip.Yang@amd.com>
> Reviewed-by: Felix Kuehling <Felix.Kuehling@amd.com>
> Signed-off-by: Felix Kuehling <Felix.Kuehling@amd.com>
> Signed-off-by: Alex Deucher <alexander.deucher@amd.com>
> ---
>  drivers/gpu/drm/amd/amdkfd/kfd_topology.c |  6 ++++++
> drivers/gpu/drm/amd/amdkfd/kfd_topology.h | 10 ++++++----
>  2 files changed, 12 insertions(+), 4 deletions(-)
>=20
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_topology.c
> b/drivers/gpu/drm/amd/amdkfd/kfd_topology.c
> index cdef608db4f4..083ac9babfa8 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_topology.c
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_topology.c
> @@ -1419,6 +1419,12 @@ int kfd_topology_add_device(struct kfd_dev
> *gpu)
>  		dev->node_props.capability |=3D (adev->ras_features !=3D 0) ?
>  			HSA_CAP_RASEVENTNOTIFY : 0;
>=20
> +	/* SVM API and HMM page migration work together, device memory
> type
> +	 * is initialized to not 0 when page migration register device memory.
> +	 */
> +	if (adev->kfd.dev->pgmap.type !=3D 0)
> +		dev->node_props.capability |=3D
> HSA_CAP_SVMAPI_SUPPORTED;
> +
>  	kfd_debug_print_topology();
>=20
>  	if (!res)
> diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_topology.h
> b/drivers/gpu/drm/amd/amdkfd/kfd_topology.h
> index b8b68087bd7a..6bd6380b0ee0 100644
> --- a/drivers/gpu/drm/amd/amdkfd/kfd_topology.h
> +++ b/drivers/gpu/drm/amd/amdkfd/kfd_topology.h
> @@ -53,8 +53,9 @@
>  #define HSA_CAP_ASIC_REVISION_MASK		0x03c00000
>  #define HSA_CAP_ASIC_REVISION_SHIFT		22
>  #define HSA_CAP_SRAM_EDCSUPPORTED		0x04000000
> +#define HSA_CAP_SVMAPI_SUPPORTED		0x08000000
>=20
> -#define HSA_CAP_RESERVED			0xf80f8000
> +#define HSA_CAP_RESERVED			0xf00f8000
>=20
>  struct kfd_node_properties {
>  	uint64_t hive_id;
> @@ -98,9 +99,10 @@ struct kfd_node_properties {
>  #define HSA_MEM_HEAP_TYPE_GPU_LDS	4
>  #define HSA_MEM_HEAP_TYPE_GPU_SCRATCH	5
>=20
> -#define HSA_MEM_FLAGS_HOT_PLUGGABLE	0x00000001
> -#define HSA_MEM_FLAGS_NON_VOLATILE	0x00000002
> -#define HSA_MEM_FLAGS_RESERVED		0xfffffffc
> +#define HSA_MEM_FLAGS_HOT_PLUGGABLE		0x00000001
> +#define HSA_MEM_FLAGS_NON_VOLATILE		0x00000002
> +#define HSA_MEM_FLAGS_COHERENTHOSTACCESS	0x00000004
> +#define HSA_MEM_FLAGS_RESERVED			0xfffffff8
>=20
>  struct kfd_mem_properties {
>  	struct list_head	list;
> --
> 2.32.0