Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) client-ip=2620:137:e000::1:18;
Message-ID: <6e0ebf4b-ffd7-df55-8ae9-472878f22605@amd.com>
Date:   Mon, 21 Feb 2022 11:32:10 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.5.0
Subject: Re: [PATCH] drm/amdgpu: check vm ready by evicting
Content-Language: en-US
To:     Qiang Yu <qiang.yu@amd.com>,
        Alex Deucher <alexander.deucher@amd.com>,
        "Pan, Xinhui" <Xinhui.Pan@amd.com>,
        David Airlie <airlied@linux.ie>,
        Daniel Vetter <daniel@ffwll.ch>
Cc:     amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org,
        linux-kernel@vger.kernel.org
References: <20220221101239.2863-1-qiang.yu@amd.com>
From:   =?UTF-8?Q?Christian_K=c3=b6nig?= <christian.koenig@amd.com>
In-Reply-To: <20220221101239.2863-1-qiang.yu@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TmNKOW03cThGSkJBS1ZueGJzaFlZeStQNmtmNlB1azBubGdocmgrNTJ4Yzd3?=
 =?utf-8?B?L05DSHFYblpiMVdFK0VXc1hDTis0Q0hPOFRBWm10bm9taW5hMS94S2Y1akJF?=
 =?utf-8?B?bUxGSy9WRUxpQUFnZi9nTm5IWmNjZnNQUi85emQzQWdTQnpwZjVzcmF1dUFR?=
 =?utf-8?B?Nm9ZY2VVZWNVaE4xRkpCQXF4RjJCdmF1NVJPdm15T0ZuSWZMN051NVozZkdp?=
 =?utf-8?B?OWc2S2x2d3d1dGhBdkV2OHBhRUEzdFIweHpTaTFLbFJlM0t6Q1FyNlQzVVh0?=
 =?utf-8?B?cW0wMDVuek9WQmwvK0h2TWJDTTF1NjhoVWtJeEZkWXBVZnk2c2NWR0lPQk1U?=
 =?utf-8?B?NHlRN0xYNHh4TWJTdS92Z2N2WVEyUkIyNzNxc1Q2UEpVUjhhTUQ0YlI2L1NI?=
 =?utf-8?B?M3RNTm1QQ3ByRkNWYWtuR3IyTEFkZGZVVGlDSUdPcWNXTVV5VURTaHgyYUxX?=
 =?utf-8?B?cnFEK0w0d1NTa1RjdWl5a0phaUhpdzJPRlo4eWZkOWVwS003U0o3UWkxUHUz?=
 =?utf-8?B?WjZINDllZndSNGZ6dTlRdk05NDNUMUZvSTUzSkVidXhPaURjcnM5dE5WdlFy?=
 =?utf-8?B?d2tiMGI2VGRhb1VDRWNKOUk4Q09rTDYwYWtIaVVPQVdRTjAvNXVkdERybmpU?=
 =?utf-8?B?ZEd6UlhpeW0zc1ovR080UCtZNXFJZm1VeGJQampnSEpkM1NtVS9VR2hBc1Y5?=
 =?utf-8?B?TzI3RC9TTlRxMGpLT0owc2V1QXd0T2NyL0FvYndQaTJodFdMZEVyMXdVS0dv?=
 =?utf-8?B?L01GdXVEZGRpUm1TdXdsaXUweHorSHNMZUhQdkZsSXljOEFaTWwyK0J0V0Jl?=
 =?utf-8?B?MHpKOVVXeWZQRnhPOEVPNXh1bXVZSGJ0VTd1MWZhMndhYnBXMzhyYzM2K2tu?=
 =?utf-8?B?MnpYUUxOTUhqQ3RWZWNkdWxFWHN0SzJpSVhRd014MWdUakpaSHVwd1hmcXZ6?=
 =?utf-8?B?cGdVK2JFeDdFUjZPU2JWMDliRERHVmcxZ2ZyditxaGxZMGlseUxKYWpNRG8w?=
 =?utf-8?B?UkJUNFZ0d1JTdWU1NW1vczdKNjdOeVB4bGozUWJKcTc3Q1dROW1aajBMQTcv?=
 =?utf-8?B?T3lWemhwR1REUW9qdy93MmhrWE90dWIxNENORU5vSWtPU0J5SThqSlhHMlNi?=
 =?utf-8?B?ek1PQndtekFyMkZtcFAwOFp0VFFzRU1GMlgyL0haTXpaMmlFcFpsYjdsdHNL?=
 =?utf-8?B?VWp3SXFqMFpGWGh2emJkWFVERCtPUU14TG8zWnYxNDdsbWIwRURjbzAwWEp3?=
 =?utf-8?B?T3U0eTBkaEFyVWVCcGZvUVFOQWF4YkQ3YXFxSWlwZm1CNW9BTENWemFoTWph?=
 =?utf-8?B?NUVhcTlqQmhiWWtFU09GRGsya3Q0VTBCRExtbXpIRVJIaVZ6Q0FCZTliWElh?=
 =?utf-8?B?Rmo3UGNwOFBZZzU0aTNPWDBRaFVQbkRMY3FTdkJObm5xRGpYV2o5ajRHclN0?=
 =?utf-8?B?aUNwcG5jVkx3K3hNVkQ2OEdEQ2VjTjI1TDZlMjlMNXZpQVMyWEMxY01nTEEz?=
 =?utf-8?B?R1pmb1lIejJWN1dmemJGaFAxMHh1QnhVMmQzNklLclFybWZ2RU8wU1NaL05v?=
 =?utf-8?B?VFpXT0psekhFU3dwRmlkc1prT3FjUDlsUHJyUTVhbFliaVZYOGFlMTJFT1F2?=
 =?utf-8?B?RUI4eE1EUHh2d3RvUDhRQTZ2bC9PSlNIdmMxYmdwdERCeU8yRUhIUVNjeHp2?=
 =?utf-8?B?elFROUNva0RRUDRjbkpSdjJjVFNvcklDKzVLK01tSkxENFBBTlR1bGppMCtv?=
 =?utf-8?B?dDhqcCttcm94R1Y1VmxCYWw0VHYweHRjdzBlREFUa05ZUHR2UWxlbHFhSEVR?=
 =?utf-8?B?SHQ1YTBEVFdXdzczTnVPemU4RXB0dHRLa2JDUFA2UmFtZkE2Wm5ZRi9IOW5T?=
 =?utf-8?B?Rk9YRWN0TnpCaU5mUU03VDBJcTFjS3hlS2dlaTdURHIxTGtnZllGUkhjTGht?=
 =?utf-8?B?OWh3WmVoYTdHT3NURXJLQ1NRUi9NRVNYSUEvVUNuaE9JQUQycWgyN3dGaTVx?=
 =?utf-8?B?b3hncFE1RHVWS2VJNU5RTVJZaWcvSGYvTVlueDd2OTlkQkJTZzJoVUVQVTRw?=
 =?utf-8?B?TTF4ZTNBQVUvYXNzcVhVc3YwYjg2cHNWSUR2ZFFmVmJVUVdDdUpaWnVPbGk3?=
 =?utf-8?Q?OI8o=3D?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9353dc55-295c-4a2b-7277-08d9f5256e19
X-MS-Exchange-CrossTenant-AuthSource: BN8PR12MB3587.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Feb 2022 10:32:16.0642
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Tlt1Aufo+vZlj0sno3JcJPl0o4VNqymRGIfTeX0lT4WaSCEuWGu4mWinpKvtTvOS
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR12MB5475
X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,NICE_REPLY_A,RDNS_NONE,SPF_HELO_NONE,
	T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
	lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


Am 21.02.22 um 11:12 schrieb Qiang Yu:
> Workstation application ANSA/META get this error dmesg:
> [drm:amdgpu_gem_va_ioctl [amdgpu]] *ERROR* Couldn't update BO_VA (-16)
>
> This is caused by:
> 1. create a 256MB buffer in invisible VRAM
> 2. CPU map the buffer and access it causes vm_fault and try to move
>     it to visible VRAM
> 3. force visible VRAM space and traverse all VRAM bos to check if
>     evicting this bo is valuable
> 4. when checking a VM bo (in invisible VRAM), amdgpu_vm_evictable()
>     will set amdgpu_vm->evicting, but latter due to not in visible
>     VRAM, won't really evict it so not add it to amdgpu_vm->evicted
> 5. before next CS to clear the amdgpu_vm->evicting, user VM ops
>     ioctl will pass amdgpu_vm_ready() (check amdgpu_vm->evicted)
>     but fail in amdgpu_vm_bo_update_mapping() (check
>     amdgpu_vm->evicting) and get this error log
>
> This error won't affect functionality as next CS will finish the
> waiting VM ops. But we'd better clear the error log by check the
> evicting flag which really stop VM ops latter.
>
> Signed-off-by: Qiang Yu <qiang.yu@amd.com>

Reviewed-by: Christian König <christian.koenig@amd.com>

Good work.

> ---
>   drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 9 +++++++--
>   1 file changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> index 37acd8911168..2cd9f1a2e5fa 100644
> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
> @@ -770,11 +770,16 @@ int amdgpu_vm_validate_pt_bos(struct amdgpu_device *adev, struct amdgpu_vm *vm,
>    * Check if all VM PDs/PTs are ready for updates
>    *
>    * Returns:
> - * True if eviction list is empty.
> + * True if VM is not evicting.
>    */
>   bool amdgpu_vm_ready(struct amdgpu_vm *vm)
>   {
> -	return list_empty(&vm->evicted);
> +	bool ret;
> +
> +	amdgpu_vm_eviction_lock(vm);
> +	ret = !vm->evicting;
> +	amdgpu_vm_eviction_unlock(vm);
> +	return ret;
>   }
>   
>   /**