Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <25ffdb03-b767-9e64-a636-55df2edc0d4c@amd.com>
Date:   Mon, 21 Feb 2022 09:24:50 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.5.0
Subject: Re: [PATCH] drm/amdgpu: check vm bo eviction valuable at last
Content-Language: en-US
To:     Qiang Yu <yuq825@gmail.com>,
        =?UTF-8?Q?Christian_K=c3=b6nig?= <ckoenig.leichtzumerken@gmail.com>
Cc:     David Airlie <airlied@linux.ie>,
        "Pan, Xinhui" <Xinhui.Pan@amd.com>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        dri-devel <dri-devel@lists.freedesktop.org>,
        linaro-mm-sig@lists.linaro.org, Qiang Yu <qiang.yu@amd.com>,
        amd-gfx@lists.freedesktop.org, Daniel Vetter <daniel@ffwll.ch>,
        Alex Deucher <alexander.deucher@amd.com>,
        Sumit Semwal <sumit.semwal@linaro.org>,
        linux-media@vger.kernel.org
References: <20220217090440.4468-1-qiang.yu@amd.com>
 <5d3fdd2c-e74a-49f4-2b28-32c06483236f@amd.com>
 <CAKGbVbtLTBJPF5eTu4rABUTBa8eqjQvqjo1AEUrzgPgYgCREuA@mail.gmail.com>
 <dac70c05-e712-d2e3-2267-278380895f1e@amd.com>
 <CAKGbVbvtLbDiKrX80-dMnipdLkTE+FP=g_mx37e12fuMtA1Y4Q@mail.gmail.com>
 <ca27a9c6-f390-a938-dd66-ac23f3b44dc4@amd.com>
 <CAKGbVbv4UFCybS_OFj5UkDgevbrB5qe3pv+0nHv9WdefYhy6Ww@mail.gmail.com>
 <6711073b-8771-5750-33f7-b72333b411c6@amd.com>
 <CAKGbVbvR+msXjrsXmDM8QTmsCP03hL5-q5CTJBYu4mm=NQd01A@mail.gmail.com>
 <a11b7073-6597-8e87-b724-33acab32e791@gmail.com>
 <CAKGbVbuJ-QdeoMTg=_O=1x5A5tbqZftsjt8aCCoVkAekci0USA@mail.gmail.com>
 <d830bb82-63ea-2de6-6d10-3a401ac0dcf0@amd.com>
 <CAKGbVbtorRius+Sq1_3SPUF3JzA00U747noSGhx7eP8Vn1rSDg@mail.gmail.com>
 <47c3a681-379e-18d4-86da-c48721081911@gmail.com>
 <CAKGbVbvmxOCZWYvB+ZSL7oHJmbm8vPgM-NJzadrEG1E=2c2Eyg@mail.gmail.com>
From:   =?UTF-8?Q?Christian_K=c3=b6nig?= <christian.koenig@amd.com>
In-Reply-To: <CAKGbVbvmxOCZWYvB+ZSL7oHJmbm8vPgM-NJzadrEG1E=2c2Eyg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?dm9IaDRjbys4b3UvNmxYYkkvVmZQbUFERUs1TWNjbFlWdlRXOWprME1sL0lE?=
 =?utf-8?B?QkhNZ0c2eXJMa1Z1UXN6LzJBakc3cnJtVjVTbXBPSE1Mcjk0Z1lHemo4TmJN?=
 =?utf-8?B?MjdQeDRPMXBnczBYZE5UbnlHVUllN3RFZHd2bk1PSG1VN2JVVXZWVHNXY042?=
 =?utf-8?B?T1l5MlNqalo1YkwwaEtFeTRYb2g2b1h4bWU1TFBVZHBOTUptMDBleVNGQVBX?=
 =?utf-8?B?bzE3RUl3ek41d0c3dHMzU3RwaS9FMWhUNmdjSVFjNlZxT2k4L2MvbDRLazR6?=
 =?utf-8?B?Q3RvamMxSUFNU2VpU2x6QStJWGxDZWU1NzlKdDdPQk00N2pNNDI5dmRKSDc4?=
 =?utf-8?B?S1RlSHY3cmFvMUhwVVBxVUJwQTl4V05UQlNOM2dRRGV0NVptUGxwdW5vQU1D?=
 =?utf-8?B?MW00KzhGU3hFa3lCYmhrWW1ITTlkb2lGQUN4VmlKZTVPOUlHcmRsL2Zyajc3?=
 =?utf-8?B?RzlFdzlSYmdUM3BLb3RacGhUS013a1pFUkJwNW50SlRrZnBMOUVVWGZmdlB4?=
 =?utf-8?B?VXdpQzhtS2dwVGNRZWpldmdCeHpTTFkxTlRGY1dmYi9ZOWUzMnlBMms1U0tt?=
 =?utf-8?B?T1d2UnI3bnAvaGlyaUIwSXczbHhmZU9WaWhob2ZUdGUxY0pXSjcxS0ZIajg2?=
 =?utf-8?B?NUlBN0Fqc0I5elFpdEo2N2hBanp3RzNqZHVNOUhIZmtIN0VTZG0xSzRiV0NW?=
 =?utf-8?B?MnRMRnJKckN1YkJNd2JDOFlIYXRZYXhjaTFET2dvenNNM3lPRlpiVzBranJh?=
 =?utf-8?B?OHpGTU9pWDA1dlpaUWh3OU1ub3cvemtKTTA3bUw0Q3loNnY1dmtFYUNwSUhR?=
 =?utf-8?B?eVpncFpKankrSEdzcTZPVmluc1NiQXprbnJuYXNPWVV1TGdHeGU5eTFqc21D?=
 =?utf-8?B?WHozUW9KNU5kTzIwblNWcGpvQnFRb1dkaWhMNjlSNGdneFZBWjJ0dkF6dWxI?=
 =?utf-8?B?RXVaYTZINU5uVVI0d3lNK1RuUW5nRUt2RVdVSjEwallNdFlDc0Q4R2xYaTBD?=
 =?utf-8?B?L2w1bi9wZHVIQlpxVmNDeGVJd25xcDBDTjN3d0lLeTQycEhoT0kxdnVTaG11?=
 =?utf-8?B?ZTVkOXZXY0VoU0YxbFpreDJjeU9WNkZGdllZODRKNXJaTTNJdmVpWFg2YWRz?=
 =?utf-8?B?MmJqYmxJOEVCdjc2eFZVUzBSQ2kwRGNsS2RyY2pod1VUTHo4YnpmM0N0TDB1?=
 =?utf-8?B?QUsrTE5RNTV6S213VVBQRzAxNDFMLzJSWmFVdVp4clo2eFdpWlo0Z3dwMGxD?=
 =?utf-8?B?WDQzVGFKOFU0OG5RS3VEUHRwNjd6RVNoaTBTdExHaGxaYU5hZ0dsWGdpR01V?=
 =?utf-8?B?OXlZaDZoSlpWRVNteXU2bmVVWHRBU2IrMExHVVR2YzdSWnJ1RkswVHZxWDZn?=
 =?utf-8?B?TEhMNEVRamhuQmhEaXRaZUdZZGcwbTUwekZQNEd4NkhkYlJQQWU1QmZwaDAr?=
 =?utf-8?B?YURXOXRjV2d3a2c4Vi8rVGhTYXJzNzVQcnlzTncwSnRJQ0o3cnhtcEZaR2x0?=
 =?utf-8?B?Mng5MzFSVXhXR1FFSmxzN3hHdzcxbThmRHg4SDZWeGcyYnNmRHNrOGl5WlNw?=
 =?utf-8?B?NnZGYm9ISlh5QThUWGlsQTBkQ2YvR0V4TFhjKzdnZ2c4YWloSU1QdkdrM094?=
 =?utf-8?B?UUZtYm5vTnlHQ1RNWlNJaERVN01oSXcvdll5SFJaOFp3UC9hTG83NEJRdXdi?=
 =?utf-8?B?RjZyNlJZdFN3bTdaWm9UMFVQVjN6WXJERTA4eXBSWEMzNGRheUNTOTFFbGM4?=
 =?utf-8?B?Tmtia1ZiMWVVN3NxSDR1cndYOHJvTDQ3UWw5QytIMmQ3aEZVRTIwNjBYbEdY?=
 =?utf-8?B?UUUyNGZsRkE3NkxXRkRLOU41TnQxZ0Ftdnh1bE1PaTdaYnpiTGZ3cjdZOGhw?=
 =?utf-8?B?cHFPZVBjdWM4dzFGeE05cktZVlRSK005b29HbVBlT3BLUmNWakFHNDZOZklJ?=
 =?utf-8?B?eFM3VFQwVXBDQWdKSDRVaDVKL0VoTnRPL2FpdlhKZkFOVFhwa09UWVpPUWJu?=
 =?utf-8?B?N1Rjb3MrWEdKQm55a2JMamJKMnpuOGpWOFRlckdNQUFLUm54VnovdEN4Zmpi?=
 =?utf-8?B?WEt5ZFhIWkdkcG5nVCt4aWgzWHgra0ZHNFMzTWx1cU9VdzdVbHhBd1hMakxi?=
 =?utf-8?Q?6ve8=3D?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f0cad0cb-c090-42f7-7e9e-08d9f513a566
X-MS-Exchange-CrossTenant-AuthSource: BN8PR12MB3587.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Feb 2022 08:24:58.0589
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: I3FeNYtehwlqtVQCeb8WZTmeMR1Pcgxmynx+qu99BYLwe/sQgq73hPjGm4J5M7vR
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB4709
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Am 21.02.22 um 04:28 schrieb Qiang Yu:
> On Fri, Feb 18, 2022 at 6:24 PM Christian König
> <ckoenig.leichtzumerken@gmail.com> wrote:
>> Am 18.02.22 um 11:16 schrieb Qiang Yu:
>>> [SNIP]
>>>>> If amdgpu_vm_ready() use evicting flag, it's still not equivalent to check
>>>>> vm idle: true -> vm idle, false -> vm may be idle or busy.
>>>> Yeah, but why should that be relevant?
>>>>
>>>> The amdgpu_vm_ready() return if we can do page table updates or not. If
>>>> the VM is idle or not is only relevant for eviction.
>>>>
>>>> In other words any CS or page table update makes the VM busy, but that
>>>> only affects if the VM can be evicted or not.
>>>>
>>> My point is: we can't use amdgpu_vm_ready() to replace vm_is_busy(), so
>>> currently we update vm even when vm is busy. So why not use:
> Sorry, should be "vm is idle".
>
>>> if (!amdgpu_vm_ready() || vm_is_busy()) return;
>>> in amdgpu_gem_va_update_vm(), as you mentioned we prefer to not
>>> update vm when it's idle.
>> Because updating the VM while it is busy is perfectly fine, we do it all
>> the time.
>>
> Yeah, as above, my typo.
>
>> We should just not update it when it is already idle and was considered
>> for eviction.
> "and", not "or"?
>
>> In this situation it makes most of the time sense to keep
>> it idle and postpone the update till the next command submission.
>>
>>>>>>> Then we can keep the evicting flag accurate (after solving your
>>>>>>> concern for this patch that eviction may fail latter by further delay
>>>>>>> the flag update after eviction success).
>>>>>> That won't work. See we need to mark the VM as evicted before we
>>>>>> actually evict them because otherwise somebody could use the VM in
>>>>>> parallel and add another fence to it.
>>>>>>
>>>>> I see, make this too accurate should cost too much like holding the
>>>>> eviction_lock when eviction. But just delay it in
>>>>> amdgpu_ttm_bo_eviction_valuable()
>>>>> could avoid most false positive case.
>>>> Partially correct. Another fundamental problem is that we can't hold the
>>>> eviction lock because that would result in lock inversion and potential
>>>> deadlock.
>>>>
>>>> We could set the flag later on, but as I said before that when we set
>>>> the evicted flag when the VM is already idle is a desired effect.
>>>>
>>> As above, this confuse me as we can explicitly check vm idle when
>>> user update vm, why bother to embed it in evicting flag implicitly?
>> Well as I said it's irrelevant for the update if the VM is idle or not.
>>
>> To summarize the rules once more:
>> 1. When VM page tables are used by CS or page tables updates it is
>> considered busy, e.g. not idle.
>>
>> 2. When we want to evict a VM it must be idle. As soon as we considered
>> this we should set the evicted flag to make sure to keep it idle as much
>> as possible.
>>
>> 3. When we want to update the page tables we just need to check if the
>> VM is idle or not.
>>
> But now we does not check vm idle directly in amdgpu_gem_va_update_vm().
> If VM bo has not been considered for eviction, it could be either idle or busy.
>
> Just want to confirm if the fix should be only change amdgpu_vm_ready()
> to use evicting flag or besides using evicting flag, also check vm_idle() in
> amdgpu_gem_va_update_vm().

Only changing the amdgpu_vm_ready() should be enough. It can be that 
this then bubbles up more issue, but those need to be taken care of 
separately then.

Regards,
Christian.

>
> Regards,
> Qiang
>
>> 4. When a CS happens we don't have another chance and make the VM busy
>> again. And do all postponed page table updates.
>>
> Anyway,
>
>> Regards,
>> Christian.
>>
>>> Check vm idle need to hold resv lock. Read your patch for adding
>>> evicting flag is to update vm without resv lock. But user vm ops in
>>> amdgpu_gem_va_update_vm() do hold the resv lock, so the difference
>>> happens when calling amdgpu_vm_bo_update_mapping() from
>>> svm_range_(un)map_to_gpu(). So embed vm idle in evicting flag
>>> is for svm_range_(un)map_to_gpu() also do nothing when vm idle?
>>
>>
>>> Regards,
>>> Qiang
>>>
>>>> Regards,
>>>> Christian.
>>>>
>>>>> Regards,
>>>>> Qiang
>>>>>
>>>>>> Regards,
>>>>>> Christian.
>>>>>>
>>>>>>> Regards,
>>>>>>> Qiang
>>>>>>>
>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Christian.
>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Qiang
>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Christian.
>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Qiang
>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Christian.
>>>>>>>>>>>>
>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>> Qiang
>>>>>>>>>>>>>
>>>>>>>>>>>>>> What we should rather do is to fix amdgpu_vm_ready() to take a look at
>>>>>>>>>>>>>> the flag instead of the linked list.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards,
>>>>>>>>>>>>>> Christian.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Signed-off-by: Qiang Yu <qiang.yu@amd.com>
>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>>>          drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 85 ++++++++++++++-----------
>>>>>>>>>>>>>>>          1 file changed, 47 insertions(+), 38 deletions(-)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
>>>>>>>>>>>>>>> index 5a32ee66d8c8..88a27911054f 100644
>>>>>>>>>>>>>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
>>>>>>>>>>>>>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
>>>>>>>>>>>>>>> @@ -1306,45 +1306,11 @@ uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm,
>>>>>>>>>>>>>>>              return flags;
>>>>>>>>>>>>>>>          }
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -/*
>>>>>>>>>>>>>>> - * amdgpu_ttm_bo_eviction_valuable - Check to see if we can evict a buffer
>>>>>>>>>>>>>>> - * object.
>>>>>>>>>>>>>>> - *
>>>>>>>>>>>>>>> - * Return true if eviction is sensible. Called by ttm_mem_evict_first() on
>>>>>>>>>>>>>>> - * behalf of ttm_bo_mem_force_space() which tries to evict buffer objects until
>>>>>>>>>>>>>>> - * it can find space for a new object and by ttm_bo_force_list_clean() which is
>>>>>>>>>>>>>>> - * used to clean out a memory space.
>>>>>>>>>>>>>>> - */
>>>>>>>>>>>>>>> -static bool amdgpu_ttm_bo_eviction_valuable(struct ttm_buffer_object *bo,
>>>>>>>>>>>>>>> -                                         const struct ttm_place *place)
>>>>>>>>>>>>>>> +static bool amdgpu_ttm_mem_eviction_valuable(struct ttm_buffer_object *bo,
>>>>>>>>>>>>>>> +                                          const struct ttm_place *place)
>>>>>>>>>>>>>>>          {
>>>>>>>>>>>>>>>              unsigned long num_pages = bo->resource->num_pages;
>>>>>>>>>>>>>>>              struct amdgpu_res_cursor cursor;
>>>>>>>>>>>>>>> -     struct dma_resv_list *flist;
>>>>>>>>>>>>>>> -     struct dma_fence *f;
>>>>>>>>>>>>>>> -     int i;
>>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>> -     /* Swapout? */
>>>>>>>>>>>>>>> -     if (bo->resource->mem_type == TTM_PL_SYSTEM)
>>>>>>>>>>>>>>> -             return true;
>>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>> -     if (bo->type == ttm_bo_type_kernel &&
>>>>>>>>>>>>>>> -         !amdgpu_vm_evictable(ttm_to_amdgpu_bo(bo)))
>>>>>>>>>>>>>>> -             return false;
>>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>> -     /* If bo is a KFD BO, check if the bo belongs to the current process.
>>>>>>>>>>>>>>> -      * If true, then return false as any KFD process needs all its BOs to
>>>>>>>>>>>>>>> -      * be resident to run successfully
>>>>>>>>>>>>>>> -      */
>>>>>>>>>>>>>>> -     flist = dma_resv_shared_list(bo->base.resv);
>>>>>>>>>>>>>>> -     if (flist) {
>>>>>>>>>>>>>>> -             for (i = 0; i < flist->shared_count; ++i) {
>>>>>>>>>>>>>>> -                     f = rcu_dereference_protected(flist->shared[i],
>>>>>>>>>>>>>>> -                             dma_resv_held(bo->base.resv));
>>>>>>>>>>>>>>> -                     if (amdkfd_fence_check_mm(f, current->mm))
>>>>>>>>>>>>>>> -                             return false;
>>>>>>>>>>>>>>> -             }
>>>>>>>>>>>>>>> -     }
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>              switch (bo->resource->mem_type) {
>>>>>>>>>>>>>>>              case AMDGPU_PL_PREEMPT:
>>>>>>>>>>>>>>> @@ -1377,10 +1343,53 @@ static bool amdgpu_ttm_bo_eviction_valuable(struct ttm_buffer_object *bo,
>>>>>>>>>>>>>>>                      return false;
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>              default:
>>>>>>>>>>>>>>> -             break;
>>>>>>>>>>>>>>> +             return ttm_bo_eviction_valuable(bo, place);
>>>>>>>>>>>>>>>              }
>>>>>>>>>>>>>>> +}
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -     return ttm_bo_eviction_valuable(bo, place);
>>>>>>>>>>>>>>> +/*
>>>>>>>>>>>>>>> + * amdgpu_ttm_bo_eviction_valuable - Check to see if we can evict a buffer
>>>>>>>>>>>>>>> + * object.
>>>>>>>>>>>>>>> + *
>>>>>>>>>>>>>>> + * Return true if eviction is sensible. Called by ttm_mem_evict_first() on
>>>>>>>>>>>>>>> + * behalf of ttm_bo_mem_force_space() which tries to evict buffer objects until
>>>>>>>>>>>>>>> + * it can find space for a new object and by ttm_bo_force_list_clean() which is
>>>>>>>>>>>>>>> + * used to clean out a memory space.
>>>>>>>>>>>>>>> + */
>>>>>>>>>>>>>>> +static bool amdgpu_ttm_bo_eviction_valuable(struct ttm_buffer_object *bo,
>>>>>>>>>>>>>>> +                                         const struct ttm_place *place)
>>>>>>>>>>>>>>> +{
>>>>>>>>>>>>>>> +     struct dma_resv_list *flist;
>>>>>>>>>>>>>>> +     struct dma_fence *f;
>>>>>>>>>>>>>>> +     int i;
>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>> +     /* Swapout? */
>>>>>>>>>>>>>>> +     if (bo->resource->mem_type == TTM_PL_SYSTEM)
>>>>>>>>>>>>>>> +             return true;
>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>> +     /* If bo is a KFD BO, check if the bo belongs to the current process.
>>>>>>>>>>>>>>> +      * If true, then return false as any KFD process needs all its BOs to
>>>>>>>>>>>>>>> +      * be resident to run successfully
>>>>>>>>>>>>>>> +      */
>>>>>>>>>>>>>>> +     flist = dma_resv_shared_list(bo->base.resv);
>>>>>>>>>>>>>>> +     if (flist) {
>>>>>>>>>>>>>>> +             for (i = 0; i < flist->shared_count; ++i) {
>>>>>>>>>>>>>>> +                     f = rcu_dereference_protected(flist->shared[i],
>>>>>>>>>>>>>>> +                             dma_resv_held(bo->base.resv));
>>>>>>>>>>>>>>> +                     if (amdkfd_fence_check_mm(f, current->mm))
>>>>>>>>>>>>>>> +                             return false;
>>>>>>>>>>>>>>> +             }
>>>>>>>>>>>>>>> +     }
>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>> +     /* Check by different mem type. */
>>>>>>>>>>>>>>> +     if (!amdgpu_ttm_mem_eviction_valuable(bo, place))
>>>>>>>>>>>>>>> +             return false;
>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>> +     /* VM bo should be checked at last because it will mark VM evicting. */
>>>>>>>>>>>>>>> +     if (bo->type == ttm_bo_type_kernel)
>>>>>>>>>>>>>>> +             return amdgpu_vm_evictable(ttm_to_amdgpu_bo(bo));
>>>>>>>>>>>>>>> +
>>>>>>>>>>>>>>> +     return true;
>>>>>>>>>>>>>>>          }
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>          static void amdgpu_ttm_vram_mm_access(struct amdgpu_device *adev, loff_t pos,