Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: drm_sched with panfrost crash on T820
To:     "Grodzovsky, Andrey" <Andrey.Grodzovsky@amd.com>,
        Hillf Danton <hdanton@sina.com>
References: <e450fbe6-dec7-2704-59c2-db7e869d67f5@baylibre.com>
 <f0ab487e-8d49-987b-12b8-7a115a6543e1@amd.com>
 <20190930145228.14000-1-hdanton@sina.com>
 <d2888614-8644-7d04-b73b-3ab7c6623e9a@amd.com>
Cc:     "daniel@ffwll.ch" <daniel@ffwll.ch>,
        "airlied@linux.ie" <airlied@linux.ie>,
        "Koenig, Christian" <Christian.Koenig@amd.com>,
        Erico Nunes <nunes.erico@gmail.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "steven.price@arm.com" <steven.price@arm.com>,
        "dri-devel@lists.freedesktop.org" <dri-devel@lists.freedesktop.org>,
        Rob Herring <robh@kernel.org>,
        Tomeu Vizoso <tomeu.vizoso@collabora.com>,
        "open list:ARM/Amlogic Meson..." <linux-amlogic@lists.infradead.org>
From:   Neil Armstrong <narmstrong@baylibre.com>
Message-ID: <7339b7a1-2d1c-4379-89a0-daf8b28d81c8@baylibre.com>
Date:   Thu, 3 Oct 2019 10:34:19 +0200
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:45.0)
 Gecko/20100101 Thunderbird/45.8.0
MIME-Version: 1.0
In-Reply-To: <d2888614-8644-7d04-b73b-3ab7c6623e9a@amd.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Hi Andrey,

Le 02/10/2019 à 16:40, Grodzovsky, Andrey a écrit :
> 
> On 9/30/19 10:52 AM, Hillf Danton wrote:
>> On Mon, 30 Sep 2019 11:17:45 +0200 Neil Armstrong wrote:
>>> Did a new run from 5.3:
>>>
>>> [   35.971972] Call trace:
>>> [   35.974391]  drm_sched_increase_karma+0x5c/0xf0
>>> 			ffff000010667f38	FFFF000010667F94
>>> 			drivers/gpu/drm/scheduler/sched_main.c:335
>>>
>>> The crashing line is :
>>>                                  if (bad->s_fence->scheduled.context ==
>>>                                      entity->fence_context) {
>>>
>>> Doesn't seem related to guilty job.
>> Bail out if s_fence is no longer fresh.
>>
>> --- a/drivers/gpu/drm/scheduler/sched_main.c
>> +++ b/drivers/gpu/drm/scheduler/sched_main.c
>> @@ -333,6 +333,10 @@ void drm_sched_increase_karma(struct drm
>>   
>>   			spin_lock(&rq->lock);
>>   			list_for_each_entry_safe(entity, tmp, &rq->entities, list) {
>> +				if (!smp_load_acquire(&bad->s_fence)) {
>> +					spin_unlock(&rq->lock);
>> +					return;
>> +				}
>>   				if (bad->s_fence->scheduled.context ==
>>   				    entity->fence_context) {
>>   					if (atomic_read(&bad->karma) >
>> @@ -543,7 +547,7 @@ EXPORT_SYMBOL(drm_sched_job_init);
>>   void drm_sched_job_cleanup(struct drm_sched_job *job)
>>   {
>>   	dma_fence_put(&job->s_fence->finished);
>> -	job->s_fence = NULL;
>> +	smp_store_release(&job->s_fence, 0);
>>   }
>>   EXPORT_SYMBOL(drm_sched_job_cleanup);
> 

This fixed the problem on the 10 CI runs.

Neil

> 
> Does this change help the problem ? Note that drm_sched_job_cleanup is 
> called from scheduler thread which is stopped at all times when work_tdr 
> thread is running and anyway the 'bad' job is still in the 
> ring_mirror_list while it's being accessed from  
> drm_sched_increase_karma so I don't think drm_sched_job_cleanup can be 
> called for it BEFORE or while drm_sched_increase_karma is executed.
> 
> Andrey
> 
> 
>>   
>> --
>>