Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: [PATCH 2/3] drm/scheduler: Don't call wait_event_killable for
 signaled process.
To:     "Eric W. Biederman" <ebiederm@xmission.com>
Cc:     Oleg Nesterov <oleg@redhat.com>, linux-kernel@vger.kernel.org,
        amd-gfx@lists.freedesktop.org, Alexander.Deucher@amd.com,
        Christian.Koenig@amd.com, David.Panariti@amd.com,
        akpm@linux-foundation.org
References: <1524583836-12130-1-git-send-email-andrey.grodzovsky@amd.com>
 <1524583836-12130-3-git-send-email-andrey.grodzovsky@amd.com>
 <87muxsbmkp.fsf@xmission.com> <8840ac96-50c4-f94d-eb7c-f007940163f3@amd.com>
 <877eowa5qh.fsf@xmission.com> <20180425135552.GD7592@redhat.com>
 <f45c7017-c75a-7f0c-b6d5-4d79600cdd33@amd.com>
 <20180425171757.GA10441@redhat.com>
 <c39cb438-1e8a-4f2c-50f6-c6e2a144487b@amd.com> <874ljyu98e.fsf@xmission.com>
 <611911a3-2858-200c-d5f8-679c5f41ee3a@amd.com>
 <d6754c66-afe8-db2a-5633-d33701e22f3c@amd.com> <87zi1qq7t1.fsf@xmission.com>
From:   Andrey Grodzovsky <Andrey.Grodzovsky@amd.com>
Message-ID: <6e7f0d52-8d31-2ebb-47c3-865d676e8158@amd.com>
Date:   Thu, 26 Apr 2018 16:43:37 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.4.0
MIME-Version: 1.0
In-Reply-To: <87zi1qq7t1.fsf@xmission.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US
Received-SPF: None (protection.outlook.com: amd.com does not designate
 permitted sender hosts)
SpamDiagnosticOutput: 1:99
SpamDiagnosticMetadata: NSPM
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Apr 2018 20:43:48.1364 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 03866ef5-91a7-42b4-5f98-08d5abb66abe
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR1201MB0119
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 04/26/2018 11:57 AM, Eric W. Biederman wrote:
> Andrey Grodzovsky <Andrey.Grodzovsky@amd.com> writes:
>
>> On 04/26/2018 08:34 AM, Andrey Grodzovsky wrote:
>>>
>>> On 04/25/2018 08:01 PM, Eric W. Biederman wrote:
>>>> Andrey Grodzovsky <Andrey.Grodzovsky@amd.com> writes:
>>>>
>>>>> On 04/25/2018 01:17 PM, Oleg Nesterov wrote:
>>>>>> On 04/25, Andrey Grodzovsky wrote:
>>>>>>> here (drm_sched_entity_fini) is also a bad idea, but we still want to be
>>>>>>> able to exit immediately
>>>>>>> and not wait for GPU jobs completion when the reason for reaching this
>>>>>>> code
>>>>>>> is because of KILL
>>>>>>> signal to the user process who opened the device file.
>>>>>> Can you hook f_op->flush method?
>>>>> But this one is called for each task releasing a reference to the the file,
>>>>> so
>>>>> not sure I see how this solves the problem.
>>>> The big question is why do you need to wait during the final closing a
>>>> file?
>>>>
>>>> The wait can be terminated so the wait does not appear to be simply a
>>>> matter of correctness.
>>> Well, as I understand it, it just means that you don't want to abruptly
>>> terminate GPU work in progress without a good
>>> reason (such as KILL signal). When we exit we are going to release various
>>> resources GPU is still using so we either
>>> wait for it to complete or terminate the remaining jobs.
> At the point of do_exit you might as well be a KILL signal however you
> got there.
>
>> Looked more into code, some correction, drm_sched_entity_fini means the SW job
>> queue itself is about to die, so we must
>> either wait for completion or terminate any outstanding jobs that are still in
>> the SW queue. Anything which already in flight in HW
>> will still complete.
> It sounds like we don't care if we block the process that had the file
> descriptor open, this is just book keeping.  Which allows having a piece
> of code that cleans up resources when the GPU is done with the queue but
> does not make userspace wait.  (option 1)
>
> For it to make sense that we let the process run there has to be
> something that cares about the results being completed.  If all of the
> file descriptors are closed and the process is killed I can't see who
> will care that the software queue will continue to be processed.  So it
> may be reasonable to simply kill the queue (option 2).
>
> If userspace really needs the wait it is probably better done in
> f_op->flush so that every close of the file descriptor blocks
> until the queue is flushed (option 3).
>
> Do you know if userspace cares about the gpu operations completing?

I don't have a good answer for that, I would assume it depends on type
of jobs still remaining unprocessed and on the general type of work the user
process is doing.

Some key people who can answer this are currently away for a few days/week
so the answer for this will have to wait a bit.

Andrey

>
> My skim of the code suggests that nothing actually cares about those
> operations, but I really don't know the gpu well.
>
> Eric