Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S946325AbcJaUJH (ORCPT ); Mon, 31 Oct 2016 16:09:07 -0400 Received: from mx1.redhat.com ([209.132.183.28]:53714 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S946313AbcJaUI0 (ORCPT ); Mon, 31 Oct 2016 16:08:26 -0400 Date: Mon, 31 Oct 2016 21:08:23 +0100 From: Oleg Nesterov To: Thomas Gleixner Cc: Andy Lutomirski , Roman Pen , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Tejun Heo , "linux-kernel@vger.kernel.org" , Chunming Zhou , Alex Deucher Subject: [PATCH 2/2] kthread: don't use to_live_kthread() in kthread_park() and kthread_unpark() Message-ID: <20161031200823.GC19430@redhat.com> References: <20161025110508.9052-1-roman.penyaev@profitbricks.com> <20161025140333.GB4326@redhat.com> <20161025154301.GA12015@redhat.com> <20161026141359.GA6893@redhat.com> <20161026155155.GA28832@redhat.com> <20161028161106.GA8933@redhat.com> <20161031200729.GA19430@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20161031200729.GA19430@redhat.com> User-Agent: Mutt/1.5.24 (2015-08-30) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Mon, 31 Oct 2016 20:08:26 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 5041 Lines: 156 Now that to_kthread() is always valid we can change kthread_park() and kthread_unpark() to use it and kill to_live_kthread(). kthread_unpark() is trivial, if KTHREAD_IS_PARKED is set we know that this kthread has called complete(&self->parked), we do not care if it exits after that if we race with kthread_stop(). kthread_park() is more tricky, but only because its semantics is not well defined. It returns -ENOSYS if the thread exited but this can never happen and as Roman pointed out kthread_park() can obviously block forever if it could race with the exiting kthread. I think we need to unexport kthread_park/unpark, and either make it return "void" or actually fix the race with kthred_stop/exit. This patch just adds WARN_ON(PF_EXITING) for now. The usage of kthread_park() in cpuhp code (cpu.c, smpboot.c, stop_machine.c) is fine. It can never see an exiting/exited kthread, smpboot_destroy_threads() clears *ht->store, smpboot_park_thread() checks it is not NULL under the same smpboot_threads_lock. cpuhp_threads and cpu_stop_threads never exit, so other callers are fine too. But it has two more users: - watchdog_park_threads() and it does not look nice. The code is actually correct, get_online_cpus() ensures that kthread_park() can't race with itself (note that kthread_park() can't handle this race correctly), but imo it should not use kthread_park() directly. - drivers/gpu/drm/amd/scheduler/gpu_scheduler.c and I think it should not use kthread_park() too. But this patch should not break this code. kthread_park() must not be called after amd_sched_fini() which does kthread_stop(), otherwise even to_live_kthread() is not safe because task_struct can be already freed and sched->thread can point to nowhere. Signed-off-by: Oleg Nesterov --- kernel/kthread.c | 69 ++++++++++++++++++++------------------------------------ 1 file changed, 24 insertions(+), 45 deletions(-) diff --git a/kernel/kthread.c b/kernel/kthread.c index 4dcbc8b..01d2716 100644 --- a/kernel/kthread.c +++ b/kernel/kthread.c @@ -78,21 +78,6 @@ void free_kthread_struct(struct task_struct *k) kfree(to_kthread(k)); } -#define __to_kthread(vfork) \ - container_of(vfork, struct kthread, exited) - -/* - * TODO: kill it and use to_kthread(). But we still need the users - * like kthread_stop() which has to sync with the exiting kthread. - */ -static struct kthread *to_live_kthread(struct task_struct *k) -{ - struct completion *vfork = ACCESS_ONCE(k->vfork_done); - if (likely(vfork)) - return __to_kthread(vfork); - return NULL; -} - /** * kthread_should_stop - should this kthread return now? * @@ -441,8 +426,18 @@ struct task_struct *kthread_create_on_cpu(int (*threadfn)(void *data), return p; } -static void __kthread_unpark(struct task_struct *k, struct kthread *kthread) +/** + * kthread_unpark - unpark a thread created by kthread_create(). + * @k: thread created by kthread_create(). + * + * Sets kthread_should_park() for @k to return false, wakes it, and + * waits for it to return. If the thread is marked percpu then its + * bound to the cpu again. + */ +void kthread_unpark(struct task_struct *k) { + struct kthread *kthread = to_kthread(k); + clear_bit(KTHREAD_SHOULD_PARK, &kthread->flags); /* * We clear the IS_PARKED bit here as we don't wait @@ -460,22 +455,6 @@ static void __kthread_unpark(struct task_struct *k, struct kthread *kthread) wake_up_state(k, TASK_PARKED); } } - -/** - * kthread_unpark - unpark a thread created by kthread_create(). - * @k: thread created by kthread_create(). - * - * Sets kthread_should_park() for @k to return false, wakes it, and - * waits for it to return. If the thread is marked percpu then its - * bound to the cpu again. - */ -void kthread_unpark(struct task_struct *k) -{ - struct kthread *kthread = to_live_kthread(k); - - if (kthread) - __kthread_unpark(k, kthread); -} EXPORT_SYMBOL_GPL(kthread_unpark); /** @@ -492,20 +471,20 @@ EXPORT_SYMBOL_GPL(kthread_unpark); */ int kthread_park(struct task_struct *k) { - struct kthread *kthread = to_live_kthread(k); - int ret = -ENOSYS; - - if (kthread) { - if (!test_bit(KTHREAD_IS_PARKED, &kthread->flags)) { - set_bit(KTHREAD_SHOULD_PARK, &kthread->flags); - if (k != current) { - wake_up_process(k); - wait_for_completion(&kthread->parked); - } + struct kthread *kthread = to_kthread(k); + + if (WARN_ON(k->flags & PF_EXITING)) + return -ENOSYS; + + if (!test_bit(KTHREAD_IS_PARKED, &kthread->flags)) { + set_bit(KTHREAD_SHOULD_PARK, &kthread->flags); + if (k != current) { + wake_up_process(k); + wait_for_completion(&kthread->parked); } - ret = 0; } - return ret; + + return 0; } EXPORT_SYMBOL_GPL(kthread_park); @@ -534,7 +513,7 @@ int kthread_stop(struct task_struct *k) get_task_struct(k); kthread = to_kthread(k); set_bit(KTHREAD_SHOULD_STOP, &kthread->flags); - __kthread_unpark(k, kthread); + kthread_unpark(k); wake_up_process(k); wait_for_completion(&kthread->exited); ret = k->exit_code; -- 2.5.0