Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [PATCH] task_work: cleanup notification modes
To:     Thomas Gleixner <tglx@linutronix.de>
Cc:     "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
References: <93292d5b-9124-d252-c81f-1f2cfbd60e7b@kernel.dk>
 <87tuutalre.fsf@nanos.tec.linutronix.de>
 <aaed79d1-972f-e4bd-f3ac-d589cd729163@kernel.dk>
 <87zh4lix8l.fsf@nanos.tec.linutronix.de>
 <2ebe7e45-b4e5-1a6b-d3ee-4a790817a119@kernel.dk>
 <87wnzpivvx.fsf@nanos.tec.linutronix.de>
From:   Jens Axboe <axboe@kernel.dk>
Message-ID: <7e32aa44-35ff-3a48-87d0-8e7df586db1d@kernel.dk>
Date:   Fri, 16 Oct 2020 17:43:42 -0600
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <87wnzpivvx.fsf@nanos.tec.linutronix.de>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 10/16/20 5:38 PM, Thomas Gleixner wrote:
> On Fri, Oct 16 2020 at 17:13, Jens Axboe wrote:
>> /**
>>  * task_work_add - ask the @task to execute @work->func()
>>  * @task: the task which should run the callback
>>  * @work: the callback to run
>>  * @notify: how to notify the targeted task
>>  *
>>  * Queue @work for task_work_run() below and notify the @task if @notify
>>  * is @TWA_RESUME or @TWA_SIGNAL. @TWA_SIGNAL work like signals, in that the
> 
> s/the//

Thanks, good catch.

>>  * it will interrupt the targeted task and run the task_work. @TWA_RESUME
>>  * work is run only when the task exits the kernel and returns to user mode.
>>  * Fails if the @task is exiting/exited and thus it can't process this @work.
>>  * Otherwise @work->func() will be called when the @task returns from kernel
>>  * mode or exits.
> 
> Yes, that makes a lot more sense.
> 
> What's still lacking is a description of the return value and how to act
> upon it.

That's really up to the caller. But we should add some explanation of
that. Most callers use some alternative if the task is exiting, like
using a work queue for example.

> Most of the call sites ignore it, some are acting upon it but I can't

If you know the task isn't exiting, then yeah you can ignore it. But
seems a bit dicey...

> make any sense of these actions:
> 
> fs/io_uring.c-	notify = 0;
> fs/io_uring.c-	if (!(ctx->flags & IORING_SETUP_SQPOLL) && twa_signal_ok)
> fs/io_uring.c-		notify = TWA_SIGNAL;
> fs/io_uring.c-
> fs/io_uring.c:	ret = task_work_add(tsk, &req->task_work, notify);
> fs/io_uring.c-	if (!ret)
> fs/io_uring.c-		wake_up_process(tsk);
> 
> ???
> 
> fs/io_uring.c-	if (unlikely(ret)) {
> fs/io_uring.c-		struct task_struct *tsk;
> fs/io_uring.c-
> fs/io_uring.c-		init_task_work(&req->task_work, io_req_task_cancel);
> fs/io_uring.c-		tsk = io_wq_get_task(req->ctx->io_wq);
> fs/io_uring.c:		task_work_add(tsk, &req->task_work, 0);
> fs/io_uring.c-		wake_up_process(tsk);
> 
> yet more magic wakeup.

It's not magic, but probably needs a comment... If we fail, that task is
exiting. But we know we have our io-wq threads, so we use that as a
fallback. Not really expected in the fast path.

> fs/io_uring.c-
> fs/io_uring.c-	init_task_work(&req->task_work, io_req_task_submit);
> fs/io_uring.c-	percpu_ref_get(&req->ctx->refs);
> fs/io_uring.c-
> fs/io_uring.c-	/* submit ref gets dropped, acquire a new one */
> fs/io_uring.c-	refcount_inc(&req->refs);
> fs/io_uring.c:	ret = io_req_task_work_add(req, true);
> fs/io_uring.c-	if (unlikely(ret)) {
> fs/io_uring.c-		struct task_struct *tsk;
> fs/io_uring.c-
> fs/io_uring.c-		/* queue just for cancelation */
> fs/io_uring.c-		init_task_work(&req->task_work, io_req_task_cancel);
> fs/io_uring.c-		tsk = io_wq_get_task(req->ctx->io_wq);
> fs/io_uring.c:		task_work_add(tsk, &req->task_work, 0);
> fs/io_uring.c-		wake_up_process(tsk);
> 
> Ditto. Why the heck is this wakeup making any sense? The initial
> task_work_add() within io_req_task_work_add() failed already ...

Right, but we're using a new task for this. And that task is a kthread
that we manage, hence no notification is needed outside of just waking
it up.

-- 
Jens Axboe