Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
To:     Nadav Amit <namit@vmware.com>, Hao Xu <haoxu@linux.alibaba.com>
Cc:     Jens Axboe <axboe@kernel.dk>,
        "io-uring@vger.kernel.org" <io-uring@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
References: <20210808001342.964634-1-namit@vmware.com>
 <20210808001342.964634-2-namit@vmware.com>
 <488f005c-fd92-9881-2700-b2eb1adbb78e@gmail.com>
 <40885F33-F35D-49E9-A79F-DB3C35278F73@vmware.com>
 <8825c48c-3574-c76f-4ff2-e68aaf657cda@linux.alibaba.com>
 <CAC2285F-0841-47F5-8567-24C1377DDFB6@vmware.com>
From:   Pavel Begunkov <asml.silence@gmail.com>
Subject: Re: [PATCH 1/2] io_uring: clear TIF_NOTIFY_SIGNAL when running task
 work
Message-ID: <f47cc1a7-eae5-df1b-a811-f2eabd6d735c@gmail.com>
Date:   Mon, 9 Aug 2021 11:35:02 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.12.0
MIME-Version: 1.0
In-Reply-To: <CAC2285F-0841-47F5-8567-24C1377DDFB6@vmware.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Precedence: bulk

On 8/9/21 5:50 AM, Nadav Amit wrote:
>> On Aug 8, 2021, at 9:07 PM, Hao Xu <haoxu@linux.alibaba.com> wrote:
>> 在 2021/8/9 上午1:31, Nadav Amit 写道:
>>>> On Aug 8, 2021, at 5:55 AM, Pavel Begunkov <asml.silence@gmail.com> wrote:
>>>> On 8/8/21 1:13 AM, Nadav Amit wrote:
>>>>> From: Nadav Amit <namit@vmware.com>
>>>>>
>>>>> When using SQPOLL, the submission queue polling thread calls
>>>>> task_work_run() to run queued work. However, when work is added with
>>>>> TWA_SIGNAL - as done by io_uring itself - the TIF_NOTIFY_SIGNAL remains
>>>>
>>>> static int io_req_task_work_add(struct io_kiocb *req)
>>>> {
>>>> 	...
>>>> 	notify = (req->ctx->flags & IORING_SETUP_SQPOLL) ? TWA_NONE : TWA_SIGNAL;
>>>> 	if (!task_work_add(tsk, &tctx->task_work, notify))
>>>> 	...
>>>> }
>>>>
>>>> io_uring doesn't set TIF_NOTIFY_SIGNAL for SQPOLL. But if you see it, I'm
>>>> rather curious who does.
>>> I was saying io-uring, but I meant io-uring in the wider sense:
>>> io_queue_worker_create().
>>> Here is a call trace for when TWA_SIGNAL is used. io_queue_worker_create()
>>> uses TWA_SIGNAL. It is called by io_wqe_dec_running(), and not shown due
>>> to inlining:
>> Hi Nadav, Pavel,
>> How about trying to make this kind of call to use TWA_NONE for sqthread,
>> though I know for this case currently there is no info to get to know if
>> task is sqthread. I think we shouldn't kick sqthread.
> 
> It is possible, but it would break the abstractions and propagating
> it would be disgusting. Let me give it some thought.

We already do io_wq_enqueue() only from the right task context,
so in theory instead of pushing it through tw, we can create
a new thread on the spot. Though, would need to be careful
with locking.

Anyway, agree that it's better to be left for next.

> Regardless, I think that this patch should go to 5.14 and stable,
> and any solution to avoid kicking the SQ should come on top (to be
> safe).

-- 
Pavel Begunkov