Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7;
Message-ID: <2d187279-0f9d-4062-a319-79c73adc5cde@kernel.dk>
Date:   Sun, 19 Nov 2023 14:23:20 -0700
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3] io_uring/fdinfo: remove need for sqpoll lock for
 thread/pid retrieval
Content-Language: en-US
To:     Xiaobing Li <xiaobing.li@samsung.com>
Cc:     asml.silence@gmail.com, linux-kernel@vger.kernel.org,
        io-uring@vger.kernel.org, kun.dou@samsung.com,
        peiwei.li@samsung.com, joshi.k@samsung.com,
        kundan.kumar@samsung.com, wenwen.chen@samsung.com,
        ruyi.zhang@samsung.com
References: <605eac76-ec47-436b-872a-f6e8b4094293@kernel.dk>
 <CGME20231118032740epcas5p20b6aad6264323376fa024bc2a56f0990@epcas5p2.samsung.com>
 <20231118031951.21764-1-xiaobing.li@samsung.com>
From:   Jens Axboe <axboe@kernel.dk>
In-Reply-To: <20231118031951.21764-1-xiaobing.li@samsung.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 11/17/23 8:19 PM, Xiaobing Li wrote:
> On 11/15/23 6:42 AM, Jens Axboe wrote:
>> 	 */
>> 	has_lock = mutex_trylock(&ctx->uring_lock);
>>
>> -	if (has_lock && (ctx->flags & IORING_SETUP_SQPOLL)) {
>> -		struct io_sq_data *sq = ctx->sq_data;
>> -
>> -		sq_pid = sq->task_pid;
>> -		sq_cpu = sq->sq_cpu;
>> +	if (ctx->flags & IORING_SETUP_SQPOLL) {
>> +		struct io_sq_data *sq;
>> +
>> +		rcu_read_lock();
>> +		sq = READ_ONCE(ctx->sq_data);
>> +		if (sq) {
>> +			sq_pid = sq->task_pid;
>> +			sq_cpu = sq->sq_cpu;
>> +		}
>> +		rcu_read_unlock();
>> 	}
>>
>> 	seq_printf(m, "SqThread:\t%d\n", sq_pid);
>> diff --git a/io_uring/sqpoll.c b/io_uring/sqpoll.c
>> index 65b5dbe3c850..583c76945cdf 100644
>> --- a/io_uring/sqpoll.c
>> +++ b/io_uring/sqpoll.c
>> @@ -70,7 +70,7 @@ void io_put_sq_data(struct io_sq_data *sqd)
>> 		WARN_ON_ONCE(atomic_read(&sqd->park_pending));
>>
>> 		io_sq_thread_stop(sqd);
>> -		kfree(sqd);
>> +		kfree_rcu(sqd, rcu);
>> 	}
>> }
>>
>> @@ -313,7 +313,7 @@ static int io_sq_thread(void *data)
>> 	}
>>
>> 	io_uring_cancel_generic(true, sqd);
>> -	sqd->thread = NULL;
>> +	WRITE_ONCE(sqd->thread, NULL);
>> 	list_for_each_entry(ctx, &sqd->ctx_list, sqd_list)
>> 		atomic_or(IORING_SQ_NEED_WAKEUP, &ctx->rings->sq_flags);
>> 	io_run_task_work();
>> @@ -411,7 +411,7 @@ __cold int io_sq_offload_create(struct io_ring_ctx *ctx,
>> 			goto err_sqpoll;
>> 		}
>>
>> -		sqd->thread = tsk;
>> +		WRITE_ONCE(sqd->thread, tsk);
>> 		ret = io_uring_alloc_task_context(tsk, ctx);
>> 		wake_up_new_task(tsk);
>> 		if (ret)
>> diff --git a/io_uring/sqpoll.h b/io_uring/sqpoll.h
>> index 8df37e8c9149..0cf0c5833a27 100644
>> --- a/io_uring/sqpoll.h
>> +++ b/io_uring/sqpoll.h
>> @@ -18,6 +18,8 @@ struct io_sq_data {
>>
>> 	unsigned long		state;
>> 	struct completion	exited;
>> +
>> +	struct rcu_head		rcu;
>> };
>>
>> int io_sq_offload_create(struct io_ring_ctx *ctx, struct io_uring_params *p);
> 
> I tested this and it worked after adding RCU lock.
> It consistently outputs correct results.
> 
> The results of a simple test are as follows:
> Every 0.5s: cat /proc/10212/fdinfo/6 | grep Sq
> SqMask: 0x3
> SqHead: 17422716
> SqTail: 17422716
> CachedSqHead:   17422716
> SqThread:       10212
> SqThreadCpu:    73
> SqBusy: 97%
> -------------------------------------------------------------
> But the name of the sq thread is "iou-sqp-" + "the PID of its parent process":
>     PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
>   10211 root      20   0  184408   8192      0 R  99.9   0.0   4:01.42 fio
>   10212 root      20   0  184408   8192      0 R  99.9   0.0   4:01.48 iou-sqp-10211
> Is this the originally desired effect?

That's how it has always been, it's the sqpoll thread belonging to that
parent.

-- 
Jens Axboe