Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Subject: Re: [PATCH v6 0/5] io_uring: remove ring quiesce in io_uring_register
To:     Usama Arif <usama.arif@bytedance.com>, io-uring@vger.kernel.org,
        asml.silence@gmail.com, linux-kernel@vger.kernel.org
Cc:     fam.zheng@bytedance.com
References: <20220204145117.1186568-1-usama.arif@bytedance.com>
From:   Jens Axboe <axboe@kernel.dk>
Message-ID: <b67f1ce6-6110-b6f3-e66e-f636d47a736d@kernel.dk>
Date:   Fri, 4 Feb 2022 08:53:02 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
MIME-Version: 1.0
In-Reply-To: <20220204145117.1186568-1-usama.arif@bytedance.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 2/4/22 7:51 AM, Usama Arif wrote:
> Ring quiesce is currently used for registering/unregistering eventfds,
> registering restrictions and enabling rings.
> 
> For opcodes relating to registering/unregistering eventfds, ring quiesce
> can be avoided by creating a new RCU data structure (io_ev_fd) as part
> of io_ring_ctx that holds the eventfd_ctx, with reads to the structure
> protected by rcu_read_lock and writes (register/unregister calls)
> protected by a mutex.
> 
> With the above approach ring quiesce can be avoided which is much more
> expensive then using RCU lock. On the system tested, io_uring_reigster with
> IORING_REGISTER_EVENTFD takes less than 1ms with RCU lock, compared to 15ms
> before with ring quiesce.
> 
> IORING_SETUP_R_DISABLED prevents submitting requests and
> so there will be no requests until IORING_REGISTER_ENABLE_RINGS
> is called. And IORING_REGISTER_RESTRICTIONS works only before
> IORING_REGISTER_ENABLE_RINGS is called. Hence ring quiesce is
> not needed for these opcodes.

I wrote a simple test case just verifying register+unregister, and also
doing a loop to catch any issues around that. Here's the current kernel:

[root@archlinux liburing]# time test/eventfd-reg 

real	0m7.980s
user	0m0.004s
sys	0m0.000s
[root@archlinux liburing]# time test/eventfd-reg 

real	0m8.197s
user	0m0.004s
sys	0m0.000s

which is around ~80ms for each register/unregister cycle, and here are
the results with this patchset:

[root@archlinux liburing]# time test/eventfd-reg

real	0m0.002s
user	0m0.001s
sys	0m0.000s
[root@archlinux liburing]# time test/eventfd-reg

real	0m0.001s
user	0m0.001s
sys	0m0.000s

which looks a lot more reasonable.

I'll look over this one and see if I've got anything to complain about,
just ran it first since I wrote the test anyway. Here's the test case,
btw:

https://git.kernel.dk/cgit/liburing/commit/?id=5bde26e4587168a439cabdbe73740454249e5204

-- 
Jens Axboe