Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: [PATCH liburing 1/1] test: add epoll test case
To:     Stefano Garzarella <sgarzare@redhat.com>
Cc:     io-uring@vger.kernel.org, linux-kernel@vger.kernel.org
References: <20200127161701.153625-1-sgarzare@redhat.com>
 <20200127161701.153625-2-sgarzare@redhat.com>
 <b1b26e79-507a-b339-2850-d2686661e669@kernel.dk>
 <20200127182534.5ljsj53vzpj6kkru@steredhat>
From:   Jens Axboe <axboe@kernel.dk>
Message-ID: <646cbb04-9bef-0d99-64ec-322d1584abe7@kernel.dk>
Date:   Mon, 27 Jan 2020 11:46:34 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.4.1
MIME-Version: 1.0
In-Reply-To: <20200127182534.5ljsj53vzpj6kkru@steredhat>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On 1/27/20 11:25 AM, Stefano Garzarella wrote:
> On Mon, Jan 27, 2020 at 09:32:43AM -0700, Jens Axboe wrote:
>> On 1/27/20 9:17 AM, Stefano Garzarella wrote:
>>> Signed-off-by: Stefano Garzarella <sgarzare@redhat.com>
>>
>> You're not reaping CQ events, and hence you overflow the ring. Once
>> overflown, an attempt to submit new IO will returns in a -16/-EBUSY
>> return value. This is io_uring telling you that it won't submit more
>> IO until you've emptied the completion ring so io_uring can flush
>> the overflown entries to the ring.
> 
> How can I reaping CQ events? (I was hoping the epoll would help me with that)
> 
> What I'm seeing is that the producer (EPOLLOUT) can fill the SQ without issues,
> the consumer (read()) is receiving all the buffers produced, but the thread
> that frees the buffers (EPOLLIN) is not woken up.
> 
> I tried to set a timeout to the epoll_wait(), but the io_uring_peek_cqe()
> returns -EAGAIN.
> 
> If I'm using a ring with 16 entries, it seems to work better, but
> sometimes I lose events and the thread that frees the buffer doesn't wake up.
> 
> Maybe I'm missing something...

OK, so that helps in terms of understanding the issue you are seeing with
it. I'll take a look at this, but it'll probably be a few days. You can
try and enable tracing, I see events completed just fine. Maybe a race
with your epoll wait and event reaping?

-- 
Jens Axboe