Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: Weird issue with epoll and kernel >= 5.0
To:     Omar Kilani <omar.kilani@gmail.com>, linux-kernel@vger.kernel.org
References: <CA+8F9hhy=WPMJLQ3Ya_w4O6xyWk7KsXi=YJofmyC577_UJTutA@mail.gmail.com>
From:   Randy Dunlap <rdunlap@infradead.org>
Cc:     Davidlohr Bueso <dave@stgolabs.net>
Message-ID: <34206eb5-1280-4aac-9a50-76f967646ca1@infradead.org>
Date:   Sat, 28 Mar 2020 12:21:55 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.6.0
MIME-Version: 1.0
In-Reply-To: <CA+8F9hhy=WPMJLQ3Ya_w4O6xyWk7KsXi=YJofmyC577_UJTutA@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On 3/28/20 11:10 AM, Omar Kilani wrote:
> Hi there,
> 
> I've observed an issue with epoll and kernels 5.0 and above when a
> system is generating a lot of epoll events.
> 
> I see this issue with nginx and jvm / netty based apps (using the
> jvm's native epoll support as well as netty's own optimized epoll
> support) but *not* with haproxy (?).
> 
> I'm not really sure what the actual problem is (nginx complains about
> epoll_wait with a generic error), but it doesn't happen on 4.19.x and
> lower.
> 
> I thought it was a netty problem at first and opened this ticket:
> 
> https://github.com/netty/netty/issues/8999
> 
> But then saw the same issue in nginx.
> 
> I haven't debugged a kernel issue in something like 20 years so I'm
> not really sure where to start myself.
> 
> I'd be more than happy to provide my test case that has a very quick
> repro to anyone who needs it.

Hi,
Please do.

> Also happy to provide a VM/machine with enough CPUs to trigger it
> easily (it seems to happen quicker with more CPUs present) to test
> with.


There have been around 10 changes in fs/eventpoll.c since v5.0 was
released in March, 2019, so it would be helpful if you could test
the latest mainline kernel to see if the problem is still present.

Hm, it looks like you have identified this commit:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?h=v5.1-rc5&id=c5a282e9635e9c7382821565083db5d260085e3e
as the/a problem.

I have Cc-ed the patch author also.

-- 
~Randy