Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: bisected: 4.18-rc* regression: x86-32 troubles (with timers?)
To:     Arnd Bergmann <arnd@arndb.de>, Meelis Roos <mroos@linux.ee>
Cc:     Linux Kernel list <linux-kernel@vger.kernel.org>,
        Networking <netdev@vger.kernel.org>
References: <alpine.LRH.2.21.1807041414080.3671@math.ut.ee>
 <CAK8P3a1RkNc_kShHBJ6QENrQheQi=C93DmnknC1S_uJDcTDqww@mail.gmail.com>
 <alpine.LRH.2.21.1807051254050.21817@math.ut.ee>
 <alpine.LRH.2.21.1807151803010.17406@math.ut.ee>
 <CAK8P3a2NtB_FmiSjrzZb=HoshEzo+MP2xwafC3QcMg79rffipg@mail.gmail.com>
 <alpine.LRH.2.21.1807161948380.27791@math.ut.ee>
 <alpine.LRH.2.21.1807162200180.27791@math.ut.ee>
 <alpine.LRH.2.21.1807202339490.8506@math.ut.ee>
 <alpine.LRH.2.21.1807210154100.8506@math.ut.ee>
 <CAK8P3a19pCbn5Ns0jVqGTiSmGj4AOgjWL0CQtAA-sYRqY4Gvjw@mail.gmail.com>
From:   Daniel Borkmann <daniel@iogearbox.net>
Message-ID: <32227088-7754-c160-8046-e860f42ca91c@iogearbox.net>
Date:   Mon, 23 Jul 2018 21:41:36 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.3.0
MIME-Version: 1.0
In-Reply-To: <CAK8P3a19pCbn5Ns0jVqGTiSmGj4AOgjWL0CQtAA-sYRqY4Gvjw@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Hello Meelis, Arnd,

On 07/23/2018 06:03 PM, Arnd Bergmann wrote:
> On Sat, Jul 21, 2018 at 1:01 AM, Meelis Roos <mroos@linux.ee> wrote:
>> Added netdev and Daniel Borkmann - please see
>> https://www.mail-archive.com/linux-kernel@vger.kernel.org/msg1724795.html
>> for the original report. It seems to be about BPF instead.
>>
>> Meanwhile I have found more machines with the trouble. Still no clear
>> mark in the config - some x86-32 machines that have
>> CONFIG_BPF=y
>> CONFIG_BPF_SYSCALL=y
>> CONFIG_BPF_JIT_ALWAYS_ON=y
>> are working fine.
>>
>>> The new bisect seems to have also led me to a strange commit. This time
>>> I tried to be careful and tested most on two reboots before classifying
>>> as good.
>>>
>>> However, f4e3ec0d573e was suspicious - it failed to autoload e1000 but
>>> had no other errors. On both boots with this kernel, modprobe e1000 and
>>> ifup -a made the system work so I assumed it was good, while it might
>>> not have been. Will try bisecting with f4e3ec0d573e marked bad.
>>
>> Now this seems more relevant:
>>
>> mroos@rx100s2:~/linux$ nice git bisect good
>> 24dea04767e6e5175f4750770281b0c17ac6a2fb is the first bad commit
>> commit 24dea04767e6e5175f4750770281b0c17ac6a2fb
>> Author: Daniel Borkmann <daniel@iogearbox.net>
>> Date:   Fri May 4 01:08:23 2018 +0200
>>
>>     bpf, x32: remove ld_abs/ld_ind
>>
>>     Since LD_ABS/LD_IND instructions are now removed from the core and
>>     reimplemented through a combination of inlined BPF instructions and
>>     a slow-path helper, we can get rid of the complexity from x32 JIT.
> 
> This does seem much more likely than the previous bisection, given
> that you ended up in an x86-32 specific commit (the subject says x32,
> but that is a mistake). I also checked that systemd indeed does
> call into bpf in a number of places, possibly for the journald socket.
> 
> OTOH, it's still hard to tell how that commit can have ended up
> corrupting the clock read function in systemd. To cross-check,
> could you try reverting that commit on the latest kernel and see
> if it still works?

I would be curious as well about that whether revert would make it
work. What's the value of sysctl net.core.bpf_jit_enable ? Does it
change anything if you set it to 0 (only interpreter) or 1 (JIT
enabled). Seems a bit strange to me that bisect ended at this commit
given the issue you have. The JIT itself was also new in this window
fwiw. In any case some more debug info would be great to have.

Thanks,
Daniel