Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Mon, 24 Jun 2019 14:24:22 -0700
From:   Stanislav Fomichev <sdf@fomichev.me>
To:     Andrii Nakryiko <andrii.nakryiko@gmail.com>
Cc:     Rong Chen <rong.a.chen@intel.com>,
        Stanislav Fomichev <sdf@google.com>,
        Daniel Borkmann <daniel@iogearbox.net>,
        Martin Lau <kafai@fb.com>, LKML <linux-kernel@vger.kernel.org>,
        Stephen Rothwell <sfr@canb.auug.org.au>,
        bpf <bpf@vger.kernel.org>, lkp@01.org
Subject: Re: [selftests/bpf] 69d96519db:
 kernel_selftests.bpf.test_socket_cookie.fail
Message-ID: <20190624212422.GA10487@mini-arch>
References: <20190621084040.GU7221@shao2-debian>
 <20190621161039.GF1383@mini-arch>
 <CAEf4Bzaajc27=YyMaOa8UFRz=xE7y6E+qLbPBPbvLADO2peXQg@mail.gmail.com>
 <20190621222745.GH1383@mini-arch>
 <f3aa0dc2-c959-1166-8b09-84781363f0e0@intel.com>
 <CAEf4BzaNBboGeU8xOxyW-aDzEPUQq-LidRzj8V08O=_TynkQOQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAEf4BzaNBboGeU8xOxyW-aDzEPUQq-LidRzj8V08O=_TynkQOQ@mail.gmail.com>
User-Agent: Mutt/1.12.1 (2019-06-15)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On 06/24, Andrii Nakryiko wrote:
> On Sun, Jun 23, 2019 at 5:59 PM Rong Chen <rong.a.chen@intel.com> wrote:
> >
> > On 6/22/19 6:27 AM, Stanislav Fomichev wrote:
> > > On 06/21, Andrii Nakryiko wrote:
> > >> )
> > >>
> > >> On Fri, Jun 21, 2019 at 9:11 AM Stanislav Fomichev <sdf@fomichev.me> wrote:
> > >>> On 06/21, kernel test robot wrote:
> > >>>> FYI, we noticed the following commit (built with gcc-7):
> > >>>>
> > >>>> commit: 69d96519dbf0bfa1868dc8597d4b9b2cdeb009d7 ("selftests/bpf: convert socket_cookie test to sk storage")
> > >>>> https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master
> > >>>>
> > >>>> in testcase: kernel_selftests
> > >>>> with following parameters:
> > >>>>
> > >>>>        group: kselftests-00
> > >>>>
> > >>>> test-description: The kernel contains a set of "self tests" under the tools/testing/selftests/ directory. These are intended to be small unit tests to exercise individual code paths in the kernel.
> > >>>> test-url: https://www.kernel.org/doc/Documentation/kselftest.txt
> > >>>>
> > >>>>
> > >>>> on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 4G
> > >>>>
> > >>>> caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
> > >>>>
> > >>>>
> > >>>> If you fix the issue, kindly add following tag
> > >>>> Reported-by: kernel test robot <rong.a.chen@intel.com>
> > >>>>
> > >>>> # selftests: bpf: test_socket_cookie
> > >>>> # libbpf: failed to create map (name: 'socket_cookies'): Invalid
> > >>>> # argument
> > >>> Another case of old clang trying to create a map that depends on BTF?
> > >>> Should we maybe switch those BTF checks in the kernel to return
> > >>> EOPNOTSUPP to make it easy to diagnose?
> > >> For older compilers that don't generate DATASEC/VAR, you'll see a clear message:
> > >>
> > >> libbpf: DATASEC '.maps' not found.
> > >>
> > >> So this must be something else. I just confirmed with clang version
> > >> 7.0.20180201 that for ./test_socket_cookie that's the first line
> > >> that's emitted on failure.
> > > Thanks for checking, I also took a look at the attached kernel_selftests.xz,
> > > here is what it has:
> > > 2019-06-21 11:58:35 ln -sf /usr/bin/clang-6.0 /usr/bin/clang
> > > 2019-06-21 11:58:35 ln -sf /usr/bin/llc-6.0 /usr/bin/llc
> > > ...
> > > # BTF libbpf test[1] (test_btf_haskv.o): SKIP. No ELF .BTF found
> > > # BTF libbpf test[2] (test_btf_nokv.o): SKIP. No ELF .BTF found
> > > ...
> > > # Test case #0 (btf_dump_test_case_syntax): test_btf_dump_case:71:FAIL
> > > # failed to load test BTF: -2
> > > # Test case #1 (btf_dump_test_case_ordering): test_btf_dump_case:71:FAIL
> > > # failed to load test BTF: -2
> > > ...
> > >
> > > And so on. So there is clearly an old clang that doesn't emit any
> > > BTF. And I also don't see your recent abd29c931459 before 69d96519dbf0 in
> > > linux-next, that's why it doesn't complain about missing/corrupt BTF.
> 
> Ah, ok, that would explain it. But in any case, clang 6&7 is too old.
> Clang 8 or better yet clang 9 (for global data, datasec/var-dependent
> stuff) would be great.
While we are it: I think I have resolved the BTF story internally,
so if you want to go ahead and convert the rest of the tests to
BTF format, I would not object anymore ;-)

(I didn't expect it to be that easy initially, so sorry if I wasted
everyones time arguing about it).

> > >
> > > We need to convince lkp people to upgrade clang, otherwise, I suppose,
> > > we'll get more of these reportings after your recent df0b77925982 :-(
> >
> > Thanks for the clarification, we'll upgrade clang asap.
> 
> Thanks Rong!
> 
> >
> > Best Regards,
> > Rong Chen
> >
> >
> > >
> > >>>> # libbpf: failed to load object './socket_cookie_prog.o'
> > >>>> # (test_socket_cookie.c:149: errno: Invalid argument) Failed to load
> > >>>> # ./socket_cookie_prog.o
> > >>>> # FAILED
> > >>>> not ok 15 selftests: bpf: test_socket_cookie
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>> To reproduce:
> > >>>>
> > >>>>          # build kernel
> > >>>>        cd linux
> > >>>>        cp config-5.2.0-rc2-00598-g69d9651 .config
> > >>>>        make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig
> > >>>>        make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 prepare
> > >>>>        make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 modules_prepare
> > >>>>        make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 SHELL=/bin/bash
> > >>>>        make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 bzImage
> > >>>>
> > >>>>
> > >>>>          git clone https://github.com/intel/lkp-tests.git
> > >>>>          cd lkp-tests
> > >>>>          bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
> > >>>>
> > >>>>
> > >>>>
> > >>>> Thanks,
> > >>>> Rong Chen
> > >>>>
> > >> <mega snip>