Received-SPF: pass (google.com: domain of linux-kernel+bounces-132075-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Message-ID: <ff36e8fa-14f4-42a6-8210-cec24a7779a0@linux.dev>
Date: Thu, 4 Apr 2024 13:06:50 -0700
Precedence: bulk
MIME-Version: 1.0
Subject: Re: [PATCH bpf-next v3] selftests/bpf: Move test_dev_cgroup to
 prog_tests
Content-Language: en-GB
To: Muhammad Usama Anjum <usama.anjum@collabora.com>,
 Alexei Starovoitov <ast@kernel.org>, Daniel Borkmann <daniel@iogearbox.net>,
 Andrii Nakryiko <andrii@kernel.org>, Martin KaFai Lau
 <martin.lau@linux.dev>, Eduard Zingerman <eddyz87@gmail.com>,
 Song Liu <song@kernel.org>, John Fastabend <john.fastabend@gmail.com>,
 KP Singh <kpsingh@kernel.org>, Stanislav Fomichev <sdf@google.com>,
 Hao Luo <haoluo@google.com>, Jiri Olsa <jolsa@kernel.org>,
 Mykola Lysenko <mykolal@fb.com>, Shuah Khan <shuah@kernel.org>
Cc: kernel@collabora.com, linux-kernel@vger.kernel.org, bpf@vger.kernel.org,
 linux-kselftest@vger.kernel.org
References: <20240401123455.1377896-1-usama.anjum@collabora.com>
 <92e1cce6-5f26-4a49-86b6-81e1e80d1aaa@linux.dev>
 <cfecd6ea-8fa3-477f-bd32-4087aefee2af@collabora.com>
 <0ff5c7d0-d5c5-4b61-ba89-8e7f9f775935@linux.dev>
 <0973bc93-7a8d-451c-9944-d91a77d68755@collabora.com>
From: Yonghong Song <yonghong.song@linux.dev>
In-Reply-To: <0973bc93-7a8d-451c-9944-d91a77d68755@collabora.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit


On 4/3/24 5:03 AM, Muhammad Usama Anjum wrote:
> On 4/3/24 7:36 AM, Yonghong Song wrote:
>> On 4/2/24 8:16 AM, Muhammad Usama Anjum wrote:
>>> Yonghong Song,
>>>
>>> Thank you so much for replying. I was missing how to run pipeline manually.
>>> Thanks a ton.
>>>
>>> On 4/1/24 11:53 PM, Yonghong Song wrote:
>>>> On 4/1/24 5:34 AM, Muhammad Usama Anjum wrote:
>>>>> Move test_dev_cgroup.c to prog_tests/dev_cgroup.c to be able to run it
>>>>> with test_progs. Replace dev_cgroup.bpf.o with skel header file,
>>>>> dev_cgroup.skel.h and load program from it accourdingly.
>>>>>
>>>>>      ./test_progs -t dev_cgroup
>>>>>      mknod: /tmp/test_dev_cgroup_null: Operation not permitted
>>>>>      64+0 records in
>>>>>      64+0 records out
>>>>>      32768 bytes (33 kB, 32 KiB) copied, 0.000856684 s, 38.2 MB/s
>>>>>      dd: failed to open '/dev/full': Operation not permitted
>>>>>      dd: failed to open '/dev/random': Operation not permitted
>>>>>      #72     test_dev_cgroup:OK
>>>>>      Summary: 1/0 PASSED, 0 SKIPPED, 0 FAILED
>>>>> Signed-off-by: Muhammad Usama Anjum <usama.anjum@collabora.com>
>>>>> ---
>>>>> Changes since v2:
>>>>> - Replace test_dev_cgroup with serial_test_dev_cgroup as there is
>>>>>      probability that the test is racing against another cgroup test
>>>>> - Minor changes to the commit message above
>>>>>
>>>>> I've tested the patch with vmtest.sh on bpf-next/for-next and linux
>>>>> next. It is passing on both. Not sure why it was failed on BPFCI.
>>>>> Test run with vmtest.h:
>>>>> sudo LDLIBS=-static PKG_CONFIG='pkg-config --static' ./vmtest.sh
>>>>> ./test_progs -t dev_cgroup
>>>>> ./test_progs -t dev_cgroup
>>>>> mknod: /tmp/test_dev_cgroup_null: Operation not permitted
>>>>> 64+0 records in
>>>>> 64+0 records out
>>>>> 32768 bytes (33 kB, 32 KiB) copied, 0.000403432 s, 81.2 MB/s
>>>>> dd: failed to open '/dev/full': Operation not permitted
>>>>> dd: failed to open '/dev/random': Operation not permitted
>>>>>     #69      dev_cgroup:OK
>>>>> Summary: 1/0 PASSED, 0 SKIPPED, 0 FAILED
>>>> The CI failure:
>>>>
>>>>
>>>> Error: #72 dev_cgroup
>>>> serial_test_dev_cgroup:PASS:skel_open_and_load 0 nsec
>>>> serial_test_dev_cgroup:PASS:cgroup_setup_and_join 0 nsec
>>>> serial_test_dev_cgroup:PASS:bpf_attach 0 nsec
>>>> serial_test_dev_cgroup:PASS:bpf_query 0 nsec
>>>> serial_test_dev_cgroup:PASS:bpf_query 0 nsec
>>>> serial_test_dev_cgroup:PASS:rm 0 nsec
>>>> serial_test_dev_cgroup:PASS:mknod 0 nsec
>>>> serial_test_dev_cgroup:PASS:rm 0 nsec
>>>> serial_test_dev_cgroup:PASS:rm 0 nsec
>>>> serial_test_dev_cgroup:FAIL:mknod unexpected mknod: actual 256 !=
>>>> expected 0
>>>> serial_test_dev_cgroup:PASS:rm 0 nsec
>>>> serial_test_dev_cgroup:PASS:dd 0 nsec
>>>> serial_test_dev_cgroup:PASS:dd 0 nsec
>>>> serial_test_dev_cgroup:PASS:dd 0 nsec
>>>>
>>>> (cgroup_helpers.c:353: errno: Device or resource busy) umount cgroup2
>>>>
>>>> The error code 256 means mknod execution has some issues. Maybe you need to
>>>> find specific errno to find out what is going on. I think you can do ci
>>>> on-demanding test to debug.
>>> errno is 2 --> No such file or directory
>>>
>>> Locally I'm unable to reproduce it until I don't remove
>>> rm -f /tmp/test_dev_cgroup_zero such that the /tmp/test_dev_cgroup_zero
>>> node is present before test execution. The error code is 256 with errno 2.
>>> I'm debugging by placing system("ls /tmp 1>&2"); to find out which files
>>> are already present in /tmp. But ls's output doesn't appear on the CI logs.
>> errno 2 means ENOENT.
>>  From mknod man page (https://linux.die.net/man/2/mknod), it means
>>    A directory component in/pathname/  does not exist or is a dangling
>> symbolic link.
>>
>> It means /tmp does not exist or a dangling symbolic link.
>> It is indeed very strange. To make the test robust, maybe creating a temp
>> directory with mkdtemp and use it as the path? The temp directory
>> creation should be done before bpf prog attach.
> I've tried following but still no luck:
> * /tmp is already present. Then I thought maybe the desired file is already
> present. I've verified that there isn't file of same name is present inside
> /tmp.
> * I thought maybe mknod isn't present in the system. But mknod --help succeeds.
> * I switched from /tmp to current directory to create the mknod. But the
> result is same error.
> * I've tried to use the same kernel config as the BPF CI is using. I'm not
> able to reproduce it.
>
> Not sure which edge case or what's going on. The problem is appearing
> because of some limitation in the rootfs.

Maybe you could collect /tmp mount options to see whether anything is
suspicious? In my vm, I have
   tmpfs on /tmp type tmpfs (rw,nosuid,nodev,size=3501540k,nr_inodes=1048576)
and the test works fine.