2023-05-17 16:33:41

by Nick Desaulniers

[permalink] [raw]
Subject: [PATCH] certs: buffer stderr from openssl unless error

Running `openssl req` prints a progress meter consisting of `.`, `*`,
and `+` characters to stderr which we redirect to stdout. During a build
with `make -j`, the output from this command becomes interspersed
throughout the rest of the quiet_cmd_* output, messing up the
indentation.

Suppress the output from this command unless the return code is
non-zero. If `openssl req` prints additional information to stderr
without setting a non-zero return code, it will be missed.

Suggested-by: Ard Biesheuvel <[email protected]>
Signed-off-by: Nick Desaulniers <[email protected]>
---
certs/Makefile | 4 +---
certs/gen_key.sh | 7 +++++++
2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/certs/Makefile b/certs/Makefile
index 799ad7b9e68a..9b4fee56780d 100644
--- a/certs/Makefile
+++ b/certs/Makefile
@@ -45,9 +45,7 @@ ifeq ($(CONFIG_MODULE_SIG_KEY),certs/signing_key.pem)
keytype-$(CONFIG_MODULE_SIG_KEY_TYPE_ECDSA) := -newkey ec -pkeyopt ec_paramgen_curve:secp384r1

quiet_cmd_gen_key = GENKEY $@
- cmd_gen_key = openssl req -new -nodes -utf8 -$(CONFIG_MODULE_SIG_HASH) -days 36500 \
- -batch -x509 -config $< \
- -outform PEM -out $@ -keyout $@ $(keytype-y) 2>&1
+ cmd_gen_key = $(srctree)/$(src)/gen_key.sh $(CONFIG_MODULE_SIG_HASH) $< $@ $(keytype-y)

$(obj)/signing_key.pem: $(obj)/x509.genkey FORCE
$(call if_changed,gen_key)
diff --git a/certs/gen_key.sh b/certs/gen_key.sh
new file mode 100755
index 000000000000..1de1f22be484
--- /dev/null
+++ b/certs/gen_key.sh
@@ -0,0 +1,7 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+OUT=$(openssl req -new -nodes -utf8 -"$1" -days 36500 -batch -x509 \
+ -config "$2" -outform PEM -out "$3" -keyout "$3" $4 2>&1)
+if [[ $? -ne 0 ]]; then
+ echo "$OUT"
+fi

---
base-commit: f1fcbaa18b28dec10281551dfe6ed3a3ed80e3d6
change-id: 20230517-genkey-24a835572835

Best regards,
--
Nick Desaulniers <[email protected]>



2023-05-18 10:49:29

by kernel test robot

[permalink] [raw]
Subject: Re: [PATCH] certs: buffer stderr from openssl unless error

Hi,

kernel test robot noticed the following build errors:

[auto build test ERROR on f1fcbaa18b28dec10281551dfe6ed3a3ed80e3d6]

url: https://github.com/intel-lab-lkp/linux/commits/ndesaulniers-google-com/certs-buffer-stderr-from-openssl-unless-error/20230518-004753
base: f1fcbaa18b28dec10281551dfe6ed3a3ed80e3d6
patch link: https://lore.kernel.org/r/20230517-genkey-v1-1-b887424da4a8%40google.com
patch subject: [PATCH] certs: buffer stderr from openssl unless error
config: csky-randconfig-m041-20230517
compiler: csky-linux-gcc (GCC) 12.1.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/0e437a41fdb41c84834de6776bf38951b197792a
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review ndesaulniers-google-com/certs-buffer-stderr-from-openssl-unless-error/20230518-004753
git checkout 0e437a41fdb41c84834de6776bf38951b197792a
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=csky olddefconfig
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=csky SHELL=/bin/bash

If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

At main.c:152:
- SSL error:FFFFFFFF80000002:system library::No such file or directory: ../crypto/bio/bss_file.c:67
- SSL error:10000080:BIO routines::no such file: ../crypto/bio/bss_file.c:75
>> extract-cert: certs/signing_key.pem: No such file or directory

--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki


Attachments:
(No filename) (1.97 kB)
config (161.28 kB)
Download all attachments

2023-05-18 18:40:09

by Jarkko Sakkinen

[permalink] [raw]
Subject: Re: [PATCH] certs: buffer stderr from openssl unless error

On Wed May 17, 2023 at 7:23 PM EEST, wrote:
> Running `openssl req` prints a progress meter consisting of `.`, `*`,
> and `+` characters to stderr which we redirect to stdout. During a build
> with `make -j`, the output from this command becomes interspersed
> throughout the rest of the quiet_cmd_* output, messing up the
> indentation.
>
> Suppress the output from this command unless the return code is
> non-zero. If `openssl req` prints additional information to stderr
> without setting a non-zero return code, it will be missed.
>
> Suggested-by: Ard Biesheuvel <[email protected]>
> Signed-off-by: Nick Desaulniers <[email protected]>
> ---
> certs/Makefile | 4 +---
> certs/gen_key.sh | 7 +++++++
> 2 files changed, 8 insertions(+), 3 deletions(-)
>
> diff --git a/certs/Makefile b/certs/Makefile
> index 799ad7b9e68a..9b4fee56780d 100644
> --- a/certs/Makefile
> +++ b/certs/Makefile
> @@ -45,9 +45,7 @@ ifeq ($(CONFIG_MODULE_SIG_KEY),certs/signing_key.pem)
> keytype-$(CONFIG_MODULE_SIG_KEY_TYPE_ECDSA) := -newkey ec -pkeyopt ec_paramgen_curve:secp384r1
>
> quiet_cmd_gen_key = GENKEY $@
> - cmd_gen_key = openssl req -new -nodes -utf8 -$(CONFIG_MODULE_SIG_HASH) -days 36500 \
> - -batch -x509 -config $< \
> - -outform PEM -out $@ -keyout $@ $(keytype-y) 2>&1
> + cmd_gen_key = $(srctree)/$(src)/gen_key.sh $(CONFIG_MODULE_SIG_HASH) $< $@ $(keytype-y)
>
> $(obj)/signing_key.pem: $(obj)/x509.genkey FORCE
> $(call if_changed,gen_key)
> diff --git a/certs/gen_key.sh b/certs/gen_key.sh
> new file mode 100755
> index 000000000000..1de1f22be484
> --- /dev/null
> +++ b/certs/gen_key.sh
> @@ -0,0 +1,7 @@
> +#!/usr/bin/env bash
> +# SPDX-License-Identifier: GPL-2.0
> +OUT=$(openssl req -new -nodes -utf8 -"$1" -days 36500 -batch -x509 \
> + -config "$2" -outform PEM -out "$3" -keyout "$3" $4 2>&1)
> +if [[ $? -ne 0 ]]; then
> + echo "$OUT"
> +fi
>
> ---
> base-commit: f1fcbaa18b28dec10281551dfe6ed3a3ed80e3d6
> change-id: 20230517-genkey-24a835572835
>
> Best regards,
> --
> Nick Desaulniers <[email protected]>

Reviewed-by: Jarkko Sakkinen <[email protected]>

BR, Jarkko

2023-05-18 22:47:02

by Nick Desaulniers

[permalink] [raw]
Subject: Re: [PATCH] certs: buffer stderr from openssl unless error

On Thu, May 18, 2023 at 3:23 AM kernel test robot <[email protected]> wrote:
>
> Hi,
>
> kernel test robot noticed the following build errors:
>
> [auto build test ERROR on f1fcbaa18b28dec10281551dfe6ed3a3ed80e3d6]
>
> url: https://github.com/intel-lab-lkp/linux/commits/ndesaulniers-google-com/certs-buffer-stderr-from-openssl-unless-error/20230518-004753
> base: f1fcbaa18b28dec10281551dfe6ed3a3ed80e3d6
> patch link: https://lore.kernel.org/r/20230517-genkey-v1-1-b887424da4a8%40google.com
> patch subject: [PATCH] certs: buffer stderr from openssl unless error
> config: csky-randconfig-m041-20230517
> compiler: csky-linux-gcc (GCC) 12.1.0
> reproduce (this is a W=1 build):
> wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross

^ should encourage `mkdir ~/bin`; the debian docker container doesn't
have `~/bin`.

> chmod +x ~/bin/make.cross
> # https://github.com/intel-lab-lkp/linux/commit/0e437a41fdb41c84834de6776bf38951b197792a
> git remote add linux-review https://github.com/intel-lab-lkp/linux
> git fetch --no-tags linux-review ndesaulniers-google-com/certs-buffer-stderr-from-openssl-unless-error/20230518-004753

^ holy crap that takes forever. maybe recommend `b4 shazam <lore link>`?

> git checkout 0e437a41fdb41c84834de6776bf38951b197792a
> # save the config file
> mkdir build_dir && cp config build_dir/.config
> COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=csky olddefconfig

^ should use `~/bin/make.cross` since `~/bin` may not exist in $PATH.

> COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=csky SHELL=/bin/bash

Hi Philip,
I've run into some issues with make.cross when trying to install the
csky toolchain. Maybe you can help?

```
$ COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0
~/bin/make.cross W=1 O=build_dir ARCH=csky olddefconfig
Compiler will be installed in /root/0day
make: gcc: No such file or directory
lftpget -c https://download.01.org/0day-ci/cross-package/./gcc-12.1.0-nolibc/x86_64-gcc-12.1.0-nolibc_csky-linux.tar.xz
/linux
tar Jxf /root/0day/gcc-12.1.0-nolibc/x86_64-gcc-12.1.0-nolibc_csky-linux.tar.xz
-C /root/0day
Please update: libc6 or glibc
ldd /root/0day/gcc-12.1.0-nolibc/csky-linux/bin/csky-linux-gcc
/root/0day/gcc-12.1.0-nolibc/csky-linux/bin/csky-linux-gcc:
/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found
(required by /root/0day/gcc-12.1.0-nolibc/csky-linux/bin/csky-linux-gcc)
/root/0day/gcc-12.1.0-nolibc/csky-linux/bin/csky-linux-gcc:
/lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found
(required by /root/0day/gcc-12.1.0-nolibc/csky-linux/bin/csky-linux-gcc)
setup_crosstool failed

$ ldd -v /lib/x86_64-linux-gnu/libc.so.6
/lib64/ld-linux-x86-64.so.2 (0x00007f11f3478000)
linux-vdso.so.1 (0x00007ffcf45b2000)

Version information:
/lib/x86_64-linux-gnu/libc.so.6:
ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2

$ cat /etc/debian_version
11.6
```
Perhaps I MUST update my debian container to 11.7?

Are the 0day toolchains not statically linked? Can they use the ones
from kernel.org which are?
https://mirrors.edge.kernel.org/pub/tools/crosstool/

>
> If you fix the issue, kindly add following tag where applicable
> | Reported-by: kernel test robot <[email protected]>
> | Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
>
> All errors (new ones prefixed by >>):
>
> At main.c:152:
> - SSL error:FFFFFFFF80000002:system library::No such file or directory: ../crypto/bio/bss_file.c:67
> - SSL error:10000080:BIO routines::no such file: ../crypto/bio/bss_file.c:75
> >> extract-cert: certs/signing_key.pem: No such file or directory
>
> --
> 0-DAY CI Kernel Test Service
> https://github.com/intel/lkp-tests/wiki



--
Thanks,
~Nick Desaulniers

2023-05-22 07:12:05

by Li, Philip

[permalink] [raw]
Subject: Re: [PATCH] certs: buffer stderr from openssl unless error

On Thu, May 18, 2023 at 03:36:53PM -0700, Nick Desaulniers wrote:
> On Thu, May 18, 2023 at 3:23 AM kernel test robot <[email protected]> wrote:
> >
> > Hi,
> >
> > kernel test robot noticed the following build errors:
> >
> > [auto build test ERROR on f1fcbaa18b28dec10281551dfe6ed3a3ed80e3d6]
> >
> > url: https://github.com/intel-lab-lkp/linux/commits/ndesaulniers-google-com/certs-buffer-stderr-from-openssl-unless-error/20230518-004753
> > base: f1fcbaa18b28dec10281551dfe6ed3a3ed80e3d6
> > patch link: https://lore.kernel.org/r/20230517-genkey-v1-1-b887424da4a8%40google.com
> > patch subject: [PATCH] certs: buffer stderr from openssl unless error
> > config: csky-randconfig-m041-20230517
> > compiler: csky-linux-gcc (GCC) 12.1.0
> > reproduce (this is a W=1 build):
> > wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
>
> ^ should encourage `mkdir ~/bin`; the debian docker container doesn't
> have `~/bin`.
>
> > chmod +x ~/bin/make.cross
> > # https://github.com/intel-lab-lkp/linux/commit/0e437a41fdb41c84834de6776bf38951b197792a
> > git remote add linux-review https://github.com/intel-lab-lkp/linux
> > git fetch --no-tags linux-review ndesaulniers-google-com/certs-buffer-stderr-from-openssl-unless-error/20230518-004753
>
> ^ holy crap that takes forever. maybe recommend `b4 shazam <lore link>`?
>
> > git checkout 0e437a41fdb41c84834de6776bf38951b197792a
> > # save the config file
> > mkdir build_dir && cp config build_dir/.config
> > COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=csky olddefconfig
>
> ^ should use `~/bin/make.cross` since `~/bin` may not exist in $PATH.
>
> > COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=csky SHELL=/bin/bash
>
> Hi Philip,
> I've run into some issues with make.cross when trying to install the
> csky toolchain. Maybe you can help?

Got it and sorry for late reply that i was in a few days sick leave. We will
go through all your comments and provide update in earliest time.

>
> ```
> $ COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0
> ~/bin/make.cross W=1 O=build_dir ARCH=csky olddefconfig
> Compiler will be installed in /root/0day
> make: gcc: No such file or directory
> lftpget -c https://download.01.org/0day-ci/cross-package/./gcc-12.1.0-nolibc/x86_64-gcc-12.1.0-nolibc_csky-linux.tar.xz
> /linux
> tar Jxf /root/0day/gcc-12.1.0-nolibc/x86_64-gcc-12.1.0-nolibc_csky-linux.tar.xz
> -C /root/0day
> Please update: libc6 or glibc
> ldd /root/0day/gcc-12.1.0-nolibc/csky-linux/bin/csky-linux-gcc
> /root/0day/gcc-12.1.0-nolibc/csky-linux/bin/csky-linux-gcc:
> /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found
> (required by /root/0day/gcc-12.1.0-nolibc/csky-linux/bin/csky-linux-gcc)
> /root/0day/gcc-12.1.0-nolibc/csky-linux/bin/csky-linux-gcc:
> /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found
> (required by /root/0day/gcc-12.1.0-nolibc/csky-linux/bin/csky-linux-gcc)
> setup_crosstool failed
>
> $ ldd -v /lib/x86_64-linux-gnu/libc.so.6
> /lib64/ld-linux-x86-64.so.2 (0x00007f11f3478000)
> linux-vdso.so.1 (0x00007ffcf45b2000)
>
> Version information:
> /lib/x86_64-linux-gnu/libc.so.6:
> ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
> ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
>
> $ cat /etc/debian_version
> 11.6
> ```
> Perhaps I MUST update my debian container to 11.7?
>
> Are the 0day toolchains not statically linked? Can they use the ones
> from kernel.org which are?
> https://mirrors.edge.kernel.org/pub/tools/crosstool/
>
> >
> > If you fix the issue, kindly add following tag where applicable
> > | Reported-by: kernel test robot <[email protected]>
> > | Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
> >
> > All errors (new ones prefixed by >>):
> >
> > At main.c:152:
> > - SSL error:FFFFFFFF80000002:system library::No such file or directory: ../crypto/bio/bss_file.c:67
> > - SSL error:10000080:BIO routines::no such file: ../crypto/bio/bss_file.c:75
> > >> extract-cert: certs/signing_key.pem: No such file or directory
> >
> > --
> > 0-DAY CI Kernel Test Service
> > https://github.com/intel/lkp-tests/wiki
>
>
>
> --
> Thanks,
> ~Nick Desaulniers
>

2023-05-22 08:58:11

by Guo Ren

[permalink] [raw]
Subject: Re: [PATCH] certs: buffer stderr from openssl unless error

F.Y.I., our compiler team, would put an eye on this.

On Fri, May 19, 2023 at 6:37 AM Nick Desaulniers
<[email protected]> wrote:
>
> On Thu, May 18, 2023 at 3:23 AM kernel test robot <[email protected]> wrote:
> >
> > Hi,
> >
> > kernel test robot noticed the following build errors:
> >
> > [auto build test ERROR on f1fcbaa18b28dec10281551dfe6ed3a3ed80e3d6]
> >
> > url: https://github.com/intel-lab-lkp/linux/commits/ndesaulniers-google-com/certs-buffer-stderr-from-openssl-unless-error/20230518-004753
> > base: f1fcbaa18b28dec10281551dfe6ed3a3ed80e3d6
> > patch link: https://lore.kernel.org/r/20230517-genkey-v1-1-b887424da4a8%40google.com
> > patch subject: [PATCH] certs: buffer stderr from openssl unless error
> > config: csky-randconfig-m041-20230517
> > compiler: csky-linux-gcc (GCC) 12.1.0
> > reproduce (this is a W=1 build):
> > wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
>
> ^ should encourage `mkdir ~/bin`; the debian docker container doesn't
> have `~/bin`.
>
> > chmod +x ~/bin/make.cross
> > # https://github.com/intel-lab-lkp/linux/commit/0e437a41fdb41c84834de6776bf38951b197792a
> > git remote add linux-review https://github.com/intel-lab-lkp/linux
> > git fetch --no-tags linux-review ndesaulniers-google-com/certs-buffer-stderr-from-openssl-unless-error/20230518-004753
>
> ^ holy crap that takes forever. maybe recommend `b4 shazam <lore link>`?
>
> > git checkout 0e437a41fdb41c84834de6776bf38951b197792a
> > # save the config file
> > mkdir build_dir && cp config build_dir/.config
> > COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=csky olddefconfig
>
> ^ should use `~/bin/make.cross` since `~/bin` may not exist in $PATH.
>
> > COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=csky SHELL=/bin/bash
>
> Hi Philip,
> I've run into some issues with make.cross when trying to install the
> csky toolchain. Maybe you can help?
>
> ```
> $ COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0
> ~/bin/make.cross W=1 O=build_dir ARCH=csky olddefconfig
> Compiler will be installed in /root/0day
> make: gcc: No such file or directory
> lftpget -c https://download.01.org/0day-ci/cross-package/./gcc-12.1.0-nolibc/x86_64-gcc-12.1.0-nolibc_csky-linux.tar.xz
> /linux
> tar Jxf /root/0day/gcc-12.1.0-nolibc/x86_64-gcc-12.1.0-nolibc_csky-linux.tar.xz
> -C /root/0day
> Please update: libc6 or glibc
> ldd /root/0day/gcc-12.1.0-nolibc/csky-linux/bin/csky-linux-gcc
> /root/0day/gcc-12.1.0-nolibc/csky-linux/bin/csky-linux-gcc:
> /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found
> (required by /root/0day/gcc-12.1.0-nolibc/csky-linux/bin/csky-linux-gcc)
> /root/0day/gcc-12.1.0-nolibc/csky-linux/bin/csky-linux-gcc:
> /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found
> (required by /root/0day/gcc-12.1.0-nolibc/csky-linux/bin/csky-linux-gcc)
> setup_crosstool failed
>
> $ ldd -v /lib/x86_64-linux-gnu/libc.so.6
> /lib64/ld-linux-x86-64.so.2 (0x00007f11f3478000)
> linux-vdso.so.1 (0x00007ffcf45b2000)
>
> Version information:
> /lib/x86_64-linux-gnu/libc.so.6:
> ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
> ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
>
> $ cat /etc/debian_version
> 11.6
> ```
> Perhaps I MUST update my debian container to 11.7?
>
> Are the 0day toolchains not statically linked? Can they use the ones
> from kernel.org which are?
> https://mirrors.edge.kernel.org/pub/tools/crosstool/
>
> >
> > If you fix the issue, kindly add following tag where applicable
> > | Reported-by: kernel test robot <[email protected]>
> > | Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
> >
> > All errors (new ones prefixed by >>):
> >
> > At main.c:152:
> > - SSL error:FFFFFFFF80000002:system library::No such file or directory: ../crypto/bio/bss_file.c:67
> > - SSL error:10000080:BIO routines::no such file: ../crypto/bio/bss_file.c:75
> > >> extract-cert: certs/signing_key.pem: No such file or directory
> >
> > --
> > 0-DAY CI Kernel Test Service
> > https://github.com/intel/lkp-tests/wiki
>
>
>
> --
> Thanks,
> ~Nick Desaulniers



--
Best Regards
Guo Ren

2023-05-22 15:48:13

by kernel test robot

[permalink] [raw]
Subject: Re: [PATCH] certs: buffer stderr from openssl unless error

Hi,

kernel test robot noticed the following build errors:

[auto build test ERROR on f1fcbaa18b28dec10281551dfe6ed3a3ed80e3d6]

url: https://github.com/intel-lab-lkp/linux/commits/ndesaulniers-google-com/certs-buffer-stderr-from-openssl-unless-error/20230518-004753
base: f1fcbaa18b28dec10281551dfe6ed3a3ed80e3d6
patch link: https://lore.kernel.org/r/20230517-genkey-v1-1-b887424da4a8%40google.com
patch subject: [PATCH] certs: buffer stderr from openssl unless error
config: arm-randconfig-r046-20230521 (https://download.01.org/0day-ci/archive/20230522/[email protected]/config)
compiler: clang version 17.0.0 (https://github.com/llvm/llvm-project b0fb98227c90adf2536c9ad644a74d5e92961111)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# install arm cross compiling tool for clang build
# apt-get install binutils-arm-linux-gnueabi
# https://github.com/intel-lab-lkp/linux/commit/0e437a41fdb41c84834de6776bf38951b197792a
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review ndesaulniers-google-com/certs-buffer-stderr-from-openssl-unless-error/20230518-004753
git checkout 0e437a41fdb41c84834de6776bf38951b197792a
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=arm olddefconfig
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=arm SHELL=/bin/bash

If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

At main.c:152:
- SSL error:FFFFFFFF80000002:system library::No such file or directory: ../crypto/bio/bss_file.c:67
- SSL error:10000080:BIO routines::no such file: ../crypto/bio/bss_file.c:75
>> extract-cert: certs/signing_key.pem: No such file or directory

--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki

2023-06-08 10:08:30

by Yujie Liu

[permalink] [raw]
Subject: Re: [PATCH] certs: buffer stderr from openssl unless error

Hi Nick,

On Thu, 2023-05-18 at 15:36 -0700, Nick Desaulniers wrote:
> On Thu, May 18, 2023 at 3:23 AM kernel test robot <[email protected]> wrote:
> >
> > Hi,
> >
> > kernel test robot noticed the following build errors:
> >
> > [auto build test ERROR on f1fcbaa18b28dec10281551dfe6ed3a3ed80e3d6]
> >
> > url:    https://github.com/intel-lab-lkp/linux/commits/ndesaulniers-google-com/certs-buffer-stderr-from-openssl-unless-error/20230518-004753
> > base:   f1fcbaa18b28dec10281551dfe6ed3a3ed80e3d6
> > patch link:    https://lore.kernel.org/r/20230517-genkey-v1-1-b887424da4a8%40google.com
> > patch subject: [PATCH] certs: buffer stderr from openssl unless error
> > config: csky-randconfig-m041-20230517
> > compiler: csky-linux-gcc (GCC) 12.1.0
> > reproduce (this is a W=1 build):
> >         wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
>
> ^ should encourage `mkdir ~/bin`; the debian docker container doesn't
> have `~/bin`.
>
> >         chmod +x ~/bin/make.cross
> >         # https://github.com/intel-lab-lkp/linux/commit/0e437a41fdb41c84834de6776bf38951b197792a
> >         git remote add linux-review https://github.com/intel-lab-lkp/linux
> >         git fetch --no-tags linux-review ndesaulniers-google-com/certs-buffer-stderr-from-openssl-unless-error/20230518-004753
>
> ^ holy crap that takes forever. maybe recommend `b4 shazam <lore link>`?
>
> >         git checkout 0e437a41fdb41c84834de6776bf38951b197792a
> >         # save the config file
> >         mkdir build_dir && cp config build_dir/.config
> >         COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=csky olddefconfig
>
> ^ should use `~/bin/make.cross` since `~/bin` may not exist in $PATH.
>
> >         COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=csky SHELL=/bin/bash

We've updated the "reproduce" section in bot's reports to be more
precise. Thanks a lot for the inputs.

> Hi Philip,
> I've run into some issues with make.cross when trying to install the
> csky toolchain. Maybe you can help?
>
> ```
> $ COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0
> ~/bin/make.cross W=1 O=build_dir ARCH=csky olddefconfig
> Compiler will be installed in /root/0day
> make: gcc: No such file or directory
> lftpget -c https://download.01.org/0day-ci/cross-package/./gcc-12.1.0-nolibc/x86_64-gcc-12.1.0-nolibc_csky-linux.tar.xz
> /linux
> tar Jxf /root/0day/gcc-12.1.0-nolibc/x86_64-gcc-12.1.0-nolibc_csky-linux.tar.xz
> -C /root/0day
> Please update: libc6 or glibc
> ldd /root/0day/gcc-12.1.0-nolibc/csky-linux/bin/csky-linux-gcc
> /root/0day/gcc-12.1.0-nolibc/csky-linux/bin/csky-linux-gcc:
> /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found
> (required by /root/0day/gcc-12.1.0-nolibc/csky-linux/bin/csky-linux-gcc)
> /root/0day/gcc-12.1.0-nolibc/csky-linux/bin/csky-linux-gcc:
> /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found
> (required by /root/0day/gcc-12.1.0-nolibc/csky-linux/bin/csky-linux-gcc)
> setup_crosstool failed
>
> $ ldd -v /lib/x86_64-linux-gnu/libc.so.6
> /lib64/ld-linux-x86-64.so.2 (0x00007f11f3478000)
> linux-vdso.so.1 (0x00007ffcf45b2000)
>
> Version information:
> /lib/x86_64-linux-gnu/libc.so.6:
> ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
> ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2
>
> $ cat /etc/debian_version
> 11.6
> ```
> Perhaps I MUST update my debian container to 11.7?
>
> Are the 0day toolchains not statically linked?
> Can they use the ones from kernel.org which are?
> https://mirrors.edge.kernel.org/pub/tools/crosstool/

Please follow below steps to use the toolchains provided by kernel.org.
We've tested this and it can work in a debian 11.6 docker env.

$ cat /etc/debian_version
11.6

$ ldd -v /lib/x86_64-linux-gnu/libc.so.6 | grep GLIBC
ld-linux-x86-64.so.2 (GLIBC_2.3) => /lib64/ld-linux-x86-64.so.2
ld-linux-x86-64.so.2 (GLIBC_PRIVATE) => /lib64/ld-linux-x86-64.so.2

$ export URL=https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64

$ COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 ./make.cross W=1 ARCH=csky olddefconfig
Compiler will be installed in /root/0day
lftpget -c https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/./12.1.0/x86_64-gcc-12.1.0-nolibc-csky-linux.tar.xz
/root/linux
tar Jxf /root/0day/12.1.0/x86_64-gcc-12.1.0-nolibc-csky-linux.tar.xz -C /root/0day
make --keep-going CONFIG_OF_ALL_DTBS=y CONFIG_DTC=y CROSS_COMPILE=/root/0day/gcc-12.1.0-nolibc/csky-linux/bin/csky-linux- --jobs=144 W=1 ARCH=csky olddefconfig
...


Thanks a lot for the suggestions. We will further improve the 0day
make.cross tool such as:

* Provide statically linked toolchains that don't depend on the libc in
the distribution
* Add hint to help user decide whether to use 0day or kernel.org
toolchains.

Best Regards,
Yujie

> >
> > If you fix the issue, kindly add following tag where applicable
> > > Reported-by: kernel test robot <[email protected]>
> > > Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
> >
> > All errors (new ones prefixed by >>):
> >
> >    At main.c:152:
> >    - SSL error:FFFFFFFF80000002:system library::No such file or directory: ../crypto/bio/bss_file.c:67
> >    - SSL error:10000080:BIO routines::no such file: ../crypto/bio/bss_file.c:75
> > > > extract-cert: certs/signing_key.pem: No such file or directory
> >
> > --
> > 0-DAY CI Kernel Test Service
> > https://github.com/intel/lkp-tests/wiki
>
>
>