2022-10-07 16:34:41

by Yves-Alexis Perez

[permalink] [raw]
Subject: crypto_alg_lookup() returning -80 (ELIBBAD)

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi list,

since some time (and I'm unsure exactly when) my strongSwan IKE/IPsec setup at
home doesn't work anymore, with charon reporting
(https://pastebin.com/7tThD1B5):

received netlink error: Accessing a corrupted shared library (80)

This is on a Debian Bullseye machine running Debian kernel 5.10.140-1 (5.10.0-
18-amd64).

With some help from Tobias Brunner (from strongSwan project) I tried to debug
the issue (I was already fairly certain it was a local issue because if the
standard Debian kernel started misbehaving like that we would have a whole lot
more reports everywhere. Removing strongSwan from the equation, I tried to
inject directly an xfrm state with ip (the parameter are a bit bogus but I
don't think it matters at that point):

ip xfrm state add src 10.0.0.1 dst 10.0.0.2 proto esp spi 123456 sel src
10.0.0.3 dst 10.0.0.4 enc aes 0xabcdefabcdef
RTNETLINK answers: Accessing a corrupted shared library

I tried on a different Debian box and the output is:

ip xfrm state add src 10.0.0.1 dst 10.0.0.2 proto esp spi 123456 sel src
10.0.0.3 dst 10.0.0.4 enc aes 0xaabbccddee
RTNETLINK answers: Invalid argument

So there's definitely something fishy in my kernel and I'm unsure why.

Would anyone have a clue about what is happening here, and any idea how to
debug further?

Regards,
- --
Yves-Alexis
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEE8vi34Qgfo83x35gF3rYcyPpXRFsFAmNAUTEACgkQ3rYcyPpX
RFtDUAf/Vu0Ls/bNtEXp1JoxmJucztLoUBGZyaLs1fnxig5TzlLAohOF2OjV1ykh
U3m4fCRqM5JThpLLyISoch4PNBRnjMaR58l/fnvpeEgI7TmBXwJiA4IiVQuvfos/
jxrlp8tAQUvJCm1Se8NGeolBTi0a+SngHig+mW0ix4jde1NKOWdVJ6MZPQgcSIM9
eI8Rdvska6RToQ6VsyscjhjdM6HS6sk7/2me3CF4ezcn3atAJwTANMwLv1Y4CS0x
UeT/HVdWCqPptYUDgyLhcF6OC49wjQ79zTfZcQOGYHJTNh9Q3rXXsLqICYsRaTiU
SXzhPZzOaALKeTfY1jXw0LAFfSwdnQ==
=kv2/
-----END PGP SIGNATURE-----


2022-11-21 12:22:29

by Yves-Alexis Perez

[permalink] [raw]
Subject: Re: crypto_alg_lookup() returning -80 (ELIBBAD)

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Fri, 2022-10-07 at 18:17 +0200, Yves-Alexis Perez wrote:
> So there's definitely something fishy in my kernel and I'm unsure why.
>
> Would anyone have a clue about what is happening here, and any idea how to
> debug further?

With some help from systemtap I managed to debug further and narrowed down the
problem to the RNG (and more specifically anssi_cprng).

When booting, first load of the module returns:

[ 7.910500] alg: cprng: Failed to load transform for ansi_cprng: -2
[ 7.917774] alg: No test for fips(ansi_cprng) (fips_ansi_cprng)

But unloading/reloading the module afterwards only shows the second line and
IPsec starts working again (whether the `ip xfrm state` lines or strongSwan
more generally).

I have yet to debug further but my feeling is that it might be TPM-related but
I'm unsure and have no clear debugging path for now. I'll let you know if I
find anything.

Regards,
- --
Yves-Alexis
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEE8vi34Qgfo83x35gF3rYcyPpXRFsFAmN7bIIACgkQ3rYcyPpX
RFsqdQgA62FGOagIAHKW000yQ/pm42+vO9DZFo7zJ17OHCsjintACME/bU3p3O+l
mmWz7yv1ib7GcCL19p1ZN/XX3ukORYwuvm3ixKy7mytRb1qwphKLKl1t08NeEceB
b7z2ZyjQAIPslkT0LL88fk5T3iOjelZg94fNTerUxDiGWCt6a8Oqz09jBUEK2yST
UgkOGVPlNQM5Frs/SUiC2HhkHQEmek/urwncKVBfBCcmJQcqaaGBKeAyZB+JEyCz
1h0HnrHhIhjWPj93SwdbaqjnT7eIOT+jQSQ67CatUWkQBEHT1FJgcyle/mzhb8hs
wGW4VaOtbfTOtdMVlXFyipNMszMG2g==
=6MCJ
-----END PGP SIGNATURE-----