2009-02-24 17:52:42

by Jan Glauber

[permalink] [raw]
Subject: hanging modprobe aes_s390

Hi Herbert,

commit 73d3864a4823abda19ebc4387b6ddcbf416e3a77 leads to a
hanging modprobe aes_s390 process. If the process is
interrupted the following oops occurs:

------------[ cut here ]------------
Badness at crypto/algapi.c:293
Modules linked in: aes_generic aes_s390(+) binfmt_misc
CPU: 0 Not tainted 2.6.27-rc5-00010-g1aa4ecd #23
Process modprobe (pid: 3401, task: 000000003ea2c048, ksp: 000000003eb13b78)
Krnl PSW : 0704100180000000 000000000013af2c (crypto_wait_for_test+0x74/0x80)
R:0 T:1 IO:1 EX:1 Key:0 M:1 W:0 P:0 AS:0 CC:1 PM:0 EA:3
Krnl GPRS: 00000000010a4000 000000003eb14000 fffffffffffffe00 0000000000000000
000000000013af12 00000000003e8900 0000000000007aa8 0000000000000000
0000000000458048 000000003eb10000 000003e000017000 000003e000000000
000000003d6cc600 00000000002dcfc8 000000000013af12 000000003eb13d18
Krnl Code: 000000000013af1c: ebcff0a00004 lmg %r12,%r15,160(%r15)
000000000013af22: c0f4ffffedf1 brcl 15,138b04
000000000013af28: a7f40001 brc 15,13af2a
>000000000013af2c: a7f4fff6 brc 15,13af18
000000000013af30: a7f40001 brc 15,13af32
000000000013af34: a7f4fff2 brc 15,13af18
000000000013af38: ebcff0780024 stmg %r12,%r15,120(%r15)
000000000013af3e: a7f13f00 tmll %r15,16128
Call Trace:
([<000000000013af12>] crypto_wait_for_test+0x5a/0x80)
[<000000000013b288>] crypto_register_alg+0x80/0x98
[<000003e000017136>] aes_s390_init+0x136/0x1e4 [aes_s390]
[<00000000000120c0>] do_one_initcall+0x40/0x17c
[<000000000006618e>] sys_init_module+0xc6/0x1e4
[<0000000000023d18>] sysc_noemu+0x10/0x16
[<000002000011cda2>] 0x2000011cda2
Last Breaking-Event-Address:
[<000000000013af28>] crypto_wait_for_test+0x70/0x80

That only happens if the aes_generic module isn't loaded. If
aes_generic is already present the aes_s390 loads without problems.

Any idea how to solve this? Is something missing in the fallback code
that uses aes_generic?

Cheers, Jan




2009-02-25 03:51:42

by Herbert Xu

[permalink] [raw]
Subject: Re: hanging modprobe aes_s390

Jan Glauber <[email protected]> wrote:
>
> That only happens if the aes_generic module isn't loaded. If
> aes_generic is already present the aes_s390 loads without problems.
>
> Any idea how to solve this? Is something missing in the fallback code
> that uses aes_generic?

Yeah it looks like it's waiting for the fallback to come up.
However, the interesting bit is in the other processes, i.e., the
one that's actually testing aes_s390 or constructing the aes-generic.

Could you get those back traces please?

Thanks,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

2009-02-25 16:34:08

by Jan Glauber

[permalink] [raw]
Subject: Re: hanging modprobe aes_s390

On Wed, 2009-02-25 at 11:51 +0800, Herbert Xu wrote:
> Jan Glauber <[email protected]> wrote:
> >
> > That only happens if the aes_generic module isn't loaded. If
> > aes_generic is already present the aes_s390 loads without problems.
> >
> > Any idea how to solve this? Is something missing in the fallback code
> > that uses aes_generic?
>
> Yeah it looks like it's waiting for the fallback to come up.
> However, the interesting bit is in the other processes, i.e., the
> one that's actually testing aes_s390 or constructing the aes-generic.
>
> Could you get those back traces please?

I've found 4 processes/threads that seem to be involved. And in case
that is relevant the oops message occured 3 times, see the attachment.

STACK TRACE FOR TASK: 0x394e37c8 (modprobe)

STACK:
0 schedule+1136 [0x2df82c]
1 schedule_timeout+198 [0x2e016e]
2 wait_for_common+256 [0x2df14c]
3 wait_for_completion_interruptible+56 [0x2df2ac]
4 crypto_wait_for_test+94 [0x16a28a]
5 crypto_register_alg+132 [0x16a598]
6 <sym not found>+<ERROR> [0x3e000daa13a]
7 do_one_initcall+68 [0x120c4]
8 sys_init_module+206 [0x825da]
9 sysc_noemu+16 [0x27a5e]

STACK TRACE FOR TASK: 0x3dcc4a60 (cryptomgr_test)

STACK:
0 schedule+1136 [0x2df82c]
1 schedule_timeout+198 [0x2e016e]
2 wait_for_common+256 [0x2df14c]
3 wait_for_completion+56 [0x2df344]
4 call_usermodehelper_exec+152 [0x5aec8]
5 request_module+272 [0x5b15c]
6 crypto_larval_lookup+176 [0x1681ec]
7 crypto_alg_mod_lookup+70 [0x1682f6]
8 crypto_alloc_base+70 [0x168416]
9 <sym not found>+<ERROR> [0x3e000d92d18]
10 __crypto_alloc_tfm+166 [0x16797a]
11 crypto_alloc_base+100 [0x168434]
12 alg_test+280 [0x17159c]
13 cryptomgr_test+80 [0x16fb4c]
14 kthread+106 [0x60d4a]
15 kernel_thread_starter+6 [0x1b01a]

STACK TRACE FOR TASK: 0x394fb7c8 (khelper)

STACK:
0 schedule+1136 [0x2df82c]
1 do_wait+672 [0x480c0]
2 sys_wait4+160 [0x48330]
3 wait_for_helper+102 [0x5afe2]
4 kernel_thread_starter+6 [0x1b01a]

STACK TRACE FOR TASK: 0x3f6137c8 (modprobe)

STACK:
0 schedule+1136 [0x2df82c]
1 fcntl_setlk+412 [0x107a0c]
2 sys_fcntl+262 [0xd7076]
3 sysc_noemu+16 [0x27a5e]

Cheers, Jan


Attachments:
oops.txt (4.93 kB)

2009-02-26 06:06:24

by Herbert Xu

[permalink] [raw]
Subject: Re: hanging modprobe aes_s390

On Wed, Feb 25, 2009 at 05:33:50PM +0000, Jan Glauber wrote:
>
> STACK TRACE FOR TASK: 0x3f6137c8 (modprobe)
>
> STACK:
> 0 schedule+1136 [0x2df82c]
> 1 fcntl_setlk+412 [0x107a0c]
> 2 sys_fcntl+262 [0xd7076]
> 3 sysc_noemu+16 [0x27a5e]

I see. Please let me know if this patch fixes it.

crypto: api - Fix module load deadlock with fallback algorithms

With the mandatory algorithm testing at registration, we have
now created a deadlock with algorithms requiring fallbacks.
This can happen if the module containing the algorithm requiring
fallback is loaded first, without the fallback module being loaded
first. The system will then try to test the new algorithm, find
that it needs to load a fallback, and then try to load that.

As both algorithms share the same module alias, it can attempt
to load the original algorithm again and block indefinitely.

As algorithms requiring fallbacks are a special case, we can fix
this by giving them a different module alias than the rest. Then
it's just a matter of using the right aliases according to what
algorithms we're trying to find.

Signed-off-by: Herbert Xu <[email protected]>

diff --git a/arch/s390/crypto/aes_s390.c b/arch/s390/crypto/aes_s390.c
index c42cd89..6118890 100644
--- a/arch/s390/crypto/aes_s390.c
+++ b/arch/s390/crypto/aes_s390.c
@@ -556,7 +556,7 @@ static void __exit aes_s390_fini(void)
module_init(aes_s390_init);
module_exit(aes_s390_fini);

-MODULE_ALIAS("aes");
+MODULE_ALIAS("aes-all");

MODULE_DESCRIPTION("Rijndael (AES) Cipher Algorithm");
MODULE_LICENSE("GPL");
diff --git a/crypto/api.c b/crypto/api.c
index efe77df..38a2bc0 100644
--- a/crypto/api.c
+++ b/crypto/api.c
@@ -215,8 +215,19 @@ struct crypto_alg *crypto_larval_lookup(const char *name, u32 type, u32 mask)
mask &= ~(CRYPTO_ALG_LARVAL | CRYPTO_ALG_DEAD);
type &= mask;

- alg = try_then_request_module(crypto_alg_lookup(name, type, mask),
- name);
+ alg = crypto_alg_lookup(name, type, mask);
+ if (!alg) {
+ char tmp[CRYPTO_MAX_ALG_NAME];
+
+ request_module(name);
+
+ if (!((type ^ CRYPTO_ALG_NEED_FALLBACK) & mask) &&
+ snprintf(tmp, sizeof(tmp), "%s-all", name) < sizeof(tmp))
+ request_module(tmp);
+
+ alg = crypto_alg_lookup(name, type, mask);
+ }
+
if (alg)
return crypto_is_larval(alg) ? crypto_larval_wait(alg) : alg;

diff --git a/drivers/crypto/padlock-aes.c b/drivers/crypto/padlock-aes.c
index 856b3cc..3f0fdd1 100644
--- a/drivers/crypto/padlock-aes.c
+++ b/drivers/crypto/padlock-aes.c
@@ -489,4 +489,4 @@ MODULE_DESCRIPTION("VIA PadLock AES algorithm support");
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Michal Ludvig");

-MODULE_ALIAS("aes");
+MODULE_ALIAS("aes-all");
diff --git a/drivers/crypto/padlock-sha.c b/drivers/crypto/padlock-sha.c
index a7fbade..a2c8e85 100644
--- a/drivers/crypto/padlock-sha.c
+++ b/drivers/crypto/padlock-sha.c
@@ -304,7 +304,7 @@ MODULE_DESCRIPTION("VIA PadLock SHA1/SHA256 algorithms support.");
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Michal Ludvig");

-MODULE_ALIAS("sha1");
-MODULE_ALIAS("sha256");
+MODULE_ALIAS("sha1-all");
+MODULE_ALIAS("sha256-all");
MODULE_ALIAS("sha1-padlock");
MODULE_ALIAS("sha256-padlock");

Thanks,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

2009-02-26 14:49:49

by Jan Glauber

[permalink] [raw]
Subject: Re: hanging modprobe aes_s390

Hi Herbert,

your patch solves the hanging modprobe (tested on top of cryptodev-2.6).
Both modules (aes_generic and aes_s390) are loaded after the modprobe
aes_s390.

Thanks a lot, Jan

On Thu, 2009-02-26 at 14:06 +0800, Herbert Xu wrote:
> On Wed, Feb 25, 2009 at 05:33:50PM +0000, Jan Glauber wrote:
> >
> > STACK TRACE FOR TASK: 0x3f6137c8 (modprobe)
> >
> > STACK:
> > 0 schedule+1136 [0x2df82c]
> > 1 fcntl_setlk+412 [0x107a0c]
> > 2 sys_fcntl+262 [0xd7076]
> > 3 sysc_noemu+16 [0x27a5e]
>
> I see. Please let me know if this patch fixes it.
>
> crypto: api - Fix module load deadlock with fallback algorithms
>
> With the mandatory algorithm testing at registration, we have
> now created a deadlock with algorithms requiring fallbacks.
> This can happen if the module containing the algorithm requiring
> fallback is loaded first, without the fallback module being loaded
> first. The system will then try to test the new algorithm, find
> that it needs to load a fallback, and then try to load that.
>
> As both algorithms share the same module alias, it can attempt
> to load the original algorithm again and block indefinitely.
>
> As algorithms requiring fallbacks are a special case, we can fix
> this by giving them a different module alias than the rest. Then
> it's just a matter of using the right aliases according to what
> algorithms we're trying to find.
>
> Signed-off-by: Herbert Xu <[email protected]>
>
> diff --git a/arch/s390/crypto/aes_s390.c b/arch/s390/crypto/aes_s390.c
> index c42cd89..6118890 100644
> --- a/arch/s390/crypto/aes_s390.c
> +++ b/arch/s390/crypto/aes_s390.c
> @@ -556,7 +556,7 @@ static void __exit aes_s390_fini(void)
> module_init(aes_s390_init);
> module_exit(aes_s390_fini);
>
> -MODULE_ALIAS("aes");
> +MODULE_ALIAS("aes-all");
>
> MODULE_DESCRIPTION("Rijndael (AES) Cipher Algorithm");
> MODULE_LICENSE("GPL");
> diff --git a/crypto/api.c b/crypto/api.c
> index efe77df..38a2bc0 100644
> --- a/crypto/api.c
> +++ b/crypto/api.c
> @@ -215,8 +215,19 @@ struct crypto_alg *crypto_larval_lookup(const char *name, u32 type, u32 mask)
> mask &= ~(CRYPTO_ALG_LARVAL | CRYPTO_ALG_DEAD);
> type &= mask;
>
> - alg = try_then_request_module(crypto_alg_lookup(name, type, mask),
> - name);
> + alg = crypto_alg_lookup(name, type, mask);
> + if (!alg) {
> + char tmp[CRYPTO_MAX_ALG_NAME];
> +
> + request_module(name);
> +
> + if (!((type ^ CRYPTO_ALG_NEED_FALLBACK) & mask) &&
> + snprintf(tmp, sizeof(tmp), "%s-all", name) < sizeof(tmp))
> + request_module(tmp);
> +
> + alg = crypto_alg_lookup(name, type, mask);
> + }
> +
> if (alg)
> return crypto_is_larval(alg) ? crypto_larval_wait(alg) : alg;
>
> diff --git a/drivers/crypto/padlock-aes.c b/drivers/crypto/padlock-aes.c
> index 856b3cc..3f0fdd1 100644
> --- a/drivers/crypto/padlock-aes.c
> +++ b/drivers/crypto/padlock-aes.c
> @@ -489,4 +489,4 @@ MODULE_DESCRIPTION("VIA PadLock AES algorithm support");
> MODULE_LICENSE("GPL");
> MODULE_AUTHOR("Michal Ludvig");
>
> -MODULE_ALIAS("aes");
> +MODULE_ALIAS("aes-all");
> diff --git a/drivers/crypto/padlock-sha.c b/drivers/crypto/padlock-sha.c
> index a7fbade..a2c8e85 100644
> --- a/drivers/crypto/padlock-sha.c
> +++ b/drivers/crypto/padlock-sha.c
> @@ -304,7 +304,7 @@ MODULE_DESCRIPTION("VIA PadLock SHA1/SHA256 algorithms support.");
> MODULE_LICENSE("GPL");
> MODULE_AUTHOR("Michal Ludvig");
>
> -MODULE_ALIAS("sha1");
> -MODULE_ALIAS("sha256");
> +MODULE_ALIAS("sha1-all");
> +MODULE_ALIAS("sha256-all");
> MODULE_ALIAS("sha1-padlock");
> MODULE_ALIAS("sha256-padlock");
>
> Thanks,


Subject: Re: hanging modprobe aes_s390

* Herbert Xu | 2009-02-26 14:06:18 [+0800]:

>As algorithms requiring fallbacks are a special case, we can fix
>this by giving them a different module alias than the rest. Then
>it's just a matter of using the right aliases according to what
>algorithms we're trying to find.

Yup. I am not sure if we have a problem with the geode here. That one
is loaded via PCI id probing. I'm going to check on that once I figured
out where the devel board is.....

Sebastian

2009-02-28 14:30:05

by Herbert Xu

[permalink] [raw]
Subject: Re: hanging modprobe aes_s390

On Sat, Feb 28, 2009 at 01:06:36PM +0100, Sebastian Andrzej Siewior wrote:
>
> Yup. I am not sure if we have a problem with the geode here. That one
> is loaded via PCI id probing. I'm going to check on that once I figured
> out where the devel board is.....

Well not everybody uses udev :)

Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt