2013-05-06 17:56:55

by Dave Jones

[permalink] [raw]
Subject: rcu_init allocation failure trace on bootup

[ 0.000000] RCU debugfs-based tracing is enabled.
[ 0.000000] RCU lockdep checking is enabled.
[ 0.000000] ------------[ cut here ]------------
[ 0.000000] WARNING: at mm/nobootmem.c:215 ___alloc_bootmem_nopanic+0x3a/0x7d()
[ 0.000000] Modules linked in:
[ 0.000000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.9.0+ #128
[ 0.000000] Hardware name: /D975XBX, BIOS BX97510J.86A.1209.2006.0601.1340 06/01/2006
[ 0.000000] ffffffff819f4246 ffffffff81c01e68 ffffffff816a0db2 ffffffff81c01ea8
[ 0.000000] ffffffff8104b1a0 ffffffff819e1078 0000000000000000 0000000000000008
[ 0.000000] ffffffffffffffff 0000000000000040 0000000000000000 ffffffff81c01eb8
[ 0.000000] Call Trace:
[ 0.000000] [<ffffffff816a0db2>] dump_stack+0x19/0x1b
[ 0.000000] [<ffffffff8104b1a0>] warn_slowpath_common+0x70/0xa0
[ 0.000000] [<ffffffff8104b1ea>] warn_slowpath_null+0x1a/0x20
[ 0.000000] [<ffffffff81ef8914>] ___alloc_bootmem_nopanic+0x3a/0x7d
[ 0.000000] [<ffffffff81ef8b87>] __alloc_bootmem+0x13/0x39
[ 0.000000] [<ffffffff81f090ad>] alloc_bootmem_cpumask_var+0x1a/0x21
[ 0.000000] [<ffffffff81ef1f5a>] rcu_init+0x7f/0x261
[ 0.000000] [<ffffffff81ef4acb>] ? perf_event_init+0x178/0x181
[ 0.000000] [<ffffffff81ed6c79>] start_kernel+0x232/0x3d3
[ 0.000000] [<ffffffff81ed6846>] ? repair_env_string+0x5e/0x5e
[ 0.000000] [<ffffffff81ed6577>] x86_64_start_reservations+0x2a/0x2c
[ 0.000000] [<ffffffff81ed6646>] x86_64_start_kernel+0xcd/0xd1
[ 0.000000] ---[ end trace 0acfc82c197165dd ]---
[ 0.000000] Experimental no-CBs for all CPUs
[ 0.000000] Experimental no-CBs CPUs: 0-1.
[ 0.000000] NO_HZ: Full dynticks CPUs: 1.


2013-05-06 19:05:48

by Paul E. McKenney

[permalink] [raw]
Subject: Re: rcu_init allocation failure trace on bootup

On Mon, May 06, 2013 at 01:56:47PM -0400, Dave Jones wrote:
> [ 0.000000] RCU debugfs-based tracing is enabled.
> [ 0.000000] RCU lockdep checking is enabled.
> [ 0.000000] ------------[ cut here ]------------
> [ 0.000000] WARNING: at mm/nobootmem.c:215 ___alloc_bootmem_nopanic+0x3a/0x7d()
> [ 0.000000] Modules linked in:
> [ 0.000000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.9.0+ #128
> [ 0.000000] Hardware name: /D975XBX, BIOS BX97510J.86A.1209.2006.0601.1340 06/01/2006
> [ 0.000000] ffffffff819f4246 ffffffff81c01e68 ffffffff816a0db2 ffffffff81c01ea8
> [ 0.000000] ffffffff8104b1a0 ffffffff819e1078 0000000000000000 0000000000000008
> [ 0.000000] ffffffffffffffff 0000000000000040 0000000000000000 ffffffff81c01eb8
> [ 0.000000] Call Trace:
> [ 0.000000] [<ffffffff816a0db2>] dump_stack+0x19/0x1b
> [ 0.000000] [<ffffffff8104b1a0>] warn_slowpath_common+0x70/0xa0
> [ 0.000000] [<ffffffff8104b1ea>] warn_slowpath_null+0x1a/0x20
> [ 0.000000] [<ffffffff81ef8914>] ___alloc_bootmem_nopanic+0x3a/0x7d
> [ 0.000000] [<ffffffff81ef8b87>] __alloc_bootmem+0x13/0x39
> [ 0.000000] [<ffffffff81f090ad>] alloc_bootmem_cpumask_var+0x1a/0x21
> [ 0.000000] [<ffffffff81ef1f5a>] rcu_init+0x7f/0x261
> [ 0.000000] [<ffffffff81ef4acb>] ? perf_event_init+0x178/0x181
> [ 0.000000] [<ffffffff81ed6c79>] start_kernel+0x232/0x3d3
> [ 0.000000] [<ffffffff81ed6846>] ? repair_env_string+0x5e/0x5e
> [ 0.000000] [<ffffffff81ed6577>] x86_64_start_reservations+0x2a/0x2c
> [ 0.000000] [<ffffffff81ed6646>] x86_64_start_kernel+0xcd/0xd1
> [ 0.000000] ---[ end trace 0acfc82c197165dd ]---
> [ 0.000000] Experimental no-CBs for all CPUs
> [ 0.000000] Experimental no-CBs CPUs: 0-1.
> [ 0.000000] NO_HZ: Full dynticks CPUs: 1.

Do you have commit fa0f93cd6a (rcu: Don't allocate bootmem from rcu_init())?
If not, could you please try applying it?

Thanx, Paul

2013-05-06 19:27:01

by Dave Jones

[permalink] [raw]
Subject: Re: rcu_init allocation failure trace on bootup

On Mon, May 06, 2013 at 12:05:22PM -0700, Paul E. McKenney wrote:
> On Mon, May 06, 2013 at 01:56:47PM -0400, Dave Jones wrote:
> > [ 0.000000] RCU debugfs-based tracing is enabled.
> > [ 0.000000] RCU lockdep checking is enabled.
> > [ 0.000000] ------------[ cut here ]------------
> > [ 0.000000] WARNING: at mm/nobootmem.c:215 ___alloc_bootmem_nopanic+0x3a/0x7d()
> > [ 0.000000] Modules linked in:
> > [ 0.000000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.9.0+ #128
> > [ 0.000000] Hardware name: /D975XBX, BIOS BX97510J.86A.1209.2006.0601.1340 06/01/2006
> > [ 0.000000] ffffffff819f4246 ffffffff81c01e68 ffffffff816a0db2 ffffffff81c01ea8
> > [ 0.000000] ffffffff8104b1a0 ffffffff819e1078 0000000000000000 0000000000000008
> > [ 0.000000] ffffffffffffffff 0000000000000040 0000000000000000 ffffffff81c01eb8
> > [ 0.000000] Call Trace:
> > [ 0.000000] [<ffffffff816a0db2>] dump_stack+0x19/0x1b
> > [ 0.000000] [<ffffffff8104b1a0>] warn_slowpath_common+0x70/0xa0
> > [ 0.000000] [<ffffffff8104b1ea>] warn_slowpath_null+0x1a/0x20
> > [ 0.000000] [<ffffffff81ef8914>] ___alloc_bootmem_nopanic+0x3a/0x7d
> > [ 0.000000] [<ffffffff81ef8b87>] __alloc_bootmem+0x13/0x39
> > [ 0.000000] [<ffffffff81f090ad>] alloc_bootmem_cpumask_var+0x1a/0x21
> > [ 0.000000] [<ffffffff81ef1f5a>] rcu_init+0x7f/0x261
> > [ 0.000000] [<ffffffff81ef4acb>] ? perf_event_init+0x178/0x181
> > [ 0.000000] [<ffffffff81ed6c79>] start_kernel+0x232/0x3d3
> > [ 0.000000] [<ffffffff81ed6846>] ? repair_env_string+0x5e/0x5e
> > [ 0.000000] [<ffffffff81ed6577>] x86_64_start_reservations+0x2a/0x2c
> > [ 0.000000] [<ffffffff81ed6646>] x86_64_start_kernel+0xcd/0xd1
> > [ 0.000000] ---[ end trace 0acfc82c197165dd ]---
> > [ 0.000000] Experimental no-CBs for all CPUs
> > [ 0.000000] Experimental no-CBs CPUs: 0-1.
> > [ 0.000000] NO_HZ: Full dynticks CPUs: 1.
>
> Do you have commit fa0f93cd6a (rcu: Don't allocate bootmem from rcu_init())?
> If not, could you please try applying it?

That commit isn't in Linus tree.

Dave

2013-05-06 19:44:12

by Paul E. McKenney

[permalink] [raw]
Subject: Re: rcu_init allocation failure trace on bootup

On Mon, May 06, 2013 at 03:26:53PM -0400, Dave Jones wrote:
> On Mon, May 06, 2013 at 12:05:22PM -0700, Paul E. McKenney wrote:
> > On Mon, May 06, 2013 at 01:56:47PM -0400, Dave Jones wrote:
> > > [ 0.000000] RCU debugfs-based tracing is enabled.
> > > [ 0.000000] RCU lockdep checking is enabled.
> > > [ 0.000000] ------------[ cut here ]------------
> > > [ 0.000000] WARNING: at mm/nobootmem.c:215 ___alloc_bootmem_nopanic+0x3a/0x7d()
> > > [ 0.000000] Modules linked in:
> > > [ 0.000000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.9.0+ #128
> > > [ 0.000000] Hardware name: /D975XBX, BIOS BX97510J.86A.1209.2006.0601.1340 06/01/2006
> > > [ 0.000000] ffffffff819f4246 ffffffff81c01e68 ffffffff816a0db2 ffffffff81c01ea8
> > > [ 0.000000] ffffffff8104b1a0 ffffffff819e1078 0000000000000000 0000000000000008
> > > [ 0.000000] ffffffffffffffff 0000000000000040 0000000000000000 ffffffff81c01eb8
> > > [ 0.000000] Call Trace:
> > > [ 0.000000] [<ffffffff816a0db2>] dump_stack+0x19/0x1b
> > > [ 0.000000] [<ffffffff8104b1a0>] warn_slowpath_common+0x70/0xa0
> > > [ 0.000000] [<ffffffff8104b1ea>] warn_slowpath_null+0x1a/0x20
> > > [ 0.000000] [<ffffffff81ef8914>] ___alloc_bootmem_nopanic+0x3a/0x7d
> > > [ 0.000000] [<ffffffff81ef8b87>] __alloc_bootmem+0x13/0x39
> > > [ 0.000000] [<ffffffff81f090ad>] alloc_bootmem_cpumask_var+0x1a/0x21
> > > [ 0.000000] [<ffffffff81ef1f5a>] rcu_init+0x7f/0x261
> > > [ 0.000000] [<ffffffff81ef4acb>] ? perf_event_init+0x178/0x181
> > > [ 0.000000] [<ffffffff81ed6c79>] start_kernel+0x232/0x3d3
> > > [ 0.000000] [<ffffffff81ed6846>] ? repair_env_string+0x5e/0x5e
> > > [ 0.000000] [<ffffffff81ed6577>] x86_64_start_reservations+0x2a/0x2c
> > > [ 0.000000] [<ffffffff81ed6646>] x86_64_start_kernel+0xcd/0xd1
> > > [ 0.000000] ---[ end trace 0acfc82c197165dd ]---
> > > [ 0.000000] Experimental no-CBs for all CPUs
> > > [ 0.000000] Experimental no-CBs CPUs: 0-1.
> > > [ 0.000000] NO_HZ: Full dynticks CPUs: 1.
> >
> > Do you have commit fa0f93cd6a (rcu: Don't allocate bootmem from rcu_init())?
> > If not, could you please try applying it?
>
> That commit isn't in Linus tree.

You are quite right, I am getting ahead of myself. Please see below
for the patch.

Thanx, Paul

------------------------------------------------------------------------

rcu: Don't allocate bootmem from rcu_init()

When rcu_init() is called we already have slab working, allocating
bootmem at that point results in warnings and an allocation from
slab. This commit therefore changes alloc_bootmem_cpumask_var() to
alloc_cpumask_var() in rcu_bootup_announce_oddness(), which is called
from rcu_init().

Signed-off-by: Sasha Levin <[email protected]>
Signed-off-by: Paul E. McKenney <[email protected]>
Reviewed-by: Josh Triplett <[email protected]>

diff --git a/kernel/rcutree_plugin.h b/kernel/rcutree_plugin.h
index ca6e39c..44b0998 100644
--- a/kernel/rcutree_plugin.h
+++ b/kernel/rcutree_plugin.h
@@ -86,7 +86,7 @@ static void __init rcu_bootup_announce_oddness(void)
#ifdef CONFIG_RCU_NOCB_CPU
#ifndef CONFIG_RCU_NOCB_CPU_NONE
if (!have_rcu_nocb_mask) {
- alloc_bootmem_cpumask_var(&rcu_nocb_mask);
+ alloc_cpumask_var(&rcu_nocb_mask, GFP_KERNEL);
have_rcu_nocb_mask = true;
}
#ifdef CONFIG_RCU_NOCB_CPU_ZERO

2013-05-06 20:09:23

by Yinghai Lu

[permalink] [raw]
Subject: Re: rcu_init allocation failure trace on bootup

On Mon, May 6, 2013 at 12:43 PM, Paul E. McKenney
<[email protected]> wrote:
> On Mon, May 06, 2013 at 03:26:53PM -0400, Dave Jones wrote:
>> On Mon, May 06, 2013 at 12:05:22PM -0700, Paul E. McKenney wrote:
>> > On Mon, May 06, 2013 at 01:56:47PM -0400, Dave Jones wrote:
>> > > [ 0.000000] RCU debugfs-based tracing is enabled.
>> > > [ 0.000000] RCU lockdep checking is enabled.
>> > > [ 0.000000] ------------[ cut here ]------------
>> > > [ 0.000000] WARNING: at mm/nobootmem.c:215 ___alloc_bootmem_nopanic+0x3a/0x7d()
>> > > [ 0.000000] Modules linked in:
>> > > [ 0.000000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.9.0+ #128
>> > > [ 0.000000] Hardware name: /D975XBX, BIOS BX97510J.86A.1209.2006.0601.1340 06/01/2006
>> > > [ 0.000000] ffffffff819f4246 ffffffff81c01e68 ffffffff816a0db2 ffffffff81c01ea8
>> > > [ 0.000000] ffffffff8104b1a0 ffffffff819e1078 0000000000000000 0000000000000008
>> > > [ 0.000000] ffffffffffffffff 0000000000000040 0000000000000000 ffffffff81c01eb8
>> > > [ 0.000000] Call Trace:
>> > > [ 0.000000] [<ffffffff816a0db2>] dump_stack+0x19/0x1b
>> > > [ 0.000000] [<ffffffff8104b1a0>] warn_slowpath_common+0x70/0xa0
>> > > [ 0.000000] [<ffffffff8104b1ea>] warn_slowpath_null+0x1a/0x20
>> > > [ 0.000000] [<ffffffff81ef8914>] ___alloc_bootmem_nopanic+0x3a/0x7d
>> > > [ 0.000000] [<ffffffff81ef8b87>] __alloc_bootmem+0x13/0x39
>> > > [ 0.000000] [<ffffffff81f090ad>] alloc_bootmem_cpumask_var+0x1a/0x21
>> > > [ 0.000000] [<ffffffff81ef1f5a>] rcu_init+0x7f/0x261
>> > > [ 0.000000] [<ffffffff81ef4acb>] ? perf_event_init+0x178/0x181
>> > > [ 0.000000] [<ffffffff81ed6c79>] start_kernel+0x232/0x3d3
>> > > [ 0.000000] [<ffffffff81ed6846>] ? repair_env_string+0x5e/0x5e
>> > > [ 0.000000] [<ffffffff81ed6577>] x86_64_start_reservations+0x2a/0x2c
>> > > [ 0.000000] [<ffffffff81ed6646>] x86_64_start_kernel+0xcd/0xd1
>> > > [ 0.000000] ---[ end trace 0acfc82c197165dd ]---
>> > > [ 0.000000] Experimental no-CBs for all CPUs
>> > > [ 0.000000] Experimental no-CBs CPUs: 0-1.
>> > > [ 0.000000] NO_HZ: Full dynticks CPUs: 1.
>> >
>> > Do you have commit fa0f93cd6a (rcu: Don't allocate bootmem from rcu_init())?
>> > If not, could you please try applying it?
>>
>> That commit isn't in Linus tree.
>
> You are quite right, I am getting ahead of myself. Please see below
> for the patch.
>
> Thanx, Paul
>
> ------------------------------------------------------------------------
>
> rcu: Don't allocate bootmem from rcu_init()
>
> When rcu_init() is called we already have slab working, allocating
> bootmem at that point results in warnings and an allocation from
> slab. This commit therefore changes alloc_bootmem_cpumask_var() to
> alloc_cpumask_var() in rcu_bootup_announce_oddness(), which is called
> from rcu_init().
>
> Signed-off-by: Sasha Levin <[email protected]>
> Signed-off-by: Paul E. McKenney <[email protected]>
> Reviewed-by: Josh Triplett <[email protected]>
>
> diff --git a/kernel/rcutree_plugin.h b/kernel/rcutree_plugin.h
> index ca6e39c..44b0998 100644
> --- a/kernel/rcutree_plugin.h
> +++ b/kernel/rcutree_plugin.h
> @@ -86,7 +86,7 @@ static void __init rcu_bootup_announce_oddness(void)
> #ifdef CONFIG_RCU_NOCB_CPU
> #ifndef CONFIG_RCU_NOCB_CPU_NONE
> if (!have_rcu_nocb_mask) {
> - alloc_bootmem_cpumask_var(&rcu_nocb_mask);
> + alloc_cpumask_var(&rcu_nocb_mask, GFP_KERNEL);

Do you need to use zalloc_cpumask_var instead?


> have_rcu_nocb_mask = true;
> }
> #ifdef CONFIG_RCU_NOCB_CPU_ZERO
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/

2013-05-06 20:24:21

by Dave Jones

[permalink] [raw]
Subject: Re: rcu_init allocation failure trace on bootup

On Mon, May 06, 2013 at 12:43:51PM -0700, Paul E. McKenney wrote:
> On Mon, May 06, 2013 at 03:26:53PM -0400, Dave Jones wrote:
> > On Mon, May 06, 2013 at 12:05:22PM -0700, Paul E. McKenney wrote:
> > > On Mon, May 06, 2013 at 01:56:47PM -0400, Dave Jones wrote:
> > > > [ 0.000000] RCU debugfs-based tracing is enabled.
> > > > [ 0.000000] RCU lockdep checking is enabled.
> > > > [ 0.000000] ------------[ cut here ]------------
> > > > [ 0.000000] WARNING: at mm/nobootmem.c:215 ___alloc_bootmem_nopanic+0x3a/0x7d()
> > > > [ 0.000000] Modules linked in:
> > > > [ 0.000000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.9.0+ #128
> > > > [ 0.000000] Hardware name: /D975XBX, BIOS BX97510J.86A.1209.2006.0601.1340 06/01/2006
> > > > [ 0.000000] ffffffff819f4246 ffffffff81c01e68 ffffffff816a0db2 ffffffff81c01ea8
> > > > [ 0.000000] ffffffff8104b1a0 ffffffff819e1078 0000000000000000 0000000000000008
> > > > [ 0.000000] ffffffffffffffff 0000000000000040 0000000000000000 ffffffff81c01eb8
> > > > [ 0.000000] Call Trace:
> > > > [ 0.000000] [<ffffffff816a0db2>] dump_stack+0x19/0x1b
> > > > [ 0.000000] [<ffffffff8104b1a0>] warn_slowpath_common+0x70/0xa0
> > > > [ 0.000000] [<ffffffff8104b1ea>] warn_slowpath_null+0x1a/0x20
> > > > [ 0.000000] [<ffffffff81ef8914>] ___alloc_bootmem_nopanic+0x3a/0x7d
> > > > [ 0.000000] [<ffffffff81ef8b87>] __alloc_bootmem+0x13/0x39
> > > > [ 0.000000] [<ffffffff81f090ad>] alloc_bootmem_cpumask_var+0x1a/0x21
> > > > [ 0.000000] [<ffffffff81ef1f5a>] rcu_init+0x7f/0x261
> > > > [ 0.000000] [<ffffffff81ef4acb>] ? perf_event_init+0x178/0x181
> > > > [ 0.000000] [<ffffffff81ed6c79>] start_kernel+0x232/0x3d3
> > > > [ 0.000000] [<ffffffff81ed6846>] ? repair_env_string+0x5e/0x5e
> > > > [ 0.000000] [<ffffffff81ed6577>] x86_64_start_reservations+0x2a/0x2c
> > > > [ 0.000000] [<ffffffff81ed6646>] x86_64_start_kernel+0xcd/0xd1
> > > > [ 0.000000] ---[ end trace 0acfc82c197165dd ]---
> > > > [ 0.000000] Experimental no-CBs for all CPUs
> > > > [ 0.000000] Experimental no-CBs CPUs: 0-1.
> > > > [ 0.000000] NO_HZ: Full dynticks CPUs: 1.
> > >
> > > Do you have commit fa0f93cd6a (rcu: Don't allocate bootmem from rcu_init())?
> > > If not, could you please try applying it?
> >
> > That commit isn't in Linus tree.
>
> You are quite right, I am getting ahead of myself. Please see below
> for the patch.

Doesn't fix it for me.

Is it this one instead perhaps ?

89 #ifndef CONFIG_RCU_NOCB_CPU_NONE
90 if (!have_rcu_nocb_mask) {
91 alloc_bootmem_cpumask_var(&rcu_nocb_mask);
92 have_rcu_nocb_mask = true;
93 }

2013-05-06 21:03:32

by Paul E. McKenney

[permalink] [raw]
Subject: Re: rcu_init allocation failure trace on bootup

On Mon, May 06, 2013 at 04:24:13PM -0400, Dave Jones wrote:
> On Mon, May 06, 2013 at 12:43:51PM -0700, Paul E. McKenney wrote:
> > On Mon, May 06, 2013 at 03:26:53PM -0400, Dave Jones wrote:
> > > On Mon, May 06, 2013 at 12:05:22PM -0700, Paul E. McKenney wrote:
> > > > On Mon, May 06, 2013 at 01:56:47PM -0400, Dave Jones wrote:
> > > > > [ 0.000000] RCU debugfs-based tracing is enabled.
> > > > > [ 0.000000] RCU lockdep checking is enabled.
> > > > > [ 0.000000] ------------[ cut here ]------------
> > > > > [ 0.000000] WARNING: at mm/nobootmem.c:215 ___alloc_bootmem_nopanic+0x3a/0x7d()
> > > > > [ 0.000000] Modules linked in:
> > > > > [ 0.000000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.9.0+ #128
> > > > > [ 0.000000] Hardware name: /D975XBX, BIOS BX97510J.86A.1209.2006.0601.1340 06/01/2006
> > > > > [ 0.000000] ffffffff819f4246 ffffffff81c01e68 ffffffff816a0db2 ffffffff81c01ea8
> > > > > [ 0.000000] ffffffff8104b1a0 ffffffff819e1078 0000000000000000 0000000000000008
> > > > > [ 0.000000] ffffffffffffffff 0000000000000040 0000000000000000 ffffffff81c01eb8
> > > > > [ 0.000000] Call Trace:
> > > > > [ 0.000000] [<ffffffff816a0db2>] dump_stack+0x19/0x1b
> > > > > [ 0.000000] [<ffffffff8104b1a0>] warn_slowpath_common+0x70/0xa0
> > > > > [ 0.000000] [<ffffffff8104b1ea>] warn_slowpath_null+0x1a/0x20
> > > > > [ 0.000000] [<ffffffff81ef8914>] ___alloc_bootmem_nopanic+0x3a/0x7d
> > > > > [ 0.000000] [<ffffffff81ef8b87>] __alloc_bootmem+0x13/0x39
> > > > > [ 0.000000] [<ffffffff81f090ad>] alloc_bootmem_cpumask_var+0x1a/0x21
> > > > > [ 0.000000] [<ffffffff81ef1f5a>] rcu_init+0x7f/0x261
> > > > > [ 0.000000] [<ffffffff81ef4acb>] ? perf_event_init+0x178/0x181
> > > > > [ 0.000000] [<ffffffff81ed6c79>] start_kernel+0x232/0x3d3
> > > > > [ 0.000000] [<ffffffff81ed6846>] ? repair_env_string+0x5e/0x5e
> > > > > [ 0.000000] [<ffffffff81ed6577>] x86_64_start_reservations+0x2a/0x2c
> > > > > [ 0.000000] [<ffffffff81ed6646>] x86_64_start_kernel+0xcd/0xd1
> > > > > [ 0.000000] ---[ end trace 0acfc82c197165dd ]---
> > > > > [ 0.000000] Experimental no-CBs for all CPUs
> > > > > [ 0.000000] Experimental no-CBs CPUs: 0-1.
> > > > > [ 0.000000] NO_HZ: Full dynticks CPUs: 1.
> > > >
> > > > Do you have commit fa0f93cd6a (rcu: Don't allocate bootmem from rcu_init())?
> > > > If not, could you please try applying it?
> > >
> > > That commit isn't in Linus tree.
> >
> > You are quite right, I am getting ahead of myself. Please see below
> > for the patch.
>
> Doesn't fix it for me.
>
> Is it this one instead perhaps ?
>
> 89 #ifndef CONFIG_RCU_NOCB_CPU_NONE
> 90 if (!have_rcu_nocb_mask) {
> 91 alloc_bootmem_cpumask_var(&rcu_nocb_mask);
> 92 have_rcu_nocb_mask = true;
> 93 }

Hmmm... That was in fact the one that the patch was intended to change.
Just out of curiosity, where did the patch apply instead?

Thanx, Paul

2013-05-06 21:09:45

by Paul E. McKenney

[permalink] [raw]
Subject: Re: rcu_init allocation failure trace on bootup

On Mon, May 06, 2013 at 01:09:22PM -0700, Yinghai Lu wrote:
> On Mon, May 6, 2013 at 12:43 PM, Paul E. McKenney
> <[email protected]> wrote:
> > On Mon, May 06, 2013 at 03:26:53PM -0400, Dave Jones wrote:
> >> On Mon, May 06, 2013 at 12:05:22PM -0700, Paul E. McKenney wrote:
> >> > On Mon, May 06, 2013 at 01:56:47PM -0400, Dave Jones wrote:
> >> > > [ 0.000000] RCU debugfs-based tracing is enabled.
> >> > > [ 0.000000] RCU lockdep checking is enabled.
> >> > > [ 0.000000] ------------[ cut here ]------------
> >> > > [ 0.000000] WARNING: at mm/nobootmem.c:215 ___alloc_bootmem_nopanic+0x3a/0x7d()
> >> > > [ 0.000000] Modules linked in:
> >> > > [ 0.000000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.9.0+ #128
> >> > > [ 0.000000] Hardware name: /D975XBX, BIOS BX97510J.86A.1209.2006.0601.1340 06/01/2006
> >> > > [ 0.000000] ffffffff819f4246 ffffffff81c01e68 ffffffff816a0db2 ffffffff81c01ea8
> >> > > [ 0.000000] ffffffff8104b1a0 ffffffff819e1078 0000000000000000 0000000000000008
> >> > > [ 0.000000] ffffffffffffffff 0000000000000040 0000000000000000 ffffffff81c01eb8
> >> > > [ 0.000000] Call Trace:
> >> > > [ 0.000000] [<ffffffff816a0db2>] dump_stack+0x19/0x1b
> >> > > [ 0.000000] [<ffffffff8104b1a0>] warn_slowpath_common+0x70/0xa0
> >> > > [ 0.000000] [<ffffffff8104b1ea>] warn_slowpath_null+0x1a/0x20
> >> > > [ 0.000000] [<ffffffff81ef8914>] ___alloc_bootmem_nopanic+0x3a/0x7d
> >> > > [ 0.000000] [<ffffffff81ef8b87>] __alloc_bootmem+0x13/0x39
> >> > > [ 0.000000] [<ffffffff81f090ad>] alloc_bootmem_cpumask_var+0x1a/0x21
> >> > > [ 0.000000] [<ffffffff81ef1f5a>] rcu_init+0x7f/0x261
> >> > > [ 0.000000] [<ffffffff81ef4acb>] ? perf_event_init+0x178/0x181
> >> > > [ 0.000000] [<ffffffff81ed6c79>] start_kernel+0x232/0x3d3
> >> > > [ 0.000000] [<ffffffff81ed6846>] ? repair_env_string+0x5e/0x5e
> >> > > [ 0.000000] [<ffffffff81ed6577>] x86_64_start_reservations+0x2a/0x2c
> >> > > [ 0.000000] [<ffffffff81ed6646>] x86_64_start_kernel+0xcd/0xd1
> >> > > [ 0.000000] ---[ end trace 0acfc82c197165dd ]---
> >> > > [ 0.000000] Experimental no-CBs for all CPUs
> >> > > [ 0.000000] Experimental no-CBs CPUs: 0-1.
> >> > > [ 0.000000] NO_HZ: Full dynticks CPUs: 1.
> >> >
> >> > Do you have commit fa0f93cd6a (rcu: Don't allocate bootmem from rcu_init())?
> >> > If not, could you please try applying it?
> >>
> >> That commit isn't in Linus tree.
> >
> > You are quite right, I am getting ahead of myself. Please see below
> > for the patch.
> >
> > Thanx, Paul
> >
> > ------------------------------------------------------------------------
> >
> > rcu: Don't allocate bootmem from rcu_init()
> >
> > When rcu_init() is called we already have slab working, allocating
> > bootmem at that point results in warnings and an allocation from
> > slab. This commit therefore changes alloc_bootmem_cpumask_var() to
> > alloc_cpumask_var() in rcu_bootup_announce_oddness(), which is called
> > from rcu_init().
> >
> > Signed-off-by: Sasha Levin <[email protected]>
> > Signed-off-by: Paul E. McKenney <[email protected]>
> > Reviewed-by: Josh Triplett <[email protected]>
> >
> > diff --git a/kernel/rcutree_plugin.h b/kernel/rcutree_plugin.h
> > index ca6e39c..44b0998 100644
> > --- a/kernel/rcutree_plugin.h
> > +++ b/kernel/rcutree_plugin.h
> > @@ -86,7 +86,7 @@ static void __init rcu_bootup_announce_oddness(void)
> > #ifdef CONFIG_RCU_NOCB_CPU
> > #ifndef CONFIG_RCU_NOCB_CPU_NONE
> > if (!have_rcu_nocb_mask) {
> > - alloc_bootmem_cpumask_var(&rcu_nocb_mask);
> > + alloc_cpumask_var(&rcu_nocb_mask, GFP_KERNEL);
>
> Do you need to use zalloc_cpumask_var instead?

Good point! I have queued up a fix for this.

Thanx, Paul

> > have_rcu_nocb_mask = true;
> > }
> > #ifdef CONFIG_RCU_NOCB_CPU_ZERO
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> > the body of a message to [email protected]
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
> > Please read the FAQ at http://www.tux.org/lkml/
>

2013-05-06 21:56:35

by Dave Jones

[permalink] [raw]
Subject: Re: rcu_init allocation failure trace on bootup

On Mon, May 06, 2013 at 02:02:49PM -0700, Paul E. McKenney wrote:

> > > You are quite right, I am getting ahead of myself. Please see below
> > > for the patch.
> >
> > Doesn't fix it for me.
> >
> > Is it this one instead perhaps ?
> >
> > 89 #ifndef CONFIG_RCU_NOCB_CPU_NONE
> > 90 if (!have_rcu_nocb_mask) {
> > 91 alloc_bootmem_cpumask_var(&rcu_nocb_mask);
> > 92 have_rcu_nocb_mask = true;
> > 93 }
>
> Hmmm... That was in fact the one that the patch was intended to change.
> Just out of curiosity, where did the patch apply instead?

I think this might be my screwup. I did it by hand, and misapplied it
to the top of rcu_nocb_setup() instead.

Dave