Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Content-Type: text/plain;
        charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 12.1 \(3445.101.1\))
Subject: Re: [PATCH] debugobjects: call debug_objects_mem_init eariler
From:   Qian Cai <cai@gmx.us>
In-Reply-To: <alpine.DEB.2.21.1811232244140.1684@nanos.tec.linutronix.de>
Date:   Fri, 23 Nov 2018 21:54:19 -0500
Cc:     Waiman Long <longman@redhat.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        Yang Shi <yang.shi@linux.alibaba.com>, arnd@arndb.de,
        linux kernel <linux-kernel@vger.kernel.org>
Content-Transfer-Encoding: 8BIT
Message-Id: <EDC1847C-1525-445C-AAFE-F07049C66565@gmx.us>
References: <alpine.DEB.2.21.1811222238270.1665@nanos.tec.linutronix.de>
 <20181123043117.992-1-cai@gmx.us>
 <dfa544fc-18d8-f1bf-fadf-46fdbda8d624@redhat.com>
 <alpine.DEB.2.21.1811232244140.1684@nanos.tec.linutronix.de>
To:     Thomas Gleixner <tglx@linutronix.de>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk


> On Nov 23, 2018, at 4:46 PM, Thomas Gleixner <tglx@linutronix.de> wrote:
> 
> On Thu, 22 Nov 2018, Waiman Long wrote:
>> On 11/22/2018 11:31 PM, Qian Cai wrote:
>>> The current value of the early boot static pool size, 1024 is not big
>>> enough for systems with large number of CPUs with timer or/and workqueue
>>> objects selected. As the results, systems have 60+ CPUs with both timer
>>> and workqueue objects enabled could trigger "ODEBUG: Out of memory.
>>> ODEBUG disabled".
>>> 
>>> However, none of the things are actually used or required beofre
> 
> before
> 
>>> debug_objects_mem_init() is invoked.
>>> 
>>> According to tglx,
>>> "the reason why the call is at this place in start_kernel() is
>>> historical. It's because back in the days when debugobjects were added
>>> the memory allocator was enabled way later than today. So we can just
>>> move the debug_objects_mem_init() call right before sched_init()."
>>> 
>>> Afterwards, when calling debug_objects_mem_init(), interrupts have
>>> already been disabled and lockdep_init() will only be called later, so
>>> no need to worry about interrupts in
>>> debug_objects_replace_static_objects().
> 
> Just out of curiosity. How many objects are allocated between early and mem
> init?

64-CPU:   68
160-CPU: 164
256-CPU: 260

INIT_WORK(&p->wq, free_work) is called per CPU:

start_kernel
   vmalloc_init
      __init_work
        __debug_object_init

Once debug_objects_mem_init() is moved just before vmalloc_init(), there
is only 1 object.

ODEBUG: 1 of 1 active objects replace

> 
>>> diff --git a/lib/debugobjects.c b/lib/debugobjects.c
>>> index 70935ed91125..cc5818ced652 100644
>>> --- a/lib/debugobjects.c
>>> +++ b/lib/debugobjects.c
>>> @@ -1132,13 +1132,6 @@ static int __init debug_objects_replace_static_objects(void)
>>> 		hlist_add_head(&obj->node, &objects);
>>> 	}
>>> 
>>> -	/*
>>> -	 * When debug_objects_mem_init() is called we know that only
>>> -	 * one CPU is up, so disabling interrupts is enough
>>> -	 * protection. This avoids the lockdep hell of lock ordering.
>>> -	 */
>>> -	local_irq_disable();
>> 
>> I think you should have a comment saying that debug_objects_mm_init() is
>> called early with only one CPU up and interrupt disabled. So it is safe
>> to replace static objects without any protection.
> 
> Yes please.
> 
> Thanks,
> 
> 	tglx