2013-08-31 13:36:04

by Martin MOKREJŠ

[permalink] [raw]
Subject: 3.10.9: kmemleak disables all CPUs except CPU0

Hi,
never realized that my CPUs are gone if I compile into kernel kmemleak.
Is that really the aim?


CONFIG_HAVE_DEBUG_KMEMLEAK=y
CONFIG_DEBUG_KMEMLEAK=y
CONFIG_DEBUG_KMEMLEAK_EARLY_LOG_SIZE=400
# CONFIG_DEBUG_KMEMLEAK_TEST is not set
# CONFIG_DEBUG_KMEMLEAK_DEFAULT_OFF is not set



1. Why isn't there /sys/devices/system/cpu/cpu0/online file?
Does not matter if it contains 0 or 1. It just should exist.

# cat /sys/devices/system/cpu/cpu0/online
cat: /sys/devices/system/cpu/cpu0/online: No such file or directory
# cat /sys/devices/system/cpu/cpu1/online
0
# cat /sys/devices/system/cpu/cpu2/online
0
# cat /sys/devices/system/cpu/cpu3/online
0
#


2. So which CPU is actually enabled now? Looks cpuinfo really say only one core is up.

# cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 42
model name : Intel(R) Core(TM) i7-2640M CPU @ 2.80GHz
stepping : 7
microcode : 0x28
cpu MHz : 3472.000
cache size : 4096 KB
physical id : 0
siblings : 1
core id : 0
cpu cores : 1
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips : 5587.30
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

#



3. Complete dmesg is attached but here are few lines:

[ 0.000000] Linux version 3.10.9-default-pciehp (root@vostro) (gcc version 4.6.3 (Gentoo 4.6.3 p1.8, pie-0.5.2) ) #8 SMP Tue Aug 27 01:37:03 MEST 2013
[ 0.000000] Command line: root=/dev/sda5 slub_debug=AFPZ pciehp.pciehp_debug=1 pciehp_debug=1

[ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs
[ 0.000000] nr_irqs_gsi: 40

[ 0.000000] setup_percpu: NR_CPUS:4 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1
[ 0.000000] PERCPU: Embedded 26 pages/cpu @ffff88041d800000 s77760 r8192 d20544 u524288
[ 0.000000] pcpu-alloc: s77760 r8192 d20544 u524288 alloc=1*2097152
[ 0.000000] pcpu-alloc: [0] 0 1 2 3

[ 0.343650] smpboot: CPU0: Intel(R) Core(TM) i7-2640M CPU @ 2.80GHz (fam: 06, model: 2a, stepping: 07)
[ 0.343879] TSC deadline timer enabled
[ 0.343893] Performance Events: PEBS fmt1+, 16-deep LBR, SandyBridge events, Intel PMU driver.
[ 0.344158] perf_event_intel: PEBS disabled due to CPU errata, please upgrade microcode
[ 0.344272] ... version: 3
[ 0.344345] ... bit width: 48
[ 0.344418] ... generic registers: 4
[ 0.344491] ... value mask: 0000ffffffffffff
[ 0.344565] ... max period: 000000007fffffff
[ 0.344639] ... fixed-purpose events: 3
[ 0.344711] ... event mask: 000000070000000f
[ 0.344840] kmemcheck: Limiting number of CPUs to 1.
[ 0.344915] kmemcheck: Initialized
[ 0.345949] Brought up 1 CPUs
[ 0.346023] smpboot: Total of 1 processors activated (5587.30 BogoMIPS)



4. If it is really intentional that kmemleak inclusion enforces only one CPU then I would
to get a fat warning in the help text kernel config and also a runtime warning in dmesg.


5. While we are at it, could somebody ensure me that in a dual physical core CPU
with hyperthreading enabled I get in linux:

cpu0 - 1st physical core
cpu1 - 2nd physical core
cpu2 - 1st hyperthreaded core adjacent to cpu0
cpu3 - 2nd hyperthreaded core adjacent to cpu1

If that is NOT case what is then in my case
CONFIG_NR_CPUS=2
would keep enabled 1st physical core and is hyperthreaded core which is of course not what I want.

So far my .config has:
CONFIG_X86_HT=y
CONFIG_NR_CPUS=4
SCHED_SMT=N
SCHED_MC=N


Thank you,
Martin


Attachments:
dmesg-3.10.9.kmemleak.txt (64.61 kB)

2013-09-02 15:32:10

by Catalin Marinas

[permalink] [raw]
Subject: Re: 3.10.9: kmemleak disables all CPUs except CPU0

On 31 August 2013 14:35, Martin MOKREJŠ <[email protected]> wrote:
> never realized that my CPUs are gone if I compile into kernel kmemleak.
> Is that really the aim?
>
> CONFIG_HAVE_DEBUG_KMEMLEAK=y
> CONFIG_DEBUG_KMEMLEAK=y
> CONFIG_DEBUG_KMEMLEAK_EARLY_LOG_SIZE=400
> # CONFIG_DEBUG_KMEMLEAK_TEST is not set
> # CONFIG_DEBUG_KMEMLEAK_DEFAULT_OFF is not set
>
> 1. Why isn't there /sys/devices/system/cpu/cpu0/online file?
> Does not matter if it contains 0 or 1. It just should exist.

I can't really see how kmemleak would do this, maybe other config
options that get set/cleared in the process of selecting kmemleak. Can
you do a diff between your config with /sys/... entries and the one
without?

--
Catalin

2013-09-02 15:45:15

by Max Filippov

[permalink] [raw]
Subject: Re: 3.10.9: kmemleak disables all CPUs except CPU0

On Mon, Sep 2, 2013 at 7:31 PM, Catalin Marinas <[email protected]> wrote:
> On 31 August 2013 14:35, Martin MOKREJŠ <[email protected]> wrote:
>> never realized that my CPUs are gone if I compile into kernel kmemleak.
>> Is that really the aim?
>>
>> CONFIG_HAVE_DEBUG_KMEMLEAK=y
>> CONFIG_DEBUG_KMEMLEAK=y
>> CONFIG_DEBUG_KMEMLEAK_EARLY_LOG_SIZE=400
>> # CONFIG_DEBUG_KMEMLEAK_TEST is not set
>> # CONFIG_DEBUG_KMEMLEAK_DEFAULT_OFF is not set
>>
>> 1. Why isn't there /sys/devices/system/cpu/cpu0/online file?
>> Does not matter if it contains 0 or 1. It just should exist.
>
> I can't really see how kmemleak would do this, maybe other config
> options that get set/cleared in the process of selecting kmemleak. Can

Seems to be kmemcheck: from arch/x86/mm/kmemcheck/kmemcheck.c:

int __init kmemcheck_init(void)
{
#ifdef CONFIG_SMP
/*
* Limit SMP to use a single CPU. We rely on the fact that this code
* runs before SMP is set up.
*/
if (setup_max_cpus > 1) {
printk(KERN_INFO
"kmemcheck: Limiting number of CPUs to 1.\n");
setup_max_cpus = 1;
}
#endif

--
Thanks.
-- Max

2013-09-02 15:46:48

by Martin MOKREJŠ

[permalink] [raw]
Subject: Re: 3.10.9: kmemleak disables all CPUs except CPU0



Catalin Marinas wrote:
> On 31 August 2013 14:35, Martin MOKREJŠ <[email protected]> wrote:
>> never realized that my CPUs are gone if I compile into kernel kmemleak.
>> Is that really the aim?
>>
>> CONFIG_HAVE_DEBUG_KMEMLEAK=y
>> CONFIG_DEBUG_KMEMLEAK=y
>> CONFIG_DEBUG_KMEMLEAK_EARLY_LOG_SIZE=400
>> # CONFIG_DEBUG_KMEMLEAK_TEST is not set
>> # CONFIG_DEBUG_KMEMLEAK_DEFAULT_OFF is not set
>>
>> 1. Why isn't there /sys/devices/system/cpu/cpu0/online file?
>> Does not matter if it contains 0 or 1. It just should exist.
>
> I can't really see how kmemleak would do this, maybe other config
> options that get set/cleared in the process of selecting kmemleak. Can
> you do a diff between your config with /sys/... entries and the one
> without?

Hi,
I tried but did not get to report back. One one these re-enabled my CPUs back.

@@ -3177,14 +3179,7 @@
CONFIG_HAVE_ARCH_KGDB=y
# CONFIG_KGDB is not set
CONFIG_HAVE_ARCH_KMEMCHECK=y
-CONFIG_KMEMCHECK=y
-# CONFIG_KMEMCHECK_DISABLED_BY_DEFAULT is not set
-# CONFIG_KMEMCHECK_ENABLED_BY_DEFAULT is not set
-CONFIG_KMEMCHECK_ONESHOT_BY_DEFAULT=y
-CONFIG_KMEMCHECK_QUEUE_SIZE=64
-CONFIG_KMEMCHECK_SHADOW_COPY_SHIFT=5
-CONFIG_KMEMCHECK_PARTIAL_OK=y
-# CONFIG_KMEMCHECK_BITOPS_OK is not set
+# CONFIG_KMEMCHECK is not set
# CONFIG_TEST_STRING_HELPERS is not set
# CONFIG_TEST_KSTRTOX is not set
# CONFIG_STRICT_DEVMEM is not set


So, the CONFIG_KMEMCHECK disables the additional CPUs I believe. I thought it
should be traceable from the dmesg output I sent to the list. Yes, I screwed the
subject line because I did not realize a difference so far. :(

Martin

2013-09-02 15:48:53

by Catalin Marinas

[permalink] [raw]
Subject: Re: 3.10.9: kmemleak disables all CPUs except CPU0

On Mon, Sep 02, 2013 at 04:44:52PM +0100, Max Filippov wrote:
> On Mon, Sep 2, 2013 at 7:31 PM, Catalin Marinas <[email protected]> wrote:
> > On 31 August 2013 14:35, Martin MOKREJŠ <[email protected]> wrote:
> >> never realized that my CPUs are gone if I compile into kernel kmemleak.
> >> Is that really the aim?
> >>
> >> CONFIG_HAVE_DEBUG_KMEMLEAK=y
> >> CONFIG_DEBUG_KMEMLEAK=y
> >> CONFIG_DEBUG_KMEMLEAK_EARLY_LOG_SIZE=400
> >> # CONFIG_DEBUG_KMEMLEAK_TEST is not set
> >> # CONFIG_DEBUG_KMEMLEAK_DEFAULT_OFF is not set
> >>
> >> 1. Why isn't there /sys/devices/system/cpu/cpu0/online file?
> >> Does not matter if it contains 0 or 1. It just should exist.
> >
> > I can't really see how kmemleak would do this, maybe other config
> > options that get set/cleared in the process of selecting kmemleak. Can
>
> Seems to be kmemcheck: from arch/x86/mm/kmemcheck/kmemcheck.c:
>
> int __init kmemcheck_init(void)
> {
> #ifdef CONFIG_SMP
> /*
> * Limit SMP to use a single CPU. We rely on the fact that this code
> * runs before SMP is set up.
> */
> if (setup_max_cpus > 1) {
> printk(KERN_INFO
> "kmemcheck: Limiting number of CPUs to 1.\n");
> setup_max_cpus = 1;
> }
> #endif

Ah, ok, not my problem then ;)

--
Catalin

2013-09-02 15:51:56

by Martin MOKREJŠ

[permalink] [raw]
Subject: Re: 3.10.9: kmemleak disables all CPUs except CPU0



Catalin Marinas wrote:
> On Mon, Sep 02, 2013 at 04:44:52PM +0100, Max Filippov wrote:
>> On Mon, Sep 2, 2013 at 7:31 PM, Catalin Marinas <[email protected]> wrote:
>>> On 31 August 2013 14:35, Martin MOKREJŠ <[email protected]> wrote:
>>>> never realized that my CPUs are gone if I compile into kernel kmemleak.
>>>> Is that really the aim?
>>>>
>>>> CONFIG_HAVE_DEBUG_KMEMLEAK=y
>>>> CONFIG_DEBUG_KMEMLEAK=y
>>>> CONFIG_DEBUG_KMEMLEAK_EARLY_LOG_SIZE=400
>>>> # CONFIG_DEBUG_KMEMLEAK_TEST is not set
>>>> # CONFIG_DEBUG_KMEMLEAK_DEFAULT_OFF is not set
>>>>
>>>> 1. Why isn't there /sys/devices/system/cpu/cpu0/online file?
>>>> Does not matter if it contains 0 or 1. It just should exist.
>>>
>>> I can't really see how kmemleak would do this, maybe other config
>>> options that get set/cleared in the process of selecting kmemleak. Can
>>
>> Seems to be kmemcheck: from arch/x86/mm/kmemcheck/kmemcheck.c:
>>
>> int __init kmemcheck_init(void)
>> {
>> #ifdef CONFIG_SMP
>> /*
>> * Limit SMP to use a single CPU. We rely on the fact that this code
>> * runs before SMP is set up.
>> */
>> if (setup_max_cpus > 1) {
>> printk(KERN_INFO
>> "kmemcheck: Limiting number of CPUs to 1.\n");
>> setup_max_cpus = 1;
>> }
>> #endif
>
> Ah, ok, not my problem then ;)


Fine, so would somebody please update the help text accessible in "menuconfig"
for this entry? It should be clear that it has a huge performance impact if enabled.
And, by compiling in it is enabled by default.

Thank you
Martin

2013-09-02 16:09:20

by Catalin Marinas

[permalink] [raw]
Subject: Re: 3.10.9: kmemleak disables all CPUs except CPU0

On Mon, Sep 02, 2013 at 04:51:17PM +0100, Martin MOKREJŠ wrote:
> Catalin Marinas wrote:
> > On Mon, Sep 02, 2013 at 04:44:52PM +0100, Max Filippov wrote:
> >> On Mon, Sep 2, 2013 at 7:31 PM, Catalin Marinas <[email protected]> wrote:
> >>> On 31 August 2013 14:35, Martin MOKREJŠ <[email protected]> wrote:
> >>>> never realized that my CPUs are gone if I compile into kernel kmemleak.
> >>>> Is that really the aim?
> >>>>
> >>>> CONFIG_HAVE_DEBUG_KMEMLEAK=y
> >>>> CONFIG_DEBUG_KMEMLEAK=y
> >>>> CONFIG_DEBUG_KMEMLEAK_EARLY_LOG_SIZE=400
> >>>> # CONFIG_DEBUG_KMEMLEAK_TEST is not set
> >>>> # CONFIG_DEBUG_KMEMLEAK_DEFAULT_OFF is not set
> >>>>
> >>>> 1. Why isn't there /sys/devices/system/cpu/cpu0/online file?
> >>>> Does not matter if it contains 0 or 1. It just should exist.
> >>>
> >>> I can't really see how kmemleak would do this, maybe other config
> >>> options that get set/cleared in the process of selecting kmemleak. Can
> >>
> >> Seems to be kmemcheck: from arch/x86/mm/kmemcheck/kmemcheck.c:
> >>
> >> int __init kmemcheck_init(void)
> >> {
> >> #ifdef CONFIG_SMP
> >> /*
> >> * Limit SMP to use a single CPU. We rely on the fact that this code
> >> * runs before SMP is set up.
> >> */
> >> if (setup_max_cpus > 1) {
> >> printk(KERN_INFO
> >> "kmemcheck: Limiting number of CPUs to 1.\n");
> >> setup_max_cpus = 1;
> >> }
> >> #endif
> >
> > Ah, ok, not my problem then ;)
>
> Fine, so would somebody please update the help text accessible in "menuconfig"
> for this entry? It should be clear that it has a huge performance impact if enabled.
> And, by compiling in it is enabled by default.

Otherwise, of no-one volunteers, please feel free to send a patch ;)

--
Catalin

2013-09-03 13:28:36

by Martin MOKREJŠ

[permalink] [raw]
Subject: Re: 3.10.9: kmemleak disables all CPUs except CPU0



Catalin Marinas wrote:
> On Mon, Sep 02, 2013 at 04:51:17PM +0100, Martin MOKREJŠ wrote:
>> Catalin Marinas wrote:
>>> On Mon, Sep 02, 2013 at 04:44:52PM +0100, Max Filippov wrote:
>>>> On Mon, Sep 2, 2013 at 7:31 PM, Catalin Marinas <[email protected]> wrote:
>>>>> On 31 August 2013 14:35, Martin MOKREJŠ <[email protected]> wrote:
>>>>>> never realized that my CPUs are gone if I compile into kernel kmemleak.
>>>>>> Is that really the aim?
>>>>>>
>>>>>> CONFIG_HAVE_DEBUG_KMEMLEAK=y
>>>>>> CONFIG_DEBUG_KMEMLEAK=y
>>>>>> CONFIG_DEBUG_KMEMLEAK_EARLY_LOG_SIZE=400
>>>>>> # CONFIG_DEBUG_KMEMLEAK_TEST is not set
>>>>>> # CONFIG_DEBUG_KMEMLEAK_DEFAULT_OFF is not set
>>>>>>
>>>>>> 1. Why isn't there /sys/devices/system/cpu/cpu0/online file?
>>>>>> Does not matter if it contains 0 or 1. It just should exist.
>>>>>
>>>>> I can't really see how kmemleak would do this, maybe other config
>>>>> options that get set/cleared in the process of selecting kmemleak. Can
>>>>
>>>> Seems to be kmemcheck: from arch/x86/mm/kmemcheck/kmemcheck.c:
>>>>
>>>> int __init kmemcheck_init(void)
>>>> {
>>>> #ifdef CONFIG_SMP
>>>> /*
>>>> * Limit SMP to use a single CPU. We rely on the fact that this code
>>>> * runs before SMP is set up.
>>>> */
>>>> if (setup_max_cpus > 1) {
>>>> printk(KERN_INFO
>>>> "kmemcheck: Limiting number of CPUs to 1.\n");
>>>> setup_max_cpus = 1;
>>>> }
>>>> #endif
>>>
>>> Ah, ok, not my problem then ;)
>>
>> Fine, so would somebody please update the help text accessible in "menuconfig"
>> for this entry? It should be clear that it has a huge performance impact if enabled.
>> And, by compiling in it is enabled by default.
>
> Otherwise, of no-one volunteers, please feel free to send a patch ;)

No, I am not a kernel developer. Please someone else do so update the "help text".

Moreover, it does not explain why /sys/devices/system/cpu/cpu0/online file is missing
while /sys/devices/system/cpu/cpu[1-3]/online do exist at the same time.

Thank you,
Martin