2002-06-11 18:13:59

by Randy.Dunlap

[permalink] [raw]
Subject: Re: [PATCH] CONFIG_NR_CPUS, redux

On 11 Jun 2002, Robert Love wrote:

| Here are the defaults I picked:
|
| CONFIG_NR_CPUS=32: i386, mips, parisc, ppc, sparc

I don't know what is "typical" for non-x86, but for x86, why not
use something more like a 'typical' NR_CPUS for SMP, like 8 (?)...
why still waste all of that memory?

| CONFIG_NR_CPUS=64: alpha, ia64, mips64, ppc64, s390, s390x, sparc64, x86-64
|
| No CONFIG_NR_CPUS: arm, cris, sh
|
| Andrew has pointed out some architectures may need minor tweaks to work
| with NR_CPUS < 32. He discovered and fixed a minor issue on i386...

What was this problem? I missed it but would like to see it.
(or do you know what the Subject: was?)

One spello (typo) below.

| diff -urN linux-2.5.21/arch/i386/Config.help linux/arch/i386/Config.help
| --- linux-2.5.21/arch/i386/Config.help Sat Jun 8 22:27:21 2002
| +++ linux/arch/i386/Config.help Sun Jun 9 13:13:02 2002
| @@ -25,6 +25,14 @@
|
| If you don't know what to do here, say N.
|
| +CONFIG_NR_CPUS
| + This allows you to specify the maximum number of CPUs which this
| + kernel will support. The maximum supported value is 32 and the
| + mimimum value which makes sense is 2.
--- minimum
| +
| + This is purely to save memory - each supported CPU adds
| + approximately eight kilobytes to the kernel image.

Thanks,
--
~Randy


2002-06-19 11:58:42

by Bill Davidsen

[permalink] [raw]
Subject: Re: [PATCH] CONFIG_NR_CPUS, redux

On Thu, 13 Jun 2002, Helge Hafting wrote:

> Why not let the boot process select the highest of two numbers,
> the (default-low) NR_CPUS and the number of CPU's detected?

That seems to address the issue. It allows use of all CPUs in the most
common case that they all are in and working at boot.

> Boot with "too many" cpu's and you still get to use them - you
> merely can't hotplug even more.

My impression is that a fair number of users don't add CPUs anyway, they
swap problem parts if runtime diagnostics indicate a failing
{fan,VRM,other}.

> Configuring a high NR_CPUS becomes something only hot-pluggers
> need to do, or those whose architecture doesn't support
> cpu detection in the early boot process. Those with a fixed
> number of detectable CPUs can simply go with a default of
> NR_CPUS=2 no matter what they actually have.

Since this would be init code the size is not an issue. I guess a
boot-time option would be desirable in case a site wants to boot with
fewer processors than are physically present (like mem=) for some reason
like benchmarking.

--
bill davidsen <[email protected]>
CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

2002-06-11 18:19:35

by Robert Love

[permalink] [raw]
Subject: Re: [PATCH] CONFIG_NR_CPUS, redux

On Tue, 2002-06-11 at 11:09, Randy.Dunlap wrote:

> I don't know what is "typical" for non-x86, but for x86, why not
> use something more like a 'typical' NR_CPUS for SMP, like 8 (?)...
> why still waste all of that memory?

Well right now we set it to 32...

I think out-of-the-box Linux, with SMP set, should support the maximum
number of CPUs. While we do save some memory I do not think it is going
to be huge with 8 vs 32.

But whatever you, Linus, and the arch maintainers say... all my boxen
are 2-way so I don't care ;)

> What was this problem? I missed it but would like to see it.
> (or do you know what the Subject: was?)

Not sure to be honest. The subject was "[patch] CONFIG_NR_CPUS"

Or ask Andrew...

> One spello (typo) below.
>
> | + mimimum value which makes sense is 2.

(cough) someone else wrote that ;)

Fixed, thanks.

Robert Love


2002-06-11 18:21:35

by Ruth Ivimey-Cook

[permalink] [raw]
Subject: Re: [PATCH] CONFIG_NR_CPUS, redux

On Tue, 11 Jun 2002, Randy.Dunlap wrote:

>On 11 Jun 2002, Robert Love wrote:
>
>| Here are the defaults I picked:
>|
>| CONFIG_NR_CPUS=32: i386, mips, parisc, ppc, sparc
>
>I don't know what is "typical" for non-x86, but for x86, why not
>use something more like a 'typical' NR_CPUS for SMP, like 8 (?)...
>why still waste all of that memory?

Perhaps it's just because I'm coming in late, but I cannot understand why
NR_CPUS cannot be as low as 4 by default, for all archs, and then in the
kernel boot messages, should more be found than is configured for a message is
emitted to say "reconfigure your kernel", and continue with the number it was
configured for. I personally only rarely see 2-way boxes, 4-way is pretty
rare, and anything more must surely count as very specialized.

Let the defaults be reasonable for 99% of users (IMO 99.9%), and let the rest
have to think about config options...

Ruth

--
Ruth Ivimey-Cook
Software engineer and technical writer.

2002-06-11 18:28:51

by Robert Love

[permalink] [raw]
Subject: Re: [PATCH] CONFIG_NR_CPUS, redux

On Tue, 2002-06-11 at 11:21, Ruth Ivimey-Cook wrote:

> Perhaps it's just because I'm coming in late, but I cannot understand why
> NR_CPUS cannot be as low as 4 by default, for all archs, and then in the
> kernel boot messages, should more be found than is configured for a message is
> emitted to say "reconfigure your kernel", and continue with the number it was
> configured for. I personally only rarely see 2-way boxes, 4-way is pretty
> rare, and anything more must surely count as very specialized.

Ugh let's stop this thread now. Two points:

(a) imo, the kernel should support out-of-the-box the maximum
number of CPUs it can handle. Be lucky we now have a
configure option to change that. But that does not matter..

(b) Right now it is 32. Now you can change it... if you want
to change the current behavior by _default_ why don't we
suggest that _after_ this is accepted into 2.5? I.e., one
battle at a time.

Thanks,

Robert Love

2002-06-11 18:31:04

by Martin Dalecki

[permalink] [raw]
Subject: Re: [PATCH] CONFIG_NR_CPUS, redux

U?ytkownik Ruth Ivimey-Cook napisa?:
> On Tue, 11 Jun 2002, Randy.Dunlap wrote:
>
>
>>On 11 Jun 2002, Robert Love wrote:
>>
>>| Here are the defaults I picked:
>>|
>>| CONFIG_NR_CPUS=32: i386, mips, parisc, ppc, sparc
>>
>>I don't know what is "typical" for non-x86, but for x86, why not
>>use something more like a 'typical' NR_CPUS for SMP, like 8 (?)...
>>why still waste all of that memory?
>
>
> Perhaps it's just because I'm coming in late, but I cannot understand why
> NR_CPUS cannot be as low as 4 by default, for all archs, and then in the
> kernel boot messages, should more be found than is configured for a message is
> emitted to say "reconfigure your kernel", and continue with the number it was
> configured for. I personally only rarely see 2-way boxes, 4-way is pretty
> rare, and anything more must surely count as very specialized.
>
> Let the defaults be reasonable for 99% of users (IMO 99.9%), and let the rest
> have to think about config options...


Actually 2 would only make sense on Intel.
Well and then you have to account for the recent
HT additions so this becoumes 4.
On Sparc 4 is quite common.
But anything above is indeed very very rare.


2002-06-11 23:20:54

by Dave Jones

[permalink] [raw]
Subject: Re: [PATCH] CONFIG_NR_CPUS, redux

On Tue, Jun 11, 2002 at 08:29:35PM +0200, Martin Dalecki wrote:
> > Let the defaults be reasonable for 99% of users (IMO 99.9%), and let the rest
> > have to think about config options...
> Actually 2 would only make sense on Intel.

I'm sure the many owners of dual Athlons would beg to differ.

Dave.

--
| Dave Jones. http://www.codemonkey.org.uk
| SuSE Labs

2002-06-12 01:36:23

by jw schultz

[permalink] [raw]
Subject: Re: [PATCH] CONFIG_NR_CPUS, redux

On Tue, Jun 11, 2002 at 11:28:36AM -0700, Robert Love wrote:
> On Tue, 2002-06-11 at 11:21, Ruth Ivimey-Cook wrote:
>
> > Perhaps it's just because I'm coming in late, but I cannot understand why
> > NR_CPUS cannot be as low as 4 by default, for all archs, and then in the
> > kernel boot messages, should more be found than is configured for a message is
> > emitted to say "reconfigure your kernel", and continue with the number it was
> > configured for. I personally only rarely see 2-way boxes, 4-way is pretty
> > rare, and anything more must surely count as very specialized.
>
> Ugh let's stop this thread now. Two points:
>
> (a) imo, the kernel should support out-of-the-box the maximum
> number of CPUs it can handle. Be lucky we now have a
> configure option to change that. But that does not matter..
>
> (b) Right now it is 32. Now you can change it... if you want
> to change the current behavior by _default_ why don't we
> suggest that _after_ this is accepted into 2.5? I.e., one
> battle at a time.

By that logic CONFIG_SMP should be "y" by default.

Now i find the name NR_CPUS a bit misleading it seems that
this should be MAX_CPUS but "legacy is as legacy does".

Using the names i prefer i would suggest in *config we
replace CONFIG_SMP with CONFIG_MAX_CPUS and give it a
default of 1. Then make CONFIG_SMP dependant on
CONFIG_MAX_CPUS > 1. That way we avoid adding yet another
option. KISS for the users.

--
________________________________________________________________
J.W. Schultz Pegasystems Technologies
email address: [email protected]

Remember Cernan and Schmitt

2002-06-12 15:07:57

by Martin J. Bligh

[permalink] [raw]
Subject: Re: [PATCH] CONFIG_NR_CPUS, redux

> Ugh let's stop this thread now. Two points:
>
> (a) imo, the kernel should support out-of-the-box the maximum
> number of CPUs it can handle. Be lucky we now have a
> configure option to change that. But that does not matter..
>
> (b) Right now it is 32. Now you can change it... if you want
> to change the current behavior by _default_ why don't we
> suggest that _after_ this is accepted into 2.5? I.e., one
> battle at a time.

Indeed. And before that gets changed, it would be necessary
to change the bootstrap procedure not to crash if you have
more than NR_CPUS cpus (as Andrew reported it did), but instead
to just not configure them ... much less troublesome.

M.

2002-06-12 15:23:07

by William Lee Irwin III

[permalink] [raw]
Subject: Re: [PATCH] CONFIG_NR_CPUS, redux

On Wed, Jun 12, 2002 at 08:06:58AM -0700, Martin J. Bligh wrote:
> Indeed. And before that gets changed, it would be necessary
> to change the bootstrap procedure not to crash if you have
> more than NR_CPUS cpus (as Andrew reported it did), but instead
> to just not configure them ... much less troublesome.
> M.

There are also fun deadlocks on i386 with "too many cpu's" as it
appears the kernel attempts to use logical APIC mode to get beyond
the eigth cpu and seems unaware that it can't IPI them that way unless
it's configured to always use the clustered hierarchical destination
format, though that's perhaps a little beyond just NR_CPUS. Perhaps
some kind of sanity checking is needed at the arch level for issues
like this as well, not just the absolute maximum number of cpu's?


Cheers,
Bill

2002-06-13 08:51:33

by Helge Hafting

[permalink] [raw]
Subject: Re: [PATCH] CONFIG_NR_CPUS, redux

Ruth Ivimey-Cook wrote:

> Perhaps it's just because I'm coming in late, but I cannot understand why
> NR_CPUS cannot be as low as 4 by default, for all archs, and then in the
> kernel boot messages, should more be found than is configured for a message is
> emitted to say "reconfigure your kernel", and continue with the number it was
> configured for. I personally only rarely see 2-way boxes, 4-way is pretty
> rare, and anything more must surely count as very specialized.
>
Why not let the boot process select the highest of two numbers,
the (default-low) NR_CPUS and the number of CPU's detected?

Boot with "too many" cpu's and you still get to use them - you
merely can't hotplug even more.

Configuring a high NR_CPUS becomes something only hot-pluggers
need to do, or those whose architecture doesn't support
cpu detection in the early boot process. Those with a fixed
number of detectable CPUs can simply go with a default of
NR_CPUS=2 no matter what they actually have.

Helge Hafting