2002-09-18 06:30:10

by Dan Kegel

[permalink] [raw]
Subject: Hardware limits on numbers of threads?

http://people.redhat.com/drepper/glibcthreads.html says:

> Hardware restrictions put hard limits on the number of
> threads the kernel can support for each process.
> Specifically this applies to IA-32 (and AMD x86_64) where the thread
> register is a segment register. The processor architecture
> puts an upper limit on the number of segment register values
> which can be used (8192 in this case).

Is this true? Where does the limit come from?
- Dan


2002-09-18 06:45:23

by Denis Vlasenko

[permalink] [raw]
Subject: Re: Hardware limits on numbers of threads?

On 18 September 2002 04:43, Dan Kegel wrote:
> http://people.redhat.com/drepper/glibcthreads.html says:
> > Hardware restrictions put hard limits on the number of
> > threads the kernel can support for each process.
> > Specifically this applies to IA-32 (and AMD x86_64) where the thread
> > register is a segment register. The processor architecture
> > puts an upper limit on the number of segment register values
> > which can be used (8192 in this case).
>
> Is this true? Where does the limit come from?

It is true that on x86 you have only 8192 different segment selectors
at a time. Nobody says you can't modify segment descriptors on demand.

If I'm not mistaken, Linux kernel does precisely this. It has per-CPU
allocated GDT entries, not per-task. So there is no limitation
unless you happen to have more than 1024 CPUs ;-).
--
vda

2002-09-18 07:00:54

by Jakub Jelinek

[permalink] [raw]
Subject: Re: Hardware limits on numbers of threads?

On Wed, Sep 18, 2002 at 09:37:58AM -0200, Denis Vlasenko wrote:
> On 18 September 2002 04:43, Dan Kegel wrote:
> > http://people.redhat.com/drepper/glibcthreads.html says:
> > > Hardware restrictions put hard limits on the number of
> > > threads the kernel can support for each process.
> > > Specifically this applies to IA-32 (and AMD x86_64) where the thread
> > > register is a segment register. The processor architecture
> > > puts an upper limit on the number of segment register values
> > > which can be used (8192 in this case).
> >
> > Is this true? Where does the limit come from?
>
> It is true that on x86 you have only 8192 different segment selectors
> at a time. Nobody says you can't modify segment descriptors on demand.
>
> If I'm not mistaken, Linux kernel does precisely this. It has per-CPU
> allocated GDT entries, not per-task.

That page has been written way before set_thread_area(2) (and other kernel
threading changes) were written.

Jakub

2002-09-18 06:57:22

by Ulrich Drepper

[permalink] [raw]
Subject: Re: Hardware limits on numbers of threads?

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dan Kegel wrote:
> http://people.redhat.com/drepper/glibcthreads.html says:
>
>
>>Hardware restrictions put hard limits on the number of
>>threads the kernel can support for each process.
>>Specifically this applies to IA-32 (and AMD x86_64) where the thread
>>register is a segment register. The processor architecture
>>puts an upper limit on the number of segment register values
>>which can be used (8192 in this case).
>
>
> Is this true? Where does the limit come from?

This was and is true with the kernel before 2.5.3<mumble> when Ingo
introduced TLS support since the thread specific data had to be
addressed via LDT entries and the LDT holds at most 8192 entries. The
GDT based solution now implemented in the kernel has no such limitation
and the number of threads you can create with the new thread library is
only limited by system resources.

- --
- ---------------. ,-. 1325 Chesapeake Terrace
Ulrich Drepper \ ,-------------------' \ Sunnyvale, CA 94089 USA
Red Hat `--' drepper at redhat.com `------------------------
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQE9iCUE2ijCOnn/RHQRAgqyAJ0e88AQXpxXoHrvZCxIhfZXJFJP9QCbBlCL
nXUfQkt00sX+XsHg89OZz+k=
=NkZd
-----END PGP SIGNATURE-----

2002-09-18 07:24:57

by Giuliano Pochini

[permalink] [raw]
Subject: RE: Hardware limits on numbers of threads?


On 18-Sep-2002 Dan Kegel wrote:
> http://people.redhat.com/drepper/glibcthreads.html says:
>
>> Hardware restrictions put hard limits on the number of
>> threads the kernel can support for each process. [...]

> Is this true? Where does the limit come from?

Threads are a software thing. If you can have 2 threads, so
there is no limit, unless you want to make use of some
hardware facility I'm not aware of.

Bye.

2002-09-18 11:53:04

by Brian Gerst

[permalink] [raw]
Subject: Re: Hardware limits on numbers of threads?

Dan Kegel wrote:
> http://people.redhat.com/drepper/glibcthreads.html says:
>
>
>>Hardware restrictions put hard limits on the number of
>>threads the kernel can support for each process.
>>Specifically this applies to IA-32 (and AMD x86_64) where the thread
>>register is a segment register. The processor architecture
>>puts an upper limit on the number of segment register values
>>which can be used (8192 in this case).
>
>
> Is this true? Where does the limit come from?
> - Dan

A long time ago Linux did use one GDT segment for a TSS and LDT for each
process. Then it was changed in 2.3.11 to have one TSS and LDT per cpu,
removing the limit on the number of processes that can exist in the system.

--
Brian Gerst


2002-09-18 16:16:55

by Dan Kegel

[permalink] [raw]
Subject: Re: Hardware limits on numbers of threads?

Ulrich Drepper wrote:
> > http://people.redhat.com/drepper/glibcthreads.html says:
> >
> >>Hardware restrictions put hard limits on the number of
> >>threads the kernel can support for each process. ...
> >
> > Is this true? Where does the limit come from?
>
> This was and is true with the kernel before 2.5.3<mumble> when Ingo
> introduced TLS support since the thread specific data had to be
> addressed via LDT entries and the LDT holds at most 8192 entries. The
> GDT based solution now implemented in the kernel has no such limitation
> and the number of threads you can create with the new thread library is
> only limited by system resources.

Thanks. I've updated http://www.kegel.com/c10k.html#threaded
accordingly.

- Dan

2002-09-18 19:15:20

by Jamie Lokier

[permalink] [raw]
Subject: Re: Hardware limits on numbers of threads?

Ulrich Drepper wrote:
> > Is this true? Where does the limit come from?
>
> This was and is true with the kernel before 2.5.3<mumble> when Ingo
> introduced TLS support since the thread specific data had to be
> addressed via LDT entries and the LDT holds at most 8192 entries. The
> GDT based solution now implemented in the kernel has no such limitation
> and the number of threads you can create with the new thread library is
> only limited by system resources.

So you did not implement LDT entry swapping through the "segment not
present" traps? Ah well, moot now :-)

-- Jamie

2002-09-19 20:20:21

by Andi Kleen

[permalink] [raw]
Subject: Re: Hardware limits on numbers of threads?

Ulrich Drepper <[email protected]> writes:


> This was and is true with the kernel before 2.5.3<mumble> when Ingo
> introduced TLS support since the thread specific data had to be
> addressed via LDT entries and the LDT holds at most 8192 entries. The
> GDT based solution now implemented in the kernel has no such
> limitation and the number of threads you can create with the new
> thread library is only limited by system resources.

It also was alwas incorrect for x86-64/64bit progreams, which do not
use a LDT entry for each thread.

-Andi