What limits the number of threads one can have on a Linux system?
I have a simple program that creates an array of threads and it locks up at the creation of somewhere between 250 and 275 threads.
The program just hangs indefinately unless a Control-C is hit.
How can I increase this number or can I?
Thanks, Joe
--------------------------------------------------------------------
mail2web - Check your email from the web at
http://mail2web.com/ .
On Thu, 21 Mar 2002 20:05:39 -0500, [email protected] wrote:
>What limits the number of threads one can have on a Linux system?
Common sense, one would hope.
>I have a simple program that creates an array of threads and it locks up at
>the creation of somewhere between 250 and 275 threads.
If it locks up, that's a bug. I remember older versions of glibc actually
had this bug. But it should simply fail to create them.
>The program just hangs indefinately unless a Control-C is hit.
>
>How can I increase this number or can I?
Why increase the number of threads you can create before you trigger a bug?
Wouldn't it make more sense to *fix* the bug so that pthread_create returns
an error like it's supposed to?
In any event, don't create so many threads. Create threads only to keep CPUs
busy or to pend I/Os that can't be done asynchronously.
DS
On Thu, 21 Mar 2002, David Schwartz wrote:
>
>
> On Thu, 21 Mar 2002 20:05:39 -0500, [email protected] wrote:
> >What limits the number of threads one can have on a Linux system?
>
> Common sense, one would hope.
>
> >I have a simple program that creates an array of threads and it locks up at
> >the creation of somewhere between 250 and 275 threads.
$ ulimit -u
- Davide
On Thu, 21 Mar 2002, Davide Libenzi wrote:
> On Thu, 21 Mar 2002, David Schwartz wrote:
>
> >
> >
> > On Thu, 21 Mar 2002 20:05:39 -0500, [email protected] wrote:
> > >What limits the number of threads one can have on a Linux system?
> >
> > Common sense, one would hope.
> >
> > >I have a simple program that creates an array of threads and it locks up at
> > >the creation of somewhere between 250 and 275 threads.
>
> $ ulimit -u
/proc/sys/kernel/threads-max is the system limit. And "locks up" is odd
unless the application is really poorly written to handle errors. Should
time out and whine ;-)
--
bill davidsen <[email protected]>
CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.
On Fri, 22 Mar 2002, Bill Davidsen wrote:
> On Thu, 21 Mar 2002, Davide Libenzi wrote:
>
> > On Thu, 21 Mar 2002, David Schwartz wrote:
> >
> > >
> > >
> > > On Thu, 21 Mar 2002 20:05:39 -0500, [email protected] wrote:
> > > >What limits the number of threads one can have on a Linux system?
> > >
> > > Common sense, one would hope.
> > >
> > > >I have a simple program that creates an array of threads and it locks up at
> > > >the creation of somewhere between 250 and 275 threads.
> >
> > $ ulimit -u
>
> /proc/sys/kernel/threads-max is the system limit. And "locks up" is odd
> unless the application is really poorly written to handle errors. Should
> time out and whine ;-)
One thing to note here, using pthreads there is a limit of 1024
threads per process. There are patches to glibc to increase this
to a larger number (4096 or 8192).
Regards
James Bourne
--
James Bourne, Supervisor Data Centre Operations
Mount Royal College, Calgary, AB, CA
http://www.mtroyal.ab.ca
******************************************************************************
This communication is intended for the use of the recipient to which it is
addressed, and may contain confidential, personal, and or privileged
information. Please contact the sender immediately if you are not the
intended recipient of this communication, and do not copy, distribute, or
take action relying on it. Any communication received in error, or
subsequent reply, should be deleted or destroyed.
******************************************************************************
On Fri, 22 Mar 2002, Bill Davidsen wrote:
> On Thu, 21 Mar 2002, Davide Libenzi wrote:
>
> > On Thu, 21 Mar 2002, David Schwartz wrote:
> >
> > >
> > >
> > > On Thu, 21 Mar 2002 20:05:39 -0500, [email protected] wrote:
> > > >What limits the number of threads one can have on a Linux system?
> > >
> > > Common sense, one would hope.
> > >
> > > >I have a simple program that creates an array of threads and it locks up at
> > > >the creation of somewhere between 250 and 275 threads.
> >
> > $ ulimit -u
>
> /proc/sys/kernel/threads-max is the system limit. And "locks up" is odd
> unless the application is really poorly written to handle errors. Should
> time out and whine ;-)
Around 250 was the old limit for max user processes ( non root ), if i
remember well.
- Davide
> > > On Thu, 21 Mar 2002 20:05:39 -0500, [email protected] wrote:
> > > >What limits the number of threads one can have on a Linux system?
> > >
> > > Common sense, one would hope.
I have none of that ;-). No this is actually a test. The only reason I'm even looking into pthreads is portability. While this may be an excessive amount of threads if the box has he capacity then I'd want to take advantage of that for batch jobs.
> > $ ulimit -u
unlimited
> /proc/sys/kernel/threads-max is the system limit. And "locks up" is odd
> unless the application is really poorly written to handle errors. Should
> time out and whine ;-)
This file shows 8192.
>Around 250 was the old limit for max user processes ( non root ), if i
>remember well.
In 2.4.17 that seems to be what I'm hitting. Actually it is 254.
How long before it times out?
thread number = 254 rc = 0
On thread 255 it does error out, however if I call exit then the program hangs.
This hangs:
if (rc > 0) { return -1;}
This hangs:
if (rc > 0) { exit(-1);}
This works:
if (rc > 0) { goto EXIT ;}
where exit is:
EXIT:
calls pthread_join();
It would seem that the functionality and behavior of threads on Sun is different from that of Linux.
Joe
--------------------------------------------------------------------
mail2web - Check your email from the web at
http://mail2web.com/ .
On Fri, 22 Mar 2002, James Bourne wrote:
> One thing to note here, using pthreads there is a limit of 1024
> threads per process. There are patches to glibc to increase this
> to a larger number (4096 or 8192).
Haven't checked to see the limit in NGPT, but I haven't hit it ;-)
--
bill davidsen <[email protected]>
CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.
On Fri, 22 Mar 2002, [email protected] wrote:
> It would seem that the functionality and behavior of threads on Sun is
> different from that of Linux.
You probably want to try NGPT, kernel 2.4.19-pre3 and later have the
kernel patches, and the new library is available from IBM. I don't know
about the limits, but the overall functionality should be closer.
--
bill davidsen <[email protected]>
CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.
Bill Davidsen wrote:
> On Fri, 22 Mar 2002, James Bourne wrote:
>
>
>>One thing to note here, using pthreads there is a limit of 1024
>>threads per process. There are patches to glibc to increase this
>>to a larger number (4096 or 8192).
>>
>
> Haven't checked to see the limit in NGPT, but I haven't hit it ;-)
>
There is another limit creeping in: pthread mmap()s 2 MB
of stack for each thread. So you run out of address space
on 32 bit systems with threads > 1024 (and smaller)