I would like to get some comments on a new syscall that I am planning on
implementing. This syscall would allow a process to register to be notified
when another process dies. The calling process would specify the pid of the
process in which it is interested and the signal which it wants to be sent when
the process with the specified pid dies. The api would be:
int sigexit(pid_t pid, int signum)
The implementation would add a new linked list to the task struct which would
store pid/signal tuples for each process requesting notification. On process
death, in do_notify_parent we walk the list and send the specified signals to
all the listeners.
I see two immediate uses for this. One would be to enable a "watcher" process
which can do useful things on the death of processes which registered with it
(logging, respawning, notifying other processes, etc). The watcher could keep a
persistant list of what its monitoring and what for in a file, and if it ever
died, the new watcher could scan the list and register to watch them all again.
The second would be to enable mutual suicide pacts between processes. (I'm not
sure when I would use this, but it sounds kind of fun.)
Anyone have any opinions on this? There is a comment in exit_notify about not
sending signals to arbitrary processes using the thread signals, but I'm not
sure if that objection was to the idea or to the implementation.
Thanks,
Chris
--
Chris Friesen | MailStop: 043/33/F10
Nortel Networks | work: (613) 765-0557
3500 Carling Avenue | fax: (613) 765-2986
Nepean, ON K2H 8E9 Canada | email: [email protected]
On Sat, May 10, 2003 at 02:29:54AM -0400, Chris Friesen wrote:
> I see two immediate uses for this. One would be to enable a "watcher"
> process which can do useful things on the death of processes which
> registered with it (logging, respawning, notifying other processes,
> etc).
Do it from user space, kill(pid, 0), check for ESRCH. I might see the
benefit of a new system call if it was synchronous (wait() semantics),
but since signal delivery is asynch anyway....
> The second would be to enable mutual
> suicide pacts between processes. (I'm not sure when I would use this, but
> it sounds kind of fun.)
Same thing, kill(pid, 0).
> Anyone have any opinions on this?
There's already a well established way to do what you want (get
non-immediate notification of process death). What benefit would your
approach give?
--
Muli Ben-Yehuda
http://www.mulix.org
Muli Ben-Yehuda wrote:
> On Sat, May 10, 2003 at 02:29:54AM -0400, Chris Friesen wrote:
>
>
>>I see two immediate uses for this. One would be to enable a "watcher"
>>process which can do useful things on the death of processes which
>>registered with it (logging, respawning, notifying other processes,
>>etc).
>>
>
> Do it from user space, kill(pid, 0), check for ESRCH. I might see the
> benefit of a new system call if it was synchronous (wait() semantics),
> but since signal delivery is asynch anyway....
Exactly. I don't want to explicitly poll each process being monitored to see if
it is still alive. That solution doesn't scale well--what happens when you are
monitoring 5000 processes and you want to make sure that you catch them within a
certain amount of time? You end up spending a lot of cpu time doing the monitoring.
> There's already a well established way to do what you want (get
> non-immediate notification of process death). What benefit would your
> approach give?
Its cheaper and faster. It only costs a single call for each process, and then
you get notified immediately when it dies.
Chris
--
Chris Friesen | MailStop: 043/33/F10
Nortel Networks | work: (613) 765-0557
3500 Carling Avenue | fax: (613) 765-2986
Nepean, ON K2H 8E9 Canada | email: [email protected]
Chris Friesen <[email protected]> writes:
> > There's already a well established way to do what you want (get
> > non-immediate notification of process death). What benefit would your
> > approach give?
>
> Its cheaper and faster. It only costs a single call for each process,
> and then you get notified immediately when it dies.
Rather than a new syscall, what about a magic file or device that you
can poll()?
-Doug
Ok i hope someone can help with this.. I have been trying to figure it out
ever since 2.4.20(this behaviour is present with all the pre and rc's and
also with all the ac patches).
I have 385844k(377mb) of SDRAM and 128512k(126mb) of SWAP. Now here is the
deal: before I upgraded to 2.4.20 swap was never touched and my ram was
always staying down at arround 3/5% usage (unless i did some compiling).
Now when I did the upgrade to 2.4.20 I did not change my config file at all
the only new thing that I added was adding usb-to-usb networking ( i have a
zaurus) ( i have tried both module and compiled into the kernel, this is
not the cause of the problem). Now if I leave the machine on for a couple
of hours my ram begins to be eaten up (top says that all goes into cache)
and the system begins to eat away into the swap, at this point if I start
anything resource intensive (like compiling or running NetBeans) RAM usage
goes down and swap usage shoots up.
These are the things I have tried so far:
1. use all the pre, rc, ac (the pres and rc with and without ac applied)
2. compile the one and only new thing (usb-tp-usb) as module and building it
into the kernel.
3. leaveing the system at the console (not running X not doing anything just
let it sit there). (the problem appears still, so I know it is not my X)
4. running X and leaving the machine to idle.(the problem presists)
5. turning swap off completely. (in this case of course ram was used but it
was never released, it always stayed above 50% usage and never came down).
None of these lead to anything... my ram was still eaten up (pretty much all
was put into cache, at least that is what top was reporting) and then once
I started using the computer instead of ram the system was using swap.
As i said i have not been albe to find out what the problem is... or where
it is coming from, I hope someone here might be able to help.
My distro is slackware 9, kernel is 2.4.20-rc2 (have not applied the ac
yet), i have X 4.3.0 and KDE 3.1.1a.
Thanks for any help.
On Sun, May 11, 2003 at 11:26:06PM -0400, Chris Friesen wrote:
>
> Its cheaper and faster. It only costs a single call for each process, and
> then you get notified immediately when it dies.
On Solaris, pwait(1) opens /proc/pid/psinfo and waits for data.
/fc
Doug McNaught wrote:
> Chris Friesen <[email protected]> writes:
>
>
>>>There's already a well established way to do what you want (get
>>>non-immediate notification of process death). What benefit would your
>>>approach give?
>>>
>>Its cheaper and faster. It only costs a single call for each process,
>>and then you get notified immediately when it dies.
>>
>
> Rather than a new syscall, what about a magic file or device that you
> can poll()?
This is definately an option to consider. The problem that I see with this is
that when you are trying to monitor large numbers of processes you have to worry
about running out of file descriptors, and select() is no longer as happy.
I have an actual real request to be able to monitor 5000 processes. This would
be a lot of file descriptors, and when select returns it would take some
processing to figure out which one had an event.
It does have easier handling of multiple simultaneous deaths though...the signal
method would probably want to use realtime signals to get signal queueing.
Chris
--
Chris Friesen | MailStop: 043/33/F10
Nortel Networks | work: (613) 765-0557
3500 Carling Avenue | fax: (613) 765-2986
Nepean, ON K2H 8E9 Canada | email: [email protected]
I forgot to add that if I leave the machine long enough on (2-3 days) the RAM
usage goes down to like the normal 5/6% BUT the swap is totaly used up with
maybe like 5/6% left free.
Also, I forgot to attach my kernel config in case it helped,, so here it is
Chris Friesen <[email protected]> writes:
> Doug McNaught wrote:
> > Rather than a new syscall, what about a magic file or device that you
> > can poll()?
>
> This is definately an option to consider. The problem that I see with
> this is that when you are trying to monitor large numbers of processes
> you have to worry about running out of file descriptors, and select()
> is no longer as happy.
No reason to have one FD per process monitored. Just a single FD, to
which you can write() a control string to to add or remove a process
from the list, and for which read() yields a small data record
describing the process event that just happened. It's a bit plan-9ish
but there's nothing wrong with that...
-Doug
Doug McNaught wrote:
> No reason to have one FD per process monitored. Just a single FD, to
> which you can write() a control string to to add or remove a process
> from the list, and for which read() yields a small data record
> describing the process event that just happened. It's a bit plan-9ish
> but there's nothing wrong with that...
Ah, okay. Interesting idea. It would get around the limitation of having to
use rt signals to get the queueing (though I'm not likely to hit that limit in
any case).
Have to think about that one...
Chris
--
Chris Friesen | MailStop: 043/33/F10
Nortel Networks | work: (613) 765-0557
3500 Carling Avenue | fax: (613) 765-2986
Nepean, ON K2H 8E9 Canada | email: [email protected]
On Sun, May 11, 2003 at 11:42:27PM -0400, Boris Kurktchiev wrote:
> Now when I did the upgrade to 2.4.20 I did not change my config file at all
> the only new thing that I added was adding usb-to-usb networking ( i have a
> zaurus)
What driver are you using for this? usbdnet? I'd recommend using the
in-kernel support for this device instead. I think the latest versions
of usbnet support it.
thanks,
greg k-h
On Tuesday May 13 2003 8:04 am, Denis Vlasenko wrote:
> On 12 May 2003 14:17, Boris Kurktchiev wrote:
> > I forgot to add that if I leave the machine long enough on (2-3 days)
> > the RAM usage goes down to like the normal 5/6% BUT the swap is
> > totaly used up with maybe like 5/6% left free.
> > Also, I forgot to attach my kernel config in case it helped,, so here
> > it is
>
> Instead of describing your problem at length just show top
> and various cat /proc/* output
> --
> vda
oh yeah and also if I turn swap off and leave the machine one besides the fact
that RAM usage does not go below 50% apps start dying from OOM (out of
memory) problems (even if there is more RAM free).
top - 11:03:41 up 4 min, 1 user, load average: 0.12, 0.20, 0.09
Tasks: 60 total, 1 running, 58 sleeping, 0 stopped, 1 zombie
Cpu(s): 8.3% user, 2.3% system, 0.0% nice, 89.4% idle
Mem: 385904k total, 173996k used, 211908k free, 14244k buffers
Swap: 128512k total, 0k used, 128512k free, 86732k cached
this is what the machine used to look like.
this is what happens when the machine has run for about 3 hours, and during
that time I have had Netbeans and Day Of Defeat(wine) running for about 15
minutes.
top - 14:14:49 up 2:31, 1 user, load average: 0.03, 0.04, 0.01
Tasks: 60 total, 2 running, 57 sleeping, 0 stopped, 1 zombie
Cpu(s): 2.7% user, 0.3% system, 0.0% nice, 97.0% idle
Mem: 385904k total, 261368k used, 124536k free, 16736k buffers
Swap: 128512k total, 8768k used, 119744k free, 175476k cached
if i leave the machine on, and say I start transcoding something.. the RAM
would not be touched and the swap usage would shoot up to 95%.
On 13 May 2003 21:15, you wrote:
>top - 11:03:41 up 4 min, ?1 user, ?load average: 0.12, 0.20, 0.09
>Tasks: ?60 total, ? 1 running, ?58 sleeping, ? 0 stopped, ? 1 zombie
>Cpu(s): ? 8.3% user, ? 2.3% system, ? 0.0% nice, ?89.4% idle
>Mem: ? ?385904k total, ? 173996k used, ? 211908k free, ? ?14244k buffers
>Swap: ? 128512k total, ? ? ? ?0k used, ? 128512k free, ? ?86732k cached
>
>this is what the machine used to look like.
>
>this is what happens when the machine has run for about 3 hours, and during
>that time I have had Netbeans and Day Of Defeat(wine) running for about 15
>minutes.
>
>top - 14:14:49 up ?2:31, ?1 user, ?load average: 0.03, 0.04, 0.01
>Tasks: ?60 total, ? 2 running, ?57 sleeping, ? 0 stopped, ? 1 zombie
>Cpu(s): ? 2.7% user, ? 0.3% system, ? 0.0% nice, ?97.0% idle
>Mem: ? ?385904k total, ? 261368k used, ? 124536k free, ? ?16736k buffers
>Swap: ? 128512k total, ? ? 8768k used, ? 119744k free, ? 175476k cached
>
>if i leave the machine on, and say I start transcoding something..
>the RAM would not be touched and the swap usage would shoot up to
>95%.
So far I see nothing abnormal. My current top:
09:51:53 up 16:40, 1 user, load average: 0.02, 0.02, 0.00
56 processes: 55 sleeping, 1 running, 0 zombie, 0 stopped
CPU states: 0.2% user, 0.4% system, 0.0% nice, 0.0% iowait, 99.3% idle
Mem: 124616k av, 114444k used, 10172k free, 0k shrd, 4k buff
53088k active, 46836k inactive
Swap: 76792k av, 1804k used, 74988k free 53632k cached
Can you show "top b n1" (unabridged) and "cat /proc/meminfo", "cat /proc/slabinfo"
of the "swap usage shoot up to 95%" event?
--
vda
> 09:51:53 up 16:40, 1 user, load average: 0.02, 0.02, 0.00
> 56 processes: 55 sleeping, 1 running, 0 zombie, 0 stopped
> CPU states: 0.2% user, 0.4% system, 0.0% nice, 0.0% iowait, 99.3% idle
> Mem: 124616k av, 114444k used, 10172k free, 0k shrd, 4k
> buff 53088k active, 46836k inactive
> Swap: 76792k av, 1804k used, 74988k free 53632k
> cached
>
> Can you show "top b n1" (unabridged) and "cat /proc/meminfo", "cat
> /proc/slabinfo" of the "swap usage shoot up to 95%" event?
heh this is very interesting.... top b n1 reports this:
top - 10:08:24 up 16:36, 2 users, load average: 0.16, 0.19, 0.08
Tasks: 62 total, 1 running, 60 sleeping, 0 stopped, 1 zombie
Cpu(s): 12.3% user, 5.1% system, 0.0% nice, 82.6% idle
Mem: 385904k total, 381572k used, 4332k free, 137244k buffers
Swap: 128512k total, 20012k used, 108500k free, 126168k cached
while gkrellm reports that my RAM used is 95MB. now this is interesting....
here is the /proc/meminfo:
total: used: free: shared: buffers: cached:
Mem: 395165696 389582848 5582848 0 141422592 138055680
Swap: 131596288 20623360 110972928
MemTotal: 385904 kB
MemFree: 5452 kB
MemShared: 0 kB
Buffers: 138108 kB
Cached: 127324 kB
SwapCached: 7496 kB
Active: 181268 kB
Inactive: 132244 kB
HighTotal: 0 kB
HighFree: 0 kB
LowTotal: 385904 kB
LowFree: 5452 kB
SwapTotal: 128512 kB
SwapFree: 108372 kB
and here is /proc/slabinfo:
slabinfo - version: 1.1
kmem_cache 57 72 108 2 2 1
ip_conntrack 3 24 320 1 2 1
tcp_tw_bucket 0 30 128 0 1 1
tcp_bind_bucket 4 112 32 1 1 1
tcp_open_request 0 0 64 0 0 1
inet_peer_cache 0 0 64 0 0 1
ip_fib_hash 14 112 32 1 1 1
ip_dst_cache 30 40 192 2 2 1
arp_cache 3 30 128 1 1 1
urb_priv 3 59 64 1 1 1
blkdev_requests 1024 1050 128 35 35 1
dnotify_cache 125 169 20 1 1 1
file_lock_cache 7 42 92 1 1 1
fasync_cache 1 202 16 1 1 1
uid_cache 1 112 32 1 1 1
skbuff_head_cache 128 220 192 11 11 1
sock 117 145 768 26 29 1
sigqueue 0 29 132 0 1 1
kiobuf 0 0 64 0 0 1
cdev_cache 19 59 64 1 1 1
bdev_cache 6 59 64 1 1 1
mnt_cache 19 59 64 1 1 1
inode_cache 45164 59479 512 8497 8497 1
dentry_cache 20481 62910 128 2097 2097 1
filp 1262 1290 128 43 43 1
names_cache 0 2 4096 0 2 1
buffer_head 94915 97440 128 3248 3248 1
mm_struct 47 60 192 3 3 1
vm_area_struct 2479 2910 128 90 97 1
fs_cache 46 59 64 1 1 1
files_cache 46 54 448 6 6 1
signal_act 51 57 1344 18 19 1
size-131072(DMA) 0 0 131072 0 0 32
size-131072 0 0 131072 0 0 32
size-65536(DMA) 0 0 65536 0 0 16
size-65536 2 2 65536 2 2 16
size-32768(DMA) 0 0 32768 0 0 8
size-32768 4 4 32768 4 4 8
size-16384(DMA) 0 0 16384 0 0 4
size-16384 9 11 16384 9 11 4
size-8192(DMA) 0 0 8192 0 0 2
size-8192 5 20 8192 5 20 2
size-4096(DMA) 0 0 4096 0 0 1
size-4096 79 95 4096 79 95 1
size-2048(DMA) 0 0 2048 0 0 1
size-2048 10 14 2048 5 7 1
size-1024(DMA) 0 0 1024 0 0 1
size-1024 70 96 1024 18 24 1
size-512(DMA) 0 0 512 0 0 1
size-512 65 80 512 10 10 1
size-256(DMA) 0 0 256 0 0 1
size-256 49 75 256 4 5 1
size-128(DMA) 2 30 128 1 1 1
size-128 883 930 128 30 31 1
size-64(DMA) 0 0 64 0 0 1
size-64 2791 2891 64 49 49 1
size-32(DMA) 36 59 64 1 1 1
size-32 414 472 64 8 8 1
On 14 May 2003 17:12, Boris Kurktchiev wrote:
> heh this is very interesting.... top b n1 reports this:
> top - 10:08:24 up 16:36, 2 users, load average: 0.16, 0.19, 0.08
> Tasks: 62 total, 1 running, 60 sleeping, 0 stopped, 1 zombie
> Cpu(s): 12.3% user, 5.1% system, 0.0% nice, 82.6% idle
> Mem: 385904k total, 381572k used, 4332k free, 137244k
> buffers Swap: 128512k total, 20012k used, 108500k free,
> 126168k cached
Typical. So what makes you think kernel leaks memory?
BTW, which version of procps do you have? Mine is 2.0.10,
2.0.11 already exists.
> while gkrellm reports that my RAM used is 95MB. now this is
> interesting....
gkrellm must be subtracting something from MemTotal trying
to account for fact that large part of RAM is used as a cache.
You may consult its source.
--
vda
On Thursday May 15 2003 1:51 am, Denis Vlasenko wrote:
> On 14 May 2003 17:12, Boris Kurktchiev wrote:
> > heh this is very interesting.... top b n1 reports this:
> > top - 10:08:24 up 16:36, 2 users, load average: 0.16, 0.19, 0.08
> > Tasks: 62 total, 1 running, 60 sleeping, 0 stopped, 1 zombie
> > Cpu(s): 12.3% user, 5.1% system, 0.0% nice, 82.6% idle
> > Mem: 385904k total, 381572k used, 4332k free, 137244k
> > buffers Swap: 128512k total, 20012k used, 108500k free,
> > 126168k cached
>
> Typical. So what makes you think kernel leaks memory?
well the fact that before my swap was never used, and now .... I need to
transcode something so I can show you how all swap is being used and non of
the RAM (thus making programs run much slower, as is the case with
transcode).
> BTW, which version of procps do you have? Mine is 2.0.10,
> 2.0.11 already exists.
I believe I have 2.0.10.
> gkrellm must be subtracting something from MemTotal trying
> to account for fact that large part of RAM is used as a cache.
> You may consult its source.
No... I forgot to tell it to count cache and buffers...
ok here is what dmesg shows:
384MB LOWMEM available.
then further down:
Memory: 385584k/393216k available (2010k kernel code, 7244k reserved, 597k
data, 128k init, 0k highmem)
now how is the little 38.../39... possible?
and then top shows this:
Mem: 385712k total
this again is different than the others...
and finaly gkrellm is telling me that I have only 377 mb actually recognized
out of the 384mb that the kernel detected above...
So the question is where does my 7mb go, why that weird 38.../39 difference
and why does top report another different value.
Boris Kurktchiev wrote:
> ok here is what dmesg shows:
> 384MB LOWMEM available.
>
> then further down:
> Memory: 385584k/393216k available
>
> now how is the little 38.../39... possible?
384 * 1024 * 1000 = 393216000
--
Chris Friesen | MailStop: 043/33/F10
Nortel Networks | work: (613) 765-0557
3500 Carling Avenue | fax: (613) 765-2986
Nepean, ON K2H 8E9 Canada | email: [email protected]
>
> ok here is what dmesg shows:
> 384MB LOWMEM available.
>
> then further down:
> Memory: 385584k/393216k available (2010k kernel code, 7244k reserved, 597k
> data, 128k init, 0k highmem)
^^^^
> now how is the little 38.../39... possible?
>
> and then top shows this:
> Mem: 385712k total
385584+128=385712
> this again is different than the others...
>
> and finaly gkrellm is telling me that I have only 377 mb actually recognized
> out of the 384mb that the kernel detected above...
# echo $((385712/1024))
376
> So the question is where does my 7mb go, why that weird 38.../39 difference
> and why does top report another different value.
--
=======================================================================
Andrzej M. Krzysztofowicz [email protected]
phone (48)(58) 347 14 61
Faculty of Applied Phys. & Math., Gdansk University of Technology
On Friday May 16 2003 1:42 am, Chris Friesen wrote:
> Boris Kurktchiev wrote:
> > ok here is what dmesg shows:
> > 384MB LOWMEM available.
> >
> > then further down:
> > Memory: 385584k/393216k available
> >
> > now how is the little 38.../39... possible?
>
> 384 * 1024 * 1000 = 393216000
ahh so I am reading it wrong... ok but where is the 7mbs going? I think I
amreading it right this time... and it says that it only uses 385584k as
opposed to the full 393216k... btw the same thing happens with swap there are
only 2mbs eaten up from it though... (I have 128mb and I get only 127mb)
On Friday May 16 2003 1:42 am, Chris Friesen wrote:
> Boris Kurktchiev wrote:
> > ok here is what dmesg shows:
> > 384MB LOWMEM available.
> >
> > then further down:
> > Memory: 385584k/393216k available
> >
> > now how is the little 38.../39... possible?
>
> 384 * 1024 * 1000 = 393216000
blah someone already answered my question about the missing 7mbs... is there a
doc that tell swhy the kernel reserves that much memory?
On 15 May 2003 17:24, Boris Kurktchiev wrote:
> > Typical. So what makes you think kernel leaks memory?
>
> well the fact that before my swap was never used, and now .... I need
> to transcode something so I can show you how all swap is being used
> and non of the RAM (thus making programs run much slower, as is the
> case with transcode).
Well there might be problems with kernel being too swap-happy.
I.e. it swaps out application pages but keeps less precious
cache pages. These problems are not that easy to debug
(How do one prove that kernel swaps out 'wrong' pages?
What is 'wrong'? It's kind of subjective).
> > BTW, which version of procps do you have? Mine is 2.0.10,
> > 2.0.11 already exists.
>
> I believe I have 2.0.10.
No. 2.0.10 top printout look different:
11:44:00 up 1 day, 18:52, 1 user, load average: 0.25, 0.29, 0.23
63 processes: 60 sleeping, 3 running, 0 zombie, 0 stopped
CPU states: 41.8% user, 3.9% system, 0.0% nice, 0.0% iowait, 54.1% idle
Mem: 124616k av, 120448k used, 4168k free, 0k shrd, 4k buff
87172k active, 18536k inactive
Swap: 76792k av, 21820k used, 54972k free 55944k cached
--
vda