2002-01-22 21:36:20

by Louis E Garcia II

[permalink] [raw]
Subject: Low latency for recent kernels

Where can I get the low latency patch for kernel-2.4.18-pre6?

--Louis





2002-01-22 21:53:11

by Andrew Morton

[permalink] [raw]
Subject: Re: Low latency for recent kernels

Louis Garcia wrote:
>
> Where can I get the low latency patch for kernel-2.4.18-pre6?
>

http://www.zip.com.au/~akpm/linux/2.4.18-pre6-low-latency.patch.gz

2002-01-22 22:00:51

by jjs

[permalink] [raw]
Subject: Re: Low latency for recent kernels

Cool, it must have just gone up.

Best to get it straight from the master then -

pls disregard my patch

;-)

Andrew Morton wrote:

>Louis Garcia wrote:
>
>>Where can I get the low latency patch for kernel-2.4.18-pre6?
>>
>
>http://www.zip.com.au/~akpm/linux/2.4.18-pre6-low-latency.patch.gz
>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to [email protected]
>More majordomo info at http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at http://www.tux.org/lkml/
>


2002-01-23 14:13:40

by Randy Hron

[permalink] [raw]
Subject: Re: Low latency for recent kernels

> http://www.zip.com.au/~akpm/linux/2.4.18-pre6-low-latency.patch.gz

2.4.18-pre3 with 2.4.17-low-latency.patch worked fine on this system
2.4.18-pre6 with 2.4.18-pre6-low-latency.patch panics at boot time.
2.4.18-pre6 is fine also.

System has reiserfs root filesystem. No modules.
/usr/src/linux/System.map was the System.map for 2.4.18pre6ll for
the ksymoops below.

No modules in ksyms, skipping objects
No ksyms, skipping lsmod
Kernel panic: can't allocate root vfsmount
<1>Unable to handle kernel NULL pointer dereference at virtual address 0000002c
c01234d3
*pde = 00000000
Oops: 0000
CPU: 0
EIP: 0010:[<c01234d3>] Not tainted
Using defaults from ksymoops -t elf32-i386 -a i386
EFLAGS: 00010046
eax: 00000000 ebx: 00000008 ecx: 00000000 edx: 00000073
esi: 00000000 edi: 00000018 ebp: 00000020 esp: c0215e78
ds: 0018 es: 0018 ss: 0018
Process . (pid: -1072541344, stackpage=c0215000)
Stack: 00000018 00000001 00000018 c0214568 c0117e6c 00000000 00000020 c0214000
00000018 00000018 00000000 c0117f4c 00000018 00000001 c0214568 00000086
00000018 c0214000 c0117ff4 00000018 00000001 c0214000 01ebb409 c0214000
Call Trace: [<c0117e6c>] [<c0117f4c>] [<c0117ff4>] [<c0118253>] [<c0117208>]
[<c0117291>] [<c011759f>] [<c010a889>] [<c0107d1c>] [<c0107e82>] [<c0105000>]
[<c0109c18>] [<c0105000>] [<c0112020>]
Code: f6 46 2c 01 74 02 0f 0b 9c 5f fa 8b 4e 08 39 d9 75 22 8b 4e

>>EIP; c01234d2 <kmem_cache_alloc+2a/b8> <=====
Trace; c0117e6c <send_signal+2c/f0>
Trace; c0117f4c <deliver_signal+1c/50>
Trace; c0117ff4 <send_sig_info+74/88>
Trace; c0118252 <send_sig+1a/20>
Trace; c0117208 <update_one_process+68/d4>
Trace; c0117290 <update_process_times+1c/88>
Trace; c011759e <do_timer+22/70>
Trace; c010a888 <timer_interrupt+60/10c>
Trace; c0107d1c <handle_IRQ_event+30/5c>
Trace; c0107e82 <do_IRQ+6a/a8>
Trace; c0105000 <_stext+0/0>
Trace; c0109c18 <call_do_IRQ+6/e>
Trace; c0105000 <_stext+0/0>
Trace; c0112020 <panic+c0/d0>
Code; c01234d2 <kmem_cache_alloc+2a/b8>
00000000 <_EIP>:
Code; c01234d2 <kmem_cache_alloc+2a/b8> <=====
0: f6 46 2c 01 testb $0x1,0x2c(%esi) <=====
Code; c01234d6 <kmem_cache_alloc+2e/b8>
4: 74 02 je 8 <_EIP+0x8> c01234da <kmem_cache_alloc+32/b8>
Code; c01234d8 <kmem_cache_alloc+30/b8>
6: 0f 0b ud2a
Code; c01234da <kmem_cache_alloc+32/b8>
8: 9c pushf
Code; c01234da <kmem_cache_alloc+32/b8>
9: 5f pop %edi
Code; c01234dc <kmem_cache_alloc+34/b8>
a: fa cli
Code; c01234dc <kmem_cache_alloc+34/b8>
b: 8b 4e 08 mov 0x8(%esi),%ecx
Code; c01234e0 <kmem_cache_alloc+38/b8>
e: 39 d9 cmp %ebx,%ecx
Code; c01234e2 <kmem_cache_alloc+3a/b8>
10: 75 22 jne 34 <_EIP+0x34> c0123506 <kmem_cache_alloc+5e/b8>
Code; c01234e4 <kmem_cache_alloc+3c/b8>
12: 8b 4e 00 mov 0x0(%esi),%ecx

<0>Kernel panic: Aiee, killing interrupt handler!

--
Randy Hron

2002-01-23 19:26:27

by jjs

[permalink] [raw]
Subject: Re: Low latency for recent kernels

Ditto here -

2.4.18-pre6 + previous low latency patch boots & runs -

2.4.18-pre6 + latest low latency panics after this line:

Mount-cache hash table entries: 8192 (order: 4, 65536 bytes)

(next line would have been "Buffer-cache hash table entries...")

Here is the oops (be advised, it's copied by hand)

Kernel panic: can't allocate root vfsmount
<1>Unable to handle kernel NULL pointer dereference at virtual address
00000000c011a830
*pde = 00000000
Oops: 0000
CPU: 0
EIP: 0010:[<c011a830>] Not tainted
Using defaults from ksymoops -t elf32-i386 -a i386
EFLAGS: 00010207
eax: 000001f3 ebx: 00000001 ecx: 00000000 edx: 00000000
esi: 00000000 edi: c02425a0 ebp: c020bf10 esp: c020bf10
ds: 0018 es: 0018 ss: 0018
Process (pid: -1072520631, stackpage=c020b000)
Stack: c020bf28 c011a8aa c020bf38 c024a5c0 00000000 c02425a0 c020bf30
c01178dd
c020bf48 c01177ef 00000000 00000000 c02425c0 fffffff6 c020bf64
c01175db
c02425c0 00000046 c020bf84 00000000 c0240900 c020bf7c c01082ac
c01f9bc8
Call Trace: [<c011a8aa>] [<c01178dd>] [<c01177ef>] [<c01175db>]
[<c01082ac>]
[<c0105000>] [<c0105000>] [<c0110018>] [<c0113382>]
Code: 8b 02 85 c0 74 07 8b 02 83 e0 02 74 06 81 c1 c0 08 00 00 8b

>>EIP; c011a830 <count_active_tasks+20/50> <=====
Trace; c011a8aa <timer_bh+4a/270>
Trace; c01178dd <bh_action+1d/50>
Trace; c01177ef <tasklet_hi_action+4f/70>
Trace; c01175db <do_softirq+5b/b0>
Trace; c01082ac <do_IRQ+ac/c0>
Trace; c0105000 <_stext+0/0>
Trace; c0105000 <_stext+0/0>
Trace; c0110018 <do_page_fault+28/555>
Trace; c0113382 <panic+e2/f0>
Code; c011a830 <count_active_tasks+20/50>
00000000 <_EIP>:
Code; c011a830 <count_active_tasks+20/50> <=====
0: 8b 02 mov (%edx),%eax <=====
Code; c011a832 <count_active_tasks+22/50>
2: 85 c0 test %eax,%eax
Code; c011a834 <count_active_tasks+24/50>
4: 74 07 je d <_EIP+0xd> c011a83d
<count_active_tasks+2d/50>
Code; c011a836 <count_active_tasks+26/50>
6: 8b 02 mov (%edx),%eax
Code; c011a838 <count_active_tasks+28/50>
8: 83 e0 02 and $0x2,%eax
Code; c011a83b <count_active_tasks+2b/50>
b: 74 06 je 13 <_EIP+0x13> c011a843
<count_active_tasks+33/50>
Code; c011a83d <count_active_tasks+2d/50>
d: 81 c1 c0 08 00 00 add $0x8c0,%ecx
Code; c011a843 <count_active_tasks+33/50>
13: 8b 00 mov (%eax),%eax

Kernel panic: Aiee, killing interrupt handler!

It looks like the newer version has substantial
changes in filemap.c and page_alloc.c relative
the the older version; if I had more time I'd
have looked into it further -

Best Regards,

joe


[email protected] wrote:

>>http://www.zip.com.au/~akpm/linux/2.4.18-pre6-low-latency.patch.gz
>>
>
>2.4.18-pre3 with 2.4.17-low-latency.patch worked fine on this system
>2.4.18-pre6 with 2.4.18-pre6-low-latency.patch panics at boot time.
>2.4.18-pre6 is fine also.
>
>System has reiserfs root filesystem. No modules.
>/usr/src/linux/System.map was the System.map for 2.4.18pre6ll for
>the ksymoops below.
>
>No modules in ksyms, skipping objects
>No ksyms, skipping lsmod
>Kernel panic: can't allocate root vfsmount
> <1>Unable to handle kernel NULL pointer dereference at virtual address 0000002c
>c01234d3
>*pde = 00000000
>Oops: 0000
>CPU: 0
>EIP: 0010:[<c01234d3>] Not tainted
>Using defaults from ksymoops -t elf32-i386 -a i386
>EFLAGS: 00010046
>eax: 00000000 ebx: 00000008 ecx: 00000000 edx: 00000073
>esi: 00000000 edi: 00000018 ebp: 00000020 esp: c0215e78
>ds: 0018 es: 0018 ss: 0018
>Process . (pid: -1072541344, stackpage=c0215000)
>Stack: 00000018 00000001 00000018 c0214568 c0117e6c 00000000 00000020 c0214000
> 00000018 00000018 00000000 c0117f4c 00000018 00000001 c0214568 00000086
> 00000018 c0214000 c0117ff4 00000018 00000001 c0214000 01ebb409 c0214000
>Call Trace: [<c0117e6c>] [<c0117f4c>] [<c0117ff4>] [<c0118253>] [<c0117208>]
> [<c0117291>] [<c011759f>] [<c010a889>] [<c0107d1c>] [<c0107e82>] [<c0105000>]
> [<c0109c18>] [<c0105000>] [<c0112020>]
>Code: f6 46 2c 01 74 02 0f 0b 9c 5f fa 8b 4e 08 39 d9 75 22 8b 4e
>
>>>EIP; c01234d2 <kmem_cache_alloc+2a/b8> <=====
>>>
>Trace; c0117e6c <send_signal+2c/f0>
>Trace; c0117f4c <deliver_signal+1c/50>
>Trace; c0117ff4 <send_sig_info+74/88>
>Trace; c0118252 <send_sig+1a/20>
>Trace; c0117208 <update_one_process+68/d4>
>Trace; c0117290 <update_process_times+1c/88>
>Trace; c011759e <do_timer+22/70>
>Trace; c010a888 <timer_interrupt+60/10c>
>Trace; c0107d1c <handle_IRQ_event+30/5c>
>Trace; c0107e82 <do_IRQ+6a/a8>
>Trace; c0105000 <_stext+0/0>
>Trace; c0109c18 <call_do_IRQ+6/e>
>Trace; c0105000 <_stext+0/0>
>Trace; c0112020 <panic+c0/d0>
>Code; c01234d2 <kmem_cache_alloc+2a/b8>
>00000000 <_EIP>:
>Code; c01234d2 <kmem_cache_alloc+2a/b8> <=====
> 0: f6 46 2c 01 testb $0x1,0x2c(%esi) <=====
>Code; c01234d6 <kmem_cache_alloc+2e/b8>
> 4: 74 02 je 8 <_EIP+0x8> c01234da <kmem_cache_alloc+32/b8>
>Code; c01234d8 <kmem_cache_alloc+30/b8>
> 6: 0f 0b ud2a
>Code; c01234da <kmem_cache_alloc+32/b8>
> 8: 9c pushf
>Code; c01234da <kmem_cache_alloc+32/b8>
> 9: 5f pop %edi
>Code; c01234dc <kmem_cache_alloc+34/b8>
> a: fa cli
>Code; c01234dc <kmem_cache_alloc+34/b8>
> b: 8b 4e 08 mov 0x8(%esi),%ecx
>Code; c01234e0 <kmem_cache_alloc+38/b8>
> e: 39 d9 cmp %ebx,%ecx
>Code; c01234e2 <kmem_cache_alloc+3a/b8>
> 10: 75 22 jne 34 <_EIP+0x34> c0123506 <kmem_cache_alloc+5e/b8>
>Code; c01234e4 <kmem_cache_alloc+3c/b8>
> 12: 8b 4e 00 mov 0x0(%esi),%ecx
>
> <0>Kernel panic: Aiee, killing interrupt handler!
>


2002-01-23 19:44:24

by Andrew Morton

[permalink] [raw]
Subject: Re: Low latency for recent kernels

J Sloan wrote:
>
> Ditto here -
>

Yes, sorry about that. That patch had a completely untested,
experimental and buggy chunk in it which kinda escaped from the
factory.

I've uploaded a saner version.

-

2002-01-23 21:03:08

by Mauricio Nuñez

[permalink] [raw]
Subject: Re: Low latency for recent kernels

Hi,

I'm sending some feedback !
:-)

I'm trying this patch... (in 2.4.18-pre6)
I'm feeling a improved performance ,as a desktop user.
I'm working with KDE2, Netbeans and VMWare.
Pentium III 666Mhz , 192M Ram, 10GB HD.

What are the tools to check this better performance?
Or my impression is sufficient ?

Thanks

Mauricio

El Mi?rcoles 23 Enero 2002 16:37, Andrew Morton escribi?:
> J Sloan wrote:
> > Ditto here -
>
> Yes, sorry about that. That patch had a completely untested,
> experimental and buggy chunk in it which kinda escaped from the
> factory.
>
> I've uploaded a saner version.
>
> -
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/

2002-01-23 21:44:34

by Andrew Morton

[permalink] [raw]
Subject: Re: Low latency for recent kernels

Mauricio Nu?ez wrote:
>
> Hi,
>
> I'm sending some feedback !
> :-)

Is good. Thanks.

> I'm trying this patch... (in 2.4.18-pre6)
> I'm feeling a improved performance ,as a desktop user.
> I'm working with KDE2, Netbeans and VMWare.
> Pentium III 666Mhz , 192M Ram, 10GB HD.

I'm a little surprised that desktop users do notice significant
benefits with all the latency/preempt patches. If you actually
instrument the kernel's behaviour, the stalls are in fact
quite small and infrequent. See the histograms in

http://www.uwsg.iu.edu/hypermail/linux/kernel/0201.0/1624.html

Probably, poor interactivity on the desktop is more due to
waiting on disk reads - a combination of bad read latency
in the presence of write traffic and unfortunate page replacement
decisions.

Try http://www.zipworld.com.au/~akpm/linux/2.4/2.4.18-pre6/read-latency2.patch

> What are the tools to check this better performance?
> Or my impression is sufficient ?

The simplest tool to use is Mark Hahn's `realfeel' app. See
http://www.zip.com.au/~akpm/linux/amlat.tar.gz

-

2002-01-24 03:48:36

by Dan Maas

[permalink] [raw]
Subject: Re: Low latency for recent kernels

> I'm a little surprised that desktop users do notice significant
> benefits with all the latency/preempt patches. If you actually
> instrument the kernel's behaviour, the stalls are in fact
> quite small and infrequent.

Havoc Pennington, Soeren Sandmann, and I have been investigating causes of
UI unresponsiveness in Xfree86/Linux. I would agree that in most situations,
on a mostly-idle machine, low-latency/preempt patches should *not* enhance
the overall feel of the desktop. (if they do measurably increase
responsiveness, then that would suggest inefficiencies in X/the WM/the X
client - a definite possibility, of course).

Two situations where I would expect low-latency/preemption to have a
positive effect on responsiveness are 1) when the system is under heavy CPU
and disk load (e.g. kernel compile); due to the interactive tasks being able
to run earlier/more often, and 2) when performing UI operations that depend
on tight synchronization between X/the WM/the X client, particularly opaque
window resizing. (my theory is that low-latency/preemption results in the
CPU switching more rapidly or evenly among these processes, reducing the
perceptible "lag" between the client window and its WM frame)

Regards,
Dan

2002-01-24 04:13:28

by Robert Love

[permalink] [raw]
Subject: Re: Low latency for recent kernels

On Wed, 2002-01-23 at 22:48, Dan Maas wrote:

> Two situations where I would expect low-latency/preemption to have a
> positive effect on responsiveness are 1) when the system is under heavy CPU
> and disk load (e.g. kernel compile); due to the interactive tasks being able
> to run earlier/more often, and 2) when performing UI operations that depend
> on tight synchronization between X/the WM/the X client, particularly opaque
> window resizing. (my theory is that low-latency/preemption results in the
> CPU switching more rapidly or evenly among these processes, reducing the
> perceptible "lag" between the client window and its WM frame)

This is exactly the area preempt/low-latency helps and I think your
theory is pretty much dead on.

With preempt-kernel, ideally, an interactive task finds itself runnable
like this: user event causes interrupt, interrupt sets need_resched, on
return from interrupt we cause a preemption of current task (which can
happen whether task is in kernel or userland, now), and schedule the
interactive task onto the CPU.

This leads to better scheduling fairness and short scheduling latency.

If you or Havoc are interested in any tests or further work with the
preemptive kernel, I'd be more than willing. Hey, I use GNOME ;)

Robert Love

2002-01-24 04:21:30

by Glendon Gross

[permalink] [raw]
Subject: Re: Low latency for recent kernels


Is this patch available for 2.5.2 ? or is it already part of the tree?

On 23 Jan 2002, Robert Love wrote:

> On Wed, 2002-01-23 at 22:48, Dan Maas wrote:
>
> > Two situations where I would expect low-latency/preemption to have a
> > positive effect on responsiveness are 1) when the system is under heavy CPU
> > and disk load (e.g. kernel compile); due to the interactive tasks being able
> > to run earlier/more often, and 2) when performing UI operations that depend
> > on tight synchronization between X/the WM/the X client, particularly opaque
> > window resizing. (my theory is that low-latency/preemption results in the
> > CPU switching more rapidly or evenly among these processes, reducing the
> > perceptible "lag" between the client window and its WM frame)
>
> This is exactly the area preempt/low-latency helps and I think your
> theory is pretty much dead on.
>
> With preempt-kernel, ideally, an interactive task finds itself runnable
> like this: user event causes interrupt, interrupt sets need_resched, on
> return from interrupt we cause a preemption of current task (which can
> happen whether task is in kernel or userland, now), and schedule the
> interactive task onto the CPU.
>
> This leads to better scheduling fairness and short scheduling latency.
>
> If you or Havoc are interested in any tests or further work with the
> preemptive kernel, I'd be more than willing. Hey, I use GNOME ;)
>
> Robert Love
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>

2002-01-24 04:24:30

by Robert Love

[permalink] [raw]
Subject: Re: Low latency for recent kernels

On Wed, 2002-01-23 at 23:20, Glendon Gross wrote:

> Is this patch available for 2.5.2 ? or is it already part of the tree?

No, its not in 2.5 at the moment. 2.5.2 patch is available at:

ftp://ftp.kernel.org/pub/linux/kernel/people/rml/preempt-kernel/v2.5

Robert Love

2002-01-24 06:35:23

by Glendon Gross

[permalink] [raw]
Subject: Re: Low latency for recent kernels


I was able to build kernel 2.5.2 after applying the patch, but only
after editing out a line from /usr/src/linux/arch/i386/kernel/i387.c.
After applying the patch, the system does seem a little faster. (This
is a P-II/233 with 512k cache.)

Prior to the edit, I got this:

Script started on Wed Jan 23 21:58:40 2002
gross@mail:/usr/src/2.5.2/linux > make bzImage
. scripts/mkversion > .tmpversion
gcc -D__KERNEL__ -I/usr/src/2.5.2/linux/include -Wall -Wstrict-prototypes -Wno-trigraphs -O2 -fomit-frame-pointer -fno-strict-aliasing -fno-common -pipe -mpreferred-stack-boundary=2 -march=i686 -DUTS_MACHINE='"i386"' -c -o init/version.o init/version.c
make CFLAGS="-D__KERNEL__ -I/usr/src/2.5.2/linux/include -Wall -Wstrict-prototypes -Wno-trigraphs -O2 -fomit-frame-pointer -fno-strict-aliasing -fno-common -pipe -mpreferred-stack-boundary=2 -march=i686 " -C kernel
make[1]: Entering directory `/usr/src/2.5.2/linux/kernel'
make all_targets

(snip)

make[2]: Entering directory `/usr/src/2.5.2/linux/kernel'
make[1]: Leaving directory `/usr/src/2.5.2/linux/arch/i386/math-emu'
ld -m elf_i386 -T /usr/src/2.5.2/linux/arch/i386/vmlinux.lds -e stext arch/i386/kernel/head.o arch/i386/kernel/init_task.o init/main.o init/version.o init/do_mounts.o \
--start-group \
arch/i386/kernel/kernel.o arch/i386/mm/mm.o kernel/kernel.o mm/mm.o fs/fs.o ipc/ipc.o \
/usr/src/2.5.2/linux/arch/i386/lib/lib.a /usr/src/2.5.2/linux/lib/lib.a /usr/src/2.5.2/linux/arch/i386/lib/lib.a \
drivers/acpi/acpi.o drivers/parport/driver.o drivers/char/char.o drivers/block/block.o drivers/misc/misc.o drivers/net/net.o drivers/media/media.o drivers/char/agp/agp.o drivers/char/drm/drm.o drivers/atm/atm.o drivers/ide/idedriver.o drivers/scsi/scsidrv.o drivers/cdrom/driver.o drivers/pci/driver.o drivers/net/pcmcia/pcmcia_net.o drivers/pnp/pnp.o drivers/video/video.o drivers/net/hamradio/hamradio.o drivers/telephony/telephony.o arch/i386/math-emu/math.o \
net/network.o \
--end-group \
-o vmlinux
arch/i386/kernel/kernel.o: In function `kernel_fpu_begin':
arch/i386/kernel/kernel.o(.text+0x7ccd): undefined reference to `preempt_disable'
make: *** [vmlinux] Error 1
gross@mail:/usr/src/2.5.2/linux > exit

Script done on Wed Jan 23 22:01:56 2002


So I looked at i387.c and commented out this line:

Script started on Wed Jan 23 22:31:34 2002
gross@mail:/usr/src/linux/arch/i386/kernel > pwd
/usr/src/linux/arch/i386/kernel
gross@mail:/usr/src/linux/arch/i386/kernel > grep preempt i387.c
/* preempt_disable(); */
gross@mail:/usr/src/linux/arch/i386/kernel > exit

Script done on Wed Jan 23 22:31:51 2002


On 23 Jan 2002, Robert Love wrote:

> On Wed, 2002-01-23 at 23:20, Glendon Gross wrote:
>
> > Is this patch available for 2.5.2 ? or is it already part of the tree?
>
> No, its not in 2.5 at the moment. 2.5.2 patch is available at:
>
> ftp://ftp.kernel.org/pub/linux/kernel/people/rml/preempt-kernel/v2.5
>
> Robert Love
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>

2002-01-24 18:19:57

by Daniel Phillips

[permalink] [raw]
Subject: Re: Low latency for recent kernels

On January 23, 2002 10:37 pm, Andrew Morton wrote:
> Probably, poor interactivity on the desktop is more due to
> waiting on disk reads - a combination of bad read latency
> in the presence of write traffic and unfortunate page replacement
> decisions.

Yep, and half-formed ideas of process scheduling. The good news is, it's all
being worked on by people who know what they're doing (including, obviously,
you).

--
Daniel