2006-05-10 20:59:41

by Mark Hedges

[permalink] [raw]
Subject: unknown io writes in 2.6.16


Apologies if this is out of place, but I can't find anyone else
who might know about this.

I'm using 2.6.16 (built from Debian unstable linux-source-2.6.16).
It's a pretty generic system with ext3 partitions.

I stop every non-kernel process except syslogd, klogd and the
tty's. Interfaces are down. It is still in default runlevel.
But the disk keeps clicking away.

`iostat` reports about 12 bytes written out to my root partition
every 5 seconds or so. But `top` and/or `sar -x ALL 1 10`
report no noticible process activity beyond what they need to
run, and there is no file on the partition that has its mtime
modified.

selinux is not compiled in the kernel.

Any idea what's doing these writes?

Is there any way to view disk i/o by process ID?

Mark


2006-05-10 21:42:26

by David Rees

[permalink] [raw]
Subject: Re: unknown io writes in 2.6.16

On 5/10/06, Mark Hedges <[email protected]> wrote:
> I stop every non-kernel process except syslogd, klogd and the
> tty's. Interfaces are down. It is still in default runlevel.
> But the disk keeps clicking away.
>
> Any idea what's doing these writes?

It's most likely atime updates. Mount the partitions with noatime
option and your writes will go away.

-Dave

2006-05-11 05:54:46

by Mark Hedges

[permalink] [raw]
Subject: Re: unknown io writes in 2.6.16


On Wed, 10 May 2006, David Rees wrote:
>
> On 5/10/06, Mark Hedges <[email protected]> wrote:
> > I stop every non-kernel process except syslogd, klogd and the
> > tty's. Interfaces are down. It is still in default runlevel.
> > But the disk keeps clicking away.
> >
> > Any idea what's doing these writes?
>
> It's most likely atime updates. Mount the partitions with noatime
> option and your writes will go away.

Yes, that's it. Never noticed this much with IDE - this is my
first SATA system.

Just a wishlist that process IO could be monitored. I hate to
say it but ctl-alt-esc in W2K can monitor io by process, and
that's really useful. (I will never go back though.)

Or is this something that could be reported with the system call
auditing infrastructure?

Mark

2006-05-11 12:46:23

by Giacomo Catenazzi

[permalink] [raw]
Subject: Re: unknown io writes in 2.6.16

Mark Hedges wrote:
> Just a wishlist that process IO could be monitored. I hate to
> say it but ctl-alt-esc in W2K can monitor io by process, and
> that's really useful. (I will never go back though.)

vmstat ?

ciao
cate

2006-05-11 13:23:24

by Avi Kivity

[permalink] [raw]
Subject: Re: unknown io writes in 2.6.16

Giacomo A. Catenazzi wrote:
> Mark Hedges wrote:
>> Just a wishlist that process IO could be monitored. I hate to say it
>> but ctl-alt-esc in W2K can monitor io by process, and that's really
>> useful. (I will never go back though.)
>
> vmstat ?

He's looking for per process stats.

--
error compiling committee.c: too many arguments to function

2006-05-11 15:22:48

by Erik Mouw

[permalink] [raw]
Subject: Re: unknown io writes in 2.6.16

On Wed, May 10, 2006 at 10:54:43PM -0700, Mark Hedges wrote:
> Just a wishlist that process IO could be monitored. I hate to
> say it but ctl-alt-esc in W2K can monitor io by process, and
> that's really useful. (I will never go back though.)

IIRC this can be done with Jens Axboe's blktrace. It got merged in
2.6.17, enable CONFIG_BLK_DEV_IO_TRACE and get the blktrace userspace
tools from git://brick.kernel.dk/data/git/blktrace.git .


Erik

--
+-- Erik Mouw -- http://www.harddisk-recovery.com -- +31 70 370 12 90 --
| Lab address: Delftechpark 26, 2628 XH, Delft, The Netherlands

2006-05-12 17:17:21

by dean gaudet

[permalink] [raw]
Subject: Re: unknown io writes in 2.6.16

On Wed, 10 May 2006, Mark Hedges wrote:

> Just a wishlist that process IO could be monitored. I hate to
> say it but ctl-alt-esc in W2K can monitor io by process, and
> that's really useful. (I will never go back though.)

/etc/init.d/klogd stop
echo 1 >/proc/sys/vm/block_dump
/sbin/klogd -n -f -

watch lots of spew

^C
echo 0 >/proc/sys/vm/block_dump
/etc/init.d/klogd start

that will tell you a lot of stuff. it will probably lead you to a process
or two as culprits... use strace(8) to attach to those and get more
specific information.

now... share with me how to do this on windoze? :) i'm trying to get rid
of the last write or two on my laptop...

-dean

2006-05-16 00:12:40

by Mark Hedges

[permalink] [raw]
Subject: Re: unknown io writes in 2.6.16



On Fri, 12 May 2006, dean gaudet wrote:
> On Wed, 10 May 2006, Mark Hedges wrote:
>
> > Just a wishlist that process IO could be monitored. I hate to
> > say it but ctl-alt-esc in W2K can monitor io by process, and
> > that's really useful. (I will never go back though.)
>
> /etc/init.d/klogd stop
> echo 1 >/proc/sys/vm/block_dump
> /sbin/klogd -n -f -
>
> watch lots of spew
>
> ^C
> echo 0 >/proc/sys/vm/block_dump
> /etc/init.d/klogd start
>
> that will tell you a lot of stuff. it will probably lead you to a process
> or two as culprits... use strace(8) to attach to those and get more
> specific information.
>
> now... share with me how to do this on windoze? :) i'm trying to get rid
> of the last write or two on my laptop...

Cool, thanks, I'll check this out. It's actually about 12k
every 5 seconds, not 12 bytes. Seems excessive for atime
updates.

In W2K at any rate, ctl-shift-esc to get the task manager.
Processes tab. View Menu > Select Columns. Sort by column.

Mark

2006-05-16 00:48:32

by dean gaudet

[permalink] [raw]
Subject: Re: unknown io writes in 2.6.16

On Mon, 15 May 2006, Mark Hedges wrote:

> In W2K at any rate, ctl-shift-esc to get the task manager.
> Processes tab. View Menu > Select Columns. Sort by column.

heh i already know about that... it's fairly useless though when the disk
writes are due to logged events going through the same services dll... i
was looking for something a little more detailed... like the block_dump +
strace.

-dean

2006-05-16 01:22:04

by Con Kolivas

[permalink] [raw]
Subject: Re: unknown io writes in 2.6.16

On Tuesday 16 May 2006 10:12, Mark Hedges wrote:
> Cool, thanks, I'll check this out. It's actually about 12k
> every 5 seconds, not 12 bytes. Seems excessive for atime
> updates.

That just sounds like the journal updating... the default journal time is 5
seconds. Try transiently disabling the journal to see if that's it:

mount -o remount,noload /mountpoint

Don't forget to re-enable it afterwards.

You could set laptop mode if the writeouts are too frequent for your liking:

echo 1 > /proc/sys/vm/laptop_mode

--
-ck