2007-08-23 00:51:07

by Joel Fuster

[permalink] [raw]
Subject: sysfs_dir_cache growing out of control

Hi,

I am running 2.6.22.3. For reasons that escape me, over time (days) the
sysfs_dir_cache, dentry, and inode_cache SLUB entries grow until they
consume all the memory on my system, requiring a reboot.

Although I did not record objective evidence of this at the time, I
suspect I had the same problem with 2.6.18 and 2.6.20...I certainly had
the same symptoms.

Here is some hopefully useful information. Let me know what else would
be helpful.

Thanks.

P.S. Please keep me CC'd as I do not subscribe to the list.

Linux periphas 2.6.22.3 #1 SMP Mon Aug 20 21:47:51 EDT 2007 x86_64 GNU/Linux

*************************************
18:30:02 up 21:27, 4 users, load average: 0.22, 0.36, 0.35
MemTotal: 1026056 kB
MemFree: 8996 kB
Buffers: 176 kB
Cached: 209524 kB
SwapCached: 32308 kB
Active: 610448 kB
Inactive: 170004 kB
SwapTotal: 1048568 kB
SwapFree: 710260 kB
Dirty: 2444 kB
Writeback: 0 kB
AnonPages: 568988 kB
Mapped: 34360 kB
Slab: 210664 kB
SReclaimable: 179460 kB
SUnreclaim: 31204 kB
PageTables: 10008 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
CommitLimit: 1561596 kB
Committed_AS: 1184628 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 1032 kB
VmallocChunk: 34359737307 kB
Name Objects Objsize Space Slabs/Part/Cpu O/S O
%Fr %Ef Flg
:0000016 2937 16 65.5K 16/5/2 256 0
31 71 *
:0000024 1222 24 32.7K 8/0/2 170 0
0 89 *
:0000032 2095 32 73.7K 18/3/2 128 0
16 90 *
:0000040 1862 40 172.0K 42/31/2 102 0
73 43 *
:0000064 4651 64 434.1K 106/46/2 64 0
43 68 *
:0000072 81 72 8.1K 2/0/2 56 0
0 71 *
:0000096 937 96 135.1K 33/14/2 42 0
42 66 *
:0000112 144 112 16.3K 4/0/1 36 0
0 98 *
:0000128 595 128 118.7K 29/19/2 32 0
65 64 *
:0000192 807 192 262.1K 64/36/2 21 0
56 59 *
:0000256 3504 256 897.0K 219/0/2 16 0
0 100 *
:0000320 47 320 16.3K 4/1/2 12 0
25 91 *A
:0000384 56 384 28.6K 7/1/2 10 0
14 75 *A
:0000512 238 512 135.1K 33/7/2 8 0
21 90 *
:0000704 133 704 114.6K 14/7/2 11 1
50 81 *A
:0000768 180 744 159.7K 39/9/2 5 0
23 83 *A
:0000832 90 832 114.6K 14/11/2 9 1
78 65 *A
:0001024 370 1024 393.2K 96/9/2 4 0
9 96 *
:0002048 474 2048 974.8K 119/1/2 4 1
0 99 *
:0004096 114 4096 483.3K 59/2/2 2 1
3 96 *
Acpi-State 102 80 8.1K 2/0/2 51 0
0 99
anon_vma 2561 24 90.1K 22/10/2 128 0
45 68
bdev_cache 53 736 45.0K 11/0/2 5 0
0 86 Aa
blkdev_queue 45 1480 73.7K 9/0/2 5 1
0 90
blkdev_requests 42 280 16.3K 4/0/2 14 0
0 71
buffer_head 2695 104 327.6K 80/25/2 39 0
31 85 a
cfq_io_context 189 152 32.7K 8/3/2 26 0
37 87
cfq_queue 203 144 32.7K 8/4/2 28 0
50 89
dentry 228340 200 46.7M 11418/0/2 20 0
0 97 a
ext3_inode_cache 16 736 24.5K 6/4/2 5 0
66 47 a
file_lock_cache 46 176 16.3K 4/2/2 22 0
50 49
idr_layer_cache 135 528 81.9K 20/1/2 7 0
5 87
inode_cache 224448 536 131.3M 32065/0/2 7 0
0 91 a
kmalloc-16384 8 16384 163.8K 10/0/2 1 2
0 80
kmalloc-32768 34 32768 1.1M 34/0/1 1 3
0 100
kmalloc-65536 3 65536 196.6K 3/0/2 1 4
0 100
kmalloc-8 893 8 8.1K 2/0/2 512 0
0 87
kmalloc-8192 14 8192 131.0K 16/0/2 1 1
0 87
kmalloc_dma-512 8 512 4.0K 1/0/1 8 0
0 100 d
mqueue_inode_cache 9 824 8.1K 1/0/1 9 1
0 90 A
proc_inode_cache 28 568 40.9K 10/8/2 7 0
80 38 a
radix_tree_node 2759 552 2.2M 544/243/2 7 0
44 68
raid5-md2 259 584 151.5K 37/0/1 7 0
0 99
shmem_inode_cache 524 728 434.1K 106/5/2 5 0
4 87
sighand_cache 135 2080 376.8K 46/2/2 3 1
4 74 A
sigqueue 8 160 8.1K 2/0/2 25 0
0 15
sock_inode_cache 261 616 192.5K 47/8/2 6 0
17 83 Aa
sysfs_dir_cache 229196 88 20.4M 4984/2/2 46 0
0 98
task_struct 152 1696 327.6K 40/6/2 4 1
15 78
TCP 27 1496 57.3K 7/2/2 5 1
28 70 A
UNIX 218 640 155.6K 38/5/2 6 0
13 89 A
vm_area_struct 7107 168 1.2M 300/19/2 24 0
6 97
xfs_acl 14 304 8.1K 2/0/2 13 0
0 51
xfs_buf_item 30 184 8.1K 2/0/2 22 0
0 67
xfs_da_state 8 488 8.1K 2/0/2 8 0
0 47
xfs_efd_item 11 360 8.1K 2/0/2 11 0
0 48
xfs_efi_item 12 352 8.1K 2/0/2 11 0
0 51
xfs_inode 4634 544 2.7M 662/0/2 7 0
0 92 Aa
xfs_vnode 4638 576 3.1M 773/0/2 6 0
0 84 Aa





2007-08-23 03:56:58

by Joel Fuster

[permalink] [raw]
Subject: Re: sysfs_dir_cache growing out of control

Joel Fuster wrote:
> Hi,
>
> I am running 2.6.22.3. For reasons that escape me, over time (days) the
> sysfs_dir_cache, dentry, and inode_cache SLUB entries grow until they
> consume all the memory on my system, requiring a reboot.
>
> Although I did not record objective evidence of this at the time, I
> suspect I had the same problem with 2.6.18 and 2.6.20...I certainly had
> the same symptoms.

An update to this...it seems to be related to "scanbuttond" which is an
app that uses libusb to poll a scanner for button presses on the scanner
itself. The number of objects in sysfs_dir_cache stops growing when I
kill the scanbuttond process.

An strace of one poll loop for scanbuttond follows:


nanosleep({0, 333000000}, NULL) = 0
open("/dev/bus/usb", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = 1
fstat(1, {st_mode=S_IFDIR|0755, st_size=80, ...}) = 0
fcntl(1, F_SETFD, FD_CLOEXEC) = 0
getdents(1, /* 4 entries */, 4096) = 96
getdents(1, /* 0 entries */, 4096) = 0
close(1) = 0
open("/dev/bus/usb/002", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = 1
fstat(1, {st_mode=S_IFDIR|0755, st_size=60, ...}) = 0
fcntl(1, F_SETFD, FD_CLOEXEC) = 0
getdents(1, /* 3 entries */, 4096) = 72
open("/dev/bus/usb/002/001", O_RDWR) = -1 EACCES (Permission denied)
open("/dev/bus/usb/002/001", O_RDONLY) = 2
ioctl(2, USBDEVFS_CONNECTINFO, 0x7fffb3a08420) = -1 EPERM (Operation not
permitted)
read(2, "\22\1\0\2\t\0\1@\0\0\0\0\6\2\3\2\1\1", 18) = 18
read(2, "\t\2\31\0\1\1\0\340", 8) = 8
read(2, "\0\t\4\0\0\1\t\0\0\0\7\5\201\3\4\0\f", 17) = 17
close(2) = 0
getdents(1, /* 0 entries */, 4096) = 0
close(1) = 0
open("/dev/bus/usb/002/001", O_RDWR) = -1 EACCES (Permission denied)
open("/dev/bus/usb/002/001", O_RDONLY) = 1
ioctl(1, USBDEVFS_IOCTL, 0x7fffb3a08420) = -1 EPERM (Operation not
permitted)
close(1) = 0
open("/dev/bus/usb/001", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = 1
fstat(1, {st_mode=S_IFDIR|0755, st_size=100, ...}) = 0
fcntl(1, F_SETFD, FD_CLOEXEC) = 0
getdents(1, /* 4 entries */, 4096) = 96
getdents(1, /* 0 entries */, 4096) = 0
close(1) = 0
open("/dev/bus/usb/002", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = 1
fstat(1, {st_mode=S_IFDIR|0755, st_size=60, ...}) = 0
fcntl(1, F_SETFD, FD_CLOEXEC) = 0
getdents(1, /* 3 entries */, 4096) = 72
open("/dev/bus/usb/002/001", O_RDWR) = -1 EACCES (Permission denied)
open("/dev/bus/usb/002/001", O_RDONLY) = 2
ioctl(2, USBDEVFS_CONNECTINFO, 0x7fffb3a08420) = -1 EPERM (Operation not
permitted)
read(2, "\22\1\0\2\t\0\1@\0\0\0\0\6\2\3\2\1\1", 18) = 18
read(2, "\t\2\31\0\1\1\0\340", 8) = 8
read(2, "\0\t\4\0\0\1\t\0\0\0\7\5\201\3\4\0\f", 17) = 17
close(2) = 0
getdents(1, /* 0 entries */, 4096) = 0
close(1) = 0
open("/dev/bus/usb/002/001", O_RDWR) = -1 EACCES (Permission denied)
open("/dev/bus/usb/002/001", O_RDONLY) = 1
ioctl(1, USBDEVFS_IOCTL, 0x7fffb3a08420) = -1 EPERM (Operation not
permitted)
close(1) = 0
open("/dev/bus/usb/001", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = 1
fstat(1, {st_mode=S_IFDIR|0755, st_size=100, ...}) = 0
fcntl(1, F_SETFD, FD_CLOEXEC) = 0
getdents(1, /* 5 entries */, 4096) = 120
open("/dev/bus/usb/001/004", O_RDWR) = 2
ioctl(2, USBDEVFS_CONNECTINFO, 0x7fffb3a08420) = 0
read(2, "\22\1\20\1\0\0\0@\330\5\3@\0\1\0\0\0\1", 18) = 18
read(2, "\t\2 \0\1\1\0\240", 8) = 8
read(2, "\372\t\4\0\0\2\377\377\377\0\7\5\201\2@\0\0\7\5\2\2@\0"..., 24)
= 24
close(2) = 0
open("/dev/bus/usb/001/003", O_RDWR) = -1 EACCES (Permission denied)
open("/dev/bus/usb/001/003", O_RDONLY) = 2
ioctl(2, USBDEVFS_CONNECTINFO, 0x7fffb3a08420) = -1 EPERM (Operation not
permitted)
read(2, "\22\1\20\1\0\0\0@{\6\3#\0\3\1\2\0\1", 18) = 18
read(2, "\t\2\'\0\1\1\0\200", 8) = 8
read(2, "2\t\4\0\0\3\377\0\0\0\7\5\201\3\n\0\1\7\5\2\2@\0\0\7\5"..., 31)
= 31
close(2) = 0
open("/dev/bus/usb/001/001", O_RDWR) = -1 EACCES (Permission denied)
open("/dev/bus/usb/001/001", O_RDONLY) = 2
ioctl(2, USBDEVFS_CONNECTINFO, 0x7fffb3a08420) = -1 EPERM (Operation not
permitted)
read(2, "\22\1\20\1\t\0\0@\0\0\0\0\6\2\3\2\1\1", 18) = 18
read(2, "\t\2\31\0\1\1\0\340", 8) = 8
read(2, "\0\t\4\0\0\1\t\0\0\0\7\5\201\3\2\0\377", 17) = 17
close(2) = 0
getdents(1, /* 0 entries */, 4096) = 0
close(1) = 0
open("/dev/bus/usb/001/004", O_RDWR) = 1
ioctl(1, USBDEVFS_IOCTL, 0x7fffb3a08420) = -1 ENOTTY (Inappropriate
ioctl for device)
close(1) = 0
open("/dev/bus/usb/001/003", O_RDWR) = -1 EACCES (Permission denied)
open("/dev/bus/usb/001/003", O_RDONLY) = 1
ioctl(1, USBDEVFS_IOCTL, 0x7fffb3a08420) = -1 EPERM (Operation not
permitted)
close(1) = 0
open("/dev/bus/usb/001/001", O_RDWR) = -1 EACCES (Permission denied)
open("/dev/bus/usb/001/001", O_RDONLY) = 1
ioctl(1, USBDEVFS_IOCTL, 0x7fffb3a08420) = -1 EPERM (Operation not
permitted)
close(1) = 0
open("/dev/bus/usb", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = 1
fstat(1, {st_mode=S_IFDIR|0755, st_size=80, ...}) = 0
fcntl(1, F_SETFD, FD_CLOEXEC) = 0
getdents(1, /* 4 entries */, 4096) = 96
getdents(1, /* 0 entries */, 4096) = 0
close(1) = 0
open("/dev/bus/usb/002", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = 1
fstat(1, {st_mode=S_IFDIR|0755, st_size=60, ...}) = 0
fcntl(1, F_SETFD, FD_CLOEXEC) = 0
getdents(1, /* 3 entries */, 4096) = 72
open("/dev/bus/usb/002/001", O_RDWR) = -1 EACCES (Permission denied)
open("/dev/bus/usb/002/001", O_RDONLY) = 2
ioctl(2, USBDEVFS_CONNECTINFO, 0x7fffb3a08420) = -1 EPERM (Operation not
permitted)
read(2, "\22\1\0\2\t\0\1@\0\0\0\0\6\2\3\2\1\1", 18) = 18
read(2, "\t\2\31\0\1\1\0\340", 8) = 8
read(2, "\0\t\4\0\0\1\t\0\0\0\7\5\201\3\4\0\f", 17) = 17
close(2) = 0
getdents(1, /* 0 entries */, 4096) = 0
close(1) = 0
open("/dev/bus/usb/002/001", O_RDWR) = -1 EACCES (Permission denied)
open("/dev/bus/usb/002/001", O_RDONLY) = 1
ioctl(1, USBDEVFS_IOCTL, 0x7fffb3a08420) = -1 EPERM (Operation not
permitted)
close(1) = 0
open("/dev/bus/usb/001", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = 1
fstat(1, {st_mode=S_IFDIR|0755, st_size=100, ...}) = 0
fcntl(1, F_SETFD, FD_CLOEXEC) = 0
getdents(1, /* 5 entries */, 4096) = 120
open("/dev/bus/usb/001/004", O_RDWR) = 2
ioctl(2, USBDEVFS_CONNECTINFO, 0x7fffb3a08420) = 0
read(2, "\22\1\20\1\0\0\0@\330\5\3@\0\1\0\0\0\1", 18) = 18
read(2, "\t\2 \0\1\1\0\240", 8) = 8
read(2, "\372\t\4\0\0\2\377\377\377\0\7\5\201\2@\0\0\7\5\2\2@\0"..., 24)
= 24
close(2) = 0
open("/dev/bus/usb/001/003", O_RDWR) = -1 EACCES (Permission denied)
open("/dev/bus/usb/001/003", O_RDONLY) = 2
ioctl(2, USBDEVFS_CONNECTINFO, 0x7fffb3a08420) = -1 EPERM (Operation not
permitted)
read(2, "\22\1\20\1\0\0\0@{\6\3#\0\3\1\2\0\1", 18) = 18
read(2, "\t\2\'\0\1\1\0\200", 8) = 8
read(2, "2\t\4\0\0\3\377\0\0\0\7\5\201\3\n\0\1\7\5\2\2@\0\0\7\5"..., 31)
= 31
close(2) = 0
open("/dev/bus/usb/001/001", O_RDWR) = -1 EACCES (Permission denied)
open("/dev/bus/usb/001/001", O_RDONLY) = 2
ioctl(2, USBDEVFS_CONNECTINFO, 0x7fffb3a08420) = -1 EPERM (Operation not
permitted)
read(2, "\22\1\20\1\t\0\0@\0\0\0\0\6\2\3\2\1\1", 18) = 18
read(2, "\t\2\31\0\1\1\0\340", 8) = 8
read(2, "\0\t\4\0\0\1\t\0\0\0\7\5\201\3\2\0\377", 17) = 17
close(2) = 0
getdents(1, /* 0 entries */, 4096) = 0
close(1) = 0
open("/dev/bus/usb/001/004", O_RDWR) = 1
ioctl(1, USBDEVFS_IOCTL, 0x7fffb3a08420) = -1 ENOTTY (Inappropriate
ioctl for device)
close(1) = 0
open("/dev/bus/usb/001/003", O_RDWR) = -1 EACCES (Permission denied)
open("/dev/bus/usb/001/003", O_RDONLY) = 1
ioctl(1, USBDEVFS_IOCTL, 0x7fffb3a08420) = -1 EPERM (Operation not
permitted)
close(1) = 0
open("/dev/bus/usb/001/001", O_RDWR) = -1 EACCES (Permission denied)
open("/dev/bus/usb/001/001", O_RDONLY) = 1
ioctl(1, USBDEVFS_IOCTL, 0x7fffb3a08420) = -1 EPERM (Operation not
permitted)
close(1) = 0
open("/dev/bus/usb/001/004", O_RDWR) = 1
ioctl(1, USBDEVFS_CLAIMINTERFACE, 0x7fffb3a0848c) = 0
ioctl(1, USBDEVFS_CONTROL, 0x7fffb3a08420) = 64
ioctl(1, USBDEVFS_CONTROL, 0x7fffb3a08420) = 8
ioctl(1, USBDEVFS_RELEASEINTERFACE, 0x7fffb3a08494) = 0
close(1) = 0
nanosleep({0, 333000000}, <unfinished ...>


2007-08-23 10:05:54

by Greg KH

[permalink] [raw]
Subject: Re: sysfs_dir_cache growing out of control

On Wed, Aug 22, 2007 at 11:56:44PM -0400, Joel Fuster wrote:
> Joel Fuster wrote:
>> Hi,
>> I am running 2.6.22.3. For reasons that escape me, over time (days) the
>> sysfs_dir_cache, dentry, and inode_cache SLUB entries grow until they
>> consume all the memory on my system, requiring a reboot.

Hm, those items should consume all the memory, but it should be freed if
you have memory pressure from other places. Does it cause the machine
to lock up, or you just got scared when seeing them?

Oh, and does the same thing happen if you do not use SLUB, but rather
the older SLAB?

>> Although I did not record objective evidence of this at the time, I
>> suspect I had the same problem with 2.6.18 and 2.6.20...I certainly had
>> the same symptoms.
>
> An update to this...it seems to be related to "scanbuttond" which is an app
> that uses libusb to poll a scanner for button presses on the scanner
> itself. The number of objects in sysfs_dir_cache stops growing when I kill
> the scanbuttond process.
>
> An strace of one poll loop for scanbuttond follows:
>
>
> nanosleep({0, 333000000}, NULL) = 0
> open("/dev/bus/usb", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = 1
> fstat(1, {st_mode=S_IFDIR|0755, st_size=80, ...}) = 0
> fcntl(1, F_SETFD, FD_CLOEXEC) = 0
> getdents(1, /* 4 entries */, 4096) = 96
> getdents(1, /* 0 entries */, 4096) = 0
> close(1) = 0
> open("/dev/bus/usb/002", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = 1
> fstat(1, {st_mode=S_IFDIR|0755, st_size=60, ...}) = 0
> fcntl(1, F_SETFD, FD_CLOEXEC) = 0
> getdents(1, /* 3 entries */, 4096) = 72
> open("/dev/bus/usb/002/001", O_RDWR) = -1 EACCES (Permission denied)
> open("/dev/bus/usb/002/001", O_RDONLY) = 2
> ioctl(2, USBDEVFS_CONNECTINFO, 0x7fffb3a08420) = -1 EPERM (Operation not
> permitted)

<snip>

I don't see any sysfs accesses there, only usbfs accesses.

And it looks like a very dumb program, iterating over all usb devices
all the time? You would think that it would not do that, otherwise your
power consumption is going to be very high, and the odds that a new usb
device is going to show up that quickly is pretty low.

thanks,

greg k-h

2007-08-24 00:44:39

by Joel Fuster

[permalink] [raw]
Subject: Re: sysfs_dir_cache growing out of control

Greg KH wrote:
> On Wed, Aug 22, 2007 at 11:56:44PM -0400, Joel Fuster wrote:
>> Joel Fuster wrote:
>>> Hi,
>>> I am running 2.6.22.3. For reasons that escape me, over time (days) the
>>> sysfs_dir_cache, dentry, and inode_cache SLUB entries grow until they
>>> consume all the memory on my system, requiring a reboot.
>
> Hm, those items should consume all the memory, but it should be freed if
> you have memory pressure from other places. Does it cause the machine
> to lock up, or you just got scared when seeing them?
>
Right. The problem is that the memory never seems to get freed no
matter what I do. I've tried setting /proc/sys/vm/vfs_cache_pressure to
10000, but after a few days all my programs are running out of swap and
I have to reboot to get things back to a usable state.

> Oh, and does the same thing happen if you do not use SLUB, but rather
> the older SLAB?

OK I just rebuilt 2.6.22.3 with SLAB and I seem to be getting the same
result..obviously I haven't waited several days, but
sysfs_dir_cache/dentry/inode_cache grow continuously when scanbuttond is
running, and stop growing when it isn't.


>>
>> An strace of one poll loop for scanbuttond follows:
>>
>>
>> nanosleep({0, 333000000}, NULL) = 0
>> open("/dev/bus/usb", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = 1
>> fstat(1, {st_mode=S_IFDIR|0755, st_size=80, ...}) = 0
>> fcntl(1, F_SETFD, FD_CLOEXEC) = 0
>> getdents(1, /* 4 entries */, 4096) = 96
>> getdents(1, /* 0 entries */, 4096) = 0
>> close(1) = 0
>> open("/dev/bus/usb/002", O_RDONLY|O_NONBLOCK|O_DIRECTORY) = 1
>> fstat(1, {st_mode=S_IFDIR|0755, st_size=60, ...}) = 0
>> fcntl(1, F_SETFD, FD_CLOEXEC) = 0
>> getdents(1, /* 3 entries */, 4096) = 72
>> open("/dev/bus/usb/002/001", O_RDWR) = -1 EACCES (Permission denied)
>> open("/dev/bus/usb/002/001", O_RDONLY) = 2
>> ioctl(2, USBDEVFS_CONNECTINFO, 0x7fffb3a08420) = -1 EPERM (Operation not
>> permitted)
>
> <snip>
>
> I don't see any sysfs accesses there, only usbfs accesses.

Yes, I don't know enough to understand why this would affect
sysfs_dir_cache, but there definitely seems to be a connection.

Thanks for the help,

Joel

2007-08-24 01:00:11

by Greg KH

[permalink] [raw]
Subject: Re: sysfs_dir_cache growing out of control

On Thu, Aug 23, 2007 at 08:44:10PM -0400, Joel Fuster wrote:
> Greg KH wrote:
>> On Wed, Aug 22, 2007 at 11:56:44PM -0400, Joel Fuster wrote:
>>> Joel Fuster wrote:
>>>> Hi,
>>>> I am running 2.6.22.3. For reasons that escape me, over time (days) the
>>>> sysfs_dir_cache, dentry, and inode_cache SLUB entries grow until they
>>>> consume all the memory on my system, requiring a reboot.
>> Hm, those items should consume all the memory, but it should be freed if
>> you have memory pressure from other places. Does it cause the machine
>> to lock up, or you just got scared when seeing them?
> Right. The problem is that the memory never seems to get freed no matter
> what I do. I've tried setting /proc/sys/vm/vfs_cache_pressure to 10000,
> but after a few days all my programs are running out of swap and I have to
> reboot to get things back to a usable state.
>
>> Oh, and does the same thing happen if you do not use SLUB, but rather
>> the older SLAB?
>
> OK I just rebuilt 2.6.22.3 with SLAB and I seem to be getting the same
> result..obviously I haven't waited several days, but
> sysfs_dir_cache/dentry/inode_cache grow continuously when scanbuttond is
> running, and stop growing when it isn't.

Do you have a pointer to the scanbuttond source code? I'll try to take
a look at this tomorrow.

thanks,

greg k-h

2007-08-24 01:28:46

by Gabriel C

[permalink] [raw]
Subject: Re: sysfs_dir_cache growing out of control

Greg KH wrote:
> On Thu, Aug 23, 2007 at 08:44:10PM -0400, Joel Fuster wrote:
>> Greg KH wrote:
>>> On Wed, Aug 22, 2007 at 11:56:44PM -0400, Joel Fuster wrote:
>>>> Joel Fuster wrote:
>>>>> Hi,
>>>>> I am running 2.6.22.3. For reasons that escape me, over time (days) the
>>>>> sysfs_dir_cache, dentry, and inode_cache SLUB entries grow until they
>>>>> consume all the memory on my system, requiring a reboot.
>>> Hm, those items should consume all the memory, but it should be freed if
>>> you have memory pressure from other places. Does it cause the machine
>>> to lock up, or you just got scared when seeing them?
>> Right. The problem is that the memory never seems to get freed no matter
>> what I do. I've tried setting /proc/sys/vm/vfs_cache_pressure to 10000,
>> but after a few days all my programs are running out of swap and I have to
>> reboot to get things back to a usable state.
>>
>>> Oh, and does the same thing happen if you do not use SLUB, but rather
>>> the older SLAB?
>> OK I just rebuilt 2.6.22.3 with SLAB and I seem to be getting the same
>> result..obviously I haven't waited several days, but
>> sysfs_dir_cache/dentry/inode_cache grow continuously when scanbuttond is
>> running, and stop growing when it isn't.
>
> Do you have a pointer to the scanbuttond source code? I'll try to take
> a look at this tomorrow.
>

I guess this one :

https://sourceforge.net/projects/scanbuttond/

> thanks,
>
> greg k-h
> -


Gabriel

2007-09-05 16:04:32

by Andrew Morton

[permalink] [raw]
Subject: Re: sysfs_dir_cache growing out of control

> On Fri, 24 Aug 2007 03:26:50 +0200 Gabriel C <[email protected]> wrote:
> Greg KH wrote:
> > On Thu, Aug 23, 2007 at 08:44:10PM -0400, Joel Fuster wrote:
> >> Greg KH wrote:
> >>> On Wed, Aug 22, 2007 at 11:56:44PM -0400, Joel Fuster wrote:
> >>>> Joel Fuster wrote:
> >>>>> Hi,
> >>>>> I am running 2.6.22.3. For reasons that escape me, over time (days) the
> >>>>> sysfs_dir_cache, dentry, and inode_cache SLUB entries grow until they
> >>>>> consume all the memory on my system, requiring a reboot.
> >>> Hm, those items should consume all the memory, but it should be freed if
> >>> you have memory pressure from other places. Does it cause the machine
> >>> to lock up, or you just got scared when seeing them?
> >> Right. The problem is that the memory never seems to get freed no matter
> >> what I do. I've tried setting /proc/sys/vm/vfs_cache_pressure to 10000,
> >> but after a few days all my programs are running out of swap and I have to
> >> reboot to get things back to a usable state.
> >>
> >>> Oh, and does the same thing happen if you do not use SLUB, but rather
> >>> the older SLAB?
> >> OK I just rebuilt 2.6.22.3 with SLAB and I seem to be getting the same
> >> result..obviously I haven't waited several days, but
> >> sysfs_dir_cache/dentry/inode_cache grow continuously when scanbuttond is
> >> running, and stop growing when it isn't.
> >
> > Do you have a pointer to the scanbuttond source code? I'll try to take
> > a look at this tomorrow.
> >

that was a long day?

> I guess this one :
>
> https://sourceforge.net/projects/scanbuttond/

I needed `setenv LD_LIBRARY_PATH /usr/local/lib' to be able to run
`scanbuttond -p 1000'

and after a few minutes I'm seeing zero growth of
/proc/spabinfo:sysfs_dir_cache over a few minutes, with 2.6.23-rc4.
Perhaps it's a specific device driver?