Hola,
We are running kernel 2.4.22 on i686 SMP box.
Inside are two Intel(R) XEON(TM) CPU 2.00GHz, with hyperthreading
enabled, so /proc/cpuinfo shows four CPUs.
This box has 4GB of RAM installed, and 2GB of swap space.
The problems we are experiencing are related to heavy usage of VM.
We have multithreaded application that uses lots of mmap()-ed memory
(one big file in size around 700 MB, and lots of anonymous mappings to
/dev/zero in average size between 64k and 1MB). This program sometimes
grow up to 1.6 GB in size (SIZE that is shown by top utility).. But,
sometimes /proc/<pid>/status shows VmSize with more than 2GB, where at
the same time top and other /proc/<pid> entries show 1.6GB. This somehow
affects pthread_create() call which fails then.
The question is, how can happen that different numbers are shown in
proc filesystem, for same pid ? (which is part of multithreaded
process), and why pthread_create() fails ? Is there maybe 2GB limit on
memory size that single process can manage on i386 ? Also, when such
program crashes, it creates 2GB core file, which is not completly usable
from gdb. (gdb complains that some addresses are not accessible)..
I suspect that this has something to do with amount of RAM (4GB), but
we are still trying to get this server tested with only 2GB running in
standard (not paged) mode.. but this can take some time, since it is one
of our production machines.
Anybody, idea ?
Thanks.
--
In addition, here are status and statm files:
[ status ]
Name: v
State: S (sleeping)
Tgid: 15675
Pid: 15675
PPid: 15651
TracerPid: 0
Uid: 502 502 502 502
Gid: 502 502 502 502
FDSize: 16384
Groups: 502 10
VmSize: 2111108 kB
VmLck: 0 kB
VmRSS: 1114900 kB
VmData: 190332 kB
VmStk: 40 kB
VmExe: 28 kB
VmLib: 5424 kB
SigPnd: 0000000000000000
SigBlk: 0000000080000000
SigIgn: 0000000000001000
SigCgt: 0000000380000002
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000
[ statm ]
425731 278725 180470 8 276340 2377 276132
On Fri, 2003-11-28 at 20:18, Vladimir Zidar wrote:
> Hola,
>
> We are running kernel 2.4.22 on i686 SMP box.
> Inside are two Intel(R) XEON(TM) CPU 2.00GHz, with hyperthreading
> enabled, so /proc/cpuinfo shows four CPUs.
> This box has 4GB of RAM installed, and 2GB of swap space.
>
> The problems we are experiencing are related to heavy usage of VM.
>
> We have multithreaded application that uses lots of mmap()-ed memory
> (one big file in size around 700 MB, and lots of anonymous mappings to
> /dev/zero in average size between 64k and 1MB). This program sometimes
> grow up to 1.6 GB in size (SIZE that is shown by top utility).. But,
> sometimes /proc/<pid>/status shows VmSize with more than 2GB, where at
> the same time top and other /proc/<pid> entries show 1.6GB. This somehow
> affects pthread_create() call which fails then.
>
> The question is, how can happen that different numbers are shown in
> proc filesystem, for same pid ? (which is part of multithreaded
> process), and why pthread_create() fails ? Is there maybe 2GB limit on
> memory size that single process can manage on i386 ? Also, when such
> program crashes, it creates 2GB core file, which is not completly usable
> from gdb. (gdb complains that some addresses are not accessible)..
> I suspect that this has something to do with amount of RAM (4GB), but
> we are still trying to get this server tested with only 2GB running in
> standard (not paged) mode.. but this can take some time, since it is one
> of our production machines.
>
>
> Anybody, idea ?
>
> Thanks.
--
On Fri, Nov 28, 2003 at 08:18:08PM +0100, Vladimir Zidar wrote:
> We have multithreaded application that uses lots of mmap()-ed memory
> (one big file in size around 700 MB, and lots of anonymous mappings to
> /dev/zero in average size between 64k and 1MB). This program sometimes
> grow up to 1.6 GB in size (SIZE that is shown by top utility).. But,
> sometimes /proc/<pid>/status shows VmSize with more than 2GB, where at
> the same time top and other /proc/<pid> entries show 1.6GB. This somehow
> affects pthread_create() call which fails then.
> The question is, how can happen that different numbers are shown in
> proc filesystem, for same pid ? (which is part of multithreaded
> process), and why pthread_create() fails ? Is there maybe 2GB limit on
> memory size that single process can manage on i386 ? Also, when such
> program crashes, it creates 2GB core file, which is not completly usable
> from gdb. (gdb complains that some addresses are not accessible)..
> I suspect that this has something to do with amount of RAM (4GB), but
> we are still trying to get this server tested with only 2GB running in
> standard (not paged) mode.. but this can take some time, since it is one
> of our production machines.
Well, some of the 2.4 proc_pid_statm() numbers are gibberish. This should
address some of the numbers being gibberish. It eliminates some of the
statm_*_range() parameters, and gets size and rss from the mm's counters,
which should be accurate, and last of all, ditches the hardcoded constant
for its intended meaning: TASK_UNMAPPED_BASE.
The real way to go is to keep count of these things as they're set up and
torn down, but that's a 100KB patch.
-- wli
===== fs/proc/array.c 1.10 vs edited =====
--- 1.10/fs/proc/array.c Tue Apr 22 10:21:00 2003
+++ edited/fs/proc/array.c Fri Nov 28 13:30:42 2003
@@ -397,7 +397,7 @@
}
static inline void statm_pte_range(pmd_t * pmd, unsigned long address, unsigned long size,
- int * pages, int * shared, int * dirty, int * total)
+ int * pages, int * shared, int * dirty)
{
pte_t * pte;
unsigned long end;
@@ -422,7 +422,6 @@
pte++;
if (pte_none(page))
continue;
- ++*total;
if (!pte_present(page))
continue;
ptpage = pte_page(page);
@@ -437,7 +436,7 @@
}
static inline void statm_pmd_range(pgd_t * pgd, unsigned long address, unsigned long size,
- int * pages, int * shared, int * dirty, int * total)
+ int * pages, int * shared, int * dirty)
{
pmd_t * pmd;
unsigned long end;
@@ -455,17 +454,17 @@
if (end > PGDIR_SIZE)
end = PGDIR_SIZE;
do {
- statm_pte_range(pmd, address, end - address, pages, shared, dirty, total);
+ statm_pte_range(pmd, address, end - address, pages, shared, dirty);
address = (address + PMD_SIZE) & PMD_MASK;
pmd++;
} while (address < end);
}
static void statm_pgd_range(pgd_t * pgd, unsigned long address, unsigned long end,
- int * pages, int * shared, int * dirty, int * total)
+ int * pages, int * shared, int * dirty)
{
while (address < end) {
- statm_pmd_range(pgd, address, end - address, pages, shared, dirty, total);
+ statm_pmd_range(pgd, address, end - address, pages, shared, dirty);
address = (address + PGDIR_SIZE) & PGDIR_MASK;
pgd++;
}
@@ -487,23 +486,23 @@
vma = mm->mmap;
while (vma) {
pgd_t *pgd = pgd_offset(mm, vma->vm_start);
- int pages = 0, shared = 0, dirty = 0, total = 0;
+ int pages = 0, shared = 0, dirty = 0;
- statm_pgd_range(pgd, vma->vm_start, vma->vm_end, &pages, &shared, &dirty, &total);
- resident += pages;
+ statm_pgd_range(pgd, vma->vm_start, vma->vm_end, &pages, &shared, &dirty);
share += shared;
dt += dirty;
- size += total;
if (vma->vm_flags & VM_EXECUTABLE)
trs += pages; /* text */
else if (vma->vm_flags & VM_GROWSDOWN)
drs += pages; /* stack */
- else if (vma->vm_end > 0x60000000)
+ else if (vma->vm_end > TASK_UNMAPPED_BASE)
lrs += pages; /* library */
else
drs += pages;
vma = vma->vm_next;
}
+ size = mm->total_vm;
+ resident = mm->rss;
up_read(&mm->mmap_sem);
mmput(mm);
}