Hi list, hi Andrew!
You wrote:
> Added a patch from Ingo which reworks the placement of mmaps within the
> ia32 virtual memory layout. Has been in RH kernels for a long time.
>
> If it breaks something, the app was already buggy. You can use
>
> setarch -L my-buggy-app <args>
>
> to run in back-compat mode. This requires a setarch patch - see the
> changelog in flexible-mmap-267-mm1-a0.patch for details.
Can someone please explain me *what* the effects of a `buggy app' would
be: Segfault I suppose. But what to do with a commerical app where I
cannot check a stack trace or whatever?
Background: I am having problems with current debian/sid on 2.6.7-mm2
with vuescan.
Best wishes
Norbert
-------------------------------------------------------------------------------
Norbert Preining <preining AT logic DOT at> Technische Universit?t Wien
gpg DSA: 0x09C5B094 fp: 14DF 2E6C 0307 BE6D AD76 A9C0 D2BF 4AA3 09C5 B094
-------------------------------------------------------------------------------
STURRY (n.,vb.)
A token run. Pedestrians who have chosen to cross a road immediately
in front of an approaching vehicle generally give a little wave and
break into a sturry. This gives the impression of hurrying without
having any practical effect on their speed whatsoever.
--- Douglas Adams, The Meaning of Liff
>
> Can someone please explain me *what* the effects of a `buggy app' would
> be: Segfault I suppose. But what to do with a commerical app where I
> cannot check a stack trace or whatever?
basically the applications that break do:
int ptr;
ptr = malloc(some_size);
if (ptr <= 0)
handle_no_memory();
> Background: I am having problems with current debian/sid on 2.6.7-mm2
> with vuescan.
can you describe these problems in somewhat more detail ?
Norbert Preining <[email protected]> wrote:
>
> Hi list, hi Andrew!
>
> You wrote:
> > Added a patch from Ingo which reworks the placement of mmaps within the
> > ia32 virtual memory layout. Has been in RH kernels for a long time.
> >
> > If it breaks something, the app was already buggy. You can use
> >
> > setarch -L my-buggy-app <args>
> >
> > to run in back-compat mode. This requires a setarch patch - see the
> > changelog in flexible-mmap-267-mm1-a0.patch for details.
>
> Can someone please explain me *what* the effects of a `buggy app' would
> be: Segfault I suppose.
Yes, that would be one symptom.
> But what to do with a commerical app where I
> cannot check a stack trace or whatever?
Use strace -f, look at the last screenful of output. That usually works.
> Background: I am having problems with current debian/sid on 2.6.7-mm2
> with vuescan.
Well I have suspicions about that patch too. Mozilla crashing
occasionally, mplayer not working at all and quitting from X left the VGA
console all black with no sync. This happened on two boots and hasn't
happened since I reverted the flexible-mmap patches.
Ingo's setarch patch wasn't a lot of use because it seems to be against a
setarch which doesn't exist. I hacked one up. Try
setarch i386 your-program
(The below program _has_ to be called "setarch")
/* Copyright (C) 2003 Red Hat, Inc. */
/* Licensed under the terms of the GPL */
/* Written by Elliot Lee <[email protected]> */
/* based on ideas from the ppc32 util by Guy Streeter (2002-01), based on the
sparc32 util by Jakub Jelinek (1998, 1999) */
#include <syscall.h>
#include <linux/personality.h>
#include <unistd.h>
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <sys/utsname.h>
#define set_pers(pers) syscall(SYS_personality, pers)
int
set_arch(const char *pers)
{
struct utsname un;
int i;
struct {
int perval;
char *target_arch, *result_arch;
} transitions[] = {
#if defined(__powerpc__) || defined(__powerpc64__)
{PER_LINUX32, "ppc32", "ppc"},
{PER_LINUX32, "ppc", "ppc"},
{PER_LINUX, "ppc64", "ppc64"},
{PER_LINUX, "ppc64pseries", "ppc64"},
{PER_LINUX, "ppc64iseries", "ppc64"},
#endif
#if defined(__x86_64__) || defined(__i386__) || defined(__ia64__)
{PER_LINUX32, "i386", "i386"},
{PER_LINUX32, "i486", "i386"},
{PER_LINUX32, "i586", "i386"},
{PER_LINUX32, "i686", "i386"},
{PER_LINUX32, "athlon", "i386"},
#endif
#if defined(__x86_64__) || defined(__i386__)
{PER_LINUX, "x86_64", "x86_64"},
#endif
#if defined(__ia64__) || defined(__i386__)
{PER_LINUX, "ia64", "ia64"},
#endif
#if defined(__s390x__) || defined(__s390__)
{PER_LINUX32, "s390", "s390"},
{PER_LINUX, "s390x", "s390x"},
#endif
#if defined(__sparc64__) || defined(__sparc__)
{PER_LINUX32, "sparc", "sparc"},
{PER_LINUX, "sparc64", "sparc64"},
#endif
{-1, NULL, NULL}
};
for(i = 0; transitions[i].perval >= 0; i++)
{
if(!strcmp(pers, transitions[i].target_arch))
break;
}
if(transitions[i].perval < 0)
{
fprintf(stderr, "Don't know how to set arch to %s\n", pers);
exit(1);
}
if(set_pers(transitions[i].perval | 0x0200000))
return 1;
printf("OK\n");
uname(&un);
if(strcmp(un.machine, transitions[i].result_arch))
{
if(strcmp(transitions[i].result_arch, "i386")
|| (strcmp(un.machine, "i486")
&& strcmp(un.machine, "i586")
&& strcmp(un.machine, "i686")
&& strcmp(un.machine, "athlon")))
{
fprintf(stderr, "Don't know how to set arch to %s\n", pers);
exit(1);
}
}
return 0;
}
int main(int argc, char *argv[])
{
char *p = strrchr(argv[0], '/');
p = p ? p + 1 : argv[0];
if(argc <= 1) {
fprintf(stderr, "Usage: %s %s program arguments\n",
p, !strcmp(p, "setarch")?"arch":"");
return 1;
}
if(!strcmp(p, "setarch"))
{
argv++;
argc--;
p = argv[0];
}
if(set_arch(p))
{
perror(argv[0]);
return 1;
}
if(argc < 2)
{
execl("/bin/sh", "-sh", NULL);
return 1;
}
execvp(argv[1], argv+1);
return 1;
}
2
> with vuescan.
if this is that scanner app that google shows as first hit; that just
works for me (well I have no scanner but it starts and I can navigate
it) with a kernel with this patch (but not the rest of -mm); I'm thus
not quite fully convinced just this patch is the cause yet....
Hi Arjan!
On Fre, 25 Jun 2004, Arjan van de Ven wrote:
> > Can someone please explain me *what* the effects of a `buggy app' would
> > be: Segfault I suppose. But what to do with a commerical app where I
> > cannot check a stack trace or whatever?
>
> basically the applications that break do:
>
> int ptr;
> ptr = malloc(some_size);
> if (ptr <= 0)
> handle_no_memory();
Mmm, this looks very common. What is the `intended' way to handle this?
> > Background: I am having problems with current debian/sid on 2.6.7-mm2
> > with vuescan.
>
> can you describe these problems in somewhat more detail ?
vuescan works on 2.6.7 and dumps core on 2.6.7-mm2 when doing memory
intense stuff like scanning, previewing or some other things.
I will try the setarch prog from Andrew after I have rebooted into mm2
(ATM scanning is in process).
Best wishes
Norbert
-------------------------------------------------------------------------------
Norbert Preining <preining AT logic DOT at> Technische Universit?t Wien
gpg DSA: 0x09C5B094 fp: 14DF 2E6C 0307 BE6D AD76 A9C0 D2BF 4AA3 09C5 B094
-------------------------------------------------------------------------------
HULL (adj.)
Descriptive of the smell of a weekend cottage.
--- Douglas Adams, The Meaning of Liff
On Fri, Jun 25, 2004 at 12:00:52PM +0200, Norbert Preining wrote:
> Hi Arjan!
>
> On Fre, 25 Jun 2004, Arjan van de Ven wrote:
> > > Can someone please explain me *what* the effects of a `buggy app' would
> > > be: Segfault I suppose. But what to do with a commerical app where I
> > > cannot check a stack trace or whatever?
> >
> > basically the applications that break do:
> >
> > int ptr;
> > ptr = malloc(some_size);
> > if (ptr <= 0)
> > handle_no_memory();
>
> Mmm, this looks very common. What is the `intended' way to handle this?
it's actually not common:
1) it stores a pointer in an int which isn't allowed
2) it uses < operator on a pointer
the correct way is
void * ptr;
ptr = malloc(some_size);
if (ptr == NULL)
handle_no_memory();
On Fre, 25 Jun 2004, Andrew Morton wrote:
> > But what to do with a commerical app where I
> > cannot check a stack trace or whatever?
>
> Use strace -f, look at the last screenful of output. That usually works.
open("/media4/scan/cam-2002.03/001-100/raw/scan0100.tif", O_RDONLY) = 6
lseek(6, 0, SEEK_END) = 43426634
lseek(6, 0, SEEK_CUR) = 43426634
lseek(6, 0, SEEK_SET) = 0
mmap2(NULL, 43426634, PROT_READ|PROT_WRITE, MAP_PRIVATE, 6, 0) = -1 ENOMEM (Cann
ot allocate memory)
lseek(6, 0, SEEK_END) = 43426634
lseek(6, 0, SEEK_CUR) = 43426634
fstat64(6, {st_mode=S_IFREG|0644, st_size=43426634, ...}) = 0
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
+++ killed by SIGSEGV +++
I guess that you are right.
> Ingo's setarch patch wasn't a lot of use because it seems to be against a
> setarch which doesn't exist. I hacked one up. Try
>
> setarch i386 your-program
This worked.
So what is the intended path:
- inform the author of the program
- remove the patch (you from mm, I from my build?)
- run setarch for all buggy programs
Best wishes and thanks a lot!
Norbert
-------------------------------------------------------------------------------
Norbert Preining <preining AT logic DOT at> Technische Universit?t Wien
gpg DSA: 0x09C5B094 fp: 14DF 2E6C 0307 BE6D AD76 A9C0 D2BF 4AA3 09C5 B094
-------------------------------------------------------------------------------
ELY (n.)
The first, tiniest inkling you get that something, somewhere, has gone
terribly wrong.
--- Douglas Adams, The Meaning of Liff
On Fri, Jun 25, 2004 at 12:33:26PM +0200, Norbert Preining wrote:
> On Fre, 25 Jun 2004, Andrew Morton wrote:
> > > But what to do with a commerical app where I
> > > cannot check a stack trace or whatever?
> >
> > Use strace -f, look at the last screenful of output. That usually works.
>
> open("/media4/scan/cam-2002.03/001-100/raw/scan0100.tif", O_RDONLY) = 6
> lseek(6, 0, SEEK_END) = 43426634
> lseek(6, 0, SEEK_CUR) = 43426634
> lseek(6, 0, SEEK_SET) = 0
> mmap2(NULL, 43426634, PROT_READ|PROT_WRITE, MAP_PRIVATE, 6, 0) = -1 ENOMEM (Cann
> ot allocate memory)
interesting.
Could you start the binary in gdb, and then when the segfault happens, take
a snapshot of /proc/<pid>/maps of this binary ?
Hi Arjan!
On Fre, 25 Jun 2004, Arjan van de Ven wrote:
> > open("/media4/scan/cam-2002.03/001-100/raw/scan0100.tif", O_RDONLY) = 6
> > lseek(6, 0, SEEK_END) = 43426634
> > lseek(6, 0, SEEK_CUR) = 43426634
> > lseek(6, 0, SEEK_SET) = 0
> > mmap2(NULL, 43426634, PROT_READ|PROT_WRITE, MAP_PRIVATE, 6, 0) = -1 ENOMEM (Cann
> > ot allocate memory)
>
> interesting.
>
> Could you start the binary in gdb, and then when the segfault happens, take
> a snapshot of /proc/<pid>/maps of this binary ?
Attached is gdb.maps and vuescan.maps, the former is the
/proc/<pid>/maps for the gdb process, the later for the vuescan which
has been started from within gdb.
> can you also mail me the output of ulimit -a ?
core file size (blocks, -c) unlimited
data seg size (kbytes, -d) unlimited
file size (blocks, -f) unlimited
max locked memory (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 6143
virtual memory (kbytes, -v) unlimited
On Fre, 25 Jun 2004, Arjan van de Ven wrote:
> if you remove the following 3 lines from mm/mmap.c
> + /* make sure it can fit in the remaining address space */
> + if (mm->free_area_cache < len)
> + return -ENOMEM;
> does it work ?
> (those lines do an optimisation that I suspect is backfiring here...)
No.
I included Ed in the Cc: to keep him uptodate:
On Fre, 25 Jun 2004, [email protected] wrote:
> > int ptr;
> > ptr = malloc(some_size);
> > if (ptr <= 0)
> > handle_no_memory();
>
> No, I never do this. I only check for NULL or non-NULL, which is correct.
Best wishes
Norbert
-------------------------------------------------------------------------------
Norbert Preining <preining AT logic DOT at> Technische Universit?t Wien
gpg DSA: 0x09C5B094 fp: 14DF 2E6C 0307 BE6D AD76 A9C0 D2BF 4AA3 09C5 B094
-------------------------------------------------------------------------------
SKELLOW (adj.)
Descriptive of the satisfaction experienced when looking at a really
good dry-stone wall.
--- Douglas Adams, The Meaning of Liff
Hi All!
Just wanted to say that with 2.6.7-mm4 the problem is gone. I don't know
which changes have been made to mm3/mm4 but now vuescan works again
without any problem.
Thanks a lot for your patience!
Best wishes
Norbert
-------------------------------------------------------------------------------
Norbert Preining <preining AT logic DOT at> Technische Universit?t Wien
gpg DSA: 0x09C5B094 fp: 14DF 2E6C 0307 BE6D AD76 A9C0 D2BF 4AA3 09C5 B094
-------------------------------------------------------------------------------
PUDSEY (n.)
The curious-shaped flat wads of dough left on a kitchen table after
someone has been cutting scones out of it.
--- Douglas Adams, The Meaning of Liff
* Norbert Preining <[email protected]> wrote:
> Hi All!
>
> Just wanted to say that with 2.6.7-mm4 the problem is gone. I don't
> know which changes have been made to mm3/mm4 but now vuescan works
> again without any problem.
there was a bug in the first iteration of the flexible-mmap patch,
introduced during a cleanup of the patch. The bug resulted in the VM
throwing -ENOMEM's after the first ~3GB worth of mmap()s are done. So if
an app did lots of repeat mmap()/munmap()s [like vuescan most likely],
it would 'run out of memory' while there's still plenty of free VM left.
This bug is fixed in the current flexible-mmap patch included in -mm4.
> Thanks a lot for your patience!
you are welcome!
Ingo