A while back, I sent out a preliminary patch
(http://thread.gmane.org/gmane.linux.ports.hppa/752) to remove the
MAX_ARG_PAGES limit on command line sizes. Since then, Peter Zijlstra
and I have fixed a number of bugs and addressed the various
outstanding issues.
The attached patch incorporates the following changes:
- Fixes a BUG_ON() assertion failure discovered by Ingo Molnar.
- Adds CONFIG_STACK_GROWSUP (parisc) support.
- Adds auditing support.
- Reverts to the old behavior on architectures with no MMU.
- Fixes broken execution of 64-bit binaries from 32-bit binaries.
- Adds elf_fdpic support.
- Fixes cache coherency bugs.
We've tested the following architectures: i386, x86_64, um/i386,
parisc, and frv. These are representative of the various scenarios
which this patch addresses, but other architecture teams should try it
out to make sure there aren't any unexpected gotchas.
Ollie
> We've tested the following architectures: i386, x86_64, um/i386,
> parisc, and frv. These are representative of the various scenarios
> which this patch addresses, but other architecture teams should try it
> out to make sure there aren't any unexpected gotchas.
Doesn't build on ia64: complaints from arch/ia64/ia32/binfmt_elf.c
(which #includes ../../../fs/binfmt_elf.c) ...
arch/ia64/ia32/binfmt_elf32.c: In function `ia32_setup_arg_pages':
arch/ia64/ia32/binfmt_elf32.c:206: error: `MAX_ARG_PAGES' undeclared (first use in this function)
arch/ia64/ia32/binfmt_elf32.c:206: error: (Each undeclared identifier is reported only once
arch/ia64/ia32/binfmt_elf32.c:206: error: for each function it appears in.)
arch/ia64/ia32/binfmt_elf32.c:240: error: structure has no member named `page'
arch/ia64/ia32/binfmt_elf32.c:242: error: structure has no member named `page'
arch/ia64/ia32/binfmt_elf32.c:243: warning: implicit declaration of function `install_arg_page'
make[1]: *** [arch/ia64/ia32/binfmt_elf32.o] Error 1
Turning off CONFIG_IA32-SUPPORT, the kernel built, but oops'd during boot.
My serial connection to my test machine is currently broken, so I didn't
get a capture of the stack trace, sorry.
-Tony
At some point in the past, Ollie Wild wrote:
>> We've tested the following architectures: i386, x86_64, um/i386,
>> parisc, and frv. These are representative of the various scenarios
>> which this patch addresses, but other architecture teams should try it
>> out to make sure there aren't any unexpected gotchas.
On Mon, May 07, 2007 at 10:46:49AM -0700, Luck, Tony wrote:
> Doesn't build on ia64: complaints from arch/ia64/ia32/binfmt_elf.c
> (which #includes ../../../fs/binfmt_elf.c) ...
[...]
> Turning off CONFIG_IA32-SUPPORT, the kernel built, but oops'd during boot.
> My serial connection to my test machine is currently broken, so I didn't
> get a capture of the stack trace, sorry.
It needs to sweep 32-bit emulation code more generally.
-- wli
On Sun, 6 May 2007 01:51:34 -0700
"Ollie Wild" <[email protected]> wrote:
> A while back, I sent out a preliminary patch
> (http://thread.gmane.org/gmane.linux.ports.hppa/752) to remove the
> MAX_ARG_PAGES limit on command line sizes. Since then, Peter Zijlstra
> and I have fixed a number of bugs and addressed the various
> outstanding issues.
>
> The attached patch incorporates the following changes:
>
> - Fixes a BUG_ON() assertion failure discovered by Ingo Molnar.
> - Adds CONFIG_STACK_GROWSUP (parisc) support.
> - Adds auditing support.
> - Reverts to the old behavior on architectures with no MMU.
> - Fixes broken execution of 64-bit binaries from 32-bit binaries.
> - Adds elf_fdpic support.
> - Fixes cache coherency bugs.
>
> We've tested the following architectures: i386, x86_64, um/i386,
> parisc, and frv. These are representative of the various scenarios
> which this patch addresses, but other architecture teams should try it
> out to make sure there aren't any unexpected gotchas.
I'll duck this for now, given the couple of problems which people have reported.
But please keep going ;) We sorely need this.
On Wednesday 09 May 2007 4:48 pm, Andrew Morton wrote:
> On Sun, 6 May 2007 01:51:34 -0700
> "Ollie Wild" <[email protected]> wrote:
>
> > A while back, I sent out a preliminary patch
> > (http://thread.gmane.org/gmane.linux.ports.hppa/752) to remove the
> > MAX_ARG_PAGES limit on command line sizes. Since then, Peter Zijlstra
> > and I have fixed a number of bugs and addressed the various
> > outstanding issues.
> >
> > The attached patch incorporates the following changes:
> >
> > - Fixes a BUG_ON() assertion failure discovered by Ingo Molnar.
> > - Adds CONFIG_STACK_GROWSUP (parisc) support.
> > - Adds auditing support.
> > - Reverts to the old behavior on architectures with no MMU.
> > - Fixes broken execution of 64-bit binaries from 32-bit binaries.
> > - Adds elf_fdpic support.
> > - Fixes cache coherency bugs.
> >
> > We've tested the following architectures: i386, x86_64, um/i386,
> > parisc, and frv. These are representative of the various scenarios
> > which this patch addresses, but other architecture teams should try it
> > out to make sure there aren't any unexpected gotchas.
>
> I'll duck this for now, given the couple of problems which people have
> reported.
Just FYI, a really really quick and dirty way of testing this sort of thing on
more architectures and you're likely to physically have?
1) Install QEMU.
2) Grab http://landley.net/code/firmware (releases in the downloads directory,
or tarball of most recent repository snapshot is
wget "http://landley.net/hg/firmware?ca=tip;type=gz").
3) Edit "download.sh" to point at the URL of your tarball instead of whatever
kernel.org version it's using. (Or add your patch to sources/patches if it
applies to the version it's already using. Note that if you set SHA1= blank
in download.sh it'll skip the checksum test, so you don't have to recalculate
the sha1sum if you don't want to.)
4) Run ./build.sh for the architecture you're interested in. (I suggest
armv4l, i686, mipsel, and x86_64. Both sparc and ppc are currently broken
for different reasons; I'm working on it.) Wait a longish time for it to
finish compiling. :)
5) "cd build; ./run-armv4l.sh" and your shell prompt should now be in qemu
running a kernel for the appropriate architecture. (You even have a native
version of gcc you can build stuff with, although you may have
to "ln -s /tools/lib /lib" to run the results, for reasons Linux From Scratch
developers will recognize. :)
This won't help you test real hardware (at least hardware qemu doesn't
emulate), but for stuff like filesystems or executable file formats, it's
handy. :)
Email me if something doesn't work...
Rob
On 5/9/07, Rob Landley <[email protected]> wrote:
> Just FYI, a really really quick and dirty way of testing this sort of thing on
> more architectures and you're likely to physically have?
Does this properly emulate caching? On parisc, cache coherency was
the main issue we ran into. I suspect this might be the case with
other architectures as well.
Ollie
On Thursday 10 May 2007 12:06 am, Ollie Wild wrote:
> On 5/9/07, Rob Landley <[email protected]> wrote:
> > Just FYI, a really really quick and dirty way of testing this sort of
thing on
> > more architectures and you're likely to physically have?
>
> Does this properly emulate caching? On parisc, cache coherency was
> the main issue we ran into. I suspect this might be the case with
> other architectures as well.
This is really a QEMU question. I've been focused on making cross-compilers
and using those to create kernels and a minimal native build environment I
could use to natively compile packages with. (The way I designed the thing
you could substitute real hardware for the qemu step, assuming you had it.
Or another emulator like armulator for a specific platform.)
I don't believe QEMU emulates parisc yet, although it adds new platforms all
the time. (It just grew an alpha emulation last month.) It's under very
active development.
Rob
On Mon, 2007-05-07 at 10:46 -0700, Luck, Tony wrote:
> > We've tested the following architectures: i386, x86_64, um/i386,
> > parisc, and frv. These are representative of the various scenarios
> > which this patch addresses, but other architecture teams should try it
> > out to make sure there aren't any unexpected gotchas.
>
> Doesn't build on ia64: complaints from arch/ia64/ia32/binfmt_elf.c
> (which #includes ../../../fs/binfmt_elf.c) ...
>
> arch/ia64/ia32/binfmt_elf32.c: In function `ia32_setup_arg_pages':
> arch/ia64/ia32/binfmt_elf32.c:206: error: `MAX_ARG_PAGES' undeclared (first use in this function)
> arch/ia64/ia32/binfmt_elf32.c:206: error: (Each undeclared identifier is reported only once
> arch/ia64/ia32/binfmt_elf32.c:206: error: for each function it appears in.)
> arch/ia64/ia32/binfmt_elf32.c:240: error: structure has no member named `page'
> arch/ia64/ia32/binfmt_elf32.c:242: error: structure has no member named `page'
> arch/ia64/ia32/binfmt_elf32.c:243: warning: implicit declaration of function `install_arg_page'
> make[1]: *** [arch/ia64/ia32/binfmt_elf32.o] Error 1
>
> Turning off CONFIG_IA32-SUPPORT, the kernel built, but oops'd during boot.
> My serial connection to my test machine is currently broken, so I didn't
> get a capture of the stack trace, sorry.
Ok, I found the problem. IA64 places constraints on virtual address
space. We initially place the stack at TASK_SIZE, and once the binfmt
tells us where it should have gone, we move it down to the new location.
However IA64 has v-space carved up in regions, and the top of the user
accessible address space is reserved for hugetlbfs.
So we should be using STACK_TOP, which provides the highest stack
address, however, some arches have conditions in the STACK_TOP macros
such that the result is not what is expected until the binfmt
personality is set.
In order to solve this, I added a STACK_TOP_MAX macro for each arch and
use that. This made IA64 boot properly.
The second patch makes the compat stuff compile, untested though, as I
have no idea how that works on ia64.
On Tue, 2007-05-22 at 16:49 -0700, Luck, Tony wrote:
> > In order to solve this, I added a STACK_TOP_MAX macro for each arch and
> > use that. This made IA64 boot properly.
>
> Yes. Patches apply to 2.6.21 cleanly, build cleanly and boot for me too.
> I can even run:
>
> $ ls -lrt `find /usr/share -type f -print`
>
> on the new kernel (thats 48020 files for a total of 2.3MB of args).
>
> > The second patch makes the compat stuff compile, untested though, as I
> > have no idea how that works on ia64.
>
> Not so good on this part. Running an ia32 binary on this patched kernel
> just hung my system :-(
I just tried this on an Altix from the test lab, and ia32 bash just
started.
native 64 bit gave this:
$ ls `find / -xdev -type f` | wc
1 89847 4200288
but the 32 bit bash kept complaining about his arg list being too long.
This might be a 32->64 issue (/bin/ls is 64 bit after all - and I
couldn't find a 32 bit ls in the compat packages), but I really don't
know. Will look into this.
The difference I had with what I posted last is below; these changes
were just me cleaning up. (and fixing one bug on STACK_GROWS_UP
introduced by the previous patches - but AFAIK that is not relevant to
ia64)
Signed-off-by: Peter Zijlstra <[email protected]>
---
fs/exec.c | 15 ++++++---------
1 file changed, 6 insertions(+), 9 deletions(-)
Index: linux-2.6-2/fs/exec.c
===================================================================
--- linux-2.6-2.orig/fs/exec.c 2007-05-24 09:15:54.000000000 +0200
+++ linux-2.6-2/fs/exec.c 2007-05-24 09:41:49.000000000 +0200
@@ -532,26 +532,23 @@ int setup_arg_pages(struct linux_binprm
stack_base = current->signal->rlim[RLIMIT_STACK].rlim_max;
if (stack_base > (1 << 30))
stack_base = 1 << 30;
- stack_base = PAGE_ALIGN(stack_top - stack_base);
/* Make sure we didn't let the argument array grow too large. */
if (vma->vm_end - vma->vm_start > stack_base)
return -ENOMEM;
+ stack_base = PAGE_ALIGN(stack_top - stack_base);
+
stack_shift = stack_base - vma->vm_start;
mm->arg_start = bprm->p + stack_shift;
bprm->p = vma->vm_end + stack_shift;
#else
BUG_ON(stack_top & ~PAGE_MASK);
- stack_base = arch_align_stack(stack_top - mm->stack_vm*PAGE_SIZE);
- stack_base = PAGE_ALIGN(stack_base);
-
- /* Make sure we didn't let the argument array grow too large. */
- if (stack_base > stack_top)
- return -ENOMEM;
+ stack_top = arch_align_stack(stack_top);
+ stack_top = PAGE_ALIGN(stack_top);
+ stack_shift = stack_top - vma->vm_end;
- stack_shift = stack_base - (bprm->p & PAGE_MASK);
bprm->p += stack_shift;
mm->arg_start = bprm->p;
#endif
@@ -598,7 +595,7 @@ int setup_arg_pages(struct linux_binprm
return -EFAULT;
}
#else
- if (expand_stack(vma, stack_base -
+ if (expand_stack(vma, vma->vm_start -
EXTRA_STACK_VM_PAGES * PAGE_SIZE)) {
up_write(&mm->mmap_sem);
return -EFAULT;
> I just tried this on an Altix from the test lab, and ia32 bash just
> started.
I don't have any native x86 binaries on my Madison-based testbox, so my
test case was to compile a simple program that counted total length of
argument strings on an x86 box, and copy it to my ia64 box. So that I
wouldn't have to copy over a bunch of libraries too, I compiled it
with -static. This is the test case that "hung" my system (re-running
it today from /dev/tty1 instead of from an xterm, I see that it actually
oopsed in rb_next()). I wasn't even running with a long arglist. Just
"*" for my home directory (19 files/directories = ~170 bytes).
-Tony
My test program. Compile on ia32 box with "cc -static -o args args.c"
---- begin args.c ----
main(int argc, char **argv)
{
int n;
printf("argc = %d\n", argc);
n = 0;
while (--argc)
n += strlen(*++argv);
printf("bytes = %d\n", n);
}