This is part of a larger series that aims at getting rid of the
copy_thread()/copy_thread_tls() split that makes the process creation
codepaths in the kernel more convoluted and error-prone than they need
to be.
I'm converting all the remaining arches that haven't yet switched and
am collecting individual acks. Once I have them, I'll send the whole series
removing the copy_thread()/copy_thread_tls() split, the
HAVE_COPY_THREAD_TLS define and the legacy do_fork() helper. The only
kernel-wide process creation entry point for anything not going directly
through the syscall path will then be based on struct kernel_clone_args.
No more danger of weird process creation abi quirks between architectures
hopefully, and easier to maintain overall.
It also unblocks implementing clone3() on architectures not support
copy_thread_tls(). Any architecture that wants to implement clone3()
will need to select HAVE_COPY_THREAD_TLS and thus need to implement
copy_thread_tls(). So both goals are connected but independently
beneficial.
HAVE_COPY_THREAD_TLS means that a given architecture supports
CLONE_SETTLS and not setting it should usually mean that the
architectures doesn't implement it but that's not how things are. In
fact all architectures support CLONE_TLS it's just that they don't
follow the calling convention that HAVE_COPY_THREAD_TLS implies. That
means all architectures can be switched over to select
HAVE_COPY_THREAD_TLS. Once that is done we can remove that macro (yay,
less code), the unnecessary do_fork() export in kernel/fork.c, and also
rename copy_thread_tls() back to copy_thread(). At this point
copy_thread() becomes the main architecture specific part of process
creation but it will be the same layout and calling convention for all
architectures. (Once that is done we can probably cleanup each
copy_thread() function even more but that's for the future.)
Since ia64 does support CLONE_SETTLS there's no reason to not select
HAVE_COPY_THREAD_TLS. This brings us one step closer to getting rid of
the copy_thread()/copy_thread_tls() split we still have and ultimately
the HAVE_COPY_THREAD_TLS define in general. A lot of architectures have
already converted and ia64 is one of the few hat haven't yet. This also
unblocks implementing the clone3() syscall on ia64. Once that is done we
can get of another ARCH_WANTS_* macro.
Once Any architecture that supports HAVE_COPY_THREAD_TLS cannot call the
do_fork() helper anymore. This is fine and intended since it should be
removed in favor of the new, cleaner _do_fork() calling convention based
on struct kernel_clone_args. In fact, most architectures have already
switched. With this patch, ia64 joins the other arches which can't use
the fork(), vfork(), clone(), clone3() syscalls directly and who follow
the new process creation calling convention that is based on struct
kernel_clone_args which we introduced a while back. This means less
custom assembly in the architectures entry path to set up the registers
before calling into the process creation helper and it is easier to to
support new features without having to adapt calling conventions. It
also unifies all process creation paths between fork(), vfork(),
clone(), and clone3(). (We can't fix the ABI nightmare that legacy
clone() is but we can prevent stuff like this happening in the future.)
I tried my very best to test this patch including installing Debian 3
and 4 to try and get my hands on a gcc version that would allow
cross-compiling for ia64. But no, even that wasn't possible let alone a
working qemu for ia64. So this is largely untested but hopefully a
straightforward change. If this breaks something we will hear from
people if they still care about new kernels on ia64 since they won't be
able to get any further than trying to start init. :) If this patch
breaks it, I'll fix it.
(I had a different patch similar to what I did for sparc where I'm passing
in struct pt_regs directly instead of keeping the in* to out* parameters
but it wasn't immediately obvious to me what the registers in struct
pt_regs would be for the syscall arguments that would correspond to out*
and the ia64 documentation for the Linux syscall convention didn't explain
this in more detail; I also doubt it matters much.)
For some more context, please see:
commit 606e9ad20094f6d500166881d301f31a51bc8aa7
Merge: ac61145a725a 457677c70c76
Author: Linus Torvalds <[email protected]>
Date: Sat Jan 11 15:33:48 2020 -0800
Merge tag 'clone3-tls-v5.5-rc6' of git://git.kernel.org/pub/scm/linux/kernel/git/brauner/linux
Pull thread fixes from Christian Brauner:
"This contains a series of patches to fix CLONE_SETTLS when used with
clone3().
The clone3() syscall passes the tls argument through struct clone_args
instead of a register. This means, all architectures that do not
implement copy_thread_tls() but still support CLONE_SETTLS via
copy_thread() expecting the tls to be located in a register argument
based on clone() are currently unfortunately broken. Their tls value
will be garbage.
The patch series fixes this on all architectures that currently define
__ARCH_WANT_SYS_CLONE3. It also adds a compile-time check to ensure
that any architecture that enables clone3() in the future is forced to
also implement copy_thread_tls().
My ultimate goal is to get rid of the copy_thread()/copy_thread_tls()
split and just have copy_thread_tls() at some point in the not too
distant future (Maybe even renaming copy_thread_tls() back to simply
copy_thread() once the old function is ripped from all arches). This
is dependent now on all arches supporting clone3().
While all relevant arches do that now there are still four missing:
ia64, m68k, sh and sparc. They have the system call reserved, but not
implemented. Once they all implement clone3() we can get rid of
ARCH_WANT_SYS_CLONE3 and HAVE_COPY_THREAD_TLS.
Note that in the meantime, m68k has already switched to the new calling
convention. And I've got sparc patches acked by Dave, too.
Cc: Tony Luck <[email protected]>
Cc: Fenghua Yu <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: Sebastian Andrzej Siewior <[email protected]>
Cc: "Peter Zijlstra (Intel)" <[email protected]>
Cc: Qais Yousef <[email protected]>
Cc: [email protected]
Cc: [email protected]
Signed-off-by: Christian Brauner <[email protected]>
---
arch/ia64/Kconfig | 1 +
arch/ia64/kernel/entry.S | 36 +++++++++++++++---------------------
arch/ia64/kernel/process.c | 33 +++++++++++++++++++++++++++------
3 files changed, 43 insertions(+), 27 deletions(-)
diff --git a/arch/ia64/Kconfig b/arch/ia64/Kconfig
index bab7cd878464..952aa4c0b556 100644
--- a/arch/ia64/Kconfig
+++ b/arch/ia64/Kconfig
@@ -56,6 +56,7 @@ config IA64
select HAVE_ARCH_AUDITSYSCALL
select NEED_DMA_MAP_STATE
select NEED_SG_DMA_LENGTH
+ select HAVE_COPY_THREAD_TLS
select NUMA if !FLATMEM
default y
help
diff --git a/arch/ia64/kernel/entry.S b/arch/ia64/kernel/entry.S
index 2ac926331500..2b846bc52483 100644
--- a/arch/ia64/kernel/entry.S
+++ b/arch/ia64/kernel/entry.S
@@ -110,21 +110,18 @@ GLOBAL_ENTRY(sys_clone2)
* Allocate 8 input registers since ptrace() may clobber them
*/
.prologue ASM_UNW_PRLG_RP|ASM_UNW_PRLG_PFS, ASM_UNW_PRLG_GRSAVE(8)
- alloc r16=ar.pfs,8,2,6,0
+ alloc r2=ar.pfs,8,2,6,0
DO_SAVE_SWITCH_STACK
- adds r2=PT(R16)+IA64_SWITCH_STACK_SIZE+16,sp
mov loc0=rp
- mov loc1=r16 // save ar.pfs across do_fork
+ mov loc1=r2 // save ar.pfs across ia64_clone
.body
+ mov out0=in0
mov out1=in1
mov out2=in2
- tbit.nz p6,p0=in0,CLONE_SETTLS_BIT
- mov out3=in3 // parent_tidptr: valid only w/CLONE_PARENT_SETTID
- ;;
-(p6) st8 [r2]=in5 // store TLS in r16 for copy_thread()
- mov out4=in4 // child_tidptr: valid only w/CLONE_CHILD_SETTID or CLONE_CHILD_CLEARTID
- mov out0=in0 // out0 = clone_flags
- br.call.sptk.many rp=do_fork
+ mov out3=in3
+ mov out4=in4
+ mov out5=in5
+ br.call.sptk.many rp=ia64_clone
.ret1: .restore sp
adds sp=IA64_SWITCH_STACK_SIZE,sp // pop the switch stack
mov ar.pfs=loc1
@@ -141,21 +138,18 @@ GLOBAL_ENTRY(sys_clone)
* Allocate 8 input registers since ptrace() may clobber them
*/
.prologue ASM_UNW_PRLG_RP|ASM_UNW_PRLG_PFS, ASM_UNW_PRLG_GRSAVE(8)
- alloc r16=ar.pfs,8,2,6,0
+ alloc r2=ar.pfs,8,2,6,0
DO_SAVE_SWITCH_STACK
- adds r2=PT(R16)+IA64_SWITCH_STACK_SIZE+16,sp
mov loc0=rp
- mov loc1=r16 // save ar.pfs across do_fork
+ mov loc1=r2 // save ar.pfs across ia64_clone
.body
+ mov out0=in0
mov out1=in1
mov out2=16 // stacksize (compensates for 16-byte scratch area)
- tbit.nz p6,p0=in0,CLONE_SETTLS_BIT
- mov out3=in2 // parent_tidptr: valid only w/CLONE_PARENT_SETTID
- ;;
-(p6) st8 [r2]=in4 // store TLS in r13 (tp)
- mov out4=in3 // child_tidptr: valid only w/CLONE_CHILD_SETTID or CLONE_CHILD_CLEARTID
- mov out0=in0 // out0 = clone_flags
- br.call.sptk.many rp=do_fork
+ mov out3=in3
+ mov out4=in4
+ mov out5=in5
+ br.call.sptk.many rp=ia64_clone
.ret2: .restore sp
adds sp=IA64_SWITCH_STACK_SIZE,sp // pop the switch stack
mov ar.pfs=loc1
@@ -590,7 +584,7 @@ GLOBAL_ENTRY(ia64_ret_from_clone)
nop.i 0
/*
* We need to call schedule_tail() to complete the scheduling process.
- * Called by ia64_switch_to() after do_fork()->copy_thread(). r8 contains the
+ * Called by ia64_switch_to() after ia64_clone()->copy_thread(). r8 contains the
* address of the previously executing task.
*/
br.call.sptk.many rp=ia64_invoke_schedule_tail
diff --git a/arch/ia64/kernel/process.c b/arch/ia64/kernel/process.c
index 10cb9382ab76..76e77f10536c 100644
--- a/arch/ia64/kernel/process.c
+++ b/arch/ia64/kernel/process.c
@@ -309,8 +309,8 @@ ia64_load_extra (struct task_struct *task)
*
* <clone syscall> <some kernel call frames>
* sys_clone :
- * do_fork do_fork
- * copy_thread copy_thread
+ * _do_fork _do_fork
+ * copy_thread_tls copy_thread_tls
*
* This means that the stack layout is as follows:
*
@@ -332,9 +332,9 @@ ia64_load_extra (struct task_struct *task)
* so there is nothing to worry about.
*/
int
-copy_thread(unsigned long clone_flags,
- unsigned long user_stack_base, unsigned long user_stack_size,
- struct task_struct *p)
+copy_thread_tls(unsigned long clone_flags, unsigned long user_stack_base,
+ unsigned long user_stack_size, struct task_struct *p,
+ unsigned long tls)
{
extern char ia64_ret_from_clone;
struct switch_stack *child_stack, *stack;
@@ -415,7 +415,7 @@ copy_thread(unsigned long clone_flags,
rbs_size = stack->ar_bspstore - rbs;
memcpy((void *) child_rbs, (void *) rbs, rbs_size);
if (clone_flags & CLONE_SETTLS)
- child_ptregs->r13 = regs->r16; /* see sys_clone2() in entry.S */
+ child_ptregs->r13 = tls;
if (user_stack_base) {
child_ptregs->r12 = user_stack_base + user_stack_size - 16;
child_ptregs->ar_bspstore = user_stack_base;
@@ -440,6 +440,27 @@ copy_thread(unsigned long clone_flags,
return retval;
}
+asmlinkage long ia64_clone(unsigned long clone_flags, unsigned long stack_start,
+ unsigned long stack_size, int __user *parent_tidptr,
+ int __user *child_tidptr, unsigned long tls)
+{
+ struct kernel_clone_args args = {
+ .flags = (lower_32_bits(clone_flags) & ~CSIGNAL),
+ .pidfd = parent_tidptr,
+ .child_tid = child_tidptr,
+ .parent_tid = parent_tidptr,
+ .exit_signal = (lower_32_bits(clone_flags) & CSIGNAL),
+ .stack = stack_start,
+ .stack_size = stack_size,
+ .tls = tls,
+ };
+
+ if (!legacy_clone_args_valid(&args))
+ return -EINVAL;
+
+ return _do_fork(&args);
+}
+
static void
do_copy_task_regs (struct task_struct *task, struct unw_frame_info *info, void *arg)
{
base-commit: 2ef96a5bb12be62ef75b5828c0aab838ebb29cb8
--
2.26.2
> I tried my very best to test this patch including installing Debian 3
> and 4 to try and get my hands on a gcc version that would allow
> cross-compiling for ia64. But no, even that wasn't possible let alone a
> working qemu for ia64. So this is largely untested but hopefully a
> straightforward change. If this breaks something we will hear from
> people if they still care about new kernels on ia64 since they won't be
> able to get any further than trying to start init. :) If this patch
> breaks it, I'll fix it.
My last functional ia64 machine no longer powers on. Unclear if it's just
a broken power supply or something more serious. With almost nobody
in offices/labs anymore my search for another machine is proceeding
slowly.
Which is to say ... it won't be me noticing any breakage (at least not for
a while).
I think Al Viro bought an ia64 on ebay a while back ... if that's still running
perhaps he'll test?
-Tony
Hi Tony!
On 5/13/20 11:19 PM, Luck, Tony wrote:
>> I tried my very best to test this patch including installing Debian 3
>> and 4 to try and get my hands on a gcc version that would allow
>> cross-compiling for ia64. But no, even that wasn't possible let alone a
>> working qemu for ia64. So this is largely untested but hopefully a
>> straightforward change. If this breaks something we will hear from
>> people if they still care about new kernels on ia64 since they won't be
>> able to get any further than trying to start init. :) If this patch
>> breaks it, I'll fix it.
>
> My last functional ia64 machine no longer powers on. Unclear if it's just
> a broken power supply or something more serious. With almost nobody
> in offices/labs anymore my search for another machine is proceeding
> slowly.
I could test it.
As for getting a working cross-compiler for ia64 in Debian, this has
been on my TODO list for a while now. Building a cross-compiler for
ia64 is a bit more tricky due to it's dependency on the external
libunwind.
Adrian
--
.''`. John Paul Adrian Glaubitz
: :' : Debian Developer - [email protected]
`. `' Freie Universitaet Berlin - [email protected]
`- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
On Wed, May 13, 2020 at 11:26:35PM +0200, John Paul Adrian Glaubitz wrote:
> Hi Tony!
>
> On 5/13/20 11:19 PM, Luck, Tony wrote:
> >> I tried my very best to test this patch including installing Debian 3
> >> and 4 to try and get my hands on a gcc version that would allow
> >> cross-compiling for ia64. But no, even that wasn't possible let alone a
> >> working qemu for ia64. So this is largely untested but hopefully a
> >> straightforward change. If this breaks something we will hear from
> >> people if they still care about new kernels on ia64 since they won't be
> >> able to get any further than trying to start init. :) If this patch
> >> breaks it, I'll fix it.
> >
> > My last functional ia64 machine no longer powers on. Unclear if it's just
> > a broken power supply or something more serious. With almost nobody
> > in offices/labs anymore my search for another machine is proceeding
> > slowly.
>
> I could test it.
Hey Adrian,
That would be excellent and much appreciated.
Do you think you can get it tested soon?
>
> As for getting a working cross-compiler for ia64 in Debian, this has
> been on my TODO list for a while now. Building a cross-compiler for
> ia64 is a bit more tricky due to it's dependency on the external
> libunwind.
I hit that roadblock as well but yeah, a cross-compiler would be
helpful.
Christian
On Wed, May 13, 2020 at 09:19:41PM +0000, Luck, Tony wrote:
> > I tried my very best to test this patch including installing Debian 3
> > and 4 to try and get my hands on a gcc version that would allow
> > cross-compiling for ia64. But no, even that wasn't possible let alone a
> > working qemu for ia64. So this is largely untested but hopefully a
> > straightforward change. If this breaks something we will hear from
> > people if they still care about new kernels on ia64 since they won't be
> > able to get any further than trying to start init. :) If this patch
> > breaks it, I'll fix it.
>
> My last functional ia64 machine no longer powers on. Unclear if it's just
> a broken power supply or something more serious. With almost nobody
> in offices/labs anymore my search for another machine is proceeding
> slowly.
Ah. :/
>
> Which is to say ... it won't be me noticing any breakage (at least not for
> a while).
If we can't get it tested on a machine soon, would you still be ok
proceeding with this patch?
>
> I think Al Viro bought an ia64 on ebay a while back ... if that's still running
> perhaps he'll test?
I have to admit, that out of pure desperation I considered buying on too
on ebay. But they are still suprisingly pricy.
Christian
Hi Christian!
On 5/14/20 9:46 AM, Christian Brauner wrote:
>>> My last functional ia64 machine no longer powers on. Unclear if it's just
>>> a broken power supply or something more serious. With almost nobody
>>> in offices/labs anymore my search for another machine is proceeding
>>> slowly.
>>
>> I could test it.
>
> Hey Adrian,
>
> That would be excellent and much appreciated.
> Do you think you can get it tested soon?
The kernel is currently building, you should get it by the evening (CEST).
The machine also serves as a Debian buildd which is why it's a bit more
busy than other servers.
>> As for getting a working cross-compiler for ia64 in Debian, this has
>> been on my TODO list for a while now. Building a cross-compiler for
>> ia64 is a bit more tricky due to it's dependency on the external
>> libunwind.
>
> I hit that roadblock as well but yeah, a cross-compiler would be
> helpful.
It's not difficult, it's just a bit of annoying package work including
some trial and error testing.
Once the cross-compiler is in Debian, it will be available in Ubuntu as well.
Adrian
--
.''`. John Paul Adrian Glaubitz
: :' : Debian Developer - [email protected]
`. `' Freie Universitaet Berlin - [email protected]
`- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
On Thu, May 14, 2020 at 09:53:49AM +0200, John Paul Adrian Glaubitz wrote:
> Hi Christian!
>
> On 5/14/20 9:46 AM, Christian Brauner wrote:
> >>> My last functional ia64 machine no longer powers on. Unclear if it's just
> >>> a broken power supply or something more serious. With almost nobody
> >>> in offices/labs anymore my search for another machine is proceeding
> >>> slowly.
> >>
> >> I could test it.
> >
> > Hey Adrian,
> >
> > That would be excellent and much appreciated.
> > Do you think you can get it tested soon?
>
> The kernel is currently building, you should get it by the evening (CEST).
Thank you!
>
> The machine also serves as a Debian buildd which is why it's a bit more
> busy than other servers.
Oh? Does it also produce Debian images for ia64 similar to what is done
for sparc64?
>
> >> As for getting a working cross-compiler for ia64 in Debian, this has
> >> been on my TODO list for a while now. Building a cross-compiler for
> >> ia64 is a bit more tricky due to it's dependency on the external
> >> libunwind.
> >
> > I hit that roadblock as well but yeah, a cross-compiler would be
> > helpful.
>
> It's not difficult, it's just a bit of annoying package work including
> some trial and error testing.
>
> Once the cross-compiler is in Debian, it will be available in Ubuntu as well.
Would that based on a recent gcc? I vaguely remember a post somwhere
that gcc 10 or 11 was planning to drop support for ia64?
Thanks for your help!
Christian
Hi!
On 5/14/20 9:58 AM, Christian Brauner wrote:
>> The machine also serves as a Debian buildd which is why it's a bit more
>> busy than other servers.
>
> Oh? Does it also produce Debian images for ia64 similar to what is done
> for sparc64?
Yes, it's actually the same person who does this - me ;).
These images should work just fine:
> https://cdimage.debian.org/cdimage/ports/2020-04-19/
The latest snapshot is currently broken due to a regression in initramfs-tools.
>>>> As for getting a working cross-compiler for ia64 in Debian, this has
>>>> been on my TODO list for a while now. Building a cross-compiler for
>>>> ia64 is a bit more tricky due to it's dependency on the external
>>>> libunwind.
>>>
>>> I hit that roadblock as well but yeah, a cross-compiler would be
>>> helpful.
>>
>> It's not difficult, it's just a bit of annoying package work including
>> some trial and error testing.
>>
>> Once the cross-compiler is in Debian, it will be available in Ubuntu as well.
>
> Would that based on a recent gcc? I vaguely remember a post somwhere
> that gcc 10 or 11 was planning to drop support for ia64?
Yes, that would be the latest gcc. There are no plans at the moment to
drop ia64 from gcc as the backend is already ported to MODE_CC.
m68k was on the brink of being removed, but I started a Bountysource campaign
to convert it from cc0 to MODE_CC and it was eventually saved ;).
> https://www.bountysource.com/issues/80706251-m68k-convert-the-backend-to-mode_cc-so-it-can-be-kept-in-future-releases
Adrian
--
.''`. John Paul Adrian Glaubitz
: :' : Debian Developer - [email protected]
`. `' Freie Universitaet Berlin - [email protected]
`- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
On Thu, May 14, 2020 at 10:24:25AM +0200, John Paul Adrian Glaubitz wrote:
> Hi!
>
> On 5/14/20 9:58 AM, Christian Brauner wrote:
> >> The machine also serves as a Debian buildd which is why it's a bit more
> >> busy than other servers.
> >
> > Oh? Does it also produce Debian images for ia64 similar to what is done
> > for sparc64?
>
> Yes, it's actually the same person who does this - me ;).
Well thank you very much. Thanks to this I was able to test my sparc
patches in qemu. :)
>
> These images should work just fine:
>
> > https://cdimage.debian.org/cdimage/ports/2020-04-19/
Oh I didn't find these images when searching for them. They would be
super helpful but there's no qemu for ia64 anymore that's useable. I had
tried building qemu from an old source based on a gsoc project for an
ia64 port but that turned out to be more involved than writing ia64
assembly itself. :)
>
> The latest snapshot is currently broken due to a regression in initramfs-tools.
>
> >>>> As for getting a working cross-compiler for ia64 in Debian, this has
> >>>> been on my TODO list for a while now. Building a cross-compiler for
> >>>> ia64 is a bit more tricky due to it's dependency on the external
> >>>> libunwind.
> >>>
> >>> I hit that roadblock as well but yeah, a cross-compiler would be
> >>> helpful.
> >>
> >> It's not difficult, it's just a bit of annoying package work including
> >> some trial and error testing.
> >>
> >> Once the cross-compiler is in Debian, it will be available in Ubuntu as well.
> >
> > Would that based on a recent gcc? I vaguely remember a post somwhere
> > that gcc 10 or 11 was planning to drop support for ia64?
>
> Yes, that would be the latest gcc. There are no plans at the moment to
> drop ia64 from gcc as the backend is already ported to MODE_CC.
>
> m68k was on the brink of being removed, but I started a Bountysource campaign
> to convert it from cc0 to MODE_CC and it was eventually saved ;).
>
> > https://www.bountysource.com/issues/80706251-m68k-convert-the-backend-to-mode_cc-so-it-can-be-kept-in-future-releases
Oh that's pretty neat. Thanks!
Christian
On 5/14/20 10:37 AM, Christian Brauner wrote:
>>> Oh? Does it also produce Debian images for ia64 similar to what is done
>>> for sparc64?
>>
>> Yes, it's actually the same person who does this - me ;).
>
> Well thank you very much. Thanks to this I was able to test my sparc
> patches in qemu. :)
I really appreciate it that kernel developers are testing their patches on
all architectures. Thank you for your work as well!
>> These images should work just fine:
>>
>>> https://cdimage.debian.org/cdimage/ports/2020-04-19/
>
> Oh I didn't find these images when searching for them. They would be
> super helpful but there's no qemu for ia64 anymore that's useable. I had
> tried building qemu from an old source based on a gsoc project for an
> ia64 port but that turned out to be more involved than writing ia64
> assembly itself. :)
There is ski but I have never used it:
> http://ski.sourceforge.net/
Adrian
--
.''`. John Paul Adrian Glaubitz
: :' : Debian Developer - [email protected]
`. `' Freie Universitaet Berlin - [email protected]
`- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
On 2020-05-14 09:46:06 [+0200], Christian Brauner wrote:
> > As for getting a working cross-compiler for ia64 in Debian, this has
> > been on my TODO list for a while now. Building a cross-compiler for
> > ia64 is a bit more tricky due to it's dependency on the external
> > libunwind.
>
> I hit that roadblock as well but yeah, a cross-compiler would be
> helpful.
There is
https://mirrors.edge.kernel.org/pub/tools/crosstool/
and I used the ia64 compiler from there while compile testing patches
for ia64.
> Christian
Sebastian
On 5/14/20 9:53 AM, John Paul Adrian Glaubitz wrote:
> The kernel is currently building, you should get it by the evening (CEST).
>
> The machine also serves as a Debian buildd which is why it's a bit more
> busy than other servers.
I applied the patch against 5.7-rc5, it applied fine and builds fine.
It seems though that systemd crashes but I'm not sure it's related:
[ 14.267489] init[1]: General Exception: IA-64 Reserved Register/Field fault (data access) 2199023255600 [1]
[ 14.267489] Modules linked in:
[ 14.267489]
[ 14.267489] CPU: 0 PID: 1 Comm: init Not tainted 5.7.0-rc5 #1
[ 14.267489] Hardware name: hp server BL860c , BIOS 04.25 07/02/2010
[ 14.267489] psr : 0000121008526030 ifs : 8000000000000510 ip : [<a00000010000c541>] Not tainted (5.7.0-rc5)
[ 14.267489] ip is at sys_clone2+0x61/0x80
[ 14.267489] unat: 0000000000000000 pfs : 0000000000000510 rsc : 0000000000000003
[ 14.267489] rnat: 0000000000000000 bsps: 0000000000000000 pr : 000000000001a959
[ 14.267489] ldrs: 0000000000000000 ccv : 0000000000000002 fpsr: 0009804c8a70033f
[ 14.267489] csd : 0000000000000000 ssd : 0000000000000000
[ 14.267489] b0 : a00000010000c540 b6 : a0000001000edf80 b7 : a000000100369560
[ 14.267489] f6 : 1003e000000000000b8a4 f7 : 1003e00000001754e8d69
[ 14.267489] f8 : 1003e0000000178311d69 f9 : 100098000000000000000
[ 14.267489] f10 : 1000eb8a3fffff475c000 f11 : 1003e000000000000b8a4
[ 14.267489] r1 : a0000001017a6f10 r2 : 0000000000000000 r3 : 0000000000000006
[ 14.267489] r8 : 0000000000000066 r9 : 0000000000000060 r10 : e0000100800d8e68
[ 14.267489] r11 : 0000000000000002 r12 : e00001008027fe30 r13 : e000010080278000
[ 14.267489] r14 : e0000100800d8e04 r15 : e0000100800d8e08 r16 : 0000000000000002
[ 14.267489] r17 : 0000000000000001 r18 : 0000000000000002 r19 : 0000000000000003
[ 14.267489] r20 : 0000000000000004 r21 : e00001008027fb98 r22 : e000000001146d02
[ 14.267489] r23 : 00000000000030c8 r24 : 00000000000030c8 r25 : 00000000000030ca
[ 14.267489] r26 : 00000000000030ca r27 : 00000000000030ca r28 : e00001008027fb90
[ 14.267489] r29 : 0000000000000400 r30 : 00000000000002e8 r31 : 0000000000000400
[ 14.267489]
[ 14.267489] Call Trace:
[ 14.267489] [<a000000100015090>] show_stack+0x90/0xc0
[ 14.267489] sp=e00001008027f9f0 bsp=e000010080279288
[ 14.267489] [<a000000100015780>] show_regs+0x6c0/0xa20
[ 14.267489] sp=e00001008027fbc0 bsp=e000010080279218
[ 14.267489] [<a000000100029590>] die+0x1b0/0x4a0
[ 14.267489] sp=e00001008027fbe0 bsp=e0000100802791d0
[ 14.267489] [<a00000010100e690>] ia64_fault+0x770/0xd80
[ 14.267489] sp=e00001008027fbe0 bsp=e000010080279180
[ 14.267489] [<a00000010000ce60>] ia64_leave_kernel+0x0/0x270
[ 14.267489] sp=e00001008027fc60 bsp=e000010080279180
[ 14.267489] [<a00000010000c540>] sys_clone2+0x60/0x80
[ 14.267489] sp=e00001008027fe30 bsp=e000010080279100
[ 14.267489] Disabling lock debugging due to kernel taint
[ 14.267489] init[102]: General Exception: IA-64 Reserved Register/Field fault (data access) 4398046511152 [2]
[ 14.350730] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[1 4 . 31540.733500]7 3-0-]- [- -e-n[d eKnedr nKeelr npealn pani[c - 1n4o.t2 6s7y4n8c9i]n g:
:Attem [D 1 4 . 2 6 7 4 8 95]. 7C.P0U-:r c15 P#I1D
i e1=002x 0C0o0m0m0:0 0ibn i]t- -T-a
nted: G D 5.7.0-rc5 #1
I have to perform more testing. The machine boots fine with a 5.5.x kernel.
Adrian
--
.''`. John Paul Adrian Glaubitz
: :' : Debian Developer - [email protected]
`. `' Freie Universitaet Berlin - [email protected]
`- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
On Thu, May 14, 2020 at 10:58:01AM +0200, Sebastian Andrzej Siewior wrote:
> On 2020-05-14 09:46:06 [+0200], Christian Brauner wrote:
> > > As for getting a working cross-compiler for ia64 in Debian, this has
> > > been on my TODO list for a while now. Building a cross-compiler for
> > > ia64 is a bit more tricky due to it's dependency on the external
> > > libunwind.
> >
> > I hit that roadblock as well but yeah, a cross-compiler would be
> > helpful.
>
> There is
> https://mirrors.edge.kernel.org/pub/tools/crosstool/
>
> and I used the ia64 compiler from there while compile testing patches
> for ia64.
Excellent, I've got it to compile on my system now. Let's see if it
succeeds!
Just a way to boot it afterwards would be great.
Christian
On Thu, May 14, 2020 at 11:48:31AM +0200, John Paul Adrian Glaubitz wrote:
>
>
> On 5/14/20 9:53 AM, John Paul Adrian Glaubitz wrote:
> > The kernel is currently building, you should get it by the evening (CEST).
> >
> > The machine also serves as a Debian buildd which is why it's a bit more
> > busy than other servers.
> I applied the patch against 5.7-rc5, it applied fine and builds fine.
>
> It seems though that systemd crashes but I'm not sure it's related:
>
> [ 14.267489] init[1]: General Exception: IA-64 Reserved Register/Field fault (data access) 2199023255600 [1]
> [ 14.267489] Modules linked in:
> [ 14.267489]
> [ 14.267489] CPU: 0 PID: 1 Comm: init Not tainted 5.7.0-rc5 #1
> [ 14.267489] Hardware name: hp server BL860c , BIOS 04.25 07/02/2010
> [ 14.267489] psr : 0000121008526030 ifs : 8000000000000510 ip : [<a00000010000c541>] Not tainted (5.7.0-rc5)
> [ 14.267489] ip is at sys_clone2+0x61/0x80
> [ 14.267489] unat: 0000000000000000 pfs : 0000000000000510 rsc : 0000000000000003
> [ 14.267489] rnat: 0000000000000000 bsps: 0000000000000000 pr : 000000000001a959
> [ 14.267489] ldrs: 0000000000000000 ccv : 0000000000000002 fpsr: 0009804c8a70033f
> [ 14.267489] csd : 0000000000000000 ssd : 0000000000000000
> [ 14.267489] b0 : a00000010000c540 b6 : a0000001000edf80 b7 : a000000100369560
> [ 14.267489] f6 : 1003e000000000000b8a4 f7 : 1003e00000001754e8d69
> [ 14.267489] f8 : 1003e0000000178311d69 f9 : 100098000000000000000
> [ 14.267489] f10 : 1000eb8a3fffff475c000 f11 : 1003e000000000000b8a4
> [ 14.267489] r1 : a0000001017a6f10 r2 : 0000000000000000 r3 : 0000000000000006
> [ 14.267489] r8 : 0000000000000066 r9 : 0000000000000060 r10 : e0000100800d8e68
> [ 14.267489] r11 : 0000000000000002 r12 : e00001008027fe30 r13 : e000010080278000
> [ 14.267489] r14 : e0000100800d8e04 r15 : e0000100800d8e08 r16 : 0000000000000002
> [ 14.267489] r17 : 0000000000000001 r18 : 0000000000000002 r19 : 0000000000000003
> [ 14.267489] r20 : 0000000000000004 r21 : e00001008027fb98 r22 : e000000001146d02
> [ 14.267489] r23 : 00000000000030c8 r24 : 00000000000030c8 r25 : 00000000000030ca
> [ 14.267489] r26 : 00000000000030ca r27 : 00000000000030ca r28 : e00001008027fb90
> [ 14.267489] r29 : 0000000000000400 r30 : 00000000000002e8 r31 : 0000000000000400
> [ 14.267489]
> [ 14.267489] Call Trace:
> [ 14.267489] [<a000000100015090>] show_stack+0x90/0xc0
> [ 14.267489] sp=e00001008027f9f0 bsp=e000010080279288
> [ 14.267489] [<a000000100015780>] show_regs+0x6c0/0xa20
> [ 14.267489] sp=e00001008027fbc0 bsp=e000010080279218
> [ 14.267489] [<a000000100029590>] die+0x1b0/0x4a0
> [ 14.267489] sp=e00001008027fbe0 bsp=e0000100802791d0
> [ 14.267489] [<a00000010100e690>] ia64_fault+0x770/0xd80
> [ 14.267489] sp=e00001008027fbe0 bsp=e000010080279180
> [ 14.267489] [<a00000010000ce60>] ia64_leave_kernel+0x0/0x270
> [ 14.267489] sp=e00001008027fc60 bsp=e000010080279180
> [ 14.267489] [<a00000010000c540>] sys_clone2+0x60/0x80
> [ 14.267489] sp=e00001008027fe30 bsp=e000010080279100
> [ 14.267489] Disabling lock debugging due to kernel taint
> [ 14.267489] init[102]: General Exception: IA-64 Reserved Register/Field fault (data access) 4398046511152 [2]
> [ 14.350730] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
> [1 4 . 31540.733500]7 3-0-]- [- -e-n[d eKnedr nKeelr npealn pani[c - 1n4o.t2 6s7y4n8c9i]n g:
> :Attem [D 1 4 . 2 6 7 4 8 95]. 7C.P0U-:r c15 P#I1D
> i e1=002x 0C0o0m0m0:0 0ibn i]t- -T-a
> nted: G D 5.7.0-rc5 #1
>
> I have to perform more testing. The machine boots fine with a 5.5.x kernel.
So that's interesting because systemd doesn't support itanium correctly afaict.
They have a raw_clone() function that they use which is not tailored to
ia64. __NR_clone should be defined as so to hit clone2() but they don't
pass a stack size argument down which is invalid on ia64:
static inline pid_t raw_clone(unsigned long flags) {
pid_t ret;
assert((flags & (CLONE_VM|CLONE_PARENT_SETTID|CLONE_CHILD_SETTID|
CLONE_CHILD_CLEARTID|CLONE_SETTLS)) == 0);
#if defined(__s390x__) || defined(__s390__) || defined(__CRIS__)
/* On s390/s390x and cris the order of the first and second arguments
* of the raw clone() system call is reversed. */
ret = (pid_t) syscall(__NR_clone, NULL, flags);
#elif defined(__sparc__)
{
/**
* sparc always returns the other process id in %o0, and
* a boolean flag whether this is the child or the parent in
* %o1. Inline assembly is needed to get the flag returned
* in %o1.
*/
int in_child, child_pid, error;
asm volatile("mov %3, %%g1\n\t"
"mov %4, %%o0\n\t"
"mov 0 , %%o1\n\t"
#if defined(__arch64__)
"t 0x6d\n\t"
#else
"t 0x10\n\t"
#endif
"addx %%g0, 0, %2\n\t"
"mov %%o1, %0\n\t"
"mov %%o0, %1" :
"=r"(in_child), "=r"(child_pid), "=r"(error) :
"i"(__NR_clone), "r"(flags) :
"%o1", "%o0", "%g1", "cc" );
if (error) {
errno = child_pid;
ret = -1;
} else
ret = in_child ? 0 : child_pid;
}
#else
ret = (pid_t) syscall(__NR_clone, flags, NULL);
#endif
if (ret == 0)
reset_cached_pid();
return ret;
}
On 5/14/20 12:04 PM, Christian Brauner wrote:
> So that's interesting because systemd doesn't support itanium correctly afaict.
> They have a raw_clone() function that they use which is not tailored to
> ia64. __NR_clone should be defined as so to hit clone2() but they don't
> pass a stack size argument down which is invalid on ia64:
Ah, I wasn't aware of this limitation. I'm surprised the systemd testsuite passes
then on ia64.
On sparc64, Michael Karcher provided the hand-written assembly you quoted to
fix the raw_clone() on this architecture.
I assume we could do the same on ia64?
Adrian
--
.''`. John Paul Adrian Glaubitz
: :' : Debian Developer - [email protected]
`. `' Freie Universitaet Berlin - [email protected]
`- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
On Thu, May 14, 2020 at 12:08:27PM +0200, John Paul Adrian Glaubitz wrote:
> On 5/14/20 12:04 PM, Christian Brauner wrote:
> > So that's interesting because systemd doesn't support itanium correctly afaict.
> > They have a raw_clone() function that they use which is not tailored to
> > ia64. __NR_clone should be defined as so to hit clone2() but they don't
> > pass a stack size argument down which is invalid on ia64:
> Ah, I wasn't aware of this limitation. I'm surprised the systemd testsuite passes
> then on ia64.
>
> On sparc64, Michael Karcher provided the hand-written assembly you quoted to
> fix the raw_clone() on this architecture.
>
> I assume we could do the same on ia64?
I think the following should be correct:
(Probably worth checking that __NR_clone and __NR_clone2 are the same
for ia64. Also note, that __NR_clone was _never_ supported by glibc on
ia64. They even have this comment:
/* clone is not supported under Linux/ia64, use clone2. */
)
static inline pid_t raw_clone(unsigned long flags) {
pid_t ret;
assert((flags & (CLONE_VM|CLONE_PARENT_SETTID|CLONE_CHILD_SETTID|
CLONE_CHILD_CLEARTID|CLONE_SETTLS)) == 0);
#if defined(__s390x__) || defined(__s390__) || defined(__CRIS__)
/* On s390/s390x and cris the order of the first and second arguments
* of the raw clone() system call is reversed. */
ret = (pid_t) syscall(__NR_clone, NULL, flags);
#elif defined(__sparc__)
{
/**
* sparc always returns the other process id in %o0, and
* a boolean flag whether this is the child or the parent in
* %o1. Inline assembly is needed to get the flag returned
* in %o1.
*/
int in_child, child_pid, error;
asm volatile("mov %3, %%g1\n\t"
"mov %4, %%o0\n\t"
"mov 0 , %%o1\n\t"
#if defined(__arch64__)
"t 0x6d\n\t"
#else
"t 0x10\n\t"
#endif
"addx %%g0, 0, %2\n\t"
"mov %%o1, %0\n\t"
"mov %%o0, %1" :
"=r"(in_child), "=r"(child_pid), "=r"(error) :
"i"(__NR_clone), "r"(flags) :
"%o1", "%o0", "%g1", "cc" );
if (error) {
errno = child_pid;
ret = -1;
} else
ret = in_child ? 0 : child_pid;
}
+#elif defined(__ia64__)
+ /* On ia64 the stack and stack size are passed as separate arguments. */
+ return (pid_t)syscall(__NR_clone, flags, NULL, 0);
+#else
+ return (pid_t)syscall(__NR_clone, flags, NULL);
+#endif
if (ret == 0)
reset_cached_pid();
return ret;
}
On 5/14/20 12:19 PM, Christian Brauner wrote:
> Scratch that. It's even worse. On ia64 it is _invalid_ to pass a NULL
> stack. That's at least what the glibc assembly assumes:
>
> cmp.eq p6,p0=0,in0
> cmp.eq p7,p0=0,in1
> mov r8=EINVAL
> mov out0=in3 /* Flags are first syscall argument. */
> mov out1=in1 /* Stack address. */
> (p6) br.cond.spnt.many __syscall_error /* no NULL function pointers */
> (p7) br.cond.spnt.many __syscall_error /* no NULL stack pointers */
> ;;
> mov out2=in2 /* Stack size. */
>
> so newer systemd just works by accident on ia64 if at all correctly
> afaict.
Hmm, interesting. I really wasn't aware of that. Thanks for the heads-up.
I'll ask Michael whether he can come up for a solution for that problem.
Maybe that's also why systemd crashes.
Adrian
--
.''`. John Paul Adrian Glaubitz
: :' : Debian Developer - [email protected]
`. `' Freie Universitaet Berlin - [email protected]
`- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
On Thu, May 14, 2020 at 12:15:42PM +0200, Christian Brauner wrote:
> On Thu, May 14, 2020 at 12:08:27PM +0200, John Paul Adrian Glaubitz wrote:
> > On 5/14/20 12:04 PM, Christian Brauner wrote:
> > > So that's interesting because systemd doesn't support itanium correctly afaict.
> > > They have a raw_clone() function that they use which is not tailored to
> > > ia64. __NR_clone should be defined as so to hit clone2() but they don't
> > > pass a stack size argument down which is invalid on ia64:
> > Ah, I wasn't aware of this limitation. I'm surprised the systemd testsuite passes
> > then on ia64.
> >
> > On sparc64, Michael Karcher provided the hand-written assembly you quoted to
> > fix the raw_clone() on this architecture.
> >
> > I assume we could do the same on ia64?
>
> I think the following should be correct:
>
> (Probably worth checking that __NR_clone and __NR_clone2 are the same
> for ia64. Also note, that __NR_clone was _never_ supported by glibc on
> ia64. They even have this comment:
>
> /* clone is not supported under Linux/ia64, use clone2. */
> )
>
> static inline pid_t raw_clone(unsigned long flags) {
> pid_t ret;
>
> assert((flags & (CLONE_VM|CLONE_PARENT_SETTID|CLONE_CHILD_SETTID|
> CLONE_CHILD_CLEARTID|CLONE_SETTLS)) == 0);
> #if defined(__s390x__) || defined(__s390__) || defined(__CRIS__)
> /* On s390/s390x and cris the order of the first and second arguments
> * of the raw clone() system call is reversed. */
> ret = (pid_t) syscall(__NR_clone, NULL, flags);
> #elif defined(__sparc__)
> {
> /**
> * sparc always returns the other process id in %o0, and
> * a boolean flag whether this is the child or the parent in
> * %o1. Inline assembly is needed to get the flag returned
> * in %o1.
> */
> int in_child, child_pid, error;
>
> asm volatile("mov %3, %%g1\n\t"
> "mov %4, %%o0\n\t"
> "mov 0 , %%o1\n\t"
> #if defined(__arch64__)
> "t 0x6d\n\t"
> #else
> "t 0x10\n\t"
> #endif
> "addx %%g0, 0, %2\n\t"
> "mov %%o1, %0\n\t"
> "mov %%o0, %1" :
> "=r"(in_child), "=r"(child_pid), "=r"(error) :
> "i"(__NR_clone), "r"(flags) :
> "%o1", "%o0", "%g1", "cc" );
>
> if (error) {
> errno = child_pid;
> ret = -1;
> } else
> ret = in_child ? 0 : child_pid;
> }
> +#elif defined(__ia64__)
> + /* On ia64 the stack and stack size are passed as separate arguments. */
> + return (pid_t)syscall(__NR_clone, flags, NULL, 0);
> +#else
> + return (pid_t)syscall(__NR_clone, flags, NULL);
> +#endif
Scratch that. It's even worse. On ia64 it is _invalid_ to pass a NULL
stack. That's at least what the glibc assembly assumes:
cmp.eq p6,p0=0,in0
cmp.eq p7,p0=0,in1
mov r8=EINVAL
mov out0=in3 /* Flags are first syscall argument. */
mov out1=in1 /* Stack address. */
(p6) br.cond.spnt.many __syscall_error /* no NULL function pointers */
(p7) br.cond.spnt.many __syscall_error /* no NULL stack pointers */
;;
mov out2=in2 /* Stack size. */
so newer systemd just works by accident on ia64 if at all correctly
afaict.
On Thu, May 14, 2020 at 12:21:13PM +0200, John Paul Adrian Glaubitz wrote:
> On 5/14/20 12:19 PM, Christian Brauner wrote:
> > Scratch that. It's even worse. On ia64 it is _invalid_ to pass a NULL
> > stack. That's at least what the glibc assembly assumes:
> >
> > cmp.eq p6,p0=0,in0
> > cmp.eq p7,p0=0,in1
> > mov r8=EINVAL
> > mov out0=in3 /* Flags are first syscall argument. */
> > mov out1=in1 /* Stack address. */
> > (p6) br.cond.spnt.many __syscall_error /* no NULL function pointers */
> > (p7) br.cond.spnt.many __syscall_error /* no NULL stack pointers */
> > ;;
> > mov out2=in2 /* Stack size. */
> >
> > so newer systemd just works by accident on ia64 if at all correctly
> > afaict.
>
> Hmm, interesting. I really wasn't aware of that. Thanks for the heads-up.
>
> I'll ask Michael whether he can come up for a solution for that problem.
>
> Maybe that's also why systemd crashes.
Do you have a very minimalistic ia64 userspace preferably without systemd where
you could simply test. That should give us an idea whether things work:
#define _GNU_SOURCE
#include <sys/wait.h>
#include <sys/utsname.h>
#include <sched.h>
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>
#define STACK_SIZE (8 * 1024 * 1024) /* standard stack size for threads in glibc */
int main(int argc, char *argv[])
{
char *stack;
pid_t pid;
stack = mmap(NULL, STACK_SIZE, PROT_READ | PROT_WRITE,
MAP_PRIVATE | MAP_ANONYMOUS | MAP_STACK, -1, 0);
if (stack == MAP_FAILED)
exit(EXIT_FAILURE);
/*
* Note that legacy clone() has different argument ordering on
* different architectures so this won't work everywhere.
*/
pid = syscall(189 /* __NR_clone2 */, SIGCHLD, stack, STACK_SIZE, NULL, NULL);
if (pid < 0)
exit(EXIT_FAILURE);
if (pid == 0)
exit(EXIT_SUCCESS);
if (wait(NULL) != pid)
exit(EXIT_FAILURE);
exit(EXIT_SUCCESS);
}
On 5/14/20 12:32 PM, Christian Brauner wrote:
> Do you have a very minimalistic ia64 userspace preferably without systemd where
> you could simply test. That should give us an idea whether things work:
>
> #define _GNU_SOURCE
> #include <sys/wait.h>
> #include <sys/utsname.h>
> #include <sched.h>
> #include <string.h>
> #include <stdio.h>
> #include <stdlib.h>
> #include <unistd.h>
> #include <sys/mman.h>
>
> #define STACK_SIZE (8 * 1024 * 1024) /* standard stack size for threads in glibc */
>
> int main(int argc, char *argv[])
> {
> char *stack;
> pid_t pid;
>
> stack = mmap(NULL, STACK_SIZE, PROT_READ | PROT_WRITE,
> MAP_PRIVATE | MAP_ANONYMOUS | MAP_STACK, -1, 0);
> if (stack == MAP_FAILED)
> exit(EXIT_FAILURE);
>
> /*
> * Note that legacy clone() has different argument ordering on
> * different architectures so this won't work everywhere.
> */
> pid = syscall(189 /* __NR_clone2 */, SIGCHLD, stack, STACK_SIZE, NULL, NULL);
> if (pid < 0)
> exit(EXIT_FAILURE);
> if (pid == 0)
> exit(EXIT_SUCCESS);
> if (wait(NULL) != pid)
> exit(EXIT_FAILURE);
>
> exit(EXIT_SUCCESS);
> }
root@titanium:~# gcc systemd_test.c -o systemd_test
root@titanium:~# ./systemd_test
root@titanium:~# echo $?
1
root@titanium:~#
I can also give you access to this machine.
Adrian
--
.''`. John Paul Adrian Glaubitz
: :' : Debian Developer - [email protected]
`. `' Freie Universitaet Berlin - [email protected]
`- GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
On Thu, May 14, 2020 at 12:33:00PM +0200, Christian Brauner wrote:
> On Thu, May 14, 2020 at 12:21:13PM +0200, John Paul Adrian Glaubitz wrote:
> > On 5/14/20 12:19 PM, Christian Brauner wrote:
> > > Scratch that. It's even worse. On ia64 it is _invalid_ to pass a NULL
> > > stack. That's at least what the glibc assembly assumes:
> > >
> > > cmp.eq p6,p0=0,in0
> > > cmp.eq p7,p0=0,in1
> > > mov r8=EINVAL
> > > mov out0=in3 /* Flags are first syscall argument. */
> > > mov out1=in1 /* Stack address. */
> > > (p6) br.cond.spnt.many __syscall_error /* no NULL function pointers */
> > > (p7) br.cond.spnt.many __syscall_error /* no NULL stack pointers */
> > > ;;
> > > mov out2=in2 /* Stack size. */
> > >
> > > so newer systemd just works by accident on ia64 if at all correctly
> > > afaict.
> >
> > Hmm, interesting. I really wasn't aware of that. Thanks for the heads-up.
> >
> > I'll ask Michael whether he can come up for a solution for that problem.
> >
> > Maybe that's also why systemd crashes.
>
> Do you have a very minimalistic ia64 userspace preferably without systemd where
> you could simply test. That should give us an idea whether things work:
>
> #define _GNU_SOURCE
> #include <sys/wait.h>
> #include <sys/utsname.h>
> #include <sched.h>
> #include <string.h>
> #include <stdio.h>
> #include <stdlib.h>
> #include <unistd.h>
> #include <sys/mman.h>
>
> #define STACK_SIZE (8 * 1024 * 1024) /* standard stack size for threads in glibc */
>
> int main(int argc, char *argv[])
> {
> char *stack;
> pid_t pid;
>
> stack = mmap(NULL, STACK_SIZE, PROT_READ | PROT_WRITE,
> MAP_PRIVATE | MAP_ANONYMOUS | MAP_STACK, -1, 0);
> if (stack == MAP_FAILED)
> exit(EXIT_FAILURE);
>
> /*
> * Note that legacy clone() has different argument ordering on
> * different architectures so this won't work everywhere.
> */
> pid = syscall(189 /* __NR_clone2 */, SIGCHLD, stack, STACK_SIZE, NULL, NULL);
Please note that even on ia64 the stack grows down but in contrast to
all other architectures ia64 expects the _lowest_ address to be given
and will add STACK_SIZE to stack itself in copy_thread{_tls}(). (This is
all fixed in clone3() where you're always expected to pass down the
lowest address and the kernel figures it out for you.)
So this is intentional.
Christian
On Mai 14 2020, Christian Brauner wrote:
>> static inline pid_t raw_clone(unsigned long flags) {
>> pid_t ret;
>>
>> assert((flags & (CLONE_VM|CLONE_PARENT_SETTID|CLONE_CHILD_SETTID|
>> CLONE_CHILD_CLEARTID|CLONE_SETTLS)) == 0);
>> #if defined(__s390x__) || defined(__s390__) || defined(__CRIS__)
>> /* On s390/s390x and cris the order of the first and second arguments
>> * of the raw clone() system call is reversed. */
>> ret = (pid_t) syscall(__NR_clone, NULL, flags);
>> #elif defined(__sparc__)
>> {
>> /**
>> * sparc always returns the other process id in %o0, and
>> * a boolean flag whether this is the child or the parent in
>> * %o1. Inline assembly is needed to get the flag returned
>> * in %o1.
>> */
>> int in_child, child_pid, error;
>>
>> asm volatile("mov %3, %%g1\n\t"
>> "mov %4, %%o0\n\t"
>> "mov 0 , %%o1\n\t"
>> #if defined(__arch64__)
>> "t 0x6d\n\t"
>> #else
>> "t 0x10\n\t"
>> #endif
>> "addx %%g0, 0, %2\n\t"
>> "mov %%o1, %0\n\t"
>> "mov %%o0, %1" :
>> "=r"(in_child), "=r"(child_pid), "=r"(error) :
>> "i"(__NR_clone), "r"(flags) :
>> "%o1", "%o0", "%g1", "cc" );
>>
>> if (error) {
>> errno = child_pid;
>> ret = -1;
>> } else
>> ret = in_child ? 0 : child_pid;
>> }
>> +#elif defined(__ia64__)
>> + /* On ia64 the stack and stack size are passed as separate arguments. */
>> + return (pid_t)syscall(__NR_clone, flags, NULL, 0);
>> +#else
>> + return (pid_t)syscall(__NR_clone, flags, NULL);
>> +#endif
>
> Scratch that. It's even worse. On ia64 it is _invalid_ to pass a NULL
> stack.
Only if you want CLONE_VM. But this raw_clone does not allow CLONE_VM,
thus it is actually a true fork.
Andreas.
--
Andreas Schwab, [email protected]
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1
"And now for something completely different."
On Thu, May 14, 2020 at 12:35:49PM +0200, John Paul Adrian Glaubitz wrote:
> On 5/14/20 12:32 PM, Christian Brauner wrote:
> > Do you have a very minimalistic ia64 userspace preferably without systemd where
> > you could simply test. That should give us an idea whether things work:
> >
> > #define _GNU_SOURCE
> > #include <sys/wait.h>
> > #include <sys/utsname.h>
> > #include <sched.h>
> > #include <string.h>
> > #include <stdio.h>
> > #include <stdlib.h>
> > #include <unistd.h>
> > #include <sys/mman.h>
> >
> > #define STACK_SIZE (8 * 1024 * 1024) /* standard stack size for threads in glibc */
> >
> > int main(int argc, char *argv[])
> > {
> > char *stack;
> > pid_t pid;
> >
> > stack = mmap(NULL, STACK_SIZE, PROT_READ | PROT_WRITE,
> > MAP_PRIVATE | MAP_ANONYMOUS | MAP_STACK, -1, 0);
> > if (stack == MAP_FAILED)
> > exit(EXIT_FAILURE);
> >
> > /*
> > * Note that legacy clone() has different argument ordering on
> > * different architectures so this won't work everywhere.
> > */
> > pid = syscall(189 /* __NR_clone2 */, SIGCHLD, stack, STACK_SIZE, NULL, NULL);
> > if (pid < 0)
> > exit(EXIT_FAILURE);
> > if (pid == 0)
> > exit(EXIT_SUCCESS);
> > if (wait(NULL) != pid)
> > exit(EXIT_FAILURE);
> >
> > exit(EXIT_SUCCESS);
> > }
>
> root@titanium:~# gcc systemd_test.c -o systemd_test
> root@titanium:~# ./systemd_test
> root@titanium:~# echo $?
> 1
> root@titanium:~#
>
> I can also give you access to this machine.
Yes please! :)
My ssh key should be on
https://launchpad.net/~cbrauner
Christian
On Mai 14 2020, Christian Brauner wrote:
> pid = syscall(189 /* __NR_clone2 */, SIGCHLD, stack, STACK_SIZE, NULL, NULL);
Syscall 189 doesn't exist on ia64, they start with 1024 (and __NR_clone2
is 1213).
Andreas.
--
Andreas Schwab, [email protected]
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510 2552 DF73 E780 A9DA AEC1
"And now for something completely different."
On Thu, May 14, 2020 at 12:45:41PM +0200, Andreas Schwab wrote:
> On Mai 14 2020, Christian Brauner wrote:
>
> > pid = syscall(189 /* __NR_clone2 */, SIGCHLD, stack, STACK_SIZE, NULL, NULL);
>
> Syscall 189 doesn't exist on ia64, they start with 1024 (and __NR_clone2
> is 1213).
Ugh, the numbering needs to be:
# Add 1024 to <number> will get the actual system call number
well ok.
Thanks!
Christian
On Thu, May 14, 2020 at 11:48:31AM +0200, John Paul Adrian Glaubitz wrote:
>
>
> On 5/14/20 9:53 AM, John Paul Adrian Glaubitz wrote:
> > The kernel is currently building, you should get it by the evening (CEST).
> >
> > The machine also serves as a Debian buildd which is why it's a bit more
> > busy than other servers.
> I applied the patch against 5.7-rc5, it applied fine and builds fine.
>
> It seems though that systemd crashes but I'm not sure it's related:
>
> [ 14.267489] init[1]: General Exception: IA-64 Reserved Register/Field fault (data access) 2199023255600 [1]
> [ 14.267489] Modules linked in:
> [ 14.267489]
> [ 14.267489] CPU: 0 PID: 1 Comm: init Not tainted 5.7.0-rc5 #1
> [ 14.267489] Hardware name: hp server BL860c , BIOS 04.25 07/02/2010
> [ 14.267489] psr : 0000121008526030 ifs : 8000000000000510 ip : [<a00000010000c541>] Not tainted (5.7.0-rc5)
> [ 14.267489] ip is at sys_clone2+0x61/0x80
> [ 14.267489] unat: 0000000000000000 pfs : 0000000000000510 rsc : 0000000000000003
> [ 14.267489] rnat: 0000000000000000 bsps: 0000000000000000 pr : 000000000001a959
> [ 14.267489] ldrs: 0000000000000000 ccv : 0000000000000002 fpsr: 0009804c8a70033f
> [ 14.267489] csd : 0000000000000000 ssd : 0000000000000000
> [ 14.267489] b0 : a00000010000c540 b6 : a0000001000edf80 b7 : a000000100369560
> [ 14.267489] f6 : 1003e000000000000b8a4 f7 : 1003e00000001754e8d69
> [ 14.267489] f8 : 1003e0000000178311d69 f9 : 100098000000000000000
> [ 14.267489] f10 : 1000eb8a3fffff475c000 f11 : 1003e000000000000b8a4
> [ 14.267489] r1 : a0000001017a6f10 r2 : 0000000000000000 r3 : 0000000000000006
> [ 14.267489] r8 : 0000000000000066 r9 : 0000000000000060 r10 : e0000100800d8e68
> [ 14.267489] r11 : 0000000000000002 r12 : e00001008027fe30 r13 : e000010080278000
> [ 14.267489] r14 : e0000100800d8e04 r15 : e0000100800d8e08 r16 : 0000000000000002
> [ 14.267489] r17 : 0000000000000001 r18 : 0000000000000002 r19 : 0000000000000003
> [ 14.267489] r20 : 0000000000000004 r21 : e00001008027fb98 r22 : e000000001146d02
> [ 14.267489] r23 : 00000000000030c8 r24 : 00000000000030c8 r25 : 00000000000030ca
> [ 14.267489] r26 : 00000000000030ca r27 : 00000000000030ca r28 : e00001008027fb90
> [ 14.267489] r29 : 0000000000000400 r30 : 00000000000002e8 r31 : 0000000000000400
> [ 14.267489]
> [ 14.267489] Call Trace:
> [ 14.267489] [<a000000100015090>] show_stack+0x90/0xc0
> [ 14.267489] sp=e00001008027f9f0 bsp=e000010080279288
> [ 14.267489] [<a000000100015780>] show_regs+0x6c0/0xa20
> [ 14.267489] sp=e00001008027fbc0 bsp=e000010080279218
> [ 14.267489] [<a000000100029590>] die+0x1b0/0x4a0
> [ 14.267489] sp=e00001008027fbe0 bsp=e0000100802791d0
> [ 14.267489] [<a00000010100e690>] ia64_fault+0x770/0xd80
> [ 14.267489] sp=e00001008027fbe0 bsp=e000010080279180
> [ 14.267489] [<a00000010000ce60>] ia64_leave_kernel+0x0/0x270
> [ 14.267489] sp=e00001008027fc60 bsp=e000010080279180
> [ 14.267489] [<a00000010000c540>] sys_clone2+0x60/0x80
> [ 14.267489] sp=e00001008027fe30 bsp=e000010080279100
> [ 14.267489] Disabling lock debugging due to kernel taint
> [ 14.267489] init[102]: General Exception: IA-64 Reserved Register/Field fault (data access) 4398046511152 [2]
> [ 14.350730] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
> [1 4 . 31540.733500]7 3-0-]- [- -e-n[d eKnedr nKeelr npealn pani[c - 1n4o.t2 6s7y4n8c9i]n g:
> :Attem [D 1 4 . 2 6 7 4 8 95]. 7C.P0U-:r c15 P#I1D
> i e1=002x 0C0o0m0m0:0 0ibn i]t- -T-a
> nted: G D 5.7.0-rc5 #1
>
> I have to perform more testing. The machine boots fine with a 5.5.x kernel.
Ok, so the good news is that from our other mail exchange we know that
it doesn't boot on 5.6 without my patch too.
Christian