Hi, Willy
nolibc for riscv is only tested for rv64 currently (see
tools/testing/selftests/nolibc/Makefile), this patchset tries to let it
compile for rv32, but still not pass the nolibc selftest:
* The first patch uses lw/sw instead of ld/sd for rv32 and verse-vice for rv64
* This patch may conflict with the stackprotector patch [1], because
both of them changed the _start assembly in arch-riscv.h
* The second patch adds __NR_llseek based sys_lseek implementation for rv32
* There is no __NR_lseek for rv32, see include/uapi/asm-generic/unistd.h
* This code is based on the version from glibc, sysdeps/unix/sysv/linux/lseek.c
* It passed the two lseek tests in nolibc selftest (write a test case manually)
* To let it compile for rv32, we still need to apply one of such actions:
* Revert the kernel commit d4c08b9776b3 ("riscv: Use latest system call ABI"),
but it is not the right direction, that commit has removed all of the time32 syscalls,
and let C lib (e.g. glibc) provide the same C APIs based on the other time64 syscalls
* If not really use any of the time32 syscalls, defining __ARCH_WANT_TIME32_SYSCALLS
macro will let it compile, but this is buggy for the current implmentations are based
on time32 syscalls!
* Really implement the C APIs for rv32, based on the time64 syscalls, just like glibc.
This commit c8ce48f06503 ("asm-generic: Make time32 syscall numbers optional") shows
us which functions should be re-implemented.
So, the work todo for rv32 is:
* Rebasing all of the old time32 syscalls based C APIs on the new time64 syscalls,
but they are not simply mapped one by one, glibc is a good reference.
* Add standalone rv32 test support in tools/testing/selftests/nolibc/
Best Regards,
Zhangjin Wu
[1]: https://lore.kernel.org/linux-riscv/mhng-1ec176a9-ec5d-470b-a278-a4e9cec728a8@palmer-ri-x1c9a/
Zhangjin Wu (2):
tools/nolibc: riscv: Fix up load/store instructions for rv32
tools/nolibc: riscv: Support __NR_llseek for rv32
tools/include/nolibc/arch-riscv.h | 14 +++++++++-----
tools/include/nolibc/std.h | 1 +
tools/include/nolibc/sys.h | 19 +++++++++++++++++++
3 files changed, 29 insertions(+), 5 deletions(-)
--
2.25.1
Signed-off-by: Zhangjin Wu <[email protected]>
---
tools/include/nolibc/arch-riscv.h | 14 +++++++++-----
1 file changed, 9 insertions(+), 5 deletions(-)
diff --git a/tools/include/nolibc/arch-riscv.h b/tools/include/nolibc/arch-riscv.h
index e197fcb10ac0..7d7c4beabb8d 100644
--- a/tools/include/nolibc/arch-riscv.h
+++ b/tools/include/nolibc/arch-riscv.h
@@ -33,9 +33,13 @@ struct sys_stat_struct {
#if __riscv_xlen == 64
#define PTRLOG "3"
#define SZREG "8"
+#define REG_L "ld"
+#define REG_S "sd"
#elif __riscv_xlen == 32
#define PTRLOG "2"
#define SZREG "4"
+#define REG_L "lw"
+#define REG_S "sw"
#endif
/* Syscalls for RISCV :
@@ -181,7 +185,7 @@ void __attribute__((weak,noreturn,optimize("omit-frame-pointer"))) _start(void)
".option norelax\n"
"lla gp, __global_pointer$\n"
".option pop\n"
- "lw a0, 0(sp)\n" // argc (a0) was in the stack
+ REG_L" a0, 0(sp)\n" // argc (a0) was in the stack
"add a1, sp, "SZREG"\n" // argv (a1) = sp
"slli a2, a0, "PTRLOG"\n" // envp (a2) = SZREG*argc ...
"add a2, a2, "SZREG"\n" // + SZREG (skip null)
@@ -189,14 +193,14 @@ void __attribute__((weak,noreturn,optimize("omit-frame-pointer"))) _start(void)
"add a3, a2, zero\n" // iterate a3 over envp to find auxv (after NULL)
"0:\n" // do {
- "ld a4, 0(a3)\n" // a4 = *a3;
+ REG_L" a4, 0(a3)\n" // a4 = *a3;
"add a3, a3, "SZREG"\n" // a3 += sizeof(void*);
"bne a4, zero, 0b\n" // } while (a4);
"lui a4, %hi(_auxv)\n" // a4 = &_auxv (high bits)
- "sd a3, %lo(_auxv)(a4)\n" // store a3 into _auxv
+ REG_S" a3, %lo(_auxv)(a4)\n" // store a3 into _auxv
- "lui a3, %hi(environ)\n" // a3 = &environ (high bits)
- "sd a2,%lo(environ)(a3)\n" // store envp(a2) into environ
+ "lui a3, %hi(environ)\n" // a3 = &environ (high bits)
+ REG_S" a2,%lo(environ)(a3)\n"// store envp(a2) into environ
"andi sp,a1,-16\n" // sp must be 16-byte aligned
"call main\n" // main() returns the status code, we'll exit with it.
"li a7, 93\n" // NR_exit == 93
--
2.25.1
Hi Zhangjin,
On Fri, May 19, 2023 at 01:00:18AM +0800, Zhangjin Wu wrote:
> Hi, Willy
>
> nolibc for riscv is only tested for rv64 currently (see
> tools/testing/selftests/nolibc/Makefile), this patchset tries to let it
> compile for rv32, but still not pass the nolibc selftest:
>
> * The first patch uses lw/sw instead of ld/sd for rv32 and verse-vice for rv64
> * This patch may conflict with the stackprotector patch [1], because
> both of them changed the _start assembly in arch-riscv.h
That's quite embarrassing, I'm having to trace of that series here. Now
I can find it in my LKML archives, but I don't have the direct message and
didn't spot the other ones. I'll have to investigate, thanks for notifying
me! I'm CCing Thomas, I will check with him how to best merge the two.
> * The second patch adds __NR_llseek based sys_lseek implementation for rv32
> * There is no __NR_lseek for rv32, see include/uapi/asm-generic/unistd.h
> * This code is based on the version from glibc, sysdeps/unix/sysv/linux/lseek.c
> * It passed the two lseek tests in nolibc selftest (write a test case manually)
OK.
> * To let it compile for rv32, we still need to apply one of such actions:
> * Revert the kernel commit d4c08b9776b3 ("riscv: Use latest system call ABI"),
> but it is not the right direction, that commit has removed all of the time32 syscalls,
> and let C lib (e.g. glibc) provide the same C APIs based on the other time64 syscalls
>
> * If not really use any of the time32 syscalls, defining __ARCH_WANT_TIME32_SYSCALLS
> macro will let it compile, but this is buggy for the current implmentations are based
> on time32 syscalls!
>
> * Really implement the C APIs for rv32, based on the time64 syscalls, just like glibc.
> This commit c8ce48f06503 ("asm-generic: Make time32 syscall numbers optional") shows
> us which functions should be re-implemented.
>
> So, the work todo for rv32 is:
>
> * Rebasing all of the old time32 syscalls based C APIs on the new time64 syscalls,
> but they are not simply mapped one by one, glibc is a good reference.
>
> * Add standalone rv32 test support in tools/testing/selftests/nolibc/
I'm not the right one to judge how to best support rv32 but at least I just
don't want to go backwards. I'm just having a probably stupid question, but
how relevant is rv32 ? I mean, all the boards I've seen to date were based
on rv64 even the smallest embedded ones, so I'm sincerely wondering if there
exists at all any rv32 devices capable of running Linux. Because if that's
not the case, maybe we should instead declare that we only support rv64 ?
If such devices exist however, I'm all for us supporting them well.
Thanks,
Willy
Hi Willy,
On 2023-05-19 11:40:30+0200, Willy Tarreau wrote:
> Hi Zhangjin,
>
> On Fri, May 19, 2023 at 01:00:18AM +0800, Zhangjin Wu wrote:
> > Hi, Willy
> >
> > nolibc for riscv is only tested for rv64 currently (see
> > tools/testing/selftests/nolibc/Makefile), this patchset tries to let it
> > compile for rv32, but still not pass the nolibc selftest:
> >
> > * The first patch uses lw/sw instead of ld/sd for rv32 and verse-vice for rv64
> > * This patch may conflict with the stackprotector patch [1], because
> > both of them changed the _start assembly in arch-riscv.h
>
> That's quite embarrassing, I'm having to trace of that series here. Now
> I can find it in my LKML archives, but I don't have the direct message and
> didn't spot the other ones. I'll have to investigate, thanks for notifying
> me! I'm CCing Thomas, I will check with him how to best merge the two.
I think the conflict should be trivial to fix.
I can also resend my series or just the single riscv patch.
<snip>
Thomas
On Fri, May 19, 2023 at 12:11:00PM +0200, Thomas Wei?schuh wrote:
> Hi Willy,
>
> On 2023-05-19 11:40:30+0200, Willy Tarreau wrote:
> > Hi Zhangjin,
> >
> > On Fri, May 19, 2023 at 01:00:18AM +0800, Zhangjin Wu wrote:
> > > Hi, Willy
> > >
> > > nolibc for riscv is only tested for rv64 currently (see
> > > tools/testing/selftests/nolibc/Makefile), this patchset tries to let it
> > > compile for rv32, but still not pass the nolibc selftest:
> > >
> > > * The first patch uses lw/sw instead of ld/sd for rv32 and verse-vice for rv64
> > > * This patch may conflict with the stackprotector patch [1], because
> > > both of them changed the _start assembly in arch-riscv.h
> >
> > That's quite embarrassing, I'm having to trace of that series here. Now
> > I can find it in my LKML archives, but I don't have the direct message and
> > didn't spot the other ones. I'll have to investigate, thanks for notifying
> > me! I'm CCing Thomas, I will check with him how to best merge the two.
>
> I think the conflict should be trivial to fix.
>
> I can also resend my series or just the single riscv patch.
OK then I'll pick Zhangjin's series and will apply yours on top of it.
Do not bother resending the whole series, only the riscv patch will be
sufficient, I have the rest of your series in my lkml mbox.
Thanks!
Willy
On 2023-05-19 12:19:10+0200, Willy Tarreau wrote:
> On Fri, May 19, 2023 at 12:11:00PM +0200, Thomas Weißschuh wrote:
> > Hi Willy,
> >
> > On 2023-05-19 11:40:30+0200, Willy Tarreau wrote:
> > > Hi Zhangjin,
> > >
> > > On Fri, May 19, 2023 at 01:00:18AM +0800, Zhangjin Wu wrote:
> > > > Hi, Willy
> > > >
> > > > nolibc for riscv is only tested for rv64 currently (see
> > > > tools/testing/selftests/nolibc/Makefile), this patchset tries to let it
> > > > compile for rv32, but still not pass the nolibc selftest:
> > > >
> > > > * The first patch uses lw/sw instead of ld/sd for rv32 and verse-vice for rv64
> > > > * This patch may conflict with the stackprotector patch [1], because
> > > > both of them changed the _start assembly in arch-riscv.h
> > >
> > > That's quite embarrassing, I'm having to trace of that series here. Now
> > > I can find it in my LKML archives, but I don't have the direct message and
> > > didn't spot the other ones. I'll have to investigate, thanks for notifying
> > > me! I'm CCing Thomas, I will check with him how to best merge the two.
> >
> > I think the conflict should be trivial to fix.
> >
> > I can also resend my series or just the single riscv patch.
>
> OK then I'll pick Zhangjin's series and will apply yours on top of it.
> Do not bother resending the whole series, only the riscv patch will be
> sufficient, I have the rest of your series in my lkml mbox.
Could you let me know if or when your are publishing your integration
branch?
Thomas
Hi Thomas,
On Sat, May 20, 2023 at 09:18:25AM +0200, Thomas Wei?schuh wrote:
> On 2023-05-19 12:19:10+0200, Willy Tarreau wrote:
> > On Fri, May 19, 2023 at 12:11:00PM +0200, Thomas Wei?schuh wrote:
> > > Hi Willy,
> > >
> > > On 2023-05-19 11:40:30+0200, Willy Tarreau wrote:
> > > > Hi Zhangjin,
> > > >
> > > > On Fri, May 19, 2023 at 01:00:18AM +0800, Zhangjin Wu wrote:
> > > > > Hi, Willy
> > > > >
> > > > > nolibc for riscv is only tested for rv64 currently (see
> > > > > tools/testing/selftests/nolibc/Makefile), this patchset tries to let it
> > > > > compile for rv32, but still not pass the nolibc selftest:
> > > > >
> > > > > * The first patch uses lw/sw instead of ld/sd for rv32 and verse-vice for rv64
> > > > > * This patch may conflict with the stackprotector patch [1], because
> > > > > both of them changed the _start assembly in arch-riscv.h
> > > >
> > > > That's quite embarrassing, I'm having to trace of that series here. Now
> > > > I can find it in my LKML archives, but I don't have the direct message and
> > > > didn't spot the other ones. I'll have to investigate, thanks for notifying
> > > > me! I'm CCing Thomas, I will check with him how to best merge the two.
> > >
> > > I think the conflict should be trivial to fix.
> > >
> > > I can also resend my series or just the single riscv patch.
> >
> > OK then I'll pick Zhangjin's series and will apply yours on top of it.
> > Do not bother resending the whole series, only the riscv patch will be
> > sufficient, I have the rest of your series in my lkml mbox.
>
> Could you let me know if or when your are publishing your integration
> branch?
Just finished it now, I had to manually rebase all patches and had to
interrupt it yesterday. You'll find it here:
https://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git 20230520-nolibc-rv32+stkp
It contains a temporary series made of:
- Zhangjin's RV32 patches
- your syscall() patch
- your stkp series except the riscv one
so that you just need to update the riscv one and that will be fine. Be
careful, I have *not* restested the riscv port. As usual, do not hesitate
to le me know if I messed up with anything.
Thanks!
Willy
Hi Zhangjin,
On Fri, May 19, 2023 at 01:02:12AM +0800, Zhangjin Wu wrote:
> Signed-off-by: Zhangjin Wu <[email protected]>
> ---
> tools/include/nolibc/arch-riscv.h | 14 +++++++++-----
> 1 file changed, 9 insertions(+), 5 deletions(-)
I'm having a comment regarding this one (which for now I've queued so that
Thomas can rebase his series), please provide a real commit message that
explains the purpose of the change and the solutions chosen. Feel free
to copy-paste from your cover letter if that fits, it's just that I do
want to see some justification for a change in the commit message itself
(i.e think about the poor person seeing a bisect stop on that one).
You can send me a paragraph or two and I'll happily copy them into my
local copy of the rebased commit.
Thank you!
Willy
Hi, Willy
This is a full commit message for this patch:
When compile for rv32, we got such error:
---
nolibc/sysroot/riscv/include/arch.h:190: Error: unrecognized opcode `ld a4,0(a3)'
nolibc/sysroot/riscv/include/arch.h:194: Error: unrecognized opcode `sd a3,%lo(_auxv)(a4)'
nolibc/sysroot/riscv/include/arch.h:196: Error: unrecognized opcode `sd a2,%lo(environ)(a3)'
Refer to arch/riscv/include/asm/asm.h and add REG_L/REG_S macros here to let
rv32 use its own lw/sw instructions.
---
I will send a new version with the above full message for you, wait for a while, very sorry ;-)
Best Regards,
Zhangjin Wu
> Hi Zhangjin,
>
> On Fri, May 19, 2023 at 01:02:12AM +0800, Zhangjin Wu wrote:
> > Signed-off-by: Zhangjin Wu <[email protected]>
> > ---
> > tools/include/nolibc/arch-riscv.h | 14 +++++++++-----
> > 1 file changed, 9 insertions(+), 5 deletions(-)
>
> I'm having a comment regarding this one (which for now I've queued so that
> Thomas can rebase his series), please provide a real commit message that
> explains the purpose of the change and the solutions chosen. Feel free
> to copy-paste from your cover letter if that fits, it's just that I do
> want to see some justification for a change in the commit message itself
> (i.e think about the poor person seeing a bisect stop on that one).
>
> You can send me a paragraph or two and I'll happily copy them into my
> local copy of the rebased commit.
>
> Thank you!
> Willy
On Sat, May 20, 2023 at 05:11:44PM +0800, Zhangjin Wu wrote:
> Hi, Willy
>
> This is a full commit message for this patch:
>
> When compile for rv32, we got such error:
>
> ---
>
> nolibc/sysroot/riscv/include/arch.h:190: Error: unrecognized opcode `ld a4,0(a3)'
> nolibc/sysroot/riscv/include/arch.h:194: Error: unrecognized opcode `sd a3,%lo(_auxv)(a4)'
> nolibc/sysroot/riscv/include/arch.h:196: Error: unrecognized opcode `sd a2,%lo(environ)(a3)'
>
> Refer to arch/riscv/include/asm/asm.h and add REG_L/REG_S macros here to let
> rv32 use its own lw/sw instructions.
>
> ---
That's fine, thank you!
> I will send a new version with the above full message for you, wait for a
> while, very sorry ;-)
Don't waste your time resending, I can perfectly take that one and
put it into the series.
Thanks!
Willy
On Sat, May 20, 2023 at 06:05:10PM +0800, Zhangjin Wu wrote:
> > On Sat, May 20, 2023 at 05:11:44PM +0800, Zhangjin Wu wrote:
> > > Hi, Willy
> > >
> > > This is a full commit message for this patch:
> > >
> > > When compile for rv32, we got such error:
> > >
> > > ---
> > >
> > > nolibc/sysroot/riscv/include/arch.h:190: Error: unrecognized opcode `ld a4,0(a3)'
> > > nolibc/sysroot/riscv/include/arch.h:194: Error: unrecognized opcode `sd a3,%lo(_auxv)(a4)'
> > > nolibc/sysroot/riscv/include/arch.h:196: Error: unrecognized opcode `sd a2,%lo(environ)(a3)'
> > >
> > > Refer to arch/riscv/include/asm/asm.h and add REG_L/REG_S macros here to let
> > > rv32 use its own lw/sw instructions.
> > >
> > > ---
> >
> > That's fine, thank you!
> >
> > > I will send a new version with the above full message for you, wait for a
> > > while, very sorry ;-)
> >
> > Don't waste your time resending, I can perfectly take that one and
> > put it into the series.
> >
>
> Thanks very much, just found the first `---` is in the wrong line, please
> remove the '---' lines manually ;-)
>
> When compile nolibc application for rv32, we got such errors:
>
> nolibc/sysroot/riscv/include/arch.h:190: Error: unrecognized opcode `ld a4,0(a3)'
> nolibc/sysroot/riscv/include/arch.h:194: Error: unrecognized opcode `sd a3,%lo(_auxv)(a4)'
> nolibc/sysroot/riscv/include/arch.h:196: Error: unrecognized opcode `sd a2,%lo(environ)(a3)'
>
> Refer to arch/riscv/include/asm/asm.h and add REG_L/REG_S macros here to let
> rv32 uses its own lw/sw instructions.
>
> Btw, I wrote some new comments for the 2nd __NR_lseek patch, it is:
>
> riscv uses the generic include/uapi/asm-generic/unistd.h, it has code
> like this:
>
> #if __BITS_PER_LONG == 64 && !defined(__SYSCALL_COMPAT)
> #define __NR_lseek __NR3264_lseek
> #else
> #define __NR_llseek __NR3264_lseek
> #endif
>
> There is no __NR_lseek for rv32, use __NR_llseek instead.
>
> This code is based on sysdeps/unix/sysv/linux/lseek.c of glibc.
Many thanks, it's indeed better this way.
> The preceeding 4 white spaces are not required for real commit messages.
Sure ;-)
Thanks
willy
> On Sat, May 20, 2023 at 05:11:44PM +0800, Zhangjin Wu wrote:
> > Hi, Willy
> >
> > This is a full commit message for this patch:
> >
> > When compile for rv32, we got such error:
> >
> > ---
> >
> > nolibc/sysroot/riscv/include/arch.h:190: Error: unrecognized opcode `ld a4,0(a3)'
> > nolibc/sysroot/riscv/include/arch.h:194: Error: unrecognized opcode `sd a3,%lo(_auxv)(a4)'
> > nolibc/sysroot/riscv/include/arch.h:196: Error: unrecognized opcode `sd a2,%lo(environ)(a3)'
> >
> > Refer to arch/riscv/include/asm/asm.h and add REG_L/REG_S macros here to let
> > rv32 use its own lw/sw instructions.
> >
> > ---
>
> That's fine, thank you!
>
> > I will send a new version with the above full message for you, wait for a
> > while, very sorry ;-)
>
> Don't waste your time resending, I can perfectly take that one and
> put it into the series.
>
Thanks very much, just found the first `---` is in the wrong line, please
remove the '---' lines manually ;-)
When compile nolibc application for rv32, we got such errors:
nolibc/sysroot/riscv/include/arch.h:190: Error: unrecognized opcode `ld a4,0(a3)'
nolibc/sysroot/riscv/include/arch.h:194: Error: unrecognized opcode `sd a3,%lo(_auxv)(a4)'
nolibc/sysroot/riscv/include/arch.h:196: Error: unrecognized opcode `sd a2,%lo(environ)(a3)'
Refer to arch/riscv/include/asm/asm.h and add REG_L/REG_S macros here to let
rv32 uses its own lw/sw instructions.
Btw, I wrote some new comments for the 2nd __NR_lseek patch, it is:
riscv uses the generic include/uapi/asm-generic/unistd.h, it has code
like this:
#if __BITS_PER_LONG == 64 && !defined(__SYSCALL_COMPAT)
#define __NR_lseek __NR3264_lseek
#else
#define __NR_llseek __NR3264_lseek
#endif
There is no __NR_lseek for rv32, use __NR_llseek instead.
This code is based on sysdeps/unix/sysv/linux/lseek.c of glibc.
The preceeding 4 white spaces are not required for real commit messages.
Best Regards,
Zhangjin Wu
> Thanks!
> Willy
Hi Willy,
> Hi Zhangjin,
>
> On Fri, May 19, 2023 at 01:00:18AM +0800, Zhangjin Wu wrote:
> ...
>
> > * To let it compile for rv32, we still need to apply one of such actions:
> > * Revert the kernel commit d4c08b9776b3 ("riscv: Use latest system call ABI"),
> > but it is not the right direction, that commit has removed all of the time32 syscalls,
> > and let C lib (e.g. glibc) provide the same C APIs based on the other time64 syscalls
> >
> > * If not really use any of the time32 syscalls, defining __ARCH_WANT_TIME32_SYSCALLS
> > macro will let it compile, but this is buggy for the current implmentations are based
> > on time32 syscalls!
> >
> > * Really implement the C APIs for rv32, based on the time64 syscalls, just like glibc.
> > This commit c8ce48f06503 ("asm-generic: Make time32 syscall numbers optional") shows
> > us which functions should be re-implemented.
> >
> > So, the work todo for rv32 is:
> >
> > * Rebasing all of the old time32 syscalls based C APIs on the new time64 syscalls,
> > but they are not simply mapped one by one, glibc is a good reference.
> >
> > * Add standalone rv32 test support in tools/testing/selftests/nolibc/
>
> I'm not the right one to judge how to best support rv32 but at least I just
> don't want to go backwards. I'm just having a probably stupid question, but
> how relevant is rv32 ? I mean, all the boards I've seen to date were based
> on rv64 even the smallest embedded ones, so I'm sincerely wondering if there
> exists at all any rv32 devices capable of running Linux. Because if that's
> not the case, maybe we should instead declare that we only support rv64 ?
> If such devices exist however, I'm all for us supporting them well.
>
Firstly, as the commit c8ce48f06503 ("asm-generic: Make time32 syscall
numbers optional") shows:
We don't want new architectures to even provide the old 32-bit time_t
based system calls any more, or define the syscall number macros.
So, this is not rv32 specific, more and more architectures are trying to
use the generic unistd.h (include/uapi/asm-generic/unistd.h), but rv32
may be the first new architecture variant who have no time32 syscalls.
Second, I did search some rv32 socs/boards from two companies, they are
bl602/bl616/bl702, esp32-c2/c3/c6, some of them even have 532KB sRAM which is
enough for nolibc-based app + linux kernel, I have gotten 334K rv64 vmlinuz
(+nolibc hello.c) in the tinylinux work for riscv, the future work may be
running linux on such a real rv32 board ;-)
Best regards,
Zhangjin Wu
> Thanks,
> Willy