Hello all,
I've uploaded my kexec-tools patches for riscv on github for
testing/review before sending them upstream.
https://github.com/riscv-collab/kexec-tools/tree/riscv
Both kexec and kdump were tested on latest rv64 qemu, using for-next
branch with this patch applied:
https://patchwork.kernel.org/project/linux-riscv/patch/[email protected]/
Regards,
Nick
On Tue, Oct 5, 2021 at 2:15 PM Nick Kossifidis <[email protected]> wrote:
>
> Hello all,
>
> I've uploaded my kexec-tools patches for riscv on github for
> testing/review before sending them upstream.
> https://github.com/riscv-collab/kexec-tools/tree/riscv
>
> Both kexec and kdump were tested on latest rv64 qemu, using for-next
> branch with this patch applied:
> https://patchwork.kernel.org/project/linux-riscv/patch/[email protected]/
>
I'll test that asap and review your patch too.
Thanks for your time Nick!
Alex
> Regards,
> Nick
>
> _______________________________________________
> linux-riscv mailing list
> [email protected]
> http://lists.infradead.org/mailman/listinfo/linux-riscv
Hi Nick,
On Tue, Oct 5, 2021 at 4:07 PM Alexandre Ghiti
<[email protected]> wrote:
>
> On Tue, Oct 5, 2021 at 2:15 PM Nick Kossifidis <[email protected]> wrote:
> >
> > Hello all,
> >
> > I've uploaded my kexec-tools patches for riscv on github for
> > testing/review before sending them upstream.
> > https://github.com/riscv-collab/kexec-tools/tree/riscv
> >
> > Both kexec and kdump were tested on latest rv64 qemu, using for-next
> > branch with this patch applied:
> > https://patchwork.kernel.org/project/linux-riscv/patch/[email protected]/
> >
>
> I'll test that asap and review your patch too.
>
> Thanks for your time Nick!
>
> Alex
>
So I followed the instructions here:
https://documentation.suse.com/fr-fr/sles/12-SP3/html/SLES-all/cha-tuning-kexec.html#cha-tuning-kexec-basic-usage,
below the output on an Unmatched board using a vmlinux stored on a sd
card:
ubuntu@ubuntu:~$ sudo sbin/kexec -l vmlinux --append="$(cat
/proc/cmdline)" --initrd=/boot/initrd.img
Warning: No cmdline provided, using append string as cmdline
Warning: No dtb provided, using /sys/firmware/fdt
[ 1813.472671] INFO: task kworker/1:0:988 blocked for more than 120 seconds.
[ 1813.478751] Not tainted 5.15.0-rc1+ #15
[ 1813.483110] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
disables this message.
Could not find a free area of memory of 0x3000 bytes...
locate_hole failed
I used the Ubuntu kernel, so this is pretty large:
-rwxrwxr-x 1 ubuntu ubuntu 277M Oct 5 15:47 vmlinux
-rw-r--r-- 1 root root 98M Sep 21 03:25 /boot/initrd.img
Then if I don't load the initrd (I sometimes have the same warning as
above) I can at least kexec the new kernel but it fails to boot:
ubuntu@ubuntu:~$ sudo ./sbin/kexec -e
Warning: No cmdline or append string provided
Warning: No dtb provided, using /sys/firmware/fdt
[...]
[ 0.000000] SBI v0.2 HSM extension detected
[ 0.000000] CPU with hartid=0 is not available
[ 0.000000] ------------[ cut here ]------------
[ 0.000000] kernel BUG at arch/riscv/kernel/smpboot.c:107!
[ 0.000000] Kernel BUG [#1]
[ 0.000000] Modules linked in:
[ 0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 5.15.0-rc1+ #15
[ 0.000000] Hardware name: SiFive HiFive Unmatched A00 (DT)
[ 0.000000] epc : setup_smp+0xcc/0x142
[ 0.000000] ra : setup_smp+0xc4/0x142
[ 0.000000] epc : ffffffff80a04080 ra : ffffffff80a04078 sp :
ffffffff81803ec0
[ 0.000000] gp : ffffffff81a23220 tp : ffffffff81810500 t0 :
ffffffff81a3551f
[ 0.000000] t1 : ffffffffffffffff t2 : 0000000000000000 s0 :
ffffffff81803f00
[ 0.000000] s1 : 0000000000000000 a0 : 0000000000000000 a1 :
0000000000000000
[ 0.000000] a2 : 0000000000000000 a3 : 0000000000000001 a4 :
0000000000000000
[ 0.000000] a5 : ffffffff80c64500 a6 : 0000000000000004 a7 :
000000000000ff00
[ 0.000000] s2 : 0000000000000005 s3 : 0000000000000000 s4 :
ffffffff8118f9a8
[ 0.000000] s5 : 0000000000000007 s6 : ffffffff80c0b790 s7 :
0000000080000200
[ 0.000000] s8 : 0000000000000fff s9 : 0000000081000200 s10:
0000000000000018
[ 0.000000] s11: 000000000000000b t3 : 0000000000ff0000 t4 :
ffffffffffffffff
[ 0.000000] t5 : ffffffff80c0b7a0 t6 : ffffffff81803bd8
[ 0.000000] status: 0000000200000100 badaddr: 0000000000000000
cause: 0000000000000003
[ 0.000000] [<ffffffff80a04080>] setup_smp+0xcc/0x142
[ 0.000000] [<ffffffff80a03d88>] setup_arch+0x56a/0x590
[ 0.000000] [<ffffffff80a00aa2>] start_kernel+0xaa/0xa5c
[ 0.000000] random: get_random_bytes called from
oops_exit+0x44/0x70 with crng_init=0
[ 0.000000] ---[ end trace 0000000000000000 ]---
[ 0.000000] Kernel panic - not syncing: Attempted to kill the idle task!
[ 0.000000] ---[ end Kernel panic - not syncing: Attempted to kill
the idle task! ]---
This reliably fails here.
> > Regards,
> > Nick
> >
> > _______________________________________________
> > linux-riscv mailing list
> > [email protected]
> > http://lists.infradead.org/mailman/listinfo/linux-riscv
Στις 2021-10-06 14:10, Alexandre Ghiti έγραψε:
>
> So I followed the instructions here:
> https://documentation.suse.com/fr-fr/sles/12-SP3/html/SLES-all/cha-tuning-kexec.html#cha-tuning-kexec-basic-usage,
> below the output on an Unmatched board using a vmlinux stored on a sd
> card:
>
> ubuntu@ubuntu:~$ sudo sbin/kexec -l vmlinux --append="$(cat
> /proc/cmdline)" --initrd=/boot/initrd.img
> Warning: No cmdline provided, using append string as cmdline
> Warning: No dtb provided, using /sys/firmware/fdt
> [ 1813.472671] INFO: task kworker/1:0:988 blocked for more than 120
> seconds.
> [ 1813.478751] Not tainted 5.15.0-rc1+ #15
> [ 1813.483110] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
> disables this message.
> Could not find a free area of memory of 0x3000 bytes...
> locate_hole failed
>
> I used the Ubuntu kernel, so this is pretty large:
> -rwxrwxr-x 1 ubuntu ubuntu 277M Oct 5 15:47 vmlinux
> -rw-r--r-- 1 root root 98M Sep 21 03:25 /boot/initrd.img
>
ACK, I haven't tested initrd much TBH, I usually don't use an initrd,
and when I do it's a small busybox-based rootfs.
> Then if I don't load the initrd (I sometimes have the same warning as
> above) I can at least kexec the new kernel but it fails to boot:
>
> ubuntu@ubuntu:~$ sudo ./sbin/kexec -e
> Warning: No cmdline or append string provided
> Warning: No dtb provided, using /sys/firmware/fdt
> [...]
> [ 0.000000] SBI v0.2 HSM extension detected
> [ 0.000000] CPU with hartid=0 is not available
> [ 0.000000] ------------[ cut here ]------------
> [ 0.000000] kernel BUG at arch/riscv/kernel/smpboot.c:107!
> [ 0.000000] Kernel BUG [#1]
> [ 0.000000] Modules linked in:
> [ 0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 5.15.0-rc1+ #15
> [ 0.000000] Hardware name: SiFive HiFive Unmatched A00 (DT)
> [ 0.000000] epc : setup_smp+0xcc/0x142
> [ 0.000000] ra : setup_smp+0xc4/0x142
> [ 0.000000] epc : ffffffff80a04080 ra : ffffffff80a04078 sp :
> ffffffff81803ec0
> [ 0.000000] gp : ffffffff81a23220 tp : ffffffff81810500 t0 :
> ffffffff81a3551f
> [ 0.000000] t1 : ffffffffffffffff t2 : 0000000000000000 s0 :
> ffffffff81803f00
> [ 0.000000] s1 : 0000000000000000 a0 : 0000000000000000 a1 :
> 0000000000000000
> [ 0.000000] a2 : 0000000000000000 a3 : 0000000000000001 a4 :
> 0000000000000000
> [ 0.000000] a5 : ffffffff80c64500 a6 : 0000000000000004 a7 :
> 000000000000ff00
> [ 0.000000] s2 : 0000000000000005 s3 : 0000000000000000 s4 :
> ffffffff8118f9a8
> [ 0.000000] s5 : 0000000000000007 s6 : ffffffff80c0b790 s7 :
> 0000000080000200
> [ 0.000000] s8 : 0000000000000fff s9 : 0000000081000200 s10:
> 0000000000000018
> [ 0.000000] s11: 000000000000000b t3 : 0000000000ff0000 t4 :
> ffffffffffffffff
> [ 0.000000] t5 : ffffffff80c0b7a0 t6 : ffffffff81803bd8
> [ 0.000000] status: 0000000200000100 badaddr: 0000000000000000
> cause: 0000000000000003
> [ 0.000000] [<ffffffff80a04080>] setup_smp+0xcc/0x142
> [ 0.000000] [<ffffffff80a03d88>] setup_arch+0x56a/0x590
> [ 0.000000] [<ffffffff80a00aa2>] start_kernel+0xaa/0xa5c
> [ 0.000000] random: get_random_bytes called from
> oops_exit+0x44/0x70 with crng_init=0
> [ 0.000000] ---[ end trace 0000000000000000 ]---
> [ 0.000000] Kernel panic - not syncing: Attempted to kill the idle
> task!
> [ 0.000000] ---[ end Kernel panic - not syncing: Attempted to kill
> the idle task! ]---
>
> This reliably fails here.
>
This looks weird, I'll check it out (we have an unmatched here so I'll
try to get my hands on it sometime next week).
Did you try kdump ? Do you get the same error ?
BTW this is what I use for testing most of the time:
For kexec:
kexec -l /mnt/shared/vmlinux --reuse-cmdline
kexec -e
For kdump:
kexec -p /mnt/shared/vmlinux
echo c > /proc/sysrq-trigger
Thanks a lot for your time !
Regards,
Nick
On Sat, Oct 9, 2021 at 3:25 PM Nick Kossifidis <[email protected]> wrote:
>
> Στις 2021-10-06 14:10, Alexandre Ghiti έγραψε:
> >
> > So I followed the instructions here:
> > https://documentation.suse.com/fr-fr/sles/12-SP3/html/SLES-all/cha-tuning-kexec.html#cha-tuning-kexec-basic-usage,
> > below the output on an Unmatched board using a vmlinux stored on a sd
> > card:
> >
> > ubuntu@ubuntu:~$ sudo sbin/kexec -l vmlinux --append="$(cat
> > /proc/cmdline)" --initrd=/boot/initrd.img
> > Warning: No cmdline provided, using append string as cmdline
> > Warning: No dtb provided, using /sys/firmware/fdt
> > [ 1813.472671] INFO: task kworker/1:0:988 blocked for more than 120
> > seconds.
> > [ 1813.478751] Not tainted 5.15.0-rc1+ #15
> > [ 1813.483110] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
> > disables this message.
> > Could not find a free area of memory of 0x3000 bytes...
> > locate_hole failed
> >
> > I used the Ubuntu kernel, so this is pretty large:
> > -rwxrwxr-x 1 ubuntu ubuntu 277M Oct 5 15:47 vmlinux
> > -rw-r--r-- 1 root root 98M Sep 21 03:25 /boot/initrd.img
> >
>
> ACK, I haven't tested initrd much TBH, I usually don't use an initrd,
> and when I do it's a small busybox-based rootfs.
>
> > Then if I don't load the initrd (I sometimes have the same warning as
> > above) I can at least kexec the new kernel but it fails to boot:
> >
> > ubuntu@ubuntu:~$ sudo ./sbin/kexec -e
> > Warning: No cmdline or append string provided
> > Warning: No dtb provided, using /sys/firmware/fdt
> > [...]
> > [ 0.000000] SBI v0.2 HSM extension detected
> > [ 0.000000] CPU with hartid=0 is not available
> > [ 0.000000] ------------[ cut here ]------------
> > [ 0.000000] kernel BUG at arch/riscv/kernel/smpboot.c:107!
> > [ 0.000000] Kernel BUG [#1]
> > [ 0.000000] Modules linked in:
> > [ 0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 5.15.0-rc1+ #15
> > [ 0.000000] Hardware name: SiFive HiFive Unmatched A00 (DT)
> > [ 0.000000] epc : setup_smp+0xcc/0x142
> > [ 0.000000] ra : setup_smp+0xc4/0x142
> > [ 0.000000] epc : ffffffff80a04080 ra : ffffffff80a04078 sp :
> > ffffffff81803ec0
> > [ 0.000000] gp : ffffffff81a23220 tp : ffffffff81810500 t0 :
> > ffffffff81a3551f
> > [ 0.000000] t1 : ffffffffffffffff t2 : 0000000000000000 s0 :
> > ffffffff81803f00
> > [ 0.000000] s1 : 0000000000000000 a0 : 0000000000000000 a1 :
> > 0000000000000000
> > [ 0.000000] a2 : 0000000000000000 a3 : 0000000000000001 a4 :
> > 0000000000000000
> > [ 0.000000] a5 : ffffffff80c64500 a6 : 0000000000000004 a7 :
> > 000000000000ff00
> > [ 0.000000] s2 : 0000000000000005 s3 : 0000000000000000 s4 :
> > ffffffff8118f9a8
> > [ 0.000000] s5 : 0000000000000007 s6 : ffffffff80c0b790 s7 :
> > 0000000080000200
> > [ 0.000000] s8 : 0000000000000fff s9 : 0000000081000200 s10:
> > 0000000000000018
> > [ 0.000000] s11: 000000000000000b t3 : 0000000000ff0000 t4 :
> > ffffffffffffffff
> > [ 0.000000] t5 : ffffffff80c0b7a0 t6 : ffffffff81803bd8
> > [ 0.000000] status: 0000000200000100 badaddr: 0000000000000000
> > cause: 0000000000000003
> > [ 0.000000] [<ffffffff80a04080>] setup_smp+0xcc/0x142
> > [ 0.000000] [<ffffffff80a03d88>] setup_arch+0x56a/0x590
> > [ 0.000000] [<ffffffff80a00aa2>] start_kernel+0xaa/0xa5c
> > [ 0.000000] random: get_random_bytes called from
> > oops_exit+0x44/0x70 with crng_init=0
> > [ 0.000000] ---[ end trace 0000000000000000 ]---
> > [ 0.000000] Kernel panic - not syncing: Attempted to kill the idle
> > task!
> > [ 0.000000] ---[ end Kernel panic - not syncing: Attempted to kill
> > the idle task! ]---
> >
> > This reliably fails here.
> >
>
> This looks weird, I'll check it out (we have an unmatched here so I'll
> try to get my hands on it sometime next week).
>
> Did you try kdump ? Do you get the same error ?
kdump works fine, it fails to find the rootfs but I think my setup is
faulty here.
I took a quick look at kexec_relocate.S, and the use of va_pa_offset
is also wrong here, we should use va_kernel_pa_offset as it is used to
modify a text address But fixing that did not work either.
Alex
>
> BTW this is what I use for testing most of the time:
>
> For kexec:
> kexec -l /mnt/shared/vmlinux --reuse-cmdline
> kexec -e
>
> For kdump:
> kexec -p /mnt/shared/vmlinux
> echo c > /proc/sysrq-trigger
>
> Thanks a lot for your time !
>
> Regards,
> Nick
Στις 2021-10-15 10:07, Alexandre Ghiti έγραψε:
> On Sat, Oct 9, 2021 at 3:25 PM Nick Kossifidis <[email protected]>
> wrote:
>>
>> Στις 2021-10-06 14:10, Alexandre Ghiti έγραψε:
>> >
>> > So I followed the instructions here:
>> > https://documentation.suse.com/fr-fr/sles/12-SP3/html/SLES-all/cha-tuning-kexec.html#cha-tuning-kexec-basic-usage,
>> > below the output on an Unmatched board using a vmlinux stored on a sd
>> > card:
>> >
>> > ubuntu@ubuntu:~$ sudo sbin/kexec -l vmlinux --append="$(cat
>> > /proc/cmdline)" --initrd=/boot/initrd.img
>> > Warning: No cmdline provided, using append string as cmdline
>> > Warning: No dtb provided, using /sys/firmware/fdt
>> > [ 1813.472671] INFO: task kworker/1:0:988 blocked for more than 120
>> > seconds.
>> > [ 1813.478751] Not tainted 5.15.0-rc1+ #15
>> > [ 1813.483110] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
>> > disables this message.
>> > Could not find a free area of memory of 0x3000 bytes...
>> > locate_hole failed
>> >
>> > I used the Ubuntu kernel, so this is pretty large:
>> > -rwxrwxr-x 1 ubuntu ubuntu 277M Oct 5 15:47 vmlinux
>> > -rw-r--r-- 1 root root 98M Sep 21 03:25 /boot/initrd.img
>> >
>>
>> ACK, I haven't tested initrd much TBH, I usually don't use an initrd,
>> and when I do it's a small busybox-based rootfs.
>>
>> > Then if I don't load the initrd (I sometimes have the same warning as
>> > above) I can at least kexec the new kernel but it fails to boot:
>> >
>> > ubuntu@ubuntu:~$ sudo ./sbin/kexec -e
>> > Warning: No cmdline or append string provided
>> > Warning: No dtb provided, using /sys/firmware/fdt
>> > [...]
>> > [ 0.000000] SBI v0.2 HSM extension detected
>> > [ 0.000000] CPU with hartid=0 is not available
>> > [ 0.000000] ------------[ cut here ]------------
>> > [ 0.000000] kernel BUG at arch/riscv/kernel/smpboot.c:107!
>> > [ 0.000000] Kernel BUG [#1]
>> > [ 0.000000] Modules linked in:
>> > [ 0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 5.15.0-rc1+ #15
>> > [ 0.000000] Hardware name: SiFive HiFive Unmatched A00 (DT)
>> > [ 0.000000] epc : setup_smp+0xcc/0x142
>> > [ 0.000000] ra : setup_smp+0xc4/0x142
>> > [ 0.000000] epc : ffffffff80a04080 ra : ffffffff80a04078 sp :
>> > ffffffff81803ec0
>> > [ 0.000000] gp : ffffffff81a23220 tp : ffffffff81810500 t0 :
>> > ffffffff81a3551f
>> > [ 0.000000] t1 : ffffffffffffffff t2 : 0000000000000000 s0 :
>> > ffffffff81803f00
>> > [ 0.000000] s1 : 0000000000000000 a0 : 0000000000000000 a1 :
>> > 0000000000000000
>> > [ 0.000000] a2 : 0000000000000000 a3 : 0000000000000001 a4 :
>> > 0000000000000000
>> > [ 0.000000] a5 : ffffffff80c64500 a6 : 0000000000000004 a7 :
>> > 000000000000ff00
>> > [ 0.000000] s2 : 0000000000000005 s3 : 0000000000000000 s4 :
>> > ffffffff8118f9a8
>> > [ 0.000000] s5 : 0000000000000007 s6 : ffffffff80c0b790 s7 :
>> > 0000000080000200
>> > [ 0.000000] s8 : 0000000000000fff s9 : 0000000081000200 s10:
>> > 0000000000000018
>> > [ 0.000000] s11: 000000000000000b t3 : 0000000000ff0000 t4 :
>> > ffffffffffffffff
>> > [ 0.000000] t5 : ffffffff80c0b7a0 t6 : ffffffff81803bd8
>> > [ 0.000000] status: 0000000200000100 badaddr: 0000000000000000
>> > cause: 0000000000000003
>> > [ 0.000000] [<ffffffff80a04080>] setup_smp+0xcc/0x142
>> > [ 0.000000] [<ffffffff80a03d88>] setup_arch+0x56a/0x590
>> > [ 0.000000] [<ffffffff80a00aa2>] start_kernel+0xaa/0xa5c
>> > [ 0.000000] random: get_random_bytes called from
>> > oops_exit+0x44/0x70 with crng_init=0
>> > [ 0.000000] ---[ end trace 0000000000000000 ]---
>> > [ 0.000000] Kernel panic - not syncing: Attempted to kill the idle
>> > task!
>> > [ 0.000000] ---[ end Kernel panic - not syncing: Attempted to kill
>> > the idle task! ]---
>> >
>> > This reliably fails here.
>> >
>>
>> This looks weird, I'll check it out (we have an unmatched here so I'll
>> try to get my hands on it sometime next week).
>>
>> Did you try kdump ? Do you get the same error ?
>
> kdump works fine, it fails to find the rootfs but I think my setup is
> faulty here.
> I took a quick look at kexec_relocate.S, and the use of va_pa_offset
> is also wrong here, we should use va_kernel_pa_offset as it is used to
> modify a text address But fixing that did not work either.
>
> Alex
>
Sorry for the delay, I finally got some time to work on this. As it
turns out I'm passing cpu id instead of hart id to the next kernel (that
comment on smp.h claiming that raw_smp_processor_id returns the hart id
didn't help much) and interestingly enough cpu id and hart id match on
qemu and also sometimes match on the unmatched / unleashed board. Also
on unmatched / unleashed, hart id 0 is used by the non-linux hart so if
we pass the cpu id of the boot cpu that's always going to be 0 it'll be
invalid, but that's not always the case, especially for kdump. I'm also
getting an error when trying to mount the rootfs on kdump on the
unmatched board, because swiotlb can't allocate bounce buffers and the
pcie driver doesn't work, so no nvme access. I'm looking for a way to
make this work without messing things up, in any case I'll send some
patches over the weekend. For the initrd issue I need to patch
kexec-tools.
Regards,
Nick
hi Nick
On Thu, Nov 25, 2021 at 11:45 PM Nick Kossifidis <[email protected]> wrote:
>
> Στις 2021-10-15 10:07, Alexandre Ghiti έγραψε:
> > On Sat, Oct 9, 2021 at 3:25 PM Nick Kossifidis <[email protected]>
> > wrote:
> >>
> >> Στις 2021-10-06 14:10, Alexandre Ghiti έγραψε:
> >> >
> >> > So I followed the instructions here:
> >> > https://documentation.suse.com/fr-fr/sles/12-SP3/html/SLES-all/cha-tuning-kexec.html#cha-tuning-kexec-basic-usage,
> >> > below the output on an Unmatched board using a vmlinux stored on a sd
> >> > card:
> >> >
> >> > ubuntu@ubuntu:~$ sudo sbin/kexec -l vmlinux --append="$(cat
> >> > /proc/cmdline)" --initrd=/boot/initrd.img
> >> > Warning: No cmdline provided, using append string as cmdline
> >> > Warning: No dtb provided, using /sys/firmware/fdt
> >> > [ 1813.472671] INFO: task kworker/1:0:988 blocked for more than 120
> >> > seconds.
> >> > [ 1813.478751] Not tainted 5.15.0-rc1+ #15
> >> > [ 1813.483110] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
> >> > disables this message.
> >> > Could not find a free area of memory of 0x3000 bytes...
> >> > locate_hole failed
> >> >
> >> > I used the Ubuntu kernel, so this is pretty large:
> >> > -rwxrwxr-x 1 ubuntu ubuntu 277M Oct 5 15:47 vmlinux
> >> > -rw-r--r-- 1 root root 98M Sep 21 03:25 /boot/initrd.img
> >> >
> >>
> >> ACK, I haven't tested initrd much TBH, I usually don't use an initrd,
> >> and when I do it's a small busybox-based rootfs.
> >>
> >> > Then if I don't load the initrd (I sometimes have the same warning as
> >> > above) I can at least kexec the new kernel but it fails to boot:
> >> >
> >> > ubuntu@ubuntu:~$ sudo ./sbin/kexec -e
> >> > Warning: No cmdline or append string provided
> >> > Warning: No dtb provided, using /sys/firmware/fdt
> >> > [...]
> >> > [ 0.000000] SBI v0.2 HSM extension detected
> >> > [ 0.000000] CPU with hartid=0 is not available
> >> > [ 0.000000] ------------[ cut here ]------------
> >> > [ 0.000000] kernel BUG at arch/riscv/kernel/smpboot.c:107!
> >> > [ 0.000000] Kernel BUG [#1]
> >> > [ 0.000000] Modules linked in:
> >> > [ 0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 5.15.0-rc1+ #15
> >> > [ 0.000000] Hardware name: SiFive HiFive Unmatched A00 (DT)
> >> > [ 0.000000] epc : setup_smp+0xcc/0x142
> >> > [ 0.000000] ra : setup_smp+0xc4/0x142
> >> > [ 0.000000] epc : ffffffff80a04080 ra : ffffffff80a04078 sp :
> >> > ffffffff81803ec0
> >> > [ 0.000000] gp : ffffffff81a23220 tp : ffffffff81810500 t0 :
> >> > ffffffff81a3551f
> >> > [ 0.000000] t1 : ffffffffffffffff t2 : 0000000000000000 s0 :
> >> > ffffffff81803f00
> >> > [ 0.000000] s1 : 0000000000000000 a0 : 0000000000000000 a1 :
> >> > 0000000000000000
> >> > [ 0.000000] a2 : 0000000000000000 a3 : 0000000000000001 a4 :
> >> > 0000000000000000
> >> > [ 0.000000] a5 : ffffffff80c64500 a6 : 0000000000000004 a7 :
> >> > 000000000000ff00
> >> > [ 0.000000] s2 : 0000000000000005 s3 : 0000000000000000 s4 :
> >> > ffffffff8118f9a8
> >> > [ 0.000000] s5 : 0000000000000007 s6 : ffffffff80c0b790 s7 :
> >> > 0000000080000200
> >> > [ 0.000000] s8 : 0000000000000fff s9 : 0000000081000200 s10:
> >> > 0000000000000018
> >> > [ 0.000000] s11: 000000000000000b t3 : 0000000000ff0000 t4 :
> >> > ffffffffffffffff
> >> > [ 0.000000] t5 : ffffffff80c0b7a0 t6 : ffffffff81803bd8
> >> > [ 0.000000] status: 0000000200000100 badaddr: 0000000000000000
> >> > cause: 0000000000000003
> >> > [ 0.000000] [<ffffffff80a04080>] setup_smp+0xcc/0x142
> >> > [ 0.000000] [<ffffffff80a03d88>] setup_arch+0x56a/0x590
> >> > [ 0.000000] [<ffffffff80a00aa2>] start_kernel+0xaa/0xa5c
> >> > [ 0.000000] random: get_random_bytes called from
> >> > oops_exit+0x44/0x70 with crng_init=0
> >> > [ 0.000000] ---[ end trace 0000000000000000 ]---
> >> > [ 0.000000] Kernel panic - not syncing: Attempted to kill the idle
> >> > task!
> >> > [ 0.000000] ---[ end Kernel panic - not syncing: Attempted to kill
> >> > the idle task! ]---
> >> >
> >> > This reliably fails here.
> >> >
> >>
> >> This looks weird, I'll check it out (we have an unmatched here so I'll
> >> try to get my hands on it sometime next week).
> >>
> >> Did you try kdump ? Do you get the same error ?
> >
> > kdump works fine, it fails to find the rootfs but I think my setup is
> > faulty here.
> > I took a quick look at kexec_relocate.S, and the use of va_pa_offset
> > is also wrong here, we should use va_kernel_pa_offset as it is used to
> > modify a text address But fixing that did not work either.
> >
> > Alex
> >
>
> Sorry for the delay, I finally got some time to work on this. As it
> turns out I'm passing cpu id instead of hart id to the next kernel (that
> comment on smp.h claiming that raw_smp_processor_id returns the hart id
> didn't help much) and interestingly enough cpu id and hart id match on
> qemu and also sometimes match on the unmatched / unleashed board. Also
> on unmatched / unleashed, hart id 0 is used by the non-linux hart so if
> we pass the cpu id of the boot cpu that's always going to be 0 it'll be
> invalid, but that's not always the case, especially for kdump. I'm also
> getting an error when trying to mount the rootfs on kdump on the
> unmatched board, because swiotlb can't allocate bounce buffers and the
> pcie driver doesn't work, so no nvme access. I'm looking for a way to
> make this work without messing things up, in any case I'll send some
> patches over the weekend. For the initrd issue I need to patch
> kexec-tools.
>
> Regards,
> Nick
>
ping, any update for this? any new patch for kexec-tools?
I'd also like to try kexec on unmatched board, and encounter the same error
as Alexandre Ghiti reported, generally I'm using almost same cmdline
Yixun Lan
HI Nick
On Tue, Jul 5, 2022 at 12:24 PM Nick Kossifidis <[email protected]> wrote:
>
> Hello Yixun,
>
> On 7/2/22 14:35, Yixun Lan wrote:
> > hi Nick
> >
> > ping, any update for this? any new patch for kexec-tools?
> >
> > I'd also like to try kexec on unmatched board, and encounter the same error
> > as Alexandre Ghiti reported, generally I'm using almost same cmdline
> >
> > Yixun Lan
>
> The kernel-side patches have already been merged upstream, for
> kexec-tools there are some people working on fixes, I'm expecting pull
> requests on the github before I do a pull request upstream. You should
> be able to use kexec without initrd.
>
> Regards,
> Nick
Thanks for the reply and updated information
I was looking forward to use "--initrd" in order to boot a complicated OS,
Anyway, I will try kexec without initrd, and please keep me posted
when there is PR
Regards
Yixun Lan
Hello Yixun,
On 7/2/22 14:35, Yixun Lan wrote:
> hi Nick
>
> ping, any update for this? any new patch for kexec-tools?
>
> I'd also like to try kexec on unmatched board, and encounter the same error
> as Alexandre Ghiti reported, generally I'm using almost same cmdline
>
> Yixun Lan
The kernel-side patches have already been merged upstream, for
kexec-tools there are some people working on fixes, I'm expecting pull
requests on the github before I do a pull request upstream. You should
be able to use kexec without initrd.
Regards,
Nick
Hi Alexandre, Nick:
On Wed, Oct 6, 2021 at 11:15 AM Alexandre Ghiti
<[email protected]> wrote:
>
> Hi Nick,
>
> On Tue, Oct 5, 2021 at 4:07 PM Alexandre Ghiti
> <[email protected]> wrote:
> >
> > On Tue, Oct 5, 2021 at 2:15 PM Nick Kossifidis <[email protected]> wrote:
> > >
> > > Hello all,
> > >
> > > I've uploaded my kexec-tools patches for riscv on github for
> > > testing/review before sending them upstream.
> > > https://github.com/riscv-collab/kexec-tools/tree/riscv
> > >
> > > Both kexec and kdump were tested on latest rv64 qemu, using for-next
> > > branch with this patch applied:
> > > https://patchwork.kernel.org/project/linux-riscv/patch/[email protected]/
> > >
> >
> > I'll test that asap and review your patch too.
> >
> > Thanks for your time Nick!
> >
> > Alex
> >
>
> So I followed the instructions here:
> https://documentation.suse.com/fr-fr/sles/12-SP3/html/SLES-all/cha-tuning-kexec.html#cha-tuning-kexec-basic-usage,
> below the output on an Unmatched board using a vmlinux stored on a sd
> card:
>
> ubuntu@ubuntu:~$ sudo sbin/kexec -l vmlinux --append="$(cat
> /proc/cmdline)" --initrd=/boot/initrd.img
> Warning: No cmdline provided, using append string as cmdline
> Warning: No dtb provided, using /sys/firmware/fdt
> [ 1813.472671] INFO: task kworker/1:0:988 blocked for more than 120 seconds.
> [ 1813.478751] Not tainted 5.15.0-rc1+ #15
> [ 1813.483110] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs"
> disables this message.
> Could not find a free area of memory of 0x3000 bytes...
> locate_hole failed
I've finally spent some time debugging this, and found the fix:
--- a/kexec/arch/riscv/kexec-riscv.c
+++ b/kexec/arch/riscv/kexec-riscv.c
@@ -120,7 +120,7 @@ int load_extra_segments(struct kexec_info *info,
uint64_t kernel_base,
initrd_base + initrd_size);
dbgprintf("Base addr for initrd image: 0x%lX\n", initrd_base);
- min_usable = initrd_base;
+ max_usable = initrd_base;
}
the initrd image has been put at end of memory space( from high
address down to low address),
for example, my case here:
the crash memory: 0x90000000 - 0xA0000000
and the initrd base: 0x9F78F000
so, the max_usable address need to update
>
> I used the Ubuntu kernel, so this is pretty large:
> -rwxrwxr-x 1 ubuntu ubuntu 277M Oct 5 15:47 vmlinux
> -rw-r--r-- 1 root root 98M Sep 21 03:25 /boot/initrd.img
>
> Then if I don't load the initrd (I sometimes have the same warning as
> above) I can at least kexec the new kernel but it fails to boot:
>
> ubuntu@ubuntu:~$ sudo ./sbin/kexec -e
> Warning: No cmdline or append string provided
> Warning: No dtb provided, using /sys/firmware/fdt
> [...]
> [ 0.000000] SBI v0.2 HSM extension detected
> [ 0.000000] CPU with hartid=0 is not available
> [ 0.000000] ------------[ cut here ]------------
> [ 0.000000] kernel BUG at arch/riscv/kernel/smpboot.c:107!
> [ 0.000000] Kernel BUG [#1]
> [ 0.000000] Modules linked in:
> [ 0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 5.15.0-rc1+ #15
> [ 0.000000] Hardware name: SiFive HiFive Unmatched A00 (DT)
> [ 0.000000] epc : setup_smp+0xcc/0x142
> [ 0.000000] ra : setup_smp+0xc4/0x142
> [ 0.000000] epc : ffffffff80a04080 ra : ffffffff80a04078 sp :
> ffffffff81803ec0
> [ 0.000000] gp : ffffffff81a23220 tp : ffffffff81810500 t0 :
> ffffffff81a3551f
> [ 0.000000] t1 : ffffffffffffffff t2 : 0000000000000000 s0 :
> ffffffff81803f00
> [ 0.000000] s1 : 0000000000000000 a0 : 0000000000000000 a1 :
> 0000000000000000
> [ 0.000000] a2 : 0000000000000000 a3 : 0000000000000001 a4 :
> 0000000000000000
> [ 0.000000] a5 : ffffffff80c64500 a6 : 0000000000000004 a7 :
> 000000000000ff00
> [ 0.000000] s2 : 0000000000000005 s3 : 0000000000000000 s4 :
> ffffffff8118f9a8
> [ 0.000000] s5 : 0000000000000007 s6 : ffffffff80c0b790 s7 :
> 0000000080000200
> [ 0.000000] s8 : 0000000000000fff s9 : 0000000081000200 s10:
> 0000000000000018
> [ 0.000000] s11: 000000000000000b t3 : 0000000000ff0000 t4 :
> ffffffffffffffff
> [ 0.000000] t5 : ffffffff80c0b7a0 t6 : ffffffff81803bd8
> [ 0.000000] status: 0000000200000100 badaddr: 0000000000000000
> cause: 0000000000000003
> [ 0.000000] [<ffffffff80a04080>] setup_smp+0xcc/0x142
> [ 0.000000] [<ffffffff80a03d88>] setup_arch+0x56a/0x590
> [ 0.000000] [<ffffffff80a00aa2>] start_kernel+0xaa/0xa5c
> [ 0.000000] random: get_random_bytes called from
> oops_exit+0x44/0x70 with crng_init=0
> [ 0.000000] ---[ end trace 0000000000000000 ]---
> [ 0.000000] Kernel panic - not syncing: Attempted to kill the idle task!
> [ 0.000000] ---[ end Kernel panic - not syncing: Attempted to kill
> the idle task! ]---
>
> This reliably fails here.
>
> > > Regards,
> > > Nick
> > >
> > > _______________________________________________
> > > linux-riscv mailing list
> > > [email protected]
> > > http://lists.infradead.org/mailman/listinfo/linux-riscv
>
> _______________________________________________
> linux-riscv mailing list
> [email protected]
> http://lists.infradead.org/mailman/listinfo/linux-riscv
On 7/5/22 15:23, Nick Kossifidis wrote:
> On 7/2/22 14:35, Yixun Lan wrote:
>> I'd also like to try kexec on unmatched board, and encounter the same error
>> as Alexandre Ghiti reported, generally I'm using almost same cmdline
>
> The kernel-side patches have already been merged upstream, for
> kexec-tools there are some people working on fixes, I'm expecting
> pull requests on the github before I do a pull request upstream. You
> should be able to use kexec without initrd.
Dear Nick, Yuxun,
first of all thank you very much for your great work.
Could you please clarify about state of makedumpfile on RISC-V?
Thank you,
Vasily Averin
Hi Vasily Averin:
On Tue, Sep 6, 2022 at 2:49 PM Vasily Averin <[email protected]> wrote:
>
> On 7/5/22 15:23, Nick Kossifidis wrote:
> > On 7/2/22 14:35, Yixun Lan wrote:
> >> I'd also like to try kexec on unmatched board, and encounter the same error
> >> as Alexandre Ghiti reported, generally I'm using almost same cmdline
> >
> > The kernel-side patches have already been merged upstream, for
> > kexec-tools there are some people working on fixes, I'm expecting
> > pull requests on the github before I do a pull request upstream. You
> > should be able to use kexec without initrd.
>
> Dear Nick, Yuxun,
> first of all thank you very much for your great work.
> Could you please clarify about state of makedumpfile on RISC-V?
>
Speak for myself, I haven't played with the makedumpfile tool yet
and a quick check [1], seems there is no RISC-V support so far
[1] https://github.com/makedumpfile/makedumpfile/tree/master/arch
Yixun