The existing upstream kernel doesn't boot for non-smp
configuration. This patch series address various issues
with non-smp configurations.
Tested on QEMU and HiFive Unleashed board.
Changes from v1->v2
1. Move the cpuid to hartd id map to smp.c from setup.c
2. Split 3rd patch into several small patches based on
logical grouping.
3. Added a new patch that fixes an issue in hwcap query.
4. Changed the title of the patch series.
Atish Patra (8):
RISC-V: Do not wait indefinitely in __cpu_up
RISC-V: Move cpuid to hartid mapping to SMP.
RISC-V: Remove NR_CPUs check during hartid search from DT
RISC-V: Allow hartid-to-cpuid function to fail.
RISC-V: Compare cpuid with NR_CPUS before mapping.
RISC-V: Add required checks during clock source init
RISC-V: Check and continue in case of an invalid cpuid.
RISC-V: Assign hwcap only according to current cpu.
arch/riscv/include/asm/smp.h | 15 ++++++++++++---
arch/riscv/kernel/cpu.c | 4 ----
arch/riscv/kernel/cpufeature.c | 11 +++++++----
arch/riscv/kernel/setup.c | 9 ---------
arch/riscv/kernel/smp.c | 10 +++++++++-
arch/riscv/kernel/smpboot.c | 19 ++++++++++++++++---
drivers/clocksource/timer-riscv.c | 23 ++++++++++++++++++++---
drivers/irqchip/irq-sifive-plic.c | 5 +++++
8 files changed, 69 insertions(+), 27 deletions(-)
--
2.7.4
Currently, clocksource registration happens for an invalid cpu
for non-smp kernels. This lead to kernel panic as cpu hotplug
registration will fail for those cpus. Moreover,
riscv_hartid_to_cpuid can return errors now.
Do not proceed if hartid or cpuid is invalid. Take this opprtunity
to print appropriate error strings for different failure cases.
Signed-off-by: Atish Patra <[email protected]>
---
drivers/clocksource/timer-riscv.c | 23 ++++++++++++++++++++---
1 file changed, 20 insertions(+), 3 deletions(-)
diff --git a/drivers/clocksource/timer-riscv.c b/drivers/clocksource/timer-riscv.c
index 43189220..d9b914e9 100644
--- a/drivers/clocksource/timer-riscv.c
+++ b/drivers/clocksource/timer-riscv.c
@@ -95,14 +95,31 @@ static int __init riscv_timer_init_dt(struct device_node *n)
struct clocksource *cs;
hartid = riscv_of_processor_hartid(n);
+ if (hartid < 0) {
+ pr_warn("Not valid hartid for node [%pOF] error = [%d]\n",
+ n, hartid);
+ return hartid;
+ }
cpuid = riscv_hartid_to_cpuid(hartid);
+ if (cpuid < 0) {
+ pr_warn("Invalid cpuid for hartid [%d]\n", hartid);
+ return cpuid;
+ }
+
if (cpuid != smp_processor_id())
return 0;
+ pr_err("%s: Registering clocksource cpuid [%d] hartid [%d]\n",
+ __func__, cpuid, hartid);
cs = per_cpu_ptr(&riscv_clocksource, cpuid);
- clocksource_register_hz(cs, riscv_timebase);
+ error = clocksource_register_hz(cs, riscv_timebase);
+ if (error) {
+ pr_err("RISCV timer register failed [%d] for cpu = [%d]\n",
+ error, cpuid);
+ return error;
+ }
sched_clock_register(riscv_sched_clock,
BITS_PER_LONG, riscv_timebase);
@@ -110,8 +127,8 @@ static int __init riscv_timer_init_dt(struct device_node *n)
"clockevents/riscv/timer:starting",
riscv_timer_starting_cpu, riscv_timer_dying_cpu);
if (error)
- pr_err("RISCV timer register failed [%d] for cpu = [%d]\n",
- error, cpuid);
+ pr_err("cpu hp setup state failed for RISCV timer [%d]\n",
+ error);
return error;
}
--
2.7.4
riscv_hartid_to_cpuid can return invalid cpuid for a hart
that is present in DT but was never brought up.
Print the appropriate warning message and continue.
Signed-off-by: Atish Patra <[email protected]>
---
drivers/irqchip/irq-sifive-plic.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/drivers/irqchip/irq-sifive-plic.c b/drivers/irqchip/irq-sifive-plic.c
index 357e9daf..254ecd76 100644
--- a/drivers/irqchip/irq-sifive-plic.c
+++ b/drivers/irqchip/irq-sifive-plic.c
@@ -237,6 +237,11 @@ static int __init plic_init(struct device_node *node,
}
cpu = riscv_hartid_to_cpuid(hartid);
+ if (cpu < 0) {
+ pr_warn("Invalid cpuid for context %d\n", i);
+ continue;
+ }
+
handler = per_cpu_ptr(&plic_handlers, cpu);
handler->present = true;
handler->ctxid = i;
--
2.7.4
We should never have a cpuid greater that NR_CPUS. Compare
with NR_CPUS before creating the mapping between logical
and physical CPU ids. This is also mandatory as NR_CPUS
check is removed from riscv_of_processor_hartid.
Signed-off-by: Atish Patra <[email protected]>
---
arch/riscv/kernel/smpboot.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/arch/riscv/kernel/smpboot.c b/arch/riscv/kernel/smpboot.c
index 32e14572..7954470b 100644
--- a/arch/riscv/kernel/smpboot.c
+++ b/arch/riscv/kernel/smpboot.c
@@ -69,6 +69,11 @@ void __init setup_smp(void)
of_node_put(dn);
continue;
}
+ if (cpuid >= NR_CPUS) {
+ pr_warn("Invalid cpuid [%d] for hartid [%d]\n",
+ cpuid, hart);
+ break;
+ }
cpuid_to_hartid_map(cpuid) = hart;
set_cpu_possible(cpuid, true);
--
2.7.4
In SMP path, __cpu_up waits for other CPU to come online
indefinitely. This is wrong as other CPU might be disabled
in machine mode and possible CPU is set to the cpus present
in DT.
Introduce a completion variable and waits only for a second.
Signed-off-by: Atish Patra <[email protected]>
Reviewed-by: Anup Patel <[email protected]>
---
arch/riscv/kernel/smpboot.c | 14 +++++++++++---
1 file changed, 11 insertions(+), 3 deletions(-)
diff --git a/arch/riscv/kernel/smpboot.c b/arch/riscv/kernel/smpboot.c
index fc185eca..32e14572 100644
--- a/arch/riscv/kernel/smpboot.c
+++ b/arch/riscv/kernel/smpboot.c
@@ -39,6 +39,7 @@
void *__cpu_up_stack_pointer[NR_CPUS];
void *__cpu_up_task_pointer[NR_CPUS];
+static DECLARE_COMPLETION(cpu_running);
void __init smp_prepare_boot_cpu(void)
{
@@ -81,6 +82,7 @@ void __init setup_smp(void)
int __cpu_up(unsigned int cpu, struct task_struct *tidle)
{
+ int ret = 0;
int hartid = cpuid_to_hartid_map(cpu);
tidle->thread_info.cpu = cpu;
@@ -96,10 +98,15 @@ int __cpu_up(unsigned int cpu, struct task_struct *tidle)
task_stack_page(tidle) + THREAD_SIZE);
WRITE_ONCE(__cpu_up_task_pointer[hartid], tidle);
- while (!cpu_online(cpu))
- cpu_relax();
+ wait_for_completion_timeout(&cpu_running,
+ msecs_to_jiffies(1000));
- return 0;
+ if (!cpu_online(cpu)) {
+ pr_crit("CPU%u: failed to come online\n", cpu);
+ ret = -EIO;
+ }
+
+ return ret;
}
void __init smp_cpus_done(unsigned int max_cpus)
@@ -125,6 +132,7 @@ asmlinkage void __init smp_callin(void)
* a local TLB flush right now just in case.
*/
local_flush_tlb_all();
+ complete(&cpu_running);
/*
* Disable preemption before enabling interrupts, so we don't try to
* schedule a CPU that hasn't actually started yet.
--
2.7.4
Currently, we set hwcap based on first valid cpu from
DT. This may not be correct always as that CPU might not
be current booting cpu.
Set hwcap based on the current cpu instead of first
valid CPU from DT.
Signed-off-by: Atish Patra <[email protected]>
---
arch/riscv/kernel/cpufeature.c | 11 +++++++----
1 file changed, 7 insertions(+), 4 deletions(-)
diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
index a6e369ed..ed6122ff 100644
--- a/arch/riscv/kernel/cpufeature.c
+++ b/arch/riscv/kernel/cpufeature.c
@@ -43,12 +43,15 @@ void riscv_fill_hwcap(void)
elf_hwcap = 0;
/*
- * We don't support running Linux on hertergenous ISA systems. For
- * now, we just check the ISA of the first "okay" processor.
+ * We don't support running Linux on hertergenous ISA systems.
+ * But first "okay" processor might not be the boot cpu.
+ * Check the ISA of boot cpu.
*/
- while ((node = of_find_node_by_type(node, "cpu")))
- if (riscv_of_processor_hartid(node) >= 0)
+ while ((node = of_find_node_by_type(node, "cpu"))) {
+ if (riscv_of_processor_hartid(node) == boot_cpu_hartid)
break;
+ }
+
if (!node) {
pr_warning("Unable to find \"cpu\" devicetree entry");
return;
--
2.7.4
Currently, logical CPU id to physical hartid mapping is
defined for both smp and non-smp configurations. This
is not required as we need this only for smp configuration.
The mapping function can define directly boot_cpu_hartid
for non-smp use case.
The reverse mapping function i.e. hartid to cpuid can be called
for any valid but not booted harts. So it should return default
cpu 0 only if it is a boot hartid.
Signed-off-by: Atish Patra <[email protected]>
Reviewed-by: Anup Patel <[email protected]>
---
arch/riscv/include/asm/smp.h | 15 ++++++++++++---
arch/riscv/kernel/setup.c | 9 ---------
arch/riscv/kernel/smp.c | 9 +++++++++
3 files changed, 21 insertions(+), 12 deletions(-)
diff --git a/arch/riscv/include/asm/smp.h b/arch/riscv/include/asm/smp.h
index 41aa73b4..8f30300f 100644
--- a/arch/riscv/include/asm/smp.h
+++ b/arch/riscv/include/asm/smp.h
@@ -22,12 +22,13 @@
/*
* Mapping between linux logical cpu index and hartid.
*/
-extern unsigned long __cpuid_to_hartid_map[NR_CPUS];
-#define cpuid_to_hartid_map(cpu) __cpuid_to_hartid_map[cpu]
+extern unsigned long boot_cpu_hartid;
struct seq_file;
#ifdef CONFIG_SMP
+extern unsigned long __cpuid_to_hartid_map[NR_CPUS];
+#define cpuid_to_hartid_map(cpu) __cpuid_to_hartid_map[cpu]
/* print IPI stats */
void show_ipi_stats(struct seq_file *p, int prec);
@@ -58,7 +59,15 @@ static inline void show_ipi_stats(struct seq_file *p, int prec)
static inline int riscv_hartid_to_cpuid(int hartid)
{
- return 0;
+ if (hartid == boot_cpu_hartid)
+ return 0;
+ else
+ return -1;
+}
+static inline unsigned long cpuid_to_hartid_map(int cpu)
+{
+
+ return boot_cpu_hartid;
}
static inline void riscv_cpuid_to_hartid_mask(const struct cpumask *in,
diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
index fc8006a0..f377add4 100644
--- a/arch/riscv/kernel/setup.c
+++ b/arch/riscv/kernel/setup.c
@@ -61,15 +61,6 @@ EXPORT_SYMBOL(empty_zero_page);
atomic_t hart_lottery;
unsigned long boot_cpu_hartid;
-unsigned long __cpuid_to_hartid_map[NR_CPUS] = {
- [0 ... NR_CPUS-1] = INVALID_HARTID
-};
-
-void __init smp_setup_processor_id(void)
-{
- cpuid_to_hartid_map(0) = boot_cpu_hartid;
-}
-
#ifdef CONFIG_BLK_DEV_INITRD
static void __init setup_initrd(void)
{
diff --git a/arch/riscv/kernel/smp.c b/arch/riscv/kernel/smp.c
index 57b1383e..e407bd4f 100644
--- a/arch/riscv/kernel/smp.c
+++ b/arch/riscv/kernel/smp.c
@@ -34,6 +34,15 @@ enum ipi_message_type {
IPI_MAX
};
+unsigned long __cpuid_to_hartid_map[NR_CPUS] = {
+ [0 ... NR_CPUS-1] = INVALID_HARTID
+};
+
+void __init smp_setup_processor_id(void)
+{
+ cpuid_to_hartid_map(0) = boot_cpu_hartid;
+}
+
/* A collection of single bit ipi messages. */
static struct {
unsigned long stats[IPI_MAX] ____cacheline_aligned;
--
2.7.4
In non-smp configuration, hartid can be higher that NR_CPUS.
riscv_of_processor_hartid should not be compared to hartid to
NR_CPUS in that case. Moreover, this function checks all the
DT properties of a hart node. NR_CPUS comparison seems out of
place.
Signed-off-by: Atish Patra <[email protected]>
---
arch/riscv/kernel/cpu.c | 4 ----
1 file changed, 4 deletions(-)
diff --git a/arch/riscv/kernel/cpu.c b/arch/riscv/kernel/cpu.c
index f8fa2c63..19edaeae 100644
--- a/arch/riscv/kernel/cpu.c
+++ b/arch/riscv/kernel/cpu.c
@@ -34,10 +34,6 @@ int riscv_of_processor_hartid(struct device_node *node)
pr_warn("Found CPU without hart ID\n");
return -(ENODEV);
}
- if (hart >= NR_CPUS) {
- pr_info("Found hart ID %d, which is above NR_CPUs. Disabling this hart\n", hart);
- return -(ENODEV);
- }
if (of_property_read_string(node, "status", &status)) {
pr_warn("CPU with hartid=%d has no \"status\" property\n", hart);
--
2.7.4
It is perfectly okay to call riscv_hartid_to_cpuid for a
hartid that is not mapped with an CPU id. It can happen
if the calling functions retrieves the hartid from DT.
However, that hartid was never brought online by the firmware
or kernel for any reasons.
No need to BUG() in the above case. A negative error return
is sufficient and the calling function should check for the
return value always.
Signed-off-by: Atish Patra <[email protected]>
---
arch/riscv/kernel/smp.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/arch/riscv/kernel/smp.c b/arch/riscv/kernel/smp.c
index e407bd4f..ce9a2e73 100644
--- a/arch/riscv/kernel/smp.c
+++ b/arch/riscv/kernel/smp.c
@@ -58,7 +58,6 @@ int riscv_hartid_to_cpuid(int hartid)
return i;
pr_err("Couldn't find cpu id for hartid [%d]\n", hartid);
- BUG();
return i;
}
--
2.7.4
On 1/8/19 1:38 AM, Atish Patra wrote:
> Currently, we set hwcap based on first valid cpu from
> DT. This may not be correct always as that CPU might not
> be current booting cpu.
>
> Set hwcap based on the current cpu instead of first
> valid CPU from DT.
>
> Signed-off-by: Atish Patra <[email protected]>
> ---
> arch/riscv/kernel/cpufeature.c | 11 +++++++----
> 1 file changed, 7 insertions(+), 4 deletions(-)
>
> diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
> index a6e369ed..ed6122ff 100644
> --- a/arch/riscv/kernel/cpufeature.c
> +++ b/arch/riscv/kernel/cpufeature.c
> @@ -43,12 +43,15 @@ void riscv_fill_hwcap(void)
> elf_hwcap = 0;
>
> /*
> - * We don't support running Linux on hertergenous ISA systems. For
> - * now, we just check the ISA of the first "okay" processor.
> + * We don't support running Linux on hertergenous ISA systems.
> + * But first "okay" processor might not be the boot cpu.
> + * Check the ISA of boot cpu.
> */
> - while ((node = of_find_node_by_type(node, "cpu")))
> - if (riscv_of_processor_hartid(node) >= 0)
> + while ((node = of_find_node_by_type(node, "cpu"))) {
> + if (riscv_of_processor_hartid(node) == boot_cpu_hartid)
> break;
> + }
> +
> if (!node) {
> pr_warning("Unable to find \"cpu\" devicetree entry");
> return;
>
Argh..Missed an include while rebasing. Following edit is required for
non SMP config. I will fix it in v2.
diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
index ed6122ff..78379ea3 100644
--- a/arch/riscv/kernel/cpufeature.c
+++ b/arch/riscv/kernel/cpufeature.c
@@ -20,6 +20,7 @@
#include <linux/of.h>
#include <asm/processor.h>
#include <asm/hwcap.h>
+#include <asm/smp.h>
Regards,
Atish
On Tue, Jan 8, 2019 at 3:08 PM Atish Patra <[email protected]> wrote:
>
> It is perfectly okay to call riscv_hartid_to_cpuid for a
> hartid that is not mapped with an CPU id. It can happen
> if the calling functions retrieves the hartid from DT.
> However, that hartid was never brought online by the firmware
> or kernel for any reasons.
>
> No need to BUG() in the above case. A negative error return
> is sufficient and the calling function should check for the
> return value always.
>
> Signed-off-by: Atish Patra <[email protected]>
> ---
> arch/riscv/kernel/smp.c | 1 -
> 1 file changed, 1 deletion(-)
>
> diff --git a/arch/riscv/kernel/smp.c b/arch/riscv/kernel/smp.c
> index e407bd4f..ce9a2e73 100644
> --- a/arch/riscv/kernel/smp.c
> +++ b/arch/riscv/kernel/smp.c
> @@ -58,7 +58,6 @@ int riscv_hartid_to_cpuid(int hartid)
> return i;
>
> pr_err("Couldn't find cpu id for hartid [%d]\n", hartid);
> - BUG();
> return i;
> }
>
> --
> 2.7.4
>
Looks good to me.
Reviewed-by: Anup Patel <[email protected]>
Regards,
Anup
Prefer, commit prefix "irqchip/irq-sifive-plic:" instead of "RISC-V:"
On Tue, Jan 8, 2019 at 3:08 PM Atish Patra <[email protected]> wrote:
>
> riscv_hartid_to_cpuid can return invalid cpuid for a hart
> that is present in DT but was never brought up.
>
> Print the appropriate warning message and continue.
>
> Signed-off-by: Atish Patra <[email protected]>
> ---
> drivers/irqchip/irq-sifive-plic.c | 5 +++++
> 1 file changed, 5 insertions(+)
>
> diff --git a/drivers/irqchip/irq-sifive-plic.c b/drivers/irqchip/irq-sifive-plic.c
> index 357e9daf..254ecd76 100644
> --- a/drivers/irqchip/irq-sifive-plic.c
> +++ b/drivers/irqchip/irq-sifive-plic.c
> @@ -237,6 +237,11 @@ static int __init plic_init(struct device_node *node,
> }
>
> cpu = riscv_hartid_to_cpuid(hartid);
> + if (cpu < 0) {
> + pr_warn("Invalid cpuid for context %d\n", i);
> + continue;
> + }
> +
> handler = per_cpu_ptr(&plic_handlers, cpu);
> handler->present = true;
> handler->ctxid = i;
> --
> 2.7.4
>
Otherwise, looks good to me.
Reviewed-by: Anup Patel <[email protected]>
Regards,
Anup
On Tue, Jan 8, 2019 at 3:08 PM Atish Patra <[email protected]> wrote:
>
> Currently, we set hwcap based on first valid cpu from
> DT. This may not be correct always as that CPU might not
> be current booting cpu.
>
> Set hwcap based on the current cpu instead of first
> valid CPU from DT.
>
> Signed-off-by: Atish Patra <[email protected]>
> ---
> arch/riscv/kernel/cpufeature.c | 11 +++++++----
> 1 file changed, 7 insertions(+), 4 deletions(-)
>
> diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
> index a6e369ed..ed6122ff 100644
> --- a/arch/riscv/kernel/cpufeature.c
> +++ b/arch/riscv/kernel/cpufeature.c
> @@ -43,12 +43,15 @@ void riscv_fill_hwcap(void)
> elf_hwcap = 0;
>
> /*
> - * We don't support running Linux on hertergenous ISA systems. For
> - * now, we just check the ISA of the first "okay" processor.
> + * We don't support running Linux on hertergenous ISA systems.
> + * But first "okay" processor might not be the boot cpu.
> + * Check the ISA of boot cpu.
> */
> - while ((node = of_find_node_by_type(node, "cpu")))
> - if (riscv_of_processor_hartid(node) >= 0)
> + while ((node = of_find_node_by_type(node, "cpu"))) {
> + if (riscv_of_processor_hartid(node) == boot_cpu_hartid)
> break;
> + }
> +
> if (!node) {
> pr_warning("Unable to find \"cpu\" devicetree entry");
> return;
> --
> 2.7.4
>
Looks good to me.
Reviewed-by: Anup Patel <[email protected]>
Regards,
Anup
On Tue, Jan 8, 2019 at 3:08 PM Atish Patra <[email protected]> wrote:
>
> In non-smp configuration, hartid can be higher that NR_CPUS.
> riscv_of_processor_hartid should not be compared to hartid to
> NR_CPUS in that case. Moreover, this function checks all the
> DT properties of a hart node. NR_CPUS comparison seems out of
> place.
>
> Signed-off-by: Atish Patra <[email protected]>
> ---
> arch/riscv/kernel/cpu.c | 4 ----
> 1 file changed, 4 deletions(-)
>
> diff --git a/arch/riscv/kernel/cpu.c b/arch/riscv/kernel/cpu.c
> index f8fa2c63..19edaeae 100644
> --- a/arch/riscv/kernel/cpu.c
> +++ b/arch/riscv/kernel/cpu.c
> @@ -34,10 +34,6 @@ int riscv_of_processor_hartid(struct device_node *node)
> pr_warn("Found CPU without hart ID\n");
> return -(ENODEV);
> }
> - if (hart >= NR_CPUS) {
> - pr_info("Found hart ID %d, which is above NR_CPUs. Disabling this hart\n", hart);
> - return -(ENODEV);
> - }
>
> if (of_property_read_string(node, "status", &status)) {
> pr_warn("CPU with hartid=%d has no \"status\" property\n", hart);
> --
> 2.7.4
>
Looks good to me.
Reviewed-by: Anup Patel <[email protected]>
Regards,
Anup
On Tue, Jan 8, 2019 at 3:08 PM Atish Patra <[email protected]> wrote:
>
> We should never have a cpuid greater that NR_CPUS. Compare
> with NR_CPUS before creating the mapping between logical
> and physical CPU ids. This is also mandatory as NR_CPUS
> check is removed from riscv_of_processor_hartid.
>
> Signed-off-by: Atish Patra <[email protected]>
> ---
> arch/riscv/kernel/smpboot.c | 5 +++++
> 1 file changed, 5 insertions(+)
>
> diff --git a/arch/riscv/kernel/smpboot.c b/arch/riscv/kernel/smpboot.c
> index 32e14572..7954470b 100644
> --- a/arch/riscv/kernel/smpboot.c
> +++ b/arch/riscv/kernel/smpboot.c
> @@ -69,6 +69,11 @@ void __init setup_smp(void)
> of_node_put(dn);
> continue;
> }
> + if (cpuid >= NR_CPUS) {
> + pr_warn("Invalid cpuid [%d] for hartid [%d]\n",
> + cpuid, hart);
> + break;
> + }
>
> cpuid_to_hartid_map(cpuid) = hart;
> set_cpu_possible(cpuid, true);
> --
> 2.7.4
>
Looks good to me.
Reviewed-by: Anup Patel <[email protected]>
Regards,
Anup
Few nit changes..
Prefer, "clocksource/drivers/riscv:" prefix instead of "RISC-V:" for
this patch.
On Tue, Jan 8, 2019 at 3:08 PM Atish Patra <[email protected]> wrote:
>
> Currently, clocksource registration happens for an invalid cpu
> for non-smp kernels. This lead to kernel panic as cpu hotplug
> registration will fail for those cpus. Moreover,
> riscv_hartid_to_cpuid can return errors now.
>
> Do not proceed if hartid or cpuid is invalid. Take this opprtunity
> to print appropriate error strings for different failure cases.
>
> Signed-off-by: Atish Patra <[email protected]>
> ---
> drivers/clocksource/timer-riscv.c | 23 ++++++++++++++++++++---
> 1 file changed, 20 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/clocksource/timer-riscv.c b/drivers/clocksource/timer-riscv.c
> index 43189220..d9b914e9 100644
> --- a/drivers/clocksource/timer-riscv.c
> +++ b/drivers/clocksource/timer-riscv.c
> @@ -95,14 +95,31 @@ static int __init riscv_timer_init_dt(struct device_node *n)
> struct clocksource *cs;
>
> hartid = riscv_of_processor_hartid(n);
> + if (hartid < 0) {
> + pr_warn("Not valid hartid for node [%pOF] error = [%d]\n",
> + n, hartid);
> + return hartid;
> + }
Add empty line here.
> cpuid = riscv_hartid_to_cpuid(hartid);
>
Remove empty line here
> + if (cpuid < 0) {
> + pr_warn("Invalid cpuid for hartid [%d]\n", hartid);
> + return cpuid;
> + }
> +
> if (cpuid != smp_processor_id())
> return 0;
>
> + pr_err("%s: Registering clocksource cpuid [%d] hartid [%d]\n",
> + __func__, cpuid, hartid);
> cs = per_cpu_ptr(&riscv_clocksource, cpuid);
> - clocksource_register_hz(cs, riscv_timebase);
> + error = clocksource_register_hz(cs, riscv_timebase);
>
Remove empty line here.
> + if (error) {
> + pr_err("RISCV timer register failed [%d] for cpu = [%d]\n",
> + error, cpuid);
> + return error;
> + }
Add empty line here.
> sched_clock_register(riscv_sched_clock,
> BITS_PER_LONG, riscv_timebase);
>
> @@ -110,8 +127,8 @@ static int __init riscv_timer_init_dt(struct device_node *n)
> "clockevents/riscv/timer:starting",
> riscv_timer_starting_cpu, riscv_timer_dying_cpu);
> if (error)
> - pr_err("RISCV timer register failed [%d] for cpu = [%d]\n",
> - error, cpuid);
> + pr_err("cpu hp setup state failed for RISCV timer [%d]\n",
> + error);
> return error;
> }
>
> --
> 2.7.4
>
Apart from above, looks good to me.
Reviewed-by: Anup Patel <[email protected]>
Regards,
Anup
> + if (hartid == boot_cpu_hartid)
> + return 0;
> + else
> + return -1;
No need for an else after a return statement.
> +}
> +static inline unsigned long cpuid_to_hartid_map(int cpu)
> +{
> +
> + return boot_cpu_hartid;
spurious empty line above.
>
> void *__cpu_up_stack_pointer[NR_CPUS];
> void *__cpu_up_task_pointer[NR_CPUS];
> +static DECLARE_COMPLETION(cpu_running);
>
> void __init smp_prepare_boot_cpu(void)
> {
> @@ -81,6 +82,7 @@ void __init setup_smp(void)
>
> int __cpu_up(unsigned int cpu, struct task_struct *tidle)
> {
> + int ret = 0;
> int hartid = cpuid_to_hartid_map(cpu);
> tidle->thread_info.cpu = cpu;
>
> @@ -96,10 +98,15 @@ int __cpu_up(unsigned int cpu, struct task_struct *tidle)
> task_stack_page(tidle) + THREAD_SIZE);
> WRITE_ONCE(__cpu_up_task_pointer[hartid], tidle);
>
> - while (!cpu_online(cpu))
> - cpu_relax();
> + wait_for_completion_timeout(&cpu_running,
> + msecs_to_jiffies(1000));
Having a global completion here worries me. I bet we have some higher
level serialization, but can we comment or even better lockdep assert on
that?
Also please use up your available lines (72 in commit logs, 80 in source
files) instead of adding spurious line wraps.
On Tue, Jan 08, 2019 at 01:38:34AM -0800, Atish Patra wrote:
> It is perfectly okay to call riscv_hartid_to_cpuid for a
> hartid that is not mapped with an CPU id. It can happen
> if the calling functions retrieves the hartid from DT.
> However, that hartid was never brought online by the firmware
> or kernel for any reasons.
>
> No need to BUG() in the above case. A negative error return
> is sufficient and the calling function should check for the
> return value always.
>
> Signed-off-by: Atish Patra <[email protected]>
Looks fine (modulo the line length issues):
Reviewed-by: Christoph Hellwig <[email protected]>
> hartid = riscv_of_processor_hartid(n);
> + if (hartid < 0) {
> + pr_warn("Not valid hartid for node [%pOF] error = [%d]\n",
> + n, hartid);
> + return hartid;
> + }
> cpuid = riscv_hartid_to_cpuid(hartid);
>
> + if (cpuid < 0) {
> + pr_warn("Invalid cpuid for hartid [%d]\n", hartid);
> + return cpuid;
> + }
> +
No need for the empty line after the riscv_hartid_to_cpuid call.
> - clocksource_register_hz(cs, riscv_timebase);
> + error = clocksource_register_hz(cs, riscv_timebase);
>
> + if (error) {
Same here.
Looks good,
Reviewed-by: Christoph Hellwig <[email protected]>
On Tue, Jan 08, 2019 at 01:38:33AM -0800, Atish Patra wrote:
> In non-smp configuration, hartid can be higher that NR_CPUS.
> riscv_of_processor_hartid should not be compared to hartid to
> NR_CPUS in that case. Moreover, this function checks all the
> DT properties of a hart node. NR_CPUS comparison seems out of
> place.
>
> Signed-off-by: Atish Patra <[email protected]>
Looks good,
Reviewed-by: Christoph Hellwig <[email protected]>
Looks fine (modulo the subject line as already pointed out):
Reviewed-by: Christoph Hellwig <[email protected]>
On Tue, Jan 08, 2019 at 01:38:38AM -0800, Atish Patra wrote:
> Currently, we set hwcap based on first valid cpu from
> DT. This may not be correct always as that CPU might not
> be current booting cpu.
>
> Set hwcap based on the current cpu instead of first
> valid CPU from DT.
This is generally the right thing to do. But can the kernel even cope
with different hwcaps per hart? I know arm land and I think x86 as well
don't, so we might want to add a sanity check that they match or reduce
them to the common subset.
On 1/15/19 5:52 AM, Christoph Hellwig wrote:
>> + if (hartid == boot_cpu_hartid)
>> + return 0;
>> + else
>> + return -1;
>
> No need for an else after a return statement.
>
Correct. Will update it.
>> +}
>> +static inline unsigned long cpuid_to_hartid_map(int cpu)
>> +{
>> +
>> + return boot_cpu_hartid;
>
> spurious empty line above.
>
Sorry for that. Will fix it.
Regards,
Atish
On 1/8/19 3:57 AM, Anup Patel wrote:
> Few nit changes..
>
> Prefer, "clocksource/drivers/riscv:" prefix instead of "RISC-V:" for
> this patch.
>
> On Tue, Jan 8, 2019 at 3:08 PM Atish Patra <[email protected]> wrote:
>>
>> Currently, clocksource registration happens for an invalid cpu
>> for non-smp kernels. This lead to kernel panic as cpu hotplug
>> registration will fail for those cpus. Moreover,
>> riscv_hartid_to_cpuid can return errors now.
>>
>> Do not proceed if hartid or cpuid is invalid. Take this opprtunity
>> to print appropriate error strings for different failure cases.
>>
>> Signed-off-by: Atish Patra <[email protected]>
>> ---
>> drivers/clocksource/timer-riscv.c | 23 ++++++++++++++++++++---
>> 1 file changed, 20 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/clocksource/timer-riscv.c b/drivers/clocksource/timer-riscv.c
>> index 43189220..d9b914e9 100644
>> --- a/drivers/clocksource/timer-riscv.c
>> +++ b/drivers/clocksource/timer-riscv.c
>> @@ -95,14 +95,31 @@ static int __init riscv_timer_init_dt(struct device_node *n)
>> struct clocksource *cs;
>>
>> hartid = riscv_of_processor_hartid(n);
>> + if (hartid < 0) {
>> + pr_warn("Not valid hartid for node [%pOF] error = [%d]\n",
>> + n, hartid);
>> + return hartid;
>> + }
>
> Add empty line here.
>
>> cpuid = riscv_hartid_to_cpuid(hartid);
>>
>
> Remove empty line here
>
>> + if (cpuid < 0) {
>> + pr_warn("Invalid cpuid for hartid [%d]\n", hartid);
>> + return cpuid;
>> + }
>> +
>> if (cpuid != smp_processor_id())
>> return 0;
>>
>> + pr_err("%s: Registering clocksource cpuid [%d] hartid [%d]\n",
>> + __func__, cpuid, hartid);
>> cs = per_cpu_ptr(&riscv_clocksource, cpuid);
>> - clocksource_register_hz(cs, riscv_timebase);
>> + error = clocksource_register_hz(cs, riscv_timebase);
>>
>
> Remove empty line here.
>
>> + if (error) {
>> + pr_err("RISCV timer register failed [%d] for cpu = [%d]\n",
>> + error, cpuid);
>> + return error;
>> + }
>
> Add empty line here.
>
>> sched_clock_register(riscv_sched_clock,
>> BITS_PER_LONG, riscv_timebase);
>>
>> @@ -110,8 +127,8 @@ static int __init riscv_timer_init_dt(struct device_node *n)
>> "clockevents/riscv/timer:starting",
>> riscv_timer_starting_cpu, riscv_timer_dying_cpu);
>> if (error)
>> - pr_err("RISCV timer register failed [%d] for cpu = [%d]\n",
>> - error, cpuid);
>> + pr_err("cpu hp setup state failed for RISCV timer [%d]\n",
>> + error);
>> return error;
>> }
>>
>> --
>> 2.7.4
>>
>
> Apart from above, looks good to me.
>
> Reviewed-by: Anup Patel <[email protected]>
>
Thanks for the review. I will fix the empty line issues and subject line.
> Regards,
> Anup
>
On 1/8/19 4:00 AM, Anup Patel wrote:
> Prefer, commit prefix "irqchip/irq-sifive-plic:" instead of "RISC-V:"
>
Will do.
Regards,
Atish
> On Tue, Jan 8, 2019 at 3:08 PM Atish Patra <[email protected]> wrote:
>>
>> riscv_hartid_to_cpuid can return invalid cpuid for a hart
>> that is present in DT but was never brought up.
>>
>> Print the appropriate warning message and continue.
>>
>> Signed-off-by: Atish Patra <[email protected]>
>> ---
>> drivers/irqchip/irq-sifive-plic.c | 5 +++++
>> 1 file changed, 5 insertions(+)
>>
>> diff --git a/drivers/irqchip/irq-sifive-plic.c b/drivers/irqchip/irq-sifive-plic.c
>> index 357e9daf..254ecd76 100644
>> --- a/drivers/irqchip/irq-sifive-plic.c
>> +++ b/drivers/irqchip/irq-sifive-plic.c
>> @@ -237,6 +237,11 @@ static int __init plic_init(struct device_node *node,
>> }
>>
>> cpu = riscv_hartid_to_cpuid(hartid);
>> + if (cpu < 0) {
>> + pr_warn("Invalid cpuid for context %d\n", i);
>> + continue;
>> + }
>> +
>> handler = per_cpu_ptr(&plic_handlers, cpu);
>> handler->present = true;
>> handler->ctxid = i;
>> --
>> 2.7.4
>>
>
> Otherwise, looks good to me.
>
> Reviewed-by: Anup Patel <[email protected]>
>
> Regards,
> Anup
>
On 1/15/19 5:56 AM, Christoph Hellwig wrote:
> On Tue, Jan 08, 2019 at 01:38:38AM -0800, Atish Patra wrote:
>> Currently, we set hwcap based on first valid cpu from
>> DT. This may not be correct always as that CPU might not
>> be current booting cpu.
>>
>> Set hwcap based on the current cpu instead of first
>> valid CPU from DT.
>
> This is generally the right thing to do. But can the kernel even cope
> with different hwcaps per hart?
I don't think so.
I know arm land and I think x86 as well
> don't, so we might want to add a sanity check that they match or reduce
> them to the common subset.
>
I will add a sanity check that they match and throw a warning if they
don't.
Regards,
Atish
On 1/15/19 5:51 AM, Christoph Hellwig wrote:
>>
>> void *__cpu_up_stack_pointer[NR_CPUS];
>> void *__cpu_up_task_pointer[NR_CPUS];
>> +static DECLARE_COMPLETION(cpu_running);
>>
>> void __init smp_prepare_boot_cpu(void)
>> {
>> @@ -81,6 +82,7 @@ void __init setup_smp(void)
>>
>> int __cpu_up(unsigned int cpu, struct task_struct *tidle)
>> {
>> + int ret = 0;
>> int hartid = cpuid_to_hartid_map(cpu);
>> tidle->thread_info.cpu = cpu;
>>
>> @@ -96,10 +98,15 @@ int __cpu_up(unsigned int cpu, struct task_struct *tidle)
>> task_stack_page(tidle) + THREAD_SIZE);
>> WRITE_ONCE(__cpu_up_task_pointer[hartid], tidle);
>>
>> - while (!cpu_online(cpu))
>> - cpu_relax();
>> + wait_for_completion_timeout(&cpu_running,
>> + msecs_to_jiffies(1000));
>
> Having a global completion here worries me. I bet we have some higher
> level serialization, but can we comment or even better lockdep assert on
> that?
>
Yes. It is serialized from smp.c in smp_init(). It brings one cpu online
at a time for preset_cpu mask.
Do we still need a lockdep assert ?
Regards,
Atish
> Also please use up your available lines (72 in commit logs, 80 in source
> files) instead of adding spurious line wraps.
>
> _______________________________________________
> linux-riscv mailing list
> [email protected]
> http://lists.infradead.org/mailman/listinfo/linux-riscv
>
On Thu, Jan 17, 2019 at 06:35:39PM -0800, Atish Patra wrote:
> On 1/15/19 5:51 AM, Christoph Hellwig wrote:
> > > void *__cpu_up_stack_pointer[NR_CPUS];
> > > void *__cpu_up_task_pointer[NR_CPUS];
> > > +static DECLARE_COMPLETION(cpu_running);
> > > void __init smp_prepare_boot_cpu(void)
> > > {
> > > @@ -81,6 +82,7 @@ void __init setup_smp(void)
> > > int __cpu_up(unsigned int cpu, struct task_struct *tidle)
> > > {
> > > + int ret = 0;
> > > int hartid = cpuid_to_hartid_map(cpu);
> > > tidle->thread_info.cpu = cpu;
> > > @@ -96,10 +98,15 @@ int __cpu_up(unsigned int cpu, struct task_struct *tidle)
> > > task_stack_page(tidle) + THREAD_SIZE);
> > > WRITE_ONCE(__cpu_up_task_pointer[hartid], tidle);
> > > - while (!cpu_online(cpu))
> > > - cpu_relax();
> > > + wait_for_completion_timeout(&cpu_running,
> > > + msecs_to_jiffies(1000));
> >
> > Having a global completion here worries me. I bet we have some higher
> > level serialization, but can we comment or even better lockdep assert on
> > that?
> >
>
> Yes. It is serialized from smp.c in smp_init(). It brings one cpu online
> at a time for preset_cpu mask.
>
> Do we still need a lockdep assert ?
I guess the real lock is through cpu_hotplug_lock. And yes, a comment
or even better lockdep assert would be good.