From: Oleksandr Tyshchenko <[email protected]>
Don't use hardcoded address, retrieve it from device-tree instead.
And besides, this patch fixes the memory error when running
on top of Xen hypervisor:
(XEN) traps.c:1999:d0v0 HSR=0x93830007 pc=0xc0b097f8 gva=0xf0805000
gpa=0x000000e6080000
Which shows that VCPU0 in Dom0 is trying to access an address in memory
it is not allowed to access (0x000000e6080000).
Put simply, Xen doesn't know that it is a device's register memory
since it wasn't described in a host device tree (which Xen parses)
and as the result this memory region wasn't assigned to Dom0 at
domain creation time.
Signed-off-by: Oleksandr Tyshchenko <[email protected]>
---
This patch is meant to get feedback from the community before
proceeding further. If we decide to go this direction, all Gen2
device-trees should be updated (add memory region) before
this patch going in.
e.g. r8a7790.dtsi:
...
timer {
compatible = "arm,armv7-timer";
interrupts-extended = <&gic GIC_PPI 13 (GIC_CPU_MASK_SIMPLE(8) | IRQ_TYPE_LEVEL_LOW)>,
<&gic GIC_PPI 14 (GIC_CPU_MASK_SIMPLE(8) | IRQ_TYPE_LEVEL_LOW)>,
<&gic GIC_PPI 11 (GIC_CPU_MASK_SIMPLE(8) | IRQ_TYPE_LEVEL_LOW)>,
<&gic GIC_PPI 10 (GIC_CPU_MASK_SIMPLE(8) | IRQ_TYPE_LEVEL_LOW)>;
+ reg = <0 0xe6080000 0 0x1000>;
};
...
---
arch/arm/mach-shmobile/setup-rcar-gen2.c | 17 ++++++++++++++++-
1 file changed, 16 insertions(+), 1 deletion(-)
diff --git a/arch/arm/mach-shmobile/setup-rcar-gen2.c b/arch/arm/mach-shmobile/setup-rcar-gen2.c
index 35dda21..153e3f5 100644
--- a/arch/arm/mach-shmobile/setup-rcar-gen2.c
+++ b/arch/arm/mach-shmobile/setup-rcar-gen2.c
@@ -15,6 +15,7 @@
#include <linux/kernel.h>
#include <linux/memblock.h>
#include <linux/of.h>
+#include <linux/of_address.h>
#include <linux/of_fdt.h>
#include <linux/of_platform.h>
#include <asm/mach/arch.h>
@@ -61,6 +62,8 @@ static unsigned int __init get_extal_freq(void)
void __init rcar_gen2_timer_init(void)
{
+ struct device_node *np;
+ struct resource res;
void __iomem *base;
u32 freq;
@@ -72,6 +75,13 @@ void __init rcar_gen2_timer_init(void)
if (!psci_smp_available())
secure_cntvoff_init();
+ np = of_find_compatible_node(NULL, NULL, "arm,armv7-timer");
+ if (!np)
+ goto out;
+
+ if (of_address_to_resource(np, 0, &res))
+ goto out;
+
if (of_machine_is_compatible("renesas,r8a7745") ||
of_machine_is_compatible("renesas,r8a77470") ||
of_machine_is_compatible("renesas,r8a7792") ||
@@ -88,7 +98,9 @@ void __init rcar_gen2_timer_init(void)
}
/* Remap "armgcnt address map" space */
- base = ioremap(0xe6080000, PAGE_SIZE);
+ base = ioremap(res.start, resource_size(&res));
+ if (!base)
+ goto out;
/*
* Update the timer if it is either not running, or is not at the
@@ -109,6 +121,9 @@ void __init rcar_gen2_timer_init(void)
iounmap(base);
+out:
+ of_node_put(np);
+
of_clk_init(NULL);
timer_probe();
}
--
2.7.4
Hi,
On 5/10/19 5:22 PM, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <[email protected]>
>
> Don't use hardcoded address, retrieve it from device-tree instead.
>
> And besides, this patch fixes the memory error when running
> on top of Xen hypervisor:
>
> (XEN) traps.c:1999:d0v0 HSR=0x93830007 pc=0xc0b097f8 gva=0xf0805000
> gpa=0x000000e6080000
>
> Which shows that VCPU0 in Dom0 is trying to access an address in memory
> it is not allowed to access (0x000000e6080000).
> Put simply, Xen doesn't know that it is a device's register memory
> since it wasn't described in a host device tree (which Xen parses)
> and as the result this memory region wasn't assigned to Dom0 at
> domain creation time.
>
> Signed-off-by: Oleksandr Tyshchenko <[email protected]>
>
> ---
>
> This patch is meant to get feedback from the community before
> proceeding further. If we decide to go this direction, all Gen2
> device-trees should be updated (add memory region) before
> this patch going in.
>
> e.g. r8a7790.dtsi:
>
> ...
> timer {
> compatible = "arm,armv7-timer";
> interrupts-extended = <&gic GIC_PPI 13 (GIC_CPU_MASK_SIMPLE(8) | IRQ_TYPE_LEVEL_LOW)>,
> <&gic GIC_PPI 14 (GIC_CPU_MASK_SIMPLE(8) | IRQ_TYPE_LEVEL_LOW)>,
> <&gic GIC_PPI 11 (GIC_CPU_MASK_SIMPLE(8) | IRQ_TYPE_LEVEL_LOW)>,
> <&gic GIC_PPI 10 (GIC_CPU_MASK_SIMPLE(8) | IRQ_TYPE_LEVEL_LOW)>;
> + reg = <0 0xe6080000 0 0x1000>;
This looks incorrect, the "arm,armv7-timer" bindings doesn't offer you
the possibility to specify an MMIO region. This makes sense because it
is meant to describe the Arch timer that is only access via co-processor
registers.
Looking at the code, I think the MMIO region corresponds to the
coresight (based on the register name). So you may want to describe the
coresight in the Device-Tree.
Also, AFAICT, the code is configuring and turning on the timer if it has
not been done yet. If you are here running on Xen, then you have
probably done something wrong. Indeed, it means Xen would not be able to
use the timer until Dom0 has booted. But, shouldn't newer U-boot do it
for you?
Cheers,
--
Julien Grall
Hi Julien,
On Mon, May 13, 2019 at 11:20 AM Julien Grall <[email protected]> wrote:
> On 5/10/19 5:22 PM, Oleksandr Tyshchenko wrote:
> > From: Oleksandr Tyshchenko <[email protected]>
> >
> > Don't use hardcoded address, retrieve it from device-tree instead.
> >
> > And besides, this patch fixes the memory error when running
> > on top of Xen hypervisor:
> >
> > (XEN) traps.c:1999:d0v0 HSR=0x93830007 pc=0xc0b097f8 gva=0xf0805000
> > gpa=0x000000e6080000
> >
> > Which shows that VCPU0 in Dom0 is trying to access an address in memory
> > it is not allowed to access (0x000000e6080000).
> > Put simply, Xen doesn't know that it is a device's register memory
> > since it wasn't described in a host device tree (which Xen parses)
> > and as the result this memory region wasn't assigned to Dom0 at
> > domain creation time.
> >
> > Signed-off-by: Oleksandr Tyshchenko <[email protected]>
> >
> > ---
> >
> > This patch is meant to get feedback from the community before
> > proceeding further. If we decide to go this direction, all Gen2
> > device-trees should be updated (add memory region) before
> > this patch going in.
> >
> > e.g. r8a7790.dtsi:
> >
> > ...
> > timer {
> > compatible = "arm,armv7-timer";
> > interrupts-extended = <&gic GIC_PPI 13 (GIC_CPU_MASK_SIMPLE(8) | IRQ_TYPE_LEVEL_LOW)>,
> > <&gic GIC_PPI 14 (GIC_CPU_MASK_SIMPLE(8) | IRQ_TYPE_LEVEL_LOW)>,
> > <&gic GIC_PPI 11 (GIC_CPU_MASK_SIMPLE(8) | IRQ_TYPE_LEVEL_LOW)>,
> > <&gic GIC_PPI 10 (GIC_CPU_MASK_SIMPLE(8) | IRQ_TYPE_LEVEL_LOW)>;
> > + reg = <0 0xe6080000 0 0x1000>;
>
> This looks incorrect, the "arm,armv7-timer" bindings doesn't offer you
> the possibility to specify an MMIO region. This makes sense because it
> is meant to describe the Arch timer that is only access via co-processor
> registers.
>
> Looking at the code, I think the MMIO region corresponds to the
> coresight (based on the register name). So you may want to describe the
> coresight in the Device-Tree.
This is the "counter module", not the ARM v7 timer, cfr. Mark Rutland's
response in an earlier discussion about describing this in DT in
"Re: Architecture Timer on R-Car Gen2"
https://lore.kernel.org/linux-renesas-soc/20170705113335.GE25115@leverpostej/
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
On 13.05.19 12:19, Julien Grall wrote:
> Hi,
Hi, Julien, Geert
>
> On 5/10/19 5:22 PM, Oleksandr Tyshchenko wrote:
>> From: Oleksandr Tyshchenko <[email protected]>
>>
>> Don't use hardcoded address, retrieve it from device-tree instead.
>>
>> And besides, this patch fixes the memory error when running
>> on top of Xen hypervisor:
>>
>> (XEN) traps.c:1999:d0v0 HSR=0x93830007 pc=0xc0b097f8 gva=0xf0805000
>> gpa=0x000000e6080000
>>
>> Which shows that VCPU0 in Dom0 is trying to access an address in memory
>> it is not allowed to access (0x000000e6080000).
>> Put simply, Xen doesn't know that it is a device's register memory
>> since it wasn't described in a host device tree (which Xen parses)
>> and as the result this memory region wasn't assigned to Dom0 at
>> domain creation time.
>>
>> Signed-off-by: Oleksandr Tyshchenko <[email protected]>
>>
>> ---
>>
>> This patch is meant to get feedback from the community before
>> proceeding further. If we decide to go this direction, all Gen2
>> device-trees should be updated (add memory region) before
>> this patch going in.
>>
>> e.g. r8a7790.dtsi:
>>
>> ...
>> timer {
>> compatible = "arm,armv7-timer";
>> interrupts-extended = <&gic GIC_PPI 13 (GIC_CPU_MASK_SIMPLE(8) |
>> IRQ_TYPE_LEVEL_LOW)>,
>> <&gic GIC_PPI 14 (GIC_CPU_MASK_SIMPLE(8) |
>> IRQ_TYPE_LEVEL_LOW)>,
>> <&gic GIC_PPI 11 (GIC_CPU_MASK_SIMPLE(8) |
>> IRQ_TYPE_LEVEL_LOW)>,
>> <&gic GIC_PPI 10 (GIC_CPU_MASK_SIMPLE(8) |
>> IRQ_TYPE_LEVEL_LOW)>;
>> + reg = <0 0xe6080000 0 0x1000>;
>
> This looks incorrect, the "arm,armv7-timer" bindings doesn't offer you
> the possibility to specify an MMIO region. This makes sense because it
> is meant to describe the Arch timer that is only access via
> co-processor registers.
>
> Looking at the code, I think the MMIO region corresponds to the
> coresight (based on the register name). So you may want to describe
> the coresight in the Device-Tree.
>
> Also, AFAICT, the code is configuring and turning on the timer if it
> has not been done yet. If you are here running on Xen, then you have
> probably done something wrong. Indeed, it means Xen would not be able
> to use the timer until Dom0 has booted. But, shouldn't newer U-boot do
> it for you?
Let me elaborate a bit more on this...
Indeed, my PSCI patch series for U-Boot includes a patch [1] for
configuring that "counter module". So, if PSCI is available
(psci_smp_available() == true), then most likely we are running on
PSCI-enabled
U-Boot which, we assume, has already taken care of configuring timer (as
well as resetting CNTVOFF). So, when running on Xen, the timer was
configured beforehand in U-Boot, and Xen is able to use it from the very
beginning, these is no need to wait for Dom0 to configure it.
(XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27 Freq: 10000 KHz
So, the code in brackets won't be called when using PSCI/running Xen,
since the timer is already both enabled and configured:
if ((ioread32(base + CNTCR) & 1) == 0 ||
ioread32(base + CNTFID0) != freq) {
/* Update registers with correct frequency */
iowrite32(freq, base + CNTFID0);
asm volatile("mcr p15, 0, %0, c14, c0, 0" : : "r" (freq));
/* make sure arch timer is started by setting bit 0 of CNTCR */
iowrite32(1, base + CNTCR);
}
But, the problem here is the first read access from timer register (when
we check whether the timer requires enabling) results in hypervisor trap:
(XEN) traps.c:1999:d0v0 HSR=0x93830007 pc=0xc0b097f8 gva=0xf0805000
gpa=0x000000e6080000
So, if the DT bindings for the counter module is not an option (if I
correctly understood a discussion pointed by Geert in another letter),
we should probably prevent all timer code here from being executed if
PSCI is in use.
What I mean is to return to [2], but with the modification to use
psci_smp_available() helper as an indicator of PSCI usage.
Julien, Geert, what do you think?
[1] https://marc.info/?l=u-boot&m=154895714510154&w=2
[2] https://lkml.org/lkml/2019/4/17/810
>
> Cheers,
>
--
Regards,
Oleksandr Tyshchenko
Hi Oleksandr,
On Mon, May 13, 2019 at 4:22 PM Oleksandr <[email protected]> wrote:
> On 13.05.19 12:19, Julien Grall wrote:
> > On 5/10/19 5:22 PM, Oleksandr Tyshchenko wrote:
> >> From: Oleksandr Tyshchenko <[email protected]>
> >>
> >> Don't use hardcoded address, retrieve it from device-tree instead.
> >>
> >> And besides, this patch fixes the memory error when running
> >> on top of Xen hypervisor:
> >>
> >> (XEN) traps.c:1999:d0v0 HSR=0x93830007 pc=0xc0b097f8 gva=0xf0805000
> >> gpa=0x000000e6080000
> >>
> >> Which shows that VCPU0 in Dom0 is trying to access an address in memory
> >> it is not allowed to access (0x000000e6080000).
> >> Put simply, Xen doesn't know that it is a device's register memory
> >> since it wasn't described in a host device tree (which Xen parses)
> >> and as the result this memory region wasn't assigned to Dom0 at
> >> domain creation time.
> >>
> >> Signed-off-by: Oleksandr Tyshchenko <[email protected]>
> >>
> >> ---
> >>
> >> This patch is meant to get feedback from the community before
> >> proceeding further. If we decide to go this direction, all Gen2
> >> device-trees should be updated (add memory region) before
> >> this patch going in.
> >>
> >> e.g. r8a7790.dtsi:
> >>
> >> ...
> >> timer {
> >> compatible = "arm,armv7-timer";
> >> interrupts-extended = <&gic GIC_PPI 13 (GIC_CPU_MASK_SIMPLE(8) |
> >> IRQ_TYPE_LEVEL_LOW)>,
> >> <&gic GIC_PPI 14 (GIC_CPU_MASK_SIMPLE(8) |
> >> IRQ_TYPE_LEVEL_LOW)>,
> >> <&gic GIC_PPI 11 (GIC_CPU_MASK_SIMPLE(8) |
> >> IRQ_TYPE_LEVEL_LOW)>,
> >> <&gic GIC_PPI 10 (GIC_CPU_MASK_SIMPLE(8) |
> >> IRQ_TYPE_LEVEL_LOW)>;
> >> + reg = <0 0xe6080000 0 0x1000>;
> >
> > This looks incorrect, the "arm,armv7-timer" bindings doesn't offer you
> > the possibility to specify an MMIO region. This makes sense because it
> > is meant to describe the Arch timer that is only access via
> > co-processor registers.
> >
> > Looking at the code, I think the MMIO region corresponds to the
> > coresight (based on the register name). So you may want to describe
> > the coresight in the Device-Tree.
> >
> > Also, AFAICT, the code is configuring and turning on the timer if it
> > has not been done yet. If you are here running on Xen, then you have
> > probably done something wrong. Indeed, it means Xen would not be able
> > to use the timer until Dom0 has booted. But, shouldn't newer U-boot do
> > it for you?
>
> Let me elaborate a bit more on this...
>
> Indeed, my PSCI patch series for U-Boot includes a patch [1] for
> configuring that "counter module". So, if PSCI is available
> (psci_smp_available() == true), then most likely we are running on
> PSCI-enabled
> U-Boot which, we assume, has already taken care of configuring timer (as
> well as resetting CNTVOFF). So, when running on Xen, the timer was
> configured beforehand in U-Boot, and Xen is able to use it from the very
> beginning, these is no need to wait for Dom0 to configure it.
>
> (XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27 Freq: 10000 KHz
>
> So, the code in brackets won't be called when using PSCI/running Xen,
> since the timer is already both enabled and configured:
>
> if ((ioread32(base + CNTCR) & 1) == 0 ||
> ioread32(base + CNTFID0) != freq) {
> /* Update registers with correct frequency */
> iowrite32(freq, base + CNTFID0);
> asm volatile("mcr p15, 0, %0, c14, c0, 0" : : "r" (freq));
>
> /* make sure arch timer is started by setting bit 0 of CNTCR */
> iowrite32(1, base + CNTCR);
> }
>
> But, the problem here is the first read access from timer register (when
> we check whether the timer requires enabling) results in hypervisor trap:
>
> (XEN) traps.c:1999:d0v0 HSR=0x93830007 pc=0xc0b097f8 gva=0xf0805000
> gpa=0x000000e6080000
>
> So, if the DT bindings for the counter module is not an option (if I
> correctly understood a discussion pointed by Geert in another letter),
> we should probably prevent all timer code here from being executed if
> PSCI is in use.
> What I mean is to return to [2], but with the modification to use
> psci_smp_available() helper as an indicator of PSCI usage.
>
> Julien, Geert, what do you think?
Yes, that sounds good to me.
Note that psci_smp_available() seems to return false if CONFIG_SMP=n,
so checking for that is not sufficient to avoid crashes when running a
uniprocessor kernel on a PSCI-enabled system.
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
On 13.05.19 18:13, Geert Uytterhoeven wrote:
> Hi Oleksandr,
Hi Geert
>> So, if the DT bindings for the counter module is not an option (if I
>> correctly understood a discussion pointed by Geert in another letter),
>> we should probably prevent all timer code here from being executed if
>> PSCI is in use.
>> What I mean is to return to [2], but with the modification to use
>> psci_smp_available() helper as an indicator of PSCI usage.
>>
>> Julien, Geert, what do you think?
> Yes, that sounds good to me.
>
> Note that psci_smp_available() seems to return false if CONFIG_SMP=n,
> so checking for that is not sufficient to avoid crashes when running a
> uniprocessor kernel on a PSCI-enabled system.
Indeed, you are right.
Nothing than just check for psci_ops.cpu_on == NULL directly comes to
mind...
Have already checked with CONFIG_SMP=n, it works.
Sounds ok?
--
Regards,
Oleksandr Tyshchenko
Hi Oleksandr,
On Mon, May 13, 2019 at 6:00 PM Oleksandr <[email protected]> wrote:
> On 13.05.19 18:13, Geert Uytterhoeven wrote:
> >> So, if the DT bindings for the counter module is not an option (if I
> >> correctly understood a discussion pointed by Geert in another letter),
> >> we should probably prevent all timer code here from being executed if
> >> PSCI is in use.
> >> What I mean is to return to [2], but with the modification to use
> >> psci_smp_available() helper as an indicator of PSCI usage.
> >>
> >> Julien, Geert, what do you think?
> > Yes, that sounds good to me.
> >
> > Note that psci_smp_available() seems to return false if CONFIG_SMP=n,
> > so checking for that is not sufficient to avoid crashes when running a
> > uniprocessor kernel on a PSCI-enabled system.
>
> Indeed, you are right.
>
>
> Nothing than just check for psci_ops.cpu_on == NULL directly comes to
> mind...
>
> Have already checked with CONFIG_SMP=n, it works.
>
> Sounds ok?
Fine for me, thanks!
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
On 14.05.19 10:16, Geert Uytterhoeven wrote:
> Hi Oleksandr,
Hi Geert
>
> On Mon, May 13, 2019 at 6:00 PM Oleksandr <[email protected]> wrote:
>> On 13.05.19 18:13, Geert Uytterhoeven wrote:
>>>> So, if the DT bindings for the counter module is not an option (if I
>>>> correctly understood a discussion pointed by Geert in another letter),
>>>> we should probably prevent all timer code here from being executed if
>>>> PSCI is in use.
>>>> What I mean is to return to [2], but with the modification to use
>>>> psci_smp_available() helper as an indicator of PSCI usage.
>>>>
>>>> Julien, Geert, what do you think?
>>> Yes, that sounds good to me.
>>>
>>> Note that psci_smp_available() seems to return false if CONFIG_SMP=n,
>>> so checking for that is not sufficient to avoid crashes when running a
>>> uniprocessor kernel on a PSCI-enabled system.
>> Indeed, you are right.
>>
>>
>> Nothing than just check for psci_ops.cpu_on == NULL directly comes to
>> mind...
>>
>> Have already checked with CONFIG_SMP=n, it works.
>>
>> Sounds ok?
> Fine for me, thanks!
Great, I will send new version.
>
> Gr{oetje,eeting}s,
>
> Geert
>
--
Regards,
Oleksandr Tyshchenko