If CPUs are marked as disabled in the devicetree, make sure they do
not exist in the system CPU information and CPU topology information.
In this case these CPUs will not be able to be added to the system later
using hot-plug. This allows a single chip with many CPUs to be easily
used in a variety of hardware devices where they may have different
actual processing requirements (eg for thermal/cost reasons).
- Change devicetree.c to ignore any cpu nodes marked as disabled,
this effectively limits the number of active cpu cores so no need
for the max_cpus=x in the chosen node.
- Change topology.c to ignore any cpu nodes marked as disabled, this
is where the scheduler would learn about big/LITTLE cores so this
effectively keeps the scheduler in sync.
Signed-off-by: James King <[email protected]>
---
Documentation/devicetree/bindings/arm/cpus.txt | 5 +++++
arch/arm/kernel/devtree.c | 6 ++++++
arch/arm/kernel/topology.c | 4 ++++
3 files changed, 15 insertions(+)
diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
index f32494d..9fbcbc5 100644
--- a/Documentation/devicetree/bindings/arm/cpus.txt
+++ b/Documentation/devicetree/bindings/arm/cpus.txt
@@ -45,6 +45,10 @@ For the ARM architecture every CPU node must contain the following properties:
"marvell,xsc3"
"marvell,xscale"
+And optionally set the following properties:
+
+- status: can be set to "disabled" to remove that CPU from the system CPU topology
+
Example:
cpus {
@@ -73,5 +77,6 @@ Example:
device_type = "cpu";
compatible = "arm,cortex-a7";
reg = <0x101>;
+ status = "disabled";
};
};
diff --git a/arch/arm/kernel/devtree.c b/arch/arm/kernel/devtree.c
index 5af04f6..f4ba8ee 100644
--- a/arch/arm/kernel/devtree.c
+++ b/arch/arm/kernel/devtree.c
@@ -112,6 +112,12 @@ void __init arm_dt_init_cpu_maps(void)
return;
/*
+ * Check if the cpu is marked as "disabled", if so ignore.
+ */
+ if (!of_device_is_available(cpu))
+ continue;
+
+ /*
* Duplicate MPIDRs are a recipe for disaster.
* Scan all initialized entries and check for
* duplicates. If any is found just bail out.
diff --git a/arch/arm/kernel/topology.c b/arch/arm/kernel/topology.c
index f10316b..90f8fb3 100644
--- a/arch/arm/kernel/topology.c
+++ b/arch/arm/kernel/topology.c
@@ -116,6 +116,10 @@ static void __init parse_dt_topology(void)
if (cpu_eff->compatible == NULL)
continue;
+ /* Check if the cpu is marked as "disabled", if so ignore. */
+ if (!of_device_is_available(cn))
+ continue;
+
rate = of_get_property(cn, "clock-frequency", &len);
if (!rate || len != 4) {
pr_err("%s missing clock-frequency property\n",
--
1.8.1.2
On Thu, 6 Jun 2013, James King wrote:
> If CPUs are marked as disabled in the devicetree, make sure they do
> not exist in the system CPU information and CPU topology information.
> In this case these CPUs will not be able to be added to the system later
> using hot-plug. This allows a single chip with many CPUs to be easily
> used in a variety of hardware devices where they may have different
> actual processing requirements (eg for thermal/cost reasons).
>
> - Change devicetree.c to ignore any cpu nodes marked as disabled,
> this effectively limits the number of active cpu cores so no need
> for the max_cpus=x in the chosen node.
> - Change topology.c to ignore any cpu nodes marked as disabled, this
> is where the scheduler would learn about big/LITTLE cores so this
> effectively keeps the scheduler in sync.
>
> Signed-off-by: James King <[email protected]>
Acked-by: Nicolas Pitre <[email protected]>
> ---
> Documentation/devicetree/bindings/arm/cpus.txt | 5 +++++
> arch/arm/kernel/devtree.c | 6 ++++++
> arch/arm/kernel/topology.c | 4 ++++
> 3 files changed, 15 insertions(+)
>
> diff --git a/Documentation/devicetree/bindings/arm/cpus.txt b/Documentation/devicetree/bindings/arm/cpus.txt
> index f32494d..9fbcbc5 100644
> --- a/Documentation/devicetree/bindings/arm/cpus.txt
> +++ b/Documentation/devicetree/bindings/arm/cpus.txt
> @@ -45,6 +45,10 @@ For the ARM architecture every CPU node must contain the following properties:
> "marvell,xsc3"
> "marvell,xscale"
>
> +And optionally set the following properties:
> +
> +- status: can be set to "disabled" to remove that CPU from the system CPU topology
> +
> Example:
>
> cpus {
> @@ -73,5 +77,6 @@ Example:
> device_type = "cpu";
> compatible = "arm,cortex-a7";
> reg = <0x101>;
> + status = "disabled";
> };
> };
> diff --git a/arch/arm/kernel/devtree.c b/arch/arm/kernel/devtree.c
> index 5af04f6..f4ba8ee 100644
> --- a/arch/arm/kernel/devtree.c
> +++ b/arch/arm/kernel/devtree.c
> @@ -112,6 +112,12 @@ void __init arm_dt_init_cpu_maps(void)
> return;
>
> /*
> + * Check if the cpu is marked as "disabled", if so ignore.
> + */
> + if (!of_device_is_available(cpu))
> + continue;
> +
> + /*
> * Duplicate MPIDRs are a recipe for disaster.
> * Scan all initialized entries and check for
> * duplicates. If any is found just bail out.
> diff --git a/arch/arm/kernel/topology.c b/arch/arm/kernel/topology.c
> index f10316b..90f8fb3 100644
> --- a/arch/arm/kernel/topology.c
> +++ b/arch/arm/kernel/topology.c
> @@ -116,6 +116,10 @@ static void __init parse_dt_topology(void)
> if (cpu_eff->compatible == NULL)
> continue;
>
> + /* Check if the cpu is marked as "disabled", if so ignore. */
> + if (!of_device_is_available(cn))
> + continue;
> +
> rate = of_get_property(cn, "clock-frequency", &len);
> if (!rate || len != 4) {
> pr_err("%s missing clock-frequency property\n",
> --
> 1.8.1.2
>
Hi James,
On Thu, Jun 06, 2013 at 06:11:25PM +0100, James King wrote:
> If CPUs are marked as disabled in the devicetree, make sure they do
> not exist in the system CPU information and CPU topology information.
> In this case these CPUs will not be able to be added to the system later
> using hot-plug. This allows a single chip with many CPUs to be easily
> used in a variety of hardware devices where they may have different
> actual processing requirements (eg for thermal/cost reasons).
>
> - Change devicetree.c to ignore any cpu nodes marked as disabled,
> this effectively limits the number of active cpu cores so no need
> for the max_cpus=x in the chosen node.
> - Change topology.c to ignore any cpu nodes marked as disabled, this
> is where the scheduler would learn about big/LITTLE cores so this
> effectively keeps the scheduler in sync.
>
I have two questions:
1) Since with this approach the DT should change anyway if on different
hardware devices based on the same chip you want to allow booting a
different number of CPUs, why do not we remove the cpu nodes instead of
disabling them ? Put it another way: cpu nodes define a cpu as
possible (currently), we can simply remove the node if we do not want
that cpu to be seen by the kernel.
2) If we go for the "status" property, why do not we use it to set present
mask ? That way the cpu is possible but not present, you cannot
hotplug it in. It is a bit of a stretch, granted, the cpu _is_ present,
we just want to disable it, do not know how this is handled in x86
and other archs though.
I am just asking, since it is something I thought about while writing
code that parses the DT cpu map, basically we do not have a way to
"disable" a cpu in the DT and that's what you are doing, I just would like
to understand the best way to put it into DT bindings.
Thanks,
Lorenzo
Hi Lorenzo,
On 7 June 2013 11:23, Lorenzo Pieralisi <[email protected]> wrote:
> Hi James,
>
> On Thu, Jun 06, 2013 at 06:11:25PM +0100, James King wrote:
>> If CPUs are marked as disabled in the devicetree, make sure they do
>> not exist in the system CPU information and CPU topology information.
>> In this case these CPUs will not be able to be added to the system later
>> using hot-plug. This allows a single chip with many CPUs to be easily
>> used in a variety of hardware devices where they may have different
>> actual processing requirements (eg for thermal/cost reasons).
>>
>> - Change devicetree.c to ignore any cpu nodes marked as disabled,
>> this effectively limits the number of active cpu cores so no need
>> for the max_cpus=x in the chosen node.
>> - Change topology.c to ignore any cpu nodes marked as disabled, this
>> is where the scheduler would learn about big/LITTLE cores so this
>> effectively keeps the scheduler in sync.
>>
>
> I have two questions:
>
> 1) Since with this approach the DT should change anyway if on different
> hardware devices based on the same chip you want to allow booting a
> different number of CPUs, why do not we remove the cpu nodes instead of
> disabling them ? Put it another way: cpu nodes define a cpu as
> possible (currently), we can simply remove the node if we do not want
> that cpu to be seen by the kernel.
The reason we want disabled status rather than just remove the nodes
is to use a common soc.dtsi file which is included in many board.dts
files - eg:
file soc.dtsi contains:
cpus {
cpu0: cpu@0 {
device_type = "cpu";
compatible = "arm,cortex-a7";
reg = <0>;
cluster = <&cluster0>;
core = <&core0>;
};
cpu1: cpu@1 {
device_type = "cpu";
compatible = "arm,cortex-a7";
reg = <1>;
cluster = <&cluster0>;
core = <&core1>;
};
cpu2: cpu@2 {
device_type = "cpu";
compatible = "arm,cortex-a15";
reg = <2>;
cluster = <&cluster0>;
core = <&core2>;
};
};
file board1.dts where we want the A15 disabled contains:
/include/ "soc.dtsi"
cpus {
cpu2: cpu@2 {
status = "disabled";
};
};
> 2) If we go for the "status" property, why do not we use it to set present
> mask ? That way the cpu is possible but not present, you cannot
> hotplug it in. It is a bit of a stretch, granted, the cpu _is_ present,
> we just want to disable it, do not know how this is handled in x86
> and other archs though.
I have been struggling to find any equivalent example for another
arch, so just tried to solve our problem. I guess in the x86 world it
is less likely to want to disable processors in a SoC for heat/battery
issues so this has just never arisen. In this case the cpu is
physically present but not possible, but I am not sure it should be in
a present mask (giving the impression it can be used). Perhaps you
could elaborate with an example what you are thinking about here?
Thanks,
James
>
> I am just asking, since it is something I thought about while writing
> code that parses the DT cpu map, basically we do not have a way to
> "disable" a cpu in the DT and that's what you are doing, I just would like
> to understand the best way to put it into DT bindings.
>
> Thanks,
> Lorenzo
>
On 06/07/2013 05:23 AM, Lorenzo Pieralisi wrote:
> Hi James,
>
> On Thu, Jun 06, 2013 at 06:11:25PM +0100, James King wrote:
>> If CPUs are marked as disabled in the devicetree, make sure they do
>> not exist in the system CPU information and CPU topology information.
>> In this case these CPUs will not be able to be added to the system later
>> using hot-plug. This allows a single chip with many CPUs to be easily
>> used in a variety of hardware devices where they may have different
>> actual processing requirements (eg for thermal/cost reasons).
>>
>> - Change devicetree.c to ignore any cpu nodes marked as disabled,
>> this effectively limits the number of active cpu cores so no need
>> for the max_cpus=x in the chosen node.
>> - Change topology.c to ignore any cpu nodes marked as disabled, this
>> is where the scheduler would learn about big/LITTLE cores so this
>> effectively keeps the scheduler in sync.
>>
>
> I have two questions:
>
> 1) Since with this approach the DT should change anyway if on different
> hardware devices based on the same chip you want to allow booting a
> different number of CPUs, why do not we remove the cpu nodes instead of
> disabling them ? Put it another way: cpu nodes define a cpu as
> possible (currently), we can simply remove the node if we do not want
> that cpu to be seen by the kernel.
> 2) If we go for the "status" property, why do not we use it to set present
> mask ? That way the cpu is possible but not present, you cannot
> hotplug it in. It is a bit of a stretch, granted, the cpu _is_ present,
> we just want to disable it, do not know how this is handled in x86
> and other archs though.
The meaning of disabled for cpus in ePAPR is that the cpu is offline
(i.e. in a spinloop or wfi), not that the cpu is unavailable. This is a
bit of a departure and inconsistency from how status for devices are
used. That would imply that we should be setting status to disabled for
all secondary cores and that possibly the status value should get
updated to reflect the state of the cpu.
Rob
> I am just asking, since it is something I thought about while writing
> code that parses the DT cpu map, basically we do not have a way to
> "disable" a cpu in the DT and that's what you are doing, I just would like
> to understand the best way to put it into DT bindings.
>
> Thanks,
> Lorenzo
>
> _______________________________________________
> devicetree-discuss mailing list
> [email protected]
> https://lists.ozlabs.org/listinfo/devicetree-discuss
>
On Fri, Jun 07, 2013 at 03:20:20PM +0100, Rob Herring wrote:
> On 06/07/2013 05:23 AM, Lorenzo Pieralisi wrote:
> > Hi James,
> >
> > On Thu, Jun 06, 2013 at 06:11:25PM +0100, James King wrote:
> >> If CPUs are marked as disabled in the devicetree, make sure they do
> >> not exist in the system CPU information and CPU topology information.
> >> In this case these CPUs will not be able to be added to the system later
> >> using hot-plug. This allows a single chip with many CPUs to be easily
> >> used in a variety of hardware devices where they may have different
> >> actual processing requirements (eg for thermal/cost reasons).
> >>
> >> - Change devicetree.c to ignore any cpu nodes marked as disabled,
> >> this effectively limits the number of active cpu cores so no need
> >> for the max_cpus=x in the chosen node.
> >> - Change topology.c to ignore any cpu nodes marked as disabled, this
> >> is where the scheduler would learn about big/LITTLE cores so this
> >> effectively keeps the scheduler in sync.
> >>
> >
> > I have two questions:
> >
> > 1) Since with this approach the DT should change anyway if on different
> > hardware devices based on the same chip you want to allow booting a
> > different number of CPUs, why do not we remove the cpu nodes instead of
> > disabling them ? Put it another way: cpu nodes define a cpu as
> > possible (currently), we can simply remove the node if we do not want
> > that cpu to be seen by the kernel.
> > 2) If we go for the "status" property, why do not we use it to set present
> > mask ? That way the cpu is possible but not present, you cannot
> > hotplug it in. It is a bit of a stretch, granted, the cpu _is_ present,
> > we just want to disable it, do not know how this is handled in x86
> > and other archs though.
>
> The meaning of disabled for cpus in ePAPR is that the cpu is offline
> (i.e. in a spinloop or wfi), not that the cpu is unavailable. This is a
> bit of a departure and inconsistency from how status for devices are
> used. That would imply that we should be setting status to disabled for
> all secondary cores and that possibly the status value should get
> updated to reflect the state of the cpu.
Yes, that's what I understood from the ePAPR as well. According to
the ePAPR, as you say, a cpu with its status property == "disabled" is a
possible CPU, since it can be enabled (through a specific enable-method).
I am not sure "status" can be reused for the purpose this patch was developed
for without changing the bindings in the ePAPR (ie if DT parsing skips
cpu nodes with status == "disabled", this is a significant departure
from what ePAPR defines, and it would force us to define an enable-method
to enable/online those CPUs which is not what this patch was developed for).
How was PowerPC tackling the problem James set about solving ?
Lorenzo
On Fri, Jun 07, 2013 at 12:48:58PM +0100, James King wrote:
> Hi Lorenzo,
>
> On 7 June 2013 11:23, Lorenzo Pieralisi <[email protected]> wrote:
> > Hi James,
> >
> > On Thu, Jun 06, 2013 at 06:11:25PM +0100, James King wrote:
> >> If CPUs are marked as disabled in the devicetree, make sure they do
> >> not exist in the system CPU information and CPU topology information.
> >> In this case these CPUs will not be able to be added to the system later
> >> using hot-plug. This allows a single chip with many CPUs to be easily
> >> used in a variety of hardware devices where they may have different
> >> actual processing requirements (eg for thermal/cost reasons).
> >>
> >> - Change devicetree.c to ignore any cpu nodes marked as disabled,
> >> this effectively limits the number of active cpu cores so no need
> >> for the max_cpus=x in the chosen node.
> >> - Change topology.c to ignore any cpu nodes marked as disabled, this
> >> is where the scheduler would learn about big/LITTLE cores so this
> >> effectively keeps the scheduler in sync.
> >>
> >
> > I have two questions:
> >
> > 1) Since with this approach the DT should change anyway if on different
> > hardware devices based on the same chip you want to allow booting a
> > different number of CPUs, why do not we remove the cpu nodes instead of
> > disabling them ? Put it another way: cpu nodes define a cpu as
> > possible (currently), we can simply remove the node if we do not want
> > that cpu to be seen by the kernel.
>
> The reason we want disabled status rather than just remove the nodes
> is to use a common soc.dtsi file which is included in many board.dts
> files - eg:
>
> file soc.dtsi contains:
>
> cpus {
> cpu0: cpu@0 {
> device_type = "cpu";
> compatible = "arm,cortex-a7";
> reg = <0>;
> cluster = <&cluster0>;
> core = <&core0>;
Minor nit, "cluster" and "core" phandles are not part of cpu the bindings
that will be merged this cycle, I know it is just an example.
> };
>
> cpu1: cpu@1 {
> device_type = "cpu";
> compatible = "arm,cortex-a7";
> reg = <1>;
> cluster = <&cluster0>;
> core = <&core1>;
> };
>
> cpu2: cpu@2 {
> device_type = "cpu";
> compatible = "arm,cortex-a15";
> reg = <2>;
> cluster = <&cluster0>;
> core = <&core2>;
> };
> };
>
> file board1.dts where we want the A15 disabled contains:
>
> /include/ "soc.dtsi"
>
> cpus {
> cpu2: cpu@2 {
> status = "disabled";
> };
> };
Understood, see the other reply as far as the status property is concerned.
> > 2) If we go for the "status" property, why do not we use it to set present
> > mask ? That way the cpu is possible but not present, you cannot
> > hotplug it in. It is a bit of a stretch, granted, the cpu _is_ present,
> > we just want to disable it, do not know how this is handled in x86
> > and other archs though.
>
> I have been struggling to find any equivalent example for another
> arch, so just tried to solve our problem. I guess in the x86 world it
> is less likely to want to disable processors in a SoC for heat/battery
> issues so this has just never arisen. In this case the cpu is
> physically present but not possible, but I am not sure it should be in
> a present mask (giving the impression it can be used). Perhaps you
> could elaborate with an example what you are thinking about here?
I was thinking about using status == "disabled" to mark a cpu as
possible but not present; that is a bad idea for the reason you mentioned
and also for the point Rob raised related to the ePAPR.
I am not sure how we can solve this issue, as I mentioned the easiest
solution consists in not defining cpu nodes in the DT or probably add
an additional property != status with proper bindings attached to it.
Lorenzo
On 7 June 2013 17:41, Lorenzo Pieralisi <[email protected]> wrote:
> On Fri, Jun 07, 2013 at 12:48:58PM +0100, James King wrote:
>> Hi Lorenzo,
>>
>> On 7 June 2013 11:23, Lorenzo Pieralisi <[email protected]> wrote:
>> > Hi James,
>> >
>> > On Thu, Jun 06, 2013 at 06:11:25PM +0100, James King wrote:
>> >> If CPUs are marked as disabled in the devicetree, make sure they do
>> >> not exist in the system CPU information and CPU topology information.
>> >> In this case these CPUs will not be able to be added to the system later
>> >> using hot-plug. This allows a single chip with many CPUs to be easily
>> >> used in a variety of hardware devices where they may have different
>> >> actual processing requirements (eg for thermal/cost reasons).
>> >>
>> >> - Change devicetree.c to ignore any cpu nodes marked as disabled,
>> >> this effectively limits the number of active cpu cores so no need
>> >> for the max_cpus=x in the chosen node.
>> >> - Change topology.c to ignore any cpu nodes marked as disabled, this
>> >> is where the scheduler would learn about big/LITTLE cores so this
>> >> effectively keeps the scheduler in sync.
>> >>
>> >
>> > I have two questions:
>> >
>> > 1) Since with this approach the DT should change anyway if on different
>> > hardware devices based on the same chip you want to allow booting a
>> > different number of CPUs, why do not we remove the cpu nodes instead of
>> > disabling them ? Put it another way: cpu nodes define a cpu as
>> > possible (currently), we can simply remove the node if we do not want
>> > that cpu to be seen by the kernel.
>>
>> The reason we want disabled status rather than just remove the nodes
>> is to use a common soc.dtsi file which is included in many board.dts
>> files - eg:
>>
>> file soc.dtsi contains:
>>
>> cpus {
>> cpu0: cpu@0 {
>> device_type = "cpu";
>> compatible = "arm,cortex-a7";
>> reg = <0>;
>> cluster = <&cluster0>;
>> core = <&core0>;
>
> Minor nit, "cluster" and "core" phandles are not part of cpu the bindings
> that will be merged this cycle, I know it is just an example.
>
>> };
>>
>> cpu1: cpu@1 {
>> device_type = "cpu";
>> compatible = "arm,cortex-a7";
>> reg = <1>;
>> cluster = <&cluster0>;
>> core = <&core1>;
>> };
>>
>> cpu2: cpu@2 {
>> device_type = "cpu";
>> compatible = "arm,cortex-a15";
>> reg = <2>;
>> cluster = <&cluster0>;
>> core = <&core2>;
>> };
>> };
>>
>> file board1.dts where we want the A15 disabled contains:
>>
>> /include/ "soc.dtsi"
>>
>> cpus {
>> cpu2: cpu@2 {
>> status = "disabled";
>> };
>> };
>
> Understood, see the other reply as far as the status property is concerned.
>
>> > 2) If we go for the "status" property, why do not we use it to set present
>> > mask ? That way the cpu is possible but not present, you cannot
>> > hotplug it in. It is a bit of a stretch, granted, the cpu _is_ present,
>> > we just want to disable it, do not know how this is handled in x86
>> > and other archs though.
>>
>> I have been struggling to find any equivalent example for another
>> arch, so just tried to solve our problem. I guess in the x86 world it
>> is less likely to want to disable processors in a SoC for heat/battery
>> issues so this has just never arisen. In this case the cpu is
>> physically present but not possible, but I am not sure it should be in
>> a present mask (giving the impression it can be used). Perhaps you
>> could elaborate with an example what you are thinking about here?
>
> I was thinking about using status == "disabled" to mark a cpu as
> possible but not present; that is a bad idea for the reason you mentioned
> and also for the point Rob raised related to the ePAPR.
>
> I am not sure how we can solve this issue, as I mentioned the easiest
> solution consists in not defining cpu nodes in the DT or probably add
> an additional property != status with proper bindings attached to it.
>
So what the ePAPR says is:
"disabled" == Indicates that the device is not presently operational,
but it might
become operational in the future (for example, something is not
plugged in, or switched off).
Refer to the device binding for details on what disabled means for
a given device.
So, would "disabled" be ok to use if those cores 'could' be re-added
to the system using hot-plug?
This would probably be fine for our usage as the user would have to
have the right permissions to hot-plug in a cpu anyway (so is just as
strong a protection as the DT anyway - if you root your device you can
burn it). The main thing is the scheduler will not try and use it by
default.
If that sounds ok - I need to go back and look at the changes to see
what will still allow a core to be hot-plugged in.
Thanks,
James
> Lorenzo
>