When CPPC is being used by ACPI on arm64, user space tools such as
cpupower report CPU frequency values from sysfs that are incorrect.
What the driver was doing was reporting the values given by ACPI tables
in whatever scale was used to provide them. However, the ACPI spec
defines the CPPC values as unitless abstract numbers. Internal kernel
structures such as struct perf_cap, in contrast, expect these values
to be in KHz. When these struct values get reported via sysfs, the
user space tools also assume they are in KHz, causing them to report
incorrect values (for example, reporting a CPU frequency of 1MHz when
it should be 1.8GHz).
While the investigation for a long term fix proceeds (several options
are being explored, some of which may require spec changes or other
much more invasive fixes), this patch forces the values read by CPPC
to be read in KHz, regardless of what they actually represent.
The downside is that this approach has some assumptions:
(1) It relies on SMBIOS3 being used, *and* that the Max Frequency
value for a processor is set to a non-zero value.
(2) It assumes that all processors run at the same speed. This
patch retrieves the first CPU Max Frequency from a type 4 DMI
record that it can find. This may not be an issue, however, as a
sampling of DMI data on x86 and arm64 indicates there is often only
one such record regardless.
For arm64 servers, this may be sufficient, but it does rely on
firmware values being set correctly. Hence, other approaches are
also being considered.
This has been tested on three arm64 servers, with and without DMI, with
and without CPPC support.
Changes for v2:
-- Corrected thinko: needed to have DEPENDS on DMI in Kconfig.arm,
not SELECT DMI (found by build daemon)
Signed-off-by: Al Stone <[email protected]>
---
drivers/acpi/cppc_acpi.c | 61 +++++++++++++++++++++++++++++++++++++++++----
drivers/cpufreq/Kconfig.arm | 1 +
2 files changed, 57 insertions(+), 5 deletions(-)
diff --git a/drivers/acpi/cppc_acpi.c b/drivers/acpi/cppc_acpi.c
index 8adac69..d61ced6 100644
--- a/drivers/acpi/cppc_acpi.c
+++ b/drivers/acpi/cppc_acpi.c
@@ -40,6 +40,9 @@
#include <linux/cpufreq.h>
#include <linux/delay.h>
#include <linux/ktime.h>
+#include <linux/dmi.h>
+
+#include <asm/unaligned.h>
#include <acpi/cppc_acpi.h>
/*
@@ -709,6 +712,47 @@ static int cpc_write(struct cpc_reg *reg, u64 val)
return ret_val;
}
+static u64 cppc_dmi_khz;
+
+static void cppc_find_dmi_mhz(const struct dmi_header *dm, void *private)
+{
+ u16 *mhz = (u16 *)private;
+ const u8 *dmi_data = (const u8 *)dm;
+
+ if (dm->type == DMI_ENTRY_PROCESSOR && dm->length >= 48)
+ *mhz = (u16)get_unaligned((const u16 *)(dmi_data + 0x14));
+}
+
+
+static u64 cppc_get_dmi_khz(void)
+{
+ u16 mhz;
+
+ dmi_walk(cppc_find_dmi_mhz, &mhz);
+
+ /*
+ * Real stupid fallback value, just in case there is no
+ * actual value set.
+ */
+ mhz = mhz ? mhz : 1;
+
+ return (1000 * mhz);
+}
+
+static u64 cppc_unitless_to_khz(u64 min, u64 max, u64 val)
+{
+ /*
+ * The incoming val should be min <= val <= max. Our
+ * job is to convert that to KHz so it can be properly
+ * reported to user space via cpufreq_policy.
+ */
+
+ if (!cppc_dmi_khz)
+ cppc_dmi_khz = cppc_get_dmi_khz();
+
+ return ((val - min) * cppc_dmi_khz) / (max - min);
+}
+
/**
* cppc_get_perf_caps - Get a CPUs performance capabilities.
* @cpunum: CPU from which to get capabilities info.
@@ -748,17 +792,24 @@ int cppc_get_perf_caps(int cpunum, struct cppc_perf_caps *perf_caps)
}
}
+ /*
+ * Since these values in perf_caps will be used in setting
+ * up the cpufreq policy, they must always be stored in units
+ * of KHz. If they are not, user space tools will become very
+ * confused since they assume these are in KHz when reading
+ * sysfs.
+ */
cpc_read(&highest_reg->cpc_entry.reg, &high);
- perf_caps->highest_perf = high;
-
cpc_read(&lowest_reg->cpc_entry.reg, &low);
- perf_caps->lowest_perf = low;
+
+ perf_caps->highest_perf = cppc_unitless_to_khz(low, high, high);
+ perf_caps->lowest_perf = cppc_unitless_to_khz(low, high, low);
cpc_read(&ref_perf->cpc_entry.reg, &ref);
- perf_caps->reference_perf = ref;
+ perf_caps->reference_perf = cppc_unitless_to_khz(low, high, ref);
cpc_read(&nom_perf->cpc_entry.reg, &nom);
- perf_caps->nominal_perf = nom;
+ perf_caps->nominal_perf = cppc_unitless_to_khz(low, high, nom);
if (!ref)
perf_caps->reference_perf = perf_caps->nominal_perf;
diff --git a/drivers/cpufreq/Kconfig.arm b/drivers/cpufreq/Kconfig.arm
index 14b1f93..5555d79 100644
--- a/drivers/cpufreq/Kconfig.arm
+++ b/drivers/cpufreq/Kconfig.arm
@@ -254,6 +254,7 @@ config ARM_PXA2xx_CPUFREQ
config ACPI_CPPC_CPUFREQ
tristate "CPUFreq driver based on the ACPI CPPC spec"
depends on ACPI
+ depends on DMI
select ACPI_CPPC_LIB
default n
help
--
2.5.5
+ Ryan
Hi Al,
On 18 April 2016 at 20:11, Al Stone <[email protected]> wrote:
> When CPPC is being used by ACPI on arm64, user space tools such as
> cpupower report CPU frequency values from sysfs that are incorrect.
>
> What the driver was doing was reporting the values given by ACPI tables
> in whatever scale was used to provide them. However, the ACPI spec
> defines the CPPC values as unitless abstract numbers. Internal kernel
> structures such as struct perf_cap, in contrast, expect these values
> to be in KHz. When these struct values get reported via sysfs, the
> user space tools also assume they are in KHz, causing them to report
> incorrect values (for example, reporting a CPU frequency of 1MHz when
> it should be 1.8GHz).
>
> While the investigation for a long term fix proceeds (several options
> are being explored, some of which may require spec changes or other
> much more invasive fixes), this patch forces the values read by CPPC
> to be read in KHz, regardless of what they actually represent.
>
> The downside is that this approach has some assumptions:
>
> (1) It relies on SMBIOS3 being used, *and* that the Max Frequency
> value for a processor is set to a non-zero value.
>
> (2) It assumes that all processors run at the same speed. This
> patch retrieves the first CPU Max Frequency from a type 4 DMI
> record that it can find. This may not be an issue, however, as a
> sampling of DMI data on x86 and arm64 indicates there is often only
> one such record regardless.
>
> For arm64 servers, this may be sufficient, but it does rely on
> firmware values being set correctly. Hence, other approaches are
> also being considered.
>
> This has been tested on three arm64 servers, with and without DMI, with
> and without CPPC support.
>
> Changes for v2:
> -- Corrected thinko: needed to have DEPENDS on DMI in Kconfig.arm,
> not SELECT DMI (found by build daemon)
>
> Signed-off-by: Al Stone <[email protected]>
This looks like a good short term solution. Does it make more sense to
move this to the cppc_cpufreq driver though? Since that ties more
closely into the cpufreq framework which requires the kHz values in
sysfs. That way we can keep the cppc_acpi.c shim compliant with the
ACPI spec. (i.e. values read in cppc structures remain abstract and
unitless).
Rafael, Viresh, others,
Any other ideas how to handle this better in the long term?
- Decouple the cpufreq sysfs from the cppc driver and introduce its
own entries. Is it possibly to do this cleanly while still allowing
usage of cpufreq registration with existing governors?
- Come up with a scaling factor using the PMU cycle counter at boot
before the CPPC drivers are initialized. This would use the current
freq set by some UEFI var. This would possibly require some messy
perfevents plumbing and added bootup time though.
- .. ?
Cheers,
Ashwin.
On 04/19/2016 02:12 PM, Ashwin Chaugule wrote:
> + Ryan
>
> Hi Al,
>
> On 18 April 2016 at 20:11, Al Stone <[email protected]> wrote:
>> When CPPC is being used by ACPI on arm64, user space tools such as
>> cpupower report CPU frequency values from sysfs that are incorrect.
>>
>> What the driver was doing was reporting the values given by ACPI tables
>> in whatever scale was used to provide them. However, the ACPI spec
>> defines the CPPC values as unitless abstract numbers. Internal kernel
>> structures such as struct perf_cap, in contrast, expect these values
>> to be in KHz. When these struct values get reported via sysfs, the
>> user space tools also assume they are in KHz, causing them to report
>> incorrect values (for example, reporting a CPU frequency of 1MHz when
>> it should be 1.8GHz).
>>
>> While the investigation for a long term fix proceeds (several options
>> are being explored, some of which may require spec changes or other
>> much more invasive fixes), this patch forces the values read by CPPC
>> to be read in KHz, regardless of what they actually represent.
>>
>> The downside is that this approach has some assumptions:
>>
>> (1) It relies on SMBIOS3 being used, *and* that the Max Frequency
>> value for a processor is set to a non-zero value.
>>
>> (2) It assumes that all processors run at the same speed. This
>> patch retrieves the first CPU Max Frequency from a type 4 DMI
>> record that it can find. This may not be an issue, however, as a
>> sampling of DMI data on x86 and arm64 indicates there is often only
>> one such record regardless.
>>
>> For arm64 servers, this may be sufficient, but it does rely on
>> firmware values being set correctly. Hence, other approaches are
>> also being considered.
>>
>> This has been tested on three arm64 servers, with and without DMI, with
>> and without CPPC support.
>>
>> Changes for v2:
>> -- Corrected thinko: needed to have DEPENDS on DMI in Kconfig.arm,
>> not SELECT DMI (found by build daemon)
>>
>> Signed-off-by: Al Stone <[email protected]>
>
> This looks like a good short term solution. Does it make more sense to
> move this to the cppc_cpufreq driver though? Since that ties more
> closely into the cpufreq framework which requires the kHz values in
> sysfs. That way we can keep the cppc_acpi.c shim compliant with the
> ACPI spec. (i.e. values read in cppc structures remain abstract and
> unitless).
Perhaps. Doing it that way made the patch a bit messier since
cppc_acpi.c would set values that then had to be replaced in
cppc_cpufreq.c, so initialization looked odd to me; that's how
I ended up here. You do raise a good point, however; I'll look
at that approach again since I could have missed an easier way
to do it.
> Rafael, Viresh, others,
>
> Any other ideas how to handle this better in the long term?
>
> - Decouple the cpufreq sysfs from the cppc driver and introduce its
> own entries. Is it possibly to do this cleanly while still allowing
> usage of cpufreq registration with existing governors?
>
> - Come up with a scaling factor using the PMU cycle counter at boot
> before the CPPC drivers are initialized. This would use the current
> freq set by some UEFI var. This would possibly require some messy
> perfevents plumbing and added bootup time though.
>
> - .. ?
>
>
> Cheers,
> Ashwin.
>
The other thought that occurs to me is to go back through the
perf_cap and cpufreq structs and make them more general -- perhaps
store the units being used and pointers to functions to convert them
to KHz. This may require separating sysfs data for perf_cap from the
cpufreq sysfs data from the cppc sysfs data. But, if units are then
reported out to sysfs, user space tools can do whatever conversions
they want, or at least know what they're reporting instead of there
being an implicit ABI between the kernel and the tools. This would
be a far more invasive patch set, I think, but it still may be the
right thing to do for the long term.
--
ciao,
al
-----------------------------------
Al Stone
Software Engineer
Red Hat, Inc.
[email protected]
-----------------------------------
On Tue, Apr 19, 2016 at 1:11 AM, Al Stone <[email protected]> wrote:
>
> When CPPC is being used by ACPI on arm64, user space tools such as
> cpupower report CPU frequency values from sysfs that are incorrect.
>
> What the driver was doing was reporting the values given by ACPI tables
> in whatever scale was used to provide them. However, the ACPI spec
> defines the CPPC values as unitless abstract numbers. Internal kernel
> structures such as struct perf_cap, in contrast, expect these values
> to be in KHz. When these struct values get reported via sysfs, the
> user space tools also assume they are in KHz, causing them to report
> incorrect values (for example, reporting a CPU frequency of 1MHz when
> it should be 1.8GHz).
>
> While the investigation for a long term fix proceeds (several options
> are being explored, some of which may require spec changes or other
> much more invasive fixes), this patch forces the values read by CPPC
> to be read in KHz, regardless of what they actually represent.
>
> The downside is that this approach has some assumptions:
>
> (1) It relies on SMBIOS3 being used, *and* that the Max Frequency
> value for a processor is set to a non-zero value.
>
> (2) It assumes that all processors run at the same speed. This
Sometimes short-term solution becomes long-term. It's worth to place
comment in code about this assumption.
> patch retrieves the first CPU Max Frequency from a type 4 DMI
> record that it can find. This may not be an issue, however, as a
> sampling of DMI data on x86 and arm64 indicates there is often only
> one such record regardless.
>
> For arm64 servers, this may be sufficient, but it does rely on
> firmware values being set correctly. Hence, other approaches are
> also being considered.
>
> This has been tested on three arm64 servers, with and without DMI, with
> and without CPPC support.
>
> Changes for v2:
> -- Corrected thinko: needed to have DEPENDS on DMI in Kconfig.arm,
> not SELECT DMI (found by build daemon)
>
> Signed-off-by: Al Stone <[email protected]>
> ---
> drivers/acpi/cppc_acpi.c | 61 +++++++++++++++++++++++++++++++++++++++++----
> drivers/cpufreq/Kconfig.arm | 1 +
> 2 files changed, 57 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/acpi/cppc_acpi.c b/drivers/acpi/cppc_acpi.c
> index 8adac69..d61ced6 100644
> --- a/drivers/acpi/cppc_acpi.c
> +++ b/drivers/acpi/cppc_acpi.c
> @@ -40,6 +40,9 @@
> #include <linux/cpufreq.h>
> #include <linux/delay.h>
> #include <linux/ktime.h>
> +#include <linux/dmi.h>
> +
> +#include <asm/unaligned.h>
>
> #include <acpi/cppc_acpi.h>
> /*
> @@ -709,6 +712,47 @@ static int cpc_write(struct cpc_reg *reg, u64 val)
> return ret_val;
> }
>
> +static u64 cppc_dmi_khz;
> +
> +static void cppc_find_dmi_mhz(const struct dmi_header *dm, void *private)
> +{
> + u16 *mhz = (u16 *)private;
> + const u8 *dmi_data = (const u8 *)dm;
> +
> + if (dm->type == DMI_ENTRY_PROCESSOR && dm->length >= 48)
> + *mhz = (u16)get_unaligned((const u16 *)(dmi_data + 0x14));
> +}
> +
> +
> +static u64 cppc_get_dmi_khz(void)
> +{
> + u16 mhz;
> +
> + dmi_walk(cppc_find_dmi_mhz, &mhz);
> +
> + /*
> + * Real stupid fallback value, just in case there is no
> + * actual value set.
> + */
> + mhz = mhz ? mhz : 1;
> +
> + return (1000 * mhz);
> +}
> +
> +static u64 cppc_unitless_to_khz(u64 min, u64 max, u64 val)
> +{
> + /*
> + * The incoming val should be min <= val <= max. Our
> + * job is to convert that to KHz so it can be properly
> + * reported to user space via cpufreq_policy.
> + */
> +
> + if (!cppc_dmi_khz)
> + cppc_dmi_khz = cppc_get_dmi_khz();
> +
> + return ((val - min) * cppc_dmi_khz) / (max - min);
How pedantic should the kernel be while dealing with this values?
This 1) can potentially divide by zero (extra care is required to
perform this in Solar System) and 2) can return 0.
Not sure if there is some benefit for firmware to export such
values.
[..]
Best regards,
Alexey
On 04/21/2016 08:53 AM, Alexey Klimov wrote:
>
> On Tue, Apr 19, 2016 at 1:11 AM, Al Stone <[email protected]> wrote:
>>
>> When CPPC is being used by ACPI on arm64, user space tools such as
>> cpupower report CPU frequency values from sysfs that are incorrect.
>>
>> What the driver was doing was reporting the values given by ACPI tables
>> in whatever scale was used to provide them. However, the ACPI spec
>> defines the CPPC values as unitless abstract numbers. Internal kernel
>> structures such as struct perf_cap, in contrast, expect these values
>> to be in KHz. When these struct values get reported via sysfs, the
>> user space tools also assume they are in KHz, causing them to report
>> incorrect values (for example, reporting a CPU frequency of 1MHz when
>> it should be 1.8GHz).
>>
>> While the investigation for a long term fix proceeds (several options
>> are being explored, some of which may require spec changes or other
>> much more invasive fixes), this patch forces the values read by CPPC
>> to be read in KHz, regardless of what they actually represent.
>>
>> The downside is that this approach has some assumptions:
>>
>> (1) It relies on SMBIOS3 being used, *and* that the Max Frequency
>> value for a processor is set to a non-zero value.
>>
>> (2) It assumes that all processors run at the same speed. This
>
> Sometimes short-term solution becomes long-term. It's worth to place
> comment in code about this assumption.
True. I'll add a comment. Thanks.
>> patch retrieves the first CPU Max Frequency from a type 4 DMI
>> record that it can find. This may not be an issue, however, as a
>> sampling of DMI data on x86 and arm64 indicates there is often only
>> one such record regardless.
>>
>> For arm64 servers, this may be sufficient, but it does rely on
>> firmware values being set correctly. Hence, other approaches are
>> also being considered.
>>
>> This has been tested on three arm64 servers, with and without DMI, with
>> and without CPPC support.
>>
>> Changes for v2:
>> -- Corrected thinko: needed to have DEPENDS on DMI in Kconfig.arm,
>> not SELECT DMI (found by build daemon)
>>
>> Signed-off-by: Al Stone <[email protected]>
>> ---
>> drivers/acpi/cppc_acpi.c | 61 +++++++++++++++++++++++++++++++++++++++++----
>> drivers/cpufreq/Kconfig.arm | 1 +
>> 2 files changed, 57 insertions(+), 5 deletions(-)
>>
>> diff --git a/drivers/acpi/cppc_acpi.c b/drivers/acpi/cppc_acpi.c
>> index 8adac69..d61ced6 100644
>> --- a/drivers/acpi/cppc_acpi.c
>> +++ b/drivers/acpi/cppc_acpi.c
>> @@ -40,6 +40,9 @@
>> #include <linux/cpufreq.h>
>> #include <linux/delay.h>
>> #include <linux/ktime.h>
>> +#include <linux/dmi.h>
>> +
>> +#include <asm/unaligned.h>
>>
>> #include <acpi/cppc_acpi.h>
>> /*
>> @@ -709,6 +712,47 @@ static int cpc_write(struct cpc_reg *reg, u64 val)
>> return ret_val;
>> }
>>
>> +static u64 cppc_dmi_khz;
>> +
>> +static void cppc_find_dmi_mhz(const struct dmi_header *dm, void *private)
>> +{
>> + u16 *mhz = (u16 *)private;
>> + const u8 *dmi_data = (const u8 *)dm;
>> +
>> + if (dm->type == DMI_ENTRY_PROCESSOR && dm->length >= 48)
>> + *mhz = (u16)get_unaligned((const u16 *)(dmi_data + 0x14));
>> +}
>> +
>> +
>> +static u64 cppc_get_dmi_khz(void)
>> +{
>> + u16 mhz;
>> +
>> + dmi_walk(cppc_find_dmi_mhz, &mhz);
>> +
>> + /*
>> + * Real stupid fallback value, just in case there is no
>> + * actual value set.
>> + */
>> + mhz = mhz ? mhz : 1;
>> +
>> + return (1000 * mhz);
>> +}
>> +
>> +static u64 cppc_unitless_to_khz(u64 min, u64 max, u64 val)
>> +{
>> + /*
>> + * The incoming val should be min <= val <= max. Our
>> + * job is to convert that to KHz so it can be properly
>> + * reported to user space via cpufreq_policy.
>> + */
>> +
>> + if (!cppc_dmi_khz)
>> + cppc_dmi_khz = cppc_get_dmi_khz();
>> +
>> + return ((val - min) * cppc_dmi_khz) / (max - min);
>
> How pedantic should the kernel be while dealing with this values?
I'm not sure it can be. By definition, the CPPC values define an abstract
range. We are only associating it with a frequency here because those are
the units assumed elsewhere in the kernel, and that user space tools make the
same assumptions. What I'm looking at for the longer term is possibly breaking
those assumptions so that maybe we can be pedantic.
> This 1) can potentially divide by zero (extra care is required to
> perform this in Solar System) and 2) can return 0.
Hrm. I'll double check the path for divide by zero; I thought that was covered
elsewhere along the path but I might have missed it.
A zero in this case would mean the processor is running at its lowest possible
level of performance, and is an artifact of mapping the CPPC abstract value
onto a linear scale from 0 to max KHz. Granted, that may not be exactly the
same as 0 KHz; I'm open to suggestions here. If there's a relatively
straightforward way to get a processor's minimum operating frequency (apart
from completely off), we could eliminate the zero.
> Not sure if there is some benefit for firmware to export such
> values.
Reporting a divide by zero would be bad and should not happen; a value
of zero, though, could be argued. Since we're using a linear scale from
zero to max KHz, it's not unexpected. That being said, though, the only
reason for this patch is so that user space does not report completely
incorrect values; we were seeing MHz values reported by cpupower when
they should have been GHz, for example.
--
ciao,
al
-----------------------------------
Al Stone
Software Engineer
Red Hat, Inc.
[email protected]
-----------------------------------
On 19-04-16, 16:12, Ashwin Chaugule wrote:
> + Ryan
>
> Hi Al,
>
> On 18 April 2016 at 20:11, Al Stone <[email protected]> wrote:
> > When CPPC is being used by ACPI on arm64, user space tools such as
> > cpupower report CPU frequency values from sysfs that are incorrect.
> >
> > What the driver was doing was reporting the values given by ACPI tables
> > in whatever scale was used to provide them. However, the ACPI spec
> > defines the CPPC values as unitless abstract numbers. Internal kernel
> > structures such as struct perf_cap, in contrast, expect these values
> > to be in KHz. When these struct values get reported via sysfs, the
> > user space tools also assume they are in KHz, causing them to report
> > incorrect values (for example, reporting a CPU frequency of 1MHz when
> > it should be 1.8GHz).
> >
> > While the investigation for a long term fix proceeds (several options
> > are being explored, some of which may require spec changes or other
> > much more invasive fixes), this patch forces the values read by CPPC
> > to be read in KHz, regardless of what they actually represent.
> >
> > The downside is that this approach has some assumptions:
> >
> > (1) It relies on SMBIOS3 being used, *and* that the Max Frequency
> > value for a processor is set to a non-zero value.
> >
> > (2) It assumes that all processors run at the same speed. This
> > patch retrieves the first CPU Max Frequency from a type 4 DMI
> > record that it can find. This may not be an issue, however, as a
> > sampling of DMI data on x86 and arm64 indicates there is often only
> > one such record regardless.
Don't we have any big LITTLE ARM servers yet ? Or we will not have them at all ?
> > For arm64 servers, this may be sufficient, but it does rely on
> > firmware values being set correctly. Hence, other approaches are
> > also being considered.
> >
> > This has been tested on three arm64 servers, with and without DMI, with
> > and without CPPC support.
> >
> > Changes for v2:
> > -- Corrected thinko: needed to have DEPENDS on DMI in Kconfig.arm,
> > not SELECT DMI (found by build daemon)
> >
> > Signed-off-by: Al Stone <[email protected]>
>
> This looks like a good short term solution. Does it make more sense to
> move this to the cppc_cpufreq driver though? Since that ties more
> closely into the cpufreq framework which requires the kHz values in
> sysfs. That way we can keep the cppc_acpi.c shim compliant with the
> ACPI spec. (i.e. values read in cppc structures remain abstract and
> unitless).
>
> Rafael, Viresh, others,
>
> Any other ideas how to handle this better in the long term?
>
> - Decouple the cpufreq sysfs from the cppc driver and introduce its
> own entries. Is it possibly to do this cleanly while still allowing
> usage of cpufreq registration with existing governors?
>
> - Come up with a scaling factor using the PMU cycle counter at boot
> before the CPPC drivers are initialized. This would use the current
> freq set by some UEFI var. This would possibly require some messy
> perfevents plumbing and added bootup time though.
I may be missing the obvious, but can't we just create the cpufreq-table from
this table in khz? We wouldn't require any further change then.
--
viresh
On Friday, April 22, 2016 11:00:20 AM Viresh Kumar wrote:
> On 19-04-16, 16:12, Ashwin Chaugule wrote:
> > + Ryan
> >
> > Hi Al,
> >
> > On 18 April 2016 at 20:11, Al Stone <[email protected]> wrote:
> > > When CPPC is being used by ACPI on arm64, user space tools such as
> > > cpupower report CPU frequency values from sysfs that are incorrect.
> > >
> > > What the driver was doing was reporting the values given by ACPI tables
> > > in whatever scale was used to provide them. However, the ACPI spec
> > > defines the CPPC values as unitless abstract numbers. Internal kernel
> > > structures such as struct perf_cap, in contrast, expect these values
> > > to be in KHz. When these struct values get reported via sysfs, the
> > > user space tools also assume they are in KHz, causing them to report
> > > incorrect values (for example, reporting a CPU frequency of 1MHz when
> > > it should be 1.8GHz).
> > >
> > > While the investigation for a long term fix proceeds (several options
> > > are being explored, some of which may require spec changes or other
> > > much more invasive fixes), this patch forces the values read by CPPC
> > > to be read in KHz, regardless of what they actually represent.
> > >
> > > The downside is that this approach has some assumptions:
> > >
> > > (1) It relies on SMBIOS3 being used, *and* that the Max Frequency
> > > value for a processor is set to a non-zero value.
> > >
> > > (2) It assumes that all processors run at the same speed. This
> > > patch retrieves the first CPU Max Frequency from a type 4 DMI
> > > record that it can find. This may not be an issue, however, as a
> > > sampling of DMI data on x86 and arm64 indicates there is often only
> > > one such record regardless.
>
> Don't we have any big LITTLE ARM servers yet ? Or we will not have them at all ?
>
> > > For arm64 servers, this may be sufficient, but it does rely on
> > > firmware values being set correctly. Hence, other approaches are
> > > also being considered.
> > >
> > > This has been tested on three arm64 servers, with and without DMI, with
> > > and without CPPC support.
> > >
> > > Changes for v2:
> > > -- Corrected thinko: needed to have DEPENDS on DMI in Kconfig.arm,
> > > not SELECT DMI (found by build daemon)
> > >
> > > Signed-off-by: Al Stone <[email protected]>
> >
> > This looks like a good short term solution. Does it make more sense to
> > move this to the cppc_cpufreq driver though? Since that ties more
> > closely into the cpufreq framework which requires the kHz values in
> > sysfs. That way we can keep the cppc_acpi.c shim compliant with the
> > ACPI spec. (i.e. values read in cppc structures remain abstract and
> > unitless).
> >
> > Rafael, Viresh, others,
> >
> > Any other ideas how to handle this better in the long term?
> >
> > - Decouple the cpufreq sysfs from the cppc driver and introduce its
> > own entries. Is it possibly to do this cleanly while still allowing
> > usage of cpufreq registration with existing governors?
> >
> > - Come up with a scaling factor using the PMU cycle counter at boot
> > before the CPPC drivers are initialized. This would use the current
> > freq set by some UEFI var. This would possibly require some messy
> > perfevents plumbing and added bootup time though.
>
> I may be missing the obvious, but can't we just create the cpufreq-table from
> this table in khz? We wouldn't require any further change then.
I wouldn't really like to do that, because the freq table would be totally
artificial then.
Thanks,
Rafael
On Tuesday, April 19, 2016 04:12:41 PM Ashwin Chaugule wrote:
> + Ryan
>
> Hi Al,
>
> On 18 April 2016 at 20:11, Al Stone <[email protected]> wrote:
> > When CPPC is being used by ACPI on arm64, user space tools such as
> > cpupower report CPU frequency values from sysfs that are incorrect.
> >
> > What the driver was doing was reporting the values given by ACPI tables
> > in whatever scale was used to provide them. However, the ACPI spec
> > defines the CPPC values as unitless abstract numbers. Internal kernel
> > structures such as struct perf_cap, in contrast, expect these values
> > to be in KHz. When these struct values get reported via sysfs, the
> > user space tools also assume they are in KHz, causing them to report
> > incorrect values (for example, reporting a CPU frequency of 1MHz when
> > it should be 1.8GHz).
> >
> > While the investigation for a long term fix proceeds (several options
> > are being explored, some of which may require spec changes or other
> > much more invasive fixes), this patch forces the values read by CPPC
> > to be read in KHz, regardless of what they actually represent.
> >
> > The downside is that this approach has some assumptions:
> >
> > (1) It relies on SMBIOS3 being used, *and* that the Max Frequency
> > value for a processor is set to a non-zero value.
> >
> > (2) It assumes that all processors run at the same speed. This
> > patch retrieves the first CPU Max Frequency from a type 4 DMI
> > record that it can find. This may not be an issue, however, as a
> > sampling of DMI data on x86 and arm64 indicates there is often only
> > one such record regardless.
> >
> > For arm64 servers, this may be sufficient, but it does rely on
> > firmware values being set correctly. Hence, other approaches are
> > also being considered.
> >
> > This has been tested on three arm64 servers, with and without DMI, with
> > and without CPPC support.
> >
> > Changes for v2:
> > -- Corrected thinko: needed to have DEPENDS on DMI in Kconfig.arm,
> > not SELECT DMI (found by build daemon)
> >
> > Signed-off-by: Al Stone <[email protected]>
>
> This looks like a good short term solution. Does it make more sense to
> move this to the cppc_cpufreq driver though? Since that ties more
> closely into the cpufreq framework which requires the kHz values in
> sysfs. That way we can keep the cppc_acpi.c shim compliant with the
> ACPI spec. (i.e. values read in cppc structures remain abstract and
> unitless).
>
> Rafael, Viresh, others,
>
> Any other ideas how to handle this better in the long term?
>
> - Decouple the cpufreq sysfs from the cppc driver and introduce its
> own entries. Is it possibly to do this cleanly while still allowing
> usage of cpufreq registration with existing governors?
>
> - Come up with a scaling factor using the PMU cycle counter at boot
> before the CPPC drivers are initialized. This would use the current
> freq set by some UEFI var. This would possibly require some messy
> perfevents plumbing and added bootup time though.
>
> - .. ?
Not sure at the moment.
Plus, there's one more thing to consider. We may end up having to use CPPC on
x86 after all (for reasons that are not relevant here), in which case it
probably would make sense to merge the acpi-cpufreq and cppc-cpufreq drivers IMO.
Thanks,
Rafael
On 2016-04-19 18:53, Al Stone wrote:
> On 04/19/2016 02:12 PM, Ashwin Chaugule wrote:
>> + Ryan
>>
>> Hi Al,
>>
>> On 18 April 2016 at 20:11, Al Stone <[email protected]> wrote:
>>> When CPPC is being used by ACPI on arm64, user space tools such as
>>> cpupower report CPU frequency values from sysfs that are incorrect.
>>>
>>> What the driver was doing was reporting the values given by ACPI
>>> tables
>>> in whatever scale was used to provide them. However, the ACPI spec
>>> defines the CPPC values as unitless abstract numbers. Internal
>>> kernel
>>> structures such as struct perf_cap, in contrast, expect these values
>>> to be in KHz. When these struct values get reported via sysfs, the
>>> user space tools also assume they are in KHz, causing them to report
>>> incorrect values (for example, reporting a CPU frequency of 1MHz when
>>> it should be 1.8GHz).
>>>
>>> While the investigation for a long term fix proceeds (several options
>>> are being explored, some of which may require spec changes or other
>>> much more invasive fixes), this patch forces the values read by CPPC
>>> to be read in KHz, regardless of what they actually represent.
>>>
>>> The downside is that this approach has some assumptions:
>>>
>>> (1) It relies on SMBIOS3 being used, *and* that the Max Frequency
>>> value for a processor is set to a non-zero value.
>>>
>>> (2) It assumes that all processors run at the same speed. This
>>> patch retrieves the first CPU Max Frequency from a type 4 DMI
>>> record that it can find. This may not be an issue, however, as a
>>> sampling of DMI data on x86 and arm64 indicates there is often
>>> only
>>> one such record regardless.
>>>
>>> For arm64 servers, this may be sufficient, but it does rely on
>>> firmware values being set correctly. Hence, other approaches are
>>> also being considered.
>>>
>>> This has been tested on three arm64 servers, with and without DMI,
>>> with
>>> and without CPPC support.
>>>
>>> Changes for v2:
>>> -- Corrected thinko: needed to have DEPENDS on DMI in
>>> Kconfig.arm,
>>> not SELECT DMI (found by build daemon)
>>>
>>> Signed-off-by: Al Stone <[email protected]>
>>
>> This looks like a good short term solution. Does it make more sense to
>> move this to the cppc_cpufreq driver though? Since that ties more
>> closely into the cpufreq framework which requires the kHz values in
>> sysfs. That way we can keep the cppc_acpi.c shim compliant with the
>> ACPI spec. (i.e. values read in cppc structures remain abstract and
>> unitless).
>
> Perhaps. Doing it that way made the patch a bit messier since
> cppc_acpi.c would set values that then had to be replaced in
> cppc_cpufreq.c, so initialization looked odd to me; that's how
> I ended up here. You do raise a good point, however; I'll look
> at that approach again since I could have missed an easier way
> to do it.
>
>> Rafael, Viresh, others,
>>
>> Any other ideas how to handle this better in the long term?
>>
>> - Decouple the cpufreq sysfs from the cppc driver and introduce its
>> own entries. Is it possibly to do this cleanly while still allowing
>> usage of cpufreq registration with existing governors?
>>
>> - Come up with a scaling factor using the PMU cycle counter at boot
>> before the CPPC drivers are initialized. This would use the current
>> freq set by some UEFI var. This would possibly require some messy
>> perfevents plumbing and added bootup time though.
>>
>> - .. ?
>>
>>
>> Cheers,
>> Ashwin.
>>
>
> The other thought that occurs to me is to go back through the
> perf_cap and cpufreq structs and make them more general -- perhaps
> store the units being used and pointers to functions to convert them
> to KHz. This may require separating sysfs data for perf_cap from the
> cpufreq sysfs data from the cppc sysfs data. But, if units are then
> reported out to sysfs, user space tools can do whatever conversions
> they want, or at least know what they're reporting instead of there
> being an implicit ABI between the kernel and the tools. This would
> be a far more invasive patch set, I think, but it still may be the
> right thing to do for the long term.
The issue is a little more fundamental than that even. We are
retrofitting a performance management interface (CPPC) into a frequency
management framework (cpufreq) and accompanying tools. Regardless of
what scheme we come up with for deriving/exposing frequency, we still
haven’t completely solved the problem as that assumes a linear
relationship between freq and performance. This will work for many but
not necessarily all CPPC systems. In fact, making that assumption is
explicitly forbidden in the ACPI spec: "OSPM must make no assumption
about the exact meaning of the performance values presented by the
platform, or how they may correlate to specific hardware metrics like
processor frequency."
So to be completely consistent with the current spec, we would need to
ween the tools off of frequency altogether and move to abstract
performance - either specifically when CPPC driver is loaded or more
generally. If we think reporting frequency is required that might still
be doable, but would need to be separate interface from the CPPC perf
scale. But I agree with Al that is a more invasive change.
For the time being, I don't think it is unreasonable to assume
performance is linear with frequency and come up with a scaling factor
via one of several mechanisms:
1) SMBIOS as Al proposed (caveats above)
2) Measure at boot using PMU or other mechanism as Ashwin floated
(more complicated but removes dependency on SMBIOS and assumption that
freq scale is same across all CPUs)
3) Just hardcode a CPPC perf to kHz mapping - maybe everyone is using
MHz today? (simplest but obviously least flexible)
4) Others?
None of these are full solution - it is a question of how many different
scenarios do we need to cover with initial solution. I think the SMBIOS
one is probably a good simplicity/flexibility compromise.
-Ryan
Qualcomm Technologies, Inc. on behalf of Qualcomm Innovation Center,
Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a
Linux Foundation Collaborative Project