It is considered good practice to call cpu_relax() in busy loops, see
Documentation/process/volatile-considered-harmful.rst. This can not
only lower CPU power consumption or yield to a hyperthreaded twin
processor, but also allows an architecture to mitigate hardware issues
(e.g. ARM Erratum 754327 for Cortex-A9 prior to r2p0) in the
architecture-specific cpu_relax() implementation.
As the iopoll helpers lack calls to cpu_relax(), people are sometimes
reluctant to use them, and may fall back to open-coded polling loops
(including cpu_relax() calls) instead.
Fix this by adding calls to cpu_relax() to the iopoll helpers:
- For the non-atomic case, it is sufficient to call cpu_relax() in
case of a zero sleep-between-reads value, as a call to
usleep_range() is a safe barrier otherwise.
- For the atomic case, cpu_relax() must be called regardless of the
sleep-between-reads value, as there is no guarantee all
architecture-specific implementations of udelay() handle this.
Signed-off-by: Geert Uytterhoeven <[email protected]>
---
Resent with a larger audience due to lack of comments.
This has been discussed before, but I am not aware of any patches moving
forward:
- "Re: [PATCH 6/7] clk: renesas: rcar-gen3: Add custom clock for PLLs"
https://lore.kernel.org/all/CAMuHMdWUEhs=nwP+a0vO2jOzkq-7FEOqcJ+SsxAGNXX1PQ2KMA@mail.gmail.com/
- "Re: [PATCH v2] clk: samsung: Prevent potential endless loop in the PLL set_rate ops"
https://lore.kernel.org/all/20200811164628.GA7958@kozik-lap
---
include/linux/iopoll.h | 3 +++
1 file changed, 3 insertions(+)
diff --git a/include/linux/iopoll.h b/include/linux/iopoll.h
index 2c8860e406bd8cae..73132721d1891a2e 100644
--- a/include/linux/iopoll.h
+++ b/include/linux/iopoll.h
@@ -53,6 +53,8 @@
} \
if (__sleep_us) \
usleep_range((__sleep_us >> 2) + 1, __sleep_us); \
+ else \
+ cpu_relax(); \
} \
(cond) ? 0 : -ETIMEDOUT; \
})
@@ -95,6 +97,7 @@
} \
if (__delay_us) \
udelay(__delay_us); \
+ cpu_relax(); \
} \
(cond) ? 0 : -ETIMEDOUT; \
})
--
2.34.1
On Thu, Jan 26, 2023 at 11:45:37AM +0100, Geert Uytterhoeven wrote:
> It is considered good practice to call cpu_relax() in busy loops, see
> Documentation/process/volatile-considered-harmful.rst. This can not
> only lower CPU power consumption or yield to a hyperthreaded twin
> processor, but also allows an architecture to mitigate hardware issues
> (e.g. ARM Erratum 754327 for Cortex-A9 prior to r2p0) in the
> architecture-specific cpu_relax() implementation.
>
> As the iopoll helpers lack calls to cpu_relax(), people are sometimes
> reluctant to use them, and may fall back to open-coded polling loops
> (including cpu_relax() calls) instead.
>
> Fix this by adding calls to cpu_relax() to the iopoll helpers:
> - For the non-atomic case, it is sufficient to call cpu_relax() in
> case of a zero sleep-between-reads value, as a call to
> usleep_range() is a safe barrier otherwise.
> - For the atomic case, cpu_relax() must be called regardless of the
> sleep-between-reads value, as there is no guarantee all
> architecture-specific implementations of udelay() handle this.
>
> Signed-off-by: Geert Uytterhoeven <[email protected]>
In addition to these dodgy architecture fails, cpu_relax() is also a
compiler barrier, it is not immediately obvious that the @op argument
'function' will result in an actual function call (inlining ftw).
Where a function call is a C sequence point, this is lost on inlining.
Therefore, with agressive enough optimization it might be possible for
the compiler to hoist the:
(val) = op(args);
'load' out of the loop because it doesn't see the value changing. The
addition of cpu_relax() will inhibit this.
Acked-by: Peter Zijlstra (Intel) <[email protected]>
> ---
> Resent with a larger audience due to lack of comments.
>
> This has been discussed before, but I am not aware of any patches moving
> forward:
> - "Re: [PATCH 6/7] clk: renesas: rcar-gen3: Add custom clock for PLLs"
> https://lore.kernel.org/all/CAMuHMdWUEhs=nwP+a0vO2jOzkq-7FEOqcJ+SsxAGNXX1PQ2KMA@mail.gmail.com/
> - "Re: [PATCH v2] clk: samsung: Prevent potential endless loop in the PLL set_rate ops"
> https://lore.kernel.org/all/20200811164628.GA7958@kozik-lap
> ---
> include/linux/iopoll.h | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/include/linux/iopoll.h b/include/linux/iopoll.h
> index 2c8860e406bd8cae..73132721d1891a2e 100644
> --- a/include/linux/iopoll.h
> +++ b/include/linux/iopoll.h
> @@ -53,6 +53,8 @@
> } \
> if (__sleep_us) \
> usleep_range((__sleep_us >> 2) + 1, __sleep_us); \
> + else \
> + cpu_relax(); \
There's a simplicitly argument to be had for making it unconditional
here too I suppose. usleep() is 'slow' anyway.
> } \
> (cond) ? 0 : -ETIMEDOUT; \
> })
> @@ -95,6 +97,7 @@
> } \
> if (__delay_us) \
> udelay(__delay_us); \
> + cpu_relax(); \
> } \
> (cond) ? 0 : -ETIMEDOUT; \
> })
> --
> 2.34.1
>
On Thu, Jan 26, 2023, at 11:45, Geert Uytterhoeven wrote:
> It is considered good practice to call cpu_relax() in busy loops, see
> Documentation/process/volatile-considered-harmful.rst. This can not
> only lower CPU power consumption or yield to a hyperthreaded twin
> processor, but also allows an architecture to mitigate hardware issues
> (e.g. ARM Erratum 754327 for Cortex-A9 prior to r2p0) in the
> architecture-specific cpu_relax() implementation.
>
> As the iopoll helpers lack calls to cpu_relax(), people are sometimes
> reluctant to use them, and may fall back to open-coded polling loops
> (including cpu_relax() calls) instead.
>
> Fix this by adding calls to cpu_relax() to the iopoll helpers:
> - For the non-atomic case, it is sufficient to call cpu_relax() in
> case of a zero sleep-between-reads value, as a call to
> usleep_range() is a safe barrier otherwise.
> - For the atomic case, cpu_relax() must be called regardless of the
> sleep-between-reads value, as there is no guarantee all
> architecture-specific implementations of udelay() handle this.
>
> Signed-off-by: Geert Uytterhoeven <[email protected]>
Acked-by: Arnd Bergmann <[email protected]>