This patchset introduces an atomic MMIO modify API.
The motivation for adding this is to allow cheap, infrastructure-less,
thread-safe access to an MMIO region, even in very early scenarios.
The chosen mask/set semantic (proposed by Russell King) is clean and flexible
enough and matches the regmap_update_bits() prototype. Consistency is good.
This series adds a simple arch-generic implementation in a new lib/atomicio.c
file. On top of that it implements an ARM-optimized variant following Will
Deacon's suggestions, that take advantage of ARM relaxed read/write functions.
Finally, just to show a few usage for this function, last two patches show
how this would solve one of the current shared-registers issues.
Since this 4th version is no longer ARM-specific but kernel-wide, and perhaps
new reviewers might jump in, please read the previous discussion on why this
is needed and why we cannot use a regmap API.
v1: https://lkml.org/lkml/2013/8/10/75
v2: http://comments.gmane.org/gmane.linux.ports.arm.kernel/261879
v3: http://www.spinics.net/lists/arm-kernel/msg269263.html
Thoughts?
Changes from v3:
* Implemented an arch-generic atomic_io_modify(), as suggested by Baruch
and Catalin.
* Add an ARM-specific variant, using relaxed R/W as Will suggested.
* Replaced spin_locks by raw_spin_locks, to protect the registers
even on RT.
* Fixed two stupid typos.
Changes from v2:
* As suggested by Will Deacon, dropped the iowmb() barrier
and use relaxed variants instead. See Will's explanation for
details: http://www.spinics.net/lists/arm-kernel/msg268775.html
* Use spin_{}_irqsave/restore to allow irq-context usage
also suggested by Will Deacon.
* Re-worked the API semantics as proposed by Russell King.
Changes from v1:
* Added an io barrier iowmb() as suggested by Will Deacon,
to ensure the writel gets completed before the spin_unlock().
Ezequiel Garcia (4):
lib: Introduce atomic MMIO modify
ARM: Add atomic_io_modify optimized routines
clocksource: orion: Use atomic access for shared registers
watchdog: orion: Use atomic access for shared registers
arch/arm/include/asm/io.h | 4 ++++
arch/arm/kernel/io.c | 29 +++++++++++++++++++++++++++++
drivers/clocksource/time-orion.c | 29 +++++++++++------------------
drivers/watchdog/orion_wdt.c | 8 ++------
include/linux/io.h | 5 +++++
lib/Makefile | 2 +-
lib/atomicio.c | 27 +++++++++++++++++++++++++++
7 files changed, 79 insertions(+), 25 deletions(-)
create mode 100644 lib/atomicio.c
--
1.8.1.5
Some platforms have MMIO regions that are shared across orthogonal
subsystems. This commit implements a possible solution for the
thread-safe access of such regions through a spinlock-protected API.
Concurrent access is protected with a single spinlock for the
entire MMIO address space. While this protects shared-registers,
it also serializes access to unrelated/unshared registers.
Signed-off-by: Ezequiel Garcia <[email protected]>
---
include/linux/io.h | 5 +++++
lib/Makefile | 2 +-
lib/atomicio.c | 27 +++++++++++++++++++++++++++
3 files changed, 33 insertions(+), 1 deletion(-)
create mode 100644 lib/atomicio.c
diff --git a/include/linux/io.h b/include/linux/io.h
index f4f42fa..c331dcb 100644
--- a/include/linux/io.h
+++ b/include/linux/io.h
@@ -101,4 +101,9 @@ static inline void arch_phys_wc_del(int handle)
#define arch_phys_wc_add arch_phys_wc_add
#endif
+#ifndef __HAVE_ARCH_ATOMIC_IO_MODIFY
+/* Atomic MMIO-wide IO modify */
+extern void atomic_io_modify(void __iomem *reg, u32 mask, u32 set);
+#endif
+
#endif /* _LINUX_IO_H */
diff --git a/lib/Makefile b/lib/Makefile
index 7baccfd..695d6e2 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -13,7 +13,7 @@ lib-y := ctype.o string.o vsprintf.o cmdline.o \
sha1.o md5.o irq_regs.o reciprocal_div.o argv_split.o \
proportions.o flex_proportions.o prio_heap.o ratelimit.o show_mem.o \
is_single_threaded.o plist.o decompress.o kobject_uevent.o \
- earlycpio.o percpu-refcount.o
+ earlycpio.o percpu-refcount.o atomicio.o
obj-$(CONFIG_ARCH_HAS_DEBUG_STRICT_USER_COPY_CHECKS) += usercopy.o
lib-$(CONFIG_MMU) += ioremap.o
diff --git a/lib/atomicio.c b/lib/atomicio.c
new file mode 100644
index 0000000..1750f9d
--- /dev/null
+++ b/lib/atomicio.c
@@ -0,0 +1,27 @@
+#include <linux/io.h>
+#include <linux/spinlock.h>
+
+#ifndef __HAVE_ARCH_ATOMIC_IO_MODIFY
+/*
+ * Generic atomic MMIO modify.
+ *
+ * Allows thread-safe access to registers shared by unrelated subsystems.
+ * The access is protected by a single MMIO-wide lock.
+ *
+ * Optimized variants can be implemented on a per-architecture basis.
+ */
+static DEFINE_RAW_SPINLOCK(__io_lock);
+void atomic_io_modify(void __iomem *reg, u32 mask, u32 set)
+{
+ unsigned long flags;
+ u32 value;
+
+ raw_spin_lock_irqsave(&__io_lock, flags);
+ value = readl(reg) & ~mask;
+ value |= (set & mask);
+ writel(value, reg);
+ raw_spin_unlock_irqrestore(&__io_lock, flags);
+
+}
+EXPORT_SYMBOL(atomic_io_modify);
+#endif
--
1.8.1.5
Since the timer control register is shared with the clocksource driver,
use the recently introduced atomic_io_clear_set() to access such register.
Signed-off-by: Ezequiel Garcia <[email protected]>
---
drivers/watchdog/orion_wdt.c | 8 ++------
1 file changed, 2 insertions(+), 6 deletions(-)
diff --git a/drivers/watchdog/orion_wdt.c b/drivers/watchdog/orion_wdt.c
index 4ea5fcc..cfc037d 100644
--- a/drivers/watchdog/orion_wdt.c
+++ b/drivers/watchdog/orion_wdt.c
@@ -73,9 +73,7 @@ static int orion_wdt_start(struct watchdog_device *wdt_dev)
writel(~WDT_INT_REQ, BRIDGE_CAUSE);
/* Enable watchdog timer */
- reg = readl(wdt_reg + TIMER_CTRL);
- reg |= WDT_EN;
- writel(reg, wdt_reg + TIMER_CTRL);
+ atomic_io_modify(wdt_reg + TIMER_CTRL, WDT_EN, WDT_EN);
/* Enable reset on watchdog */
reg = readl(RSTOUTn_MASK);
@@ -98,9 +96,7 @@ static int orion_wdt_stop(struct watchdog_device *wdt_dev)
writel(reg, RSTOUTn_MASK);
/* Disable watchdog timer */
- reg = readl(wdt_reg + TIMER_CTRL);
- reg &= ~WDT_EN;
- writel(reg, wdt_reg + TIMER_CTRL);
+ atomic_io_modify(wdt_reg + TIMER_CTRL, WDT_EN, 0);
spin_unlock(&wdt_lock);
return 0;
--
1.8.1.5
Replace the driver-specific thread-safe shared register API
by the recently introduced atomic_io_modify().
Signed-off-by: Ezequiel Garcia <[email protected]>
---
drivers/clocksource/time-orion.c | 29 +++++++++++------------------
1 file changed, 11 insertions(+), 18 deletions(-)
diff --git a/drivers/clocksource/time-orion.c b/drivers/clocksource/time-orion.c
index ecbeb68..f69f697 100644
--- a/drivers/clocksource/time-orion.c
+++ b/drivers/clocksource/time-orion.c
@@ -18,6 +18,7 @@
#include <linux/interrupt.h>
#include <linux/of_address.h>
#include <linux/of_irq.h>
+#include <linux/io.h>
#include <linux/spinlock.h>
#include <asm/sched_clock.h>
@@ -35,20 +36,6 @@
#define ORION_ONESHOT_MAX 0xfffffffe
static void __iomem *timer_base;
-static DEFINE_SPINLOCK(timer_ctrl_lock);
-
-/*
- * Thread-safe access to TIMER_CTRL register
- * (shared with watchdog timer)
- */
-void orion_timer_ctrl_clrset(u32 clr, u32 set)
-{
- spin_lock(&timer_ctrl_lock);
- writel((readl(timer_base + TIMER_CTRL) & ~clr) | set,
- timer_base + TIMER_CTRL);
- spin_unlock(&timer_ctrl_lock);
-}
-EXPORT_SYMBOL(orion_timer_ctrl_clrset);
/*
* Free-running clocksource handling.
@@ -68,7 +55,8 @@ static int orion_clkevt_next_event(unsigned long delta,
{
/* setup and enable one-shot timer */
writel(delta, timer_base + TIMER1_VAL);
- orion_timer_ctrl_clrset(TIMER1_RELOAD_EN, TIMER1_EN);
+ atomic_io_modify(timer_base + TIMER_CTRL,
+ TIMER1_RELOAD_EN | TIMER1_EN, TIMER1_EN);
return 0;
}
@@ -80,10 +68,13 @@ static void orion_clkevt_mode(enum clock_event_mode mode,
/* setup and enable periodic timer at 1/HZ intervals */
writel(ticks_per_jiffy - 1, timer_base + TIMER1_RELOAD);
writel(ticks_per_jiffy - 1, timer_base + TIMER1_VAL);
- orion_timer_ctrl_clrset(0, TIMER1_RELOAD_EN | TIMER1_EN);
+ atomic_io_modify(timer_base + TIMER_CTRL,
+ TIMER1_RELOAD_EN | TIMER1_EN,
+ TIMER1_RELOAD_EN | TIMER1_EN);
} else {
/* disable timer */
- orion_timer_ctrl_clrset(TIMER1_RELOAD_EN | TIMER1_EN, 0);
+ atomic_io_modify(timer_base + TIMER_CTRL,
+ TIMER1_RELOAD_EN | TIMER1_EN, 0);
}
}
@@ -131,7 +122,9 @@ static void __init orion_timer_init(struct device_node *np)
/* setup timer0 as free-running clocksource */
writel(~0, timer_base + TIMER0_VAL);
writel(~0, timer_base + TIMER0_RELOAD);
- orion_timer_ctrl_clrset(0, TIMER0_RELOAD_EN | TIMER0_EN);
+ atomic_io_modify(timer_base + TIMER_CTRL,
+ TIMER0_RELOAD_EN | TIMER0_EN,
+ TIMER0_RELOAD_EN | TIMER0_EN);
clocksource_mmio_init(timer_base + TIMER0_VAL, "orion_clocksource",
clk_get_rate(clk), 300, 32,
clocksource_mmio_readl_down);
--
1.8.1.5
Implement arch-specific atomic_io_modify and atomic_io_modify_relaxed,
which are based on writel/readl_relaxed and writel_relaxed/readl_relaxed,
respectively.
In both cases, by relaxing the readl, perfomance can be improved.
Signed-off-by: Ezequiel Garcia <[email protected]>
---
arch/arm/include/asm/io.h | 4 ++++
arch/arm/kernel/io.c | 29 +++++++++++++++++++++++++++++
2 files changed, 33 insertions(+)
diff --git a/arch/arm/include/asm/io.h b/arch/arm/include/asm/io.h
index d070741..53637b6 100644
--- a/arch/arm/include/asm/io.h
+++ b/arch/arm/include/asm/io.h
@@ -397,5 +397,9 @@ extern int devmem_is_allowed(unsigned long pfn);
extern void register_isa_ports(unsigned int mmio, unsigned int io,
unsigned int io_shift);
+#define __HAVE_ARCH_ATOMIC_IO_MODIFY
+extern void atomic_io_modify(void __iomem *reg, u32 mask, u32 set);
+extern void atomic_io_modify_relaxed(void __iomem *reg, u32 mask, u32 set);
+
#endif /* __KERNEL__ */
#endif /* __ASM_ARM_IO_H */
diff --git a/arch/arm/kernel/io.c b/arch/arm/kernel/io.c
index dcd5b4d..a8c9c9b 100644
--- a/arch/arm/kernel/io.c
+++ b/arch/arm/kernel/io.c
@@ -1,6 +1,35 @@
#include <linux/export.h>
#include <linux/types.h>
#include <linux/io.h>
+#include <linux/spinlock.h>
+
+static DEFINE_RAW_SPINLOCK(__io_lock);
+
+void atomic_io_modify_relaxed(void __iomem *reg, u32 mask, u32 set)
+{
+ unsigned long flags;
+ u32 value;
+
+ raw_spin_lock_irqsave(&__io_lock, flags);
+ value = readl_relaxed(reg) & ~mask;
+ value |= (set & mask);
+ writel_relaxed(value, reg);
+ raw_spin_unlock_irqrestore(&__io_lock, flags);
+}
+EXPORT_SYMBOL(atomic_io_modify_relaxed);
+
+void atomic_io_modify(void __iomem *reg, u32 mask, u32 set)
+{
+ unsigned long flags;
+ u32 value;
+
+ raw_spin_lock_irqsave(&__io_lock, flags);
+ value = readl_relaxed(reg) & ~mask;
+ value |= (set & mask);
+ writel(value, reg);
+ raw_spin_unlock_irqrestore(&__io_lock, flags);
+}
+EXPORT_SYMBOL(atomic_io_modify);
/*
* Copy data from IO memory space to "real" memory space.
--
1.8.1.5
On Sat, Aug 24, 2013 at 5:35 PM, Ezequiel Garcia
<[email protected]> wrote:
> Some platforms have MMIO regions that are shared across orthogonal
> subsystems. This commit implements a possible solution for the
> thread-safe access of such regions through a spinlock-protected API.
>
> Concurrent access is protected with a single spinlock for the
> entire MMIO address space. While this protects shared-registers,
> it also serializes access to unrelated/unshared registers.
>
> Signed-off-by: Ezequiel Garcia <[email protected]>
> ---
> include/linux/io.h | 5 +++++
> lib/Makefile | 2 +-
> lib/atomicio.c | 27 +++++++++++++++++++++++++++
> 3 files changed, 33 insertions(+), 1 deletion(-)
> create mode 100644 lib/atomicio.c
>
> diff --git a/include/linux/io.h b/include/linux/io.h
> index f4f42fa..c331dcb 100644
> --- a/include/linux/io.h
> +++ b/include/linux/io.h
> @@ -101,4 +101,9 @@ static inline void arch_phys_wc_del(int handle)
> #define arch_phys_wc_add arch_phys_wc_add
> #endif
>
> +#ifndef __HAVE_ARCH_ATOMIC_IO_MODIFY
> +/* Atomic MMIO-wide IO modify */
> +extern void atomic_io_modify(void __iomem *reg, u32 mask, u32 set);
> +#endif
> +
> #endif /* _LINUX_IO_H */
> diff --git a/lib/Makefile b/lib/Makefile
> index 7baccfd..695d6e2 100644
> --- a/lib/Makefile
> +++ b/lib/Makefile
> @@ -13,7 +13,7 @@ lib-y := ctype.o string.o vsprintf.o cmdline.o \
> sha1.o md5.o irq_regs.o reciprocal_div.o argv_split.o \
> proportions.o flex_proportions.o prio_heap.o ratelimit.o show_mem.o \
> is_single_threaded.o plist.o decompress.o kobject_uevent.o \
> - earlycpio.o percpu-refcount.o
> + earlycpio.o percpu-refcount.o atomicio.o
>
> obj-$(CONFIG_ARCH_HAS_DEBUG_STRICT_USER_COPY_CHECKS) += usercopy.o
> lib-$(CONFIG_MMU) += ioremap.o
> diff --git a/lib/atomicio.c b/lib/atomicio.c
> new file mode 100644
> index 0000000..1750f9d
> --- /dev/null
> +++ b/lib/atomicio.c
> @@ -0,0 +1,27 @@
> +#include <linux/io.h>
> +#include <linux/spinlock.h>
> +
> +#ifndef __HAVE_ARCH_ATOMIC_IO_MODIFY
> +/*
> + * Generic atomic MMIO modify.
> + *
> + * Allows thread-safe access to registers shared by unrelated subsystems.
> + * The access is protected by a single MMIO-wide lock.
> + *
> + * Optimized variants can be implemented on a per-architecture basis.
> + */
> +static DEFINE_RAW_SPINLOCK(__io_lock);
> +void atomic_io_modify(void __iomem *reg, u32 mask, u32 set)
> +{
> + unsigned long flags;
> + u32 value;
> +
> + raw_spin_lock_irqsave(&__io_lock, flags);
> + value = readl(reg) & ~mask;
> + value |= (set & mask);
> + writel(value, reg);
> + raw_spin_unlock_irqrestore(&__io_lock, flags);
> +
> +}
> +EXPORT_SYMBOL(atomic_io_modify);
Why not the default case EXPORT_SYMBOL_GPL()?
--
Thanks,
//richard
On Sat, Aug 24, 2013 at 08:27:10PM +0200, richard -rw- weinberger wrote:
> On Sat, Aug 24, 2013 at 5:35 PM, Ezequiel Garcia
> <[email protected]> wrote:
> > Some platforms have MMIO regions that are shared across orthogonal
> > subsystems. This commit implements a possible solution for the
> > thread-safe access of such regions through a spinlock-protected API.
> >
> > Concurrent access is protected with a single spinlock for the
> > entire MMIO address space. While this protects shared-registers,
> > it also serializes access to unrelated/unshared registers.
> >
> > Signed-off-by: Ezequiel Garcia <[email protected]>
> > ---
> > include/linux/io.h | 5 +++++
> > lib/Makefile | 2 +-
> > lib/atomicio.c | 27 +++++++++++++++++++++++++++
> > 3 files changed, 33 insertions(+), 1 deletion(-)
> > create mode 100644 lib/atomicio.c
> >
> > diff --git a/include/linux/io.h b/include/linux/io.h
> > index f4f42fa..c331dcb 100644
> > --- a/include/linux/io.h
> > +++ b/include/linux/io.h
> > @@ -101,4 +101,9 @@ static inline void arch_phys_wc_del(int handle)
> > #define arch_phys_wc_add arch_phys_wc_add
> > #endif
> >
> > +#ifndef __HAVE_ARCH_ATOMIC_IO_MODIFY
> > +/* Atomic MMIO-wide IO modify */
> > +extern void atomic_io_modify(void __iomem *reg, u32 mask, u32 set);
> > +#endif
> > +
> > #endif /* _LINUX_IO_H */
> > diff --git a/lib/Makefile b/lib/Makefile
> > index 7baccfd..695d6e2 100644
> > --- a/lib/Makefile
> > +++ b/lib/Makefile
> > @@ -13,7 +13,7 @@ lib-y := ctype.o string.o vsprintf.o cmdline.o \
> > sha1.o md5.o irq_regs.o reciprocal_div.o argv_split.o \
> > proportions.o flex_proportions.o prio_heap.o ratelimit.o show_mem.o \
> > is_single_threaded.o plist.o decompress.o kobject_uevent.o \
> > - earlycpio.o percpu-refcount.o
> > + earlycpio.o percpu-refcount.o atomicio.o
> >
> > obj-$(CONFIG_ARCH_HAS_DEBUG_STRICT_USER_COPY_CHECKS) += usercopy.o
> > lib-$(CONFIG_MMU) += ioremap.o
> > diff --git a/lib/atomicio.c b/lib/atomicio.c
> > new file mode 100644
> > index 0000000..1750f9d
> > --- /dev/null
> > +++ b/lib/atomicio.c
> > @@ -0,0 +1,27 @@
> > +#include <linux/io.h>
> > +#include <linux/spinlock.h>
> > +
> > +#ifndef __HAVE_ARCH_ATOMIC_IO_MODIFY
> > +/*
> > + * Generic atomic MMIO modify.
> > + *
> > + * Allows thread-safe access to registers shared by unrelated subsystems.
> > + * The access is protected by a single MMIO-wide lock.
> > + *
> > + * Optimized variants can be implemented on a per-architecture basis.
> > + */
> > +static DEFINE_RAW_SPINLOCK(__io_lock);
> > +void atomic_io_modify(void __iomem *reg, u32 mask, u32 set)
> > +{
> > + unsigned long flags;
> > + u32 value;
> > +
> > + raw_spin_lock_irqsave(&__io_lock, flags);
> > + value = readl(reg) & ~mask;
> > + value |= (set & mask);
> > + writel(value, reg);
> > + raw_spin_unlock_irqrestore(&__io_lock, flags);
> > +
> > +}
> > +EXPORT_SYMBOL(atomic_io_modify);
>
> Why not the default case EXPORT_SYMBOL_GPL()?
>
Because I copy-pasted the export from some other lib/.. :-)
Mind explaining me the difference, why you say _GPL it's the default,
and why EXPORT_SYMBOL is more frequently used in lib/ ?
$ git grep EXPORT_SYMBOL lib/ | wc -l
598
$ git grep EXPORT_SYMBOL_GPL lib/ | wc -l
117
Thanks!
--
Ezequiel García, Free Electrons
Embedded Linux, Kernel and Android Engineering
http://free-electrons.com
Am 24.08.2013 21:58, schrieb Ezequiel Garcia:
> On Sat, Aug 24, 2013 at 08:27:10PM +0200, richard -rw- weinberger wrote:
>> On Sat, Aug 24, 2013 at 5:35 PM, Ezequiel Garcia
>> <[email protected]> wrote:
>>> Some platforms have MMIO regions that are shared across orthogonal
>>> subsystems. This commit implements a possible solution for the
>>> thread-safe access of such regions through a spinlock-protected API.
>>>
>>> Concurrent access is protected with a single spinlock for the
>>> entire MMIO address space. While this protects shared-registers,
>>> it also serializes access to unrelated/unshared registers.
>>>
>>> Signed-off-by: Ezequiel Garcia <[email protected]>
>>> ---
>>> include/linux/io.h | 5 +++++
>>> lib/Makefile | 2 +-
>>> lib/atomicio.c | 27 +++++++++++++++++++++++++++
>>> 3 files changed, 33 insertions(+), 1 deletion(-)
>>> create mode 100644 lib/atomicio.c
>>>
>>> diff --git a/include/linux/io.h b/include/linux/io.h
>>> index f4f42fa..c331dcb 100644
>>> --- a/include/linux/io.h
>>> +++ b/include/linux/io.h
>>> @@ -101,4 +101,9 @@ static inline void arch_phys_wc_del(int handle)
>>> #define arch_phys_wc_add arch_phys_wc_add
>>> #endif
>>>
>>> +#ifndef __HAVE_ARCH_ATOMIC_IO_MODIFY
>>> +/* Atomic MMIO-wide IO modify */
>>> +extern void atomic_io_modify(void __iomem *reg, u32 mask, u32 set);
>>> +#endif
>>> +
>>> #endif /* _LINUX_IO_H */
>>> diff --git a/lib/Makefile b/lib/Makefile
>>> index 7baccfd..695d6e2 100644
>>> --- a/lib/Makefile
>>> +++ b/lib/Makefile
>>> @@ -13,7 +13,7 @@ lib-y := ctype.o string.o vsprintf.o cmdline.o \
>>> sha1.o md5.o irq_regs.o reciprocal_div.o argv_split.o \
>>> proportions.o flex_proportions.o prio_heap.o ratelimit.o show_mem.o \
>>> is_single_threaded.o plist.o decompress.o kobject_uevent.o \
>>> - earlycpio.o percpu-refcount.o
>>> + earlycpio.o percpu-refcount.o atomicio.o
>>>
>>> obj-$(CONFIG_ARCH_HAS_DEBUG_STRICT_USER_COPY_CHECKS) += usercopy.o
>>> lib-$(CONFIG_MMU) += ioremap.o
>>> diff --git a/lib/atomicio.c b/lib/atomicio.c
>>> new file mode 100644
>>> index 0000000..1750f9d
>>> --- /dev/null
>>> +++ b/lib/atomicio.c
>>> @@ -0,0 +1,27 @@
>>> +#include <linux/io.h>
>>> +#include <linux/spinlock.h>
>>> +
>>> +#ifndef __HAVE_ARCH_ATOMIC_IO_MODIFY
>>> +/*
>>> + * Generic atomic MMIO modify.
>>> + *
>>> + * Allows thread-safe access to registers shared by unrelated subsystems.
>>> + * The access is protected by a single MMIO-wide lock.
>>> + *
>>> + * Optimized variants can be implemented on a per-architecture basis.
>>> + */
>>> +static DEFINE_RAW_SPINLOCK(__io_lock);
>>> +void atomic_io_modify(void __iomem *reg, u32 mask, u32 set)
>>> +{
>>> + unsigned long flags;
>>> + u32 value;
>>> +
>>> + raw_spin_lock_irqsave(&__io_lock, flags);
>>> + value = readl(reg) & ~mask;
>>> + value |= (set & mask);
>>> + writel(value, reg);
>>> + raw_spin_unlock_irqrestore(&__io_lock, flags);
>>> +
>>> +}
>>> +EXPORT_SYMBOL(atomic_io_modify);
>>
>> Why not the default case EXPORT_SYMBOL_GPL()?
>>
>
> Because I copy-pasted the export from some other lib/.. :-)
>
> Mind explaining me the difference, why you say _GPL it's the default,
> and why EXPORT_SYMBOL is more frequently used in lib/ ?
As the kernel is GPL it is the default case to mark new things as GPL symbols.
If your new feature is a core feature which is used by mostly everyone, EXPORT_SYMBOL()
is appropriate.
I.e. having kmalloc() and friends EXPORT_SYMBOL_GPL() would be a bad idea. :)
EXPORT_SYMBOL() seems to be often used in lib/ because most lib/ things are core features.
Thanks,
//richard
On Sat, Aug 24, 2013 at 10:35:34PM +0200, Richard Weinberger wrote:
> Am 24.08.2013 21:58, schrieb Ezequiel Garcia:
> > On Sat, Aug 24, 2013 at 08:27:10PM +0200, richard -rw- weinberger wrote:
> >> On Sat, Aug 24, 2013 at 5:35 PM, Ezequiel Garcia
> >> <[email protected]> wrote:
> >>> Some platforms have MMIO regions that are shared across orthogonal
> >>> subsystems. This commit implements a possible solution for the
> >>> thread-safe access of such regions through a spinlock-protected API.
> >>>
> >>> Concurrent access is protected with a single spinlock for the
> >>> entire MMIO address space. While this protects shared-registers,
> >>> it also serializes access to unrelated/unshared registers.
> >>>
> >>> Signed-off-by: Ezequiel Garcia <[email protected]>
> >>> ---
> >>> include/linux/io.h | 5 +++++
> >>> lib/Makefile | 2 +-
> >>> lib/atomicio.c | 27 +++++++++++++++++++++++++++
> >>> 3 files changed, 33 insertions(+), 1 deletion(-)
> >>> create mode 100644 lib/atomicio.c
> >>>
> >>> diff --git a/include/linux/io.h b/include/linux/io.h
> >>> index f4f42fa..c331dcb 100644
> >>> --- a/include/linux/io.h
> >>> +++ b/include/linux/io.h
> >>> @@ -101,4 +101,9 @@ static inline void arch_phys_wc_del(int handle)
> >>> #define arch_phys_wc_add arch_phys_wc_add
> >>> #endif
> >>>
> >>> +#ifndef __HAVE_ARCH_ATOMIC_IO_MODIFY
> >>> +/* Atomic MMIO-wide IO modify */
> >>> +extern void atomic_io_modify(void __iomem *reg, u32 mask, u32 set);
> >>> +#endif
> >>> +
> >>> #endif /* _LINUX_IO_H */
> >>> diff --git a/lib/Makefile b/lib/Makefile
> >>> index 7baccfd..695d6e2 100644
> >>> --- a/lib/Makefile
> >>> +++ b/lib/Makefile
> >>> @@ -13,7 +13,7 @@ lib-y := ctype.o string.o vsprintf.o cmdline.o \
> >>> sha1.o md5.o irq_regs.o reciprocal_div.o argv_split.o \
> >>> proportions.o flex_proportions.o prio_heap.o ratelimit.o show_mem.o \
> >>> is_single_threaded.o plist.o decompress.o kobject_uevent.o \
> >>> - earlycpio.o percpu-refcount.o
> >>> + earlycpio.o percpu-refcount.o atomicio.o
> >>>
> >>> obj-$(CONFIG_ARCH_HAS_DEBUG_STRICT_USER_COPY_CHECKS) += usercopy.o
> >>> lib-$(CONFIG_MMU) += ioremap.o
> >>> diff --git a/lib/atomicio.c b/lib/atomicio.c
> >>> new file mode 100644
> >>> index 0000000..1750f9d
> >>> --- /dev/null
> >>> +++ b/lib/atomicio.c
> >>> @@ -0,0 +1,27 @@
> >>> +#include <linux/io.h>
> >>> +#include <linux/spinlock.h>
> >>> +
> >>> +#ifndef __HAVE_ARCH_ATOMIC_IO_MODIFY
> >>> +/*
> >>> + * Generic atomic MMIO modify.
> >>> + *
> >>> + * Allows thread-safe access to registers shared by unrelated subsystems.
> >>> + * The access is protected by a single MMIO-wide lock.
> >>> + *
> >>> + * Optimized variants can be implemented on a per-architecture basis.
> >>> + */
> >>> +static DEFINE_RAW_SPINLOCK(__io_lock);
> >>> +void atomic_io_modify(void __iomem *reg, u32 mask, u32 set)
> >>> +{
> >>> + unsigned long flags;
> >>> + u32 value;
> >>> +
> >>> + raw_spin_lock_irqsave(&__io_lock, flags);
> >>> + value = readl(reg) & ~mask;
> >>> + value |= (set & mask);
> >>> + writel(value, reg);
> >>> + raw_spin_unlock_irqrestore(&__io_lock, flags);
> >>> +
> >>> +}
> >>> +EXPORT_SYMBOL(atomic_io_modify);
> >>
> >> Why not the default case EXPORT_SYMBOL_GPL()?
> >>
> >
> > Because I copy-pasted the export from some other lib/.. :-)
> >
> > Mind explaining me the difference, why you say _GPL it's the default,
> > and why EXPORT_SYMBOL is more frequently used in lib/ ?
>
> As the kernel is GPL it is the default case to mark new things as GPL symbols.
> If your new feature is a core feature which is used by mostly everyone, EXPORT_SYMBOL()
> is appropriate.
> I.e. having kmalloc() and friends EXPORT_SYMBOL_GPL() would be a bad idea. :)
> EXPORT_SYMBOL() seems to be often used in lib/ because most lib/ things are core features.
>
In that case, EXPORT_SYMBOL is certainly the most appropriate one,
for the reasons you stated above.
--
Ezequiel García, Free Electrons
Embedded Linux, Kernel and Android Engineering
http://free-electrons.com
Am 24.08.2013 22:49, schrieb Ezequiel Garcia:
> On Sat, Aug 24, 2013 at 10:35:34PM +0200, Richard Weinberger wrote:
>> Am 24.08.2013 21:58, schrieb Ezequiel Garcia:
>>> On Sat, Aug 24, 2013 at 08:27:10PM +0200, richard -rw- weinberger wrote:
>>>> On Sat, Aug 24, 2013 at 5:35 PM, Ezequiel Garcia
>>>> <[email protected]> wrote:
>>>>> Some platforms have MMIO regions that are shared across orthogonal
>>>>> subsystems. This commit implements a possible solution for the
>>>>> thread-safe access of such regions through a spinlock-protected API.
>>>>>
>>>>> Concurrent access is protected with a single spinlock for the
>>>>> entire MMIO address space. While this protects shared-registers,
>>>>> it also serializes access to unrelated/unshared registers.
>>>>>
>>>>> Signed-off-by: Ezequiel Garcia <[email protected]>
>>>>> ---
>>>>> include/linux/io.h | 5 +++++
>>>>> lib/Makefile | 2 +-
>>>>> lib/atomicio.c | 27 +++++++++++++++++++++++++++
>>>>> 3 files changed, 33 insertions(+), 1 deletion(-)
>>>>> create mode 100644 lib/atomicio.c
>>>>>
>>>>> diff --git a/include/linux/io.h b/include/linux/io.h
>>>>> index f4f42fa..c331dcb 100644
>>>>> --- a/include/linux/io.h
>>>>> +++ b/include/linux/io.h
>>>>> @@ -101,4 +101,9 @@ static inline void arch_phys_wc_del(int handle)
>>>>> #define arch_phys_wc_add arch_phys_wc_add
>>>>> #endif
>>>>>
>>>>> +#ifndef __HAVE_ARCH_ATOMIC_IO_MODIFY
>>>>> +/* Atomic MMIO-wide IO modify */
>>>>> +extern void atomic_io_modify(void __iomem *reg, u32 mask, u32 set);
>>>>> +#endif
>>>>> +
>>>>> #endif /* _LINUX_IO_H */
>>>>> diff --git a/lib/Makefile b/lib/Makefile
>>>>> index 7baccfd..695d6e2 100644
>>>>> --- a/lib/Makefile
>>>>> +++ b/lib/Makefile
>>>>> @@ -13,7 +13,7 @@ lib-y := ctype.o string.o vsprintf.o cmdline.o \
>>>>> sha1.o md5.o irq_regs.o reciprocal_div.o argv_split.o \
>>>>> proportions.o flex_proportions.o prio_heap.o ratelimit.o show_mem.o \
>>>>> is_single_threaded.o plist.o decompress.o kobject_uevent.o \
>>>>> - earlycpio.o percpu-refcount.o
>>>>> + earlycpio.o percpu-refcount.o atomicio.o
>>>>>
>>>>> obj-$(CONFIG_ARCH_HAS_DEBUG_STRICT_USER_COPY_CHECKS) += usercopy.o
>>>>> lib-$(CONFIG_MMU) += ioremap.o
>>>>> diff --git a/lib/atomicio.c b/lib/atomicio.c
>>>>> new file mode 100644
>>>>> index 0000000..1750f9d
>>>>> --- /dev/null
>>>>> +++ b/lib/atomicio.c
>>>>> @@ -0,0 +1,27 @@
>>>>> +#include <linux/io.h>
>>>>> +#include <linux/spinlock.h>
>>>>> +
>>>>> +#ifndef __HAVE_ARCH_ATOMIC_IO_MODIFY
>>>>> +/*
>>>>> + * Generic atomic MMIO modify.
>>>>> + *
>>>>> + * Allows thread-safe access to registers shared by unrelated subsystems.
>>>>> + * The access is protected by a single MMIO-wide lock.
>>>>> + *
>>>>> + * Optimized variants can be implemented on a per-architecture basis.
>>>>> + */
>>>>> +static DEFINE_RAW_SPINLOCK(__io_lock);
>>>>> +void atomic_io_modify(void __iomem *reg, u32 mask, u32 set)
>>>>> +{
>>>>> + unsigned long flags;
>>>>> + u32 value;
>>>>> +
>>>>> + raw_spin_lock_irqsave(&__io_lock, flags);
>>>>> + value = readl(reg) & ~mask;
>>>>> + value |= (set & mask);
>>>>> + writel(value, reg);
>>>>> + raw_spin_unlock_irqrestore(&__io_lock, flags);
>>>>> +
>>>>> +}
>>>>> +EXPORT_SYMBOL(atomic_io_modify);
>>>>
>>>> Why not the default case EXPORT_SYMBOL_GPL()?
>>>>
>>>
>>> Because I copy-pasted the export from some other lib/.. :-)
>>>
>>> Mind explaining me the difference, why you say _GPL it's the default,
>>> and why EXPORT_SYMBOL is more frequently used in lib/ ?
>>
>> As the kernel is GPL it is the default case to mark new things as GPL symbols.
>> If your new feature is a core feature which is used by mostly everyone, EXPORT_SYMBOL()
>> is appropriate.
>> I.e. having kmalloc() and friends EXPORT_SYMBOL_GPL() would be a bad idea. :)
>> EXPORT_SYMBOL() seems to be often used in lib/ because most lib/ things are core features.
>>
>
> In that case, EXPORT_SYMBOL is certainly the most appropriate one,
> for the reasons you stated above.
Okay, did not realize that your feature is that fundamental. :-)
Thanks,
//richard
On Sat, Aug 24, 2013 at 04:58:59PM -0300, Ezequiel Garcia wrote:
> On Sat, Aug 24, 2013 at 08:27:10PM +0200, richard -rw- weinberger wrote:
> > On Sat, Aug 24, 2013 at 5:35 PM, Ezequiel Garcia
> > <[email protected]> wrote:
> > > Some platforms have MMIO regions that are shared across orthogonal
> > > subsystems. This commit implements a possible solution for the
> > > thread-safe access of such regions through a spinlock-protected API.
> > >
> > > Concurrent access is protected with a single spinlock for the
> > > entire MMIO address space. While this protects shared-registers,
> > > it also serializes access to unrelated/unshared registers.
> > >
> > > Signed-off-by: Ezequiel Garcia <[email protected]>
> > > ---
> > > include/linux/io.h | 5 +++++
> > > lib/Makefile | 2 +-
> > > lib/atomicio.c | 27 +++++++++++++++++++++++++++
> > > 3 files changed, 33 insertions(+), 1 deletion(-)
> > > create mode 100644 lib/atomicio.c
> > >
> > > diff --git a/include/linux/io.h b/include/linux/io.h
> > > index f4f42fa..c331dcb 100644
> > > --- a/include/linux/io.h
> > > +++ b/include/linux/io.h
> > > @@ -101,4 +101,9 @@ static inline void arch_phys_wc_del(int handle)
> > > #define arch_phys_wc_add arch_phys_wc_add
> > > #endif
> > >
> > > +#ifndef __HAVE_ARCH_ATOMIC_IO_MODIFY
> > > +/* Atomic MMIO-wide IO modify */
> > > +extern void atomic_io_modify(void __iomem *reg, u32 mask, u32 set);
> > > +#endif
> > > +
> > > #endif /* _LINUX_IO_H */
> > > diff --git a/lib/Makefile b/lib/Makefile
> > > index 7baccfd..695d6e2 100644
> > > --- a/lib/Makefile
> > > +++ b/lib/Makefile
> > > @@ -13,7 +13,7 @@ lib-y := ctype.o string.o vsprintf.o cmdline.o \
> > > sha1.o md5.o irq_regs.o reciprocal_div.o argv_split.o \
> > > proportions.o flex_proportions.o prio_heap.o ratelimit.o show_mem.o \
> > > is_single_threaded.o plist.o decompress.o kobject_uevent.o \
> > > - earlycpio.o percpu-refcount.o
> > > + earlycpio.o percpu-refcount.o atomicio.o
> > >
> > > obj-$(CONFIG_ARCH_HAS_DEBUG_STRICT_USER_COPY_CHECKS) += usercopy.o
> > > lib-$(CONFIG_MMU) += ioremap.o
> > > diff --git a/lib/atomicio.c b/lib/atomicio.c
> > > new file mode 100644
> > > index 0000000..1750f9d
> > > --- /dev/null
> > > +++ b/lib/atomicio.c
> > > @@ -0,0 +1,27 @@
> > > +#include <linux/io.h>
> > > +#include <linux/spinlock.h>
> > > +
> > > +#ifndef __HAVE_ARCH_ATOMIC_IO_MODIFY
> > > +/*
> > > + * Generic atomic MMIO modify.
> > > + *
> > > + * Allows thread-safe access to registers shared by unrelated subsystems.
> > > + * The access is protected by a single MMIO-wide lock.
> > > + *
> > > + * Optimized variants can be implemented on a per-architecture basis.
> > > + */
> > > +static DEFINE_RAW_SPINLOCK(__io_lock);
> > > +void atomic_io_modify(void __iomem *reg, u32 mask, u32 set)
> > > +{
> > > + unsigned long flags;
> > > + u32 value;
> > > +
> > > + raw_spin_lock_irqsave(&__io_lock, flags);
> > > + value = readl(reg) & ~mask;
> > > + value |= (set & mask);
> > > + writel(value, reg);
> > > + raw_spin_unlock_irqrestore(&__io_lock, flags);
> > > +
> > > +}
> > > +EXPORT_SYMBOL(atomic_io_modify);
> >
> > Why not the default case EXPORT_SYMBOL_GPL()?
> >
>
> Because I copy-pasted the export from some other lib/.. :-)
>
> Mind explaining me the difference, why you say _GPL it's the default,
> and why EXPORT_SYMBOL is more frequently used in lib/ ?
This is actually a decision solely for the author of the code which is
being created: it's a statement about the _use_ of the symbol.
Do you wish to permit your code to be used by modules with non-GPL
compatible licenses - such as closed source modules? If so, then use
EXPORT_SYMBOL().
If you wish your code to only be used by GPL compatible modules, then
use EXPORT_SYMBOL_GPL().
Things get a little murkey if you call code which has been exported with
EXPORT_SYMBOL_GPL() and you wish to mark your new function with
EXPORT_SYMBOL(), because you're effectively bypassing acess restrictions
to taht code put in place by the original code author (you're not in this
case, but I'm including this statement for completeness.)
(Adding Andrew Morton in Cc)
On Sat, Aug 24, 2013 at 12:35:29PM -0300, Ezequiel Garcia wrote:
> Some platforms have MMIO regions that are shared across orthogonal
> subsystems. This commit implements a possible solution for the
> thread-safe access of such regions through a spinlock-protected API.
>
> Concurrent access is protected with a single spinlock for the
> entire MMIO address space. While this protects shared-registers,
> it also serializes access to unrelated/unshared registers.
>
> Signed-off-by: Ezequiel Garcia <[email protected]>
> ---
> include/linux/io.h | 5 +++++
> lib/Makefile | 2 +-
> lib/atomicio.c | 27 +++++++++++++++++++++++++++
> 3 files changed, 33 insertions(+), 1 deletion(-)
> create mode 100644 lib/atomicio.c
>
> diff --git a/include/linux/io.h b/include/linux/io.h
> index f4f42fa..c331dcb 100644
> --- a/include/linux/io.h
> +++ b/include/linux/io.h
> @@ -101,4 +101,9 @@ static inline void arch_phys_wc_del(int handle)
> #define arch_phys_wc_add arch_phys_wc_add
> #endif
>
> +#ifndef __HAVE_ARCH_ATOMIC_IO_MODIFY
> +/* Atomic MMIO-wide IO modify */
> +extern void atomic_io_modify(void __iomem *reg, u32 mask, u32 set);
> +#endif
> +
> #endif /* _LINUX_IO_H */
> diff --git a/lib/Makefile b/lib/Makefile
> index 7baccfd..695d6e2 100644
> --- a/lib/Makefile
> +++ b/lib/Makefile
> @@ -13,7 +13,7 @@ lib-y := ctype.o string.o vsprintf.o cmdline.o \
> sha1.o md5.o irq_regs.o reciprocal_div.o argv_split.o \
> proportions.o flex_proportions.o prio_heap.o ratelimit.o show_mem.o \
> is_single_threaded.o plist.o decompress.o kobject_uevent.o \
> - earlycpio.o percpu-refcount.o
> + earlycpio.o percpu-refcount.o atomicio.o
>
> obj-$(CONFIG_ARCH_HAS_DEBUG_STRICT_USER_COPY_CHECKS) += usercopy.o
> lib-$(CONFIG_MMU) += ioremap.o
> diff --git a/lib/atomicio.c b/lib/atomicio.c
> new file mode 100644
> index 0000000..1750f9d
> --- /dev/null
> +++ b/lib/atomicio.c
> @@ -0,0 +1,27 @@
> +#include <linux/io.h>
> +#include <linux/spinlock.h>
> +
> +#ifndef __HAVE_ARCH_ATOMIC_IO_MODIFY
> +/*
> + * Generic atomic MMIO modify.
> + *
> + * Allows thread-safe access to registers shared by unrelated subsystems.
> + * The access is protected by a single MMIO-wide lock.
> + *
> + * Optimized variants can be implemented on a per-architecture basis.
> + */
> +static DEFINE_RAW_SPINLOCK(__io_lock);
> +void atomic_io_modify(void __iomem *reg, u32 mask, u32 set)
> +{
> + unsigned long flags;
> + u32 value;
> +
> + raw_spin_lock_irqsave(&__io_lock, flags);
> + value = readl(reg) & ~mask;
> + value |= (set & mask);
> + writel(value, reg);
> + raw_spin_unlock_irqrestore(&__io_lock, flags);
> +
> +}
> +EXPORT_SYMBOL(atomic_io_modify);
> +#endif
> --
> 1.8.1.5
>
So, assuming this is ready to be merged (and knowing right now it's probably
a very bad time in the release cylce to ask), who is supposed to pick
this patch?
Thanks!
--
Ezequiel García, Free Electrons
Embedded Linux, Kernel and Android Engineering
http://free-electrons.com
On Sat, 24 Aug 2013 12:35:29 -0300 Ezequiel Garcia <[email protected]> wrote:
> Some platforms have MMIO regions that are shared across orthogonal
> subsystems. This commit implements a possible solution for the
> thread-safe access of such regions through a spinlock-protected API.
Seem sensible. Perhaps.
It only works if both subsystems agree to use atomic_io_modify(). And
if they're both capable of doing that, they are both capable of
implementing an agreed-upon internal locking scheme, so why bother?
And the advantage of having the two subsystems agree on a locking
scheme is that they then will not share a lock with all other
subsystems which use atomic IO. That lock you use in lib/atomicio.c
could become heavily contended. Although such contention problems
could be easily fixed up by using an array of locks within
atomic_io_modify(), keyed by a hash of the io address.
> Concurrent access is protected with a single spinlock for the
> entire MMIO address space. While this protects shared-registers,
> it also serializes access to unrelated/unshared registers.
I'd suggest cc'ing Linus on this material - he might have opinons.
> --- a/include/linux/io.h
> +++ b/include/linux/io.h
> @@ -101,4 +101,9 @@ static inline void arch_phys_wc_del(int handle)
> #define arch_phys_wc_add arch_phys_wc_add
> #endif
>
> +#ifndef __HAVE_ARCH_ATOMIC_IO_MODIFY
> +/* Atomic MMIO-wide IO modify */
> +extern void atomic_io_modify(void __iomem *reg, u32 mask, u32 set);
> +#endif
I disagree with the presence of the ifndef. If
__HAVE_ARCH_ATOMIC_IO_MODIFY is undefined, the architecture must still
implement the identical function signature. The best way to ensure that
is to use the same prototype in both cases.
> #endif /* _LINUX_IO_H */
> diff --git a/lib/Makefile b/lib/Makefile
> index 7baccfd..695d6e2 100644
> --- a/lib/Makefile
> +++ b/lib/Makefile
> @@ -13,7 +13,7 @@ lib-y := ctype.o string.o vsprintf.o cmdline.o \
> sha1.o md5.o irq_regs.o reciprocal_div.o argv_split.o \
> proportions.o flex_proportions.o prio_heap.o ratelimit.o show_mem.o \
> is_single_threaded.o plist.o decompress.o kobject_uevent.o \
> - earlycpio.o percpu-refcount.o
> + earlycpio.o percpu-refcount.o atomicio.o
>
> obj-$(CONFIG_ARCH_HAS_DEBUG_STRICT_USER_COPY_CHECKS) += usercopy.o
> lib-$(CONFIG_MMU) += ioremap.o
> diff --git a/lib/atomicio.c b/lib/atomicio.c
> new file mode 100644
> index 0000000..1750f9d
> --- /dev/null
> +++ b/lib/atomicio.c
> @@ -0,0 +1,27 @@
> +#include <linux/io.h>
> +#include <linux/spinlock.h>
> +
> +#ifndef __HAVE_ARCH_ATOMIC_IO_MODIFY
> +/*
> + * Generic atomic MMIO modify.
> + *
> + * Allows thread-safe access to registers shared by unrelated subsystems.
> + * The access is protected by a single MMIO-wide lock.
> + *
> + * Optimized variants can be implemented on a per-architecture basis.
> + */
Some kerneldoc would be nice.
> +static DEFINE_RAW_SPINLOCK(__io_lock);
This could be local to atomic_io_modify().
> +void atomic_io_modify(void __iomem *reg, u32 mask, u32 set)
> +{
> + unsigned long flags;
> + u32 value;
> +
> + raw_spin_lock_irqsave(&__io_lock, flags);
> + value = readl(reg) & ~mask;
> + value |= (set & mask);
Could just be "value |= set". The code as you have it permits callers
to set bits in "set" which aren't permitted in "mask", which would be
pretty darn stupid of them.
> + writel(value, reg);
> + raw_spin_unlock_irqrestore(&__io_lock, flags);
> +
> +}
> +EXPORT_SYMBOL(atomic_io_modify);
> +#endif
What about 8, 16 and 64-bit IO addresses?
On Sat, Aug 24, 2013 at 04:35:30PM +0100, Ezequiel Garcia wrote:
> Implement arch-specific atomic_io_modify and atomic_io_modify_relaxed,
> which are based on writel/readl_relaxed and writel_relaxed/readl_relaxed,
> respectively.
> In both cases, by relaxing the readl, perfomance can be improved.
>
> Signed-off-by: Ezequiel Garcia <[email protected]>
> ---
> arch/arm/include/asm/io.h | 4 ++++
> arch/arm/kernel/io.c | 29 +++++++++++++++++++++++++++++
> 2 files changed, 33 insertions(+)
>
> diff --git a/arch/arm/include/asm/io.h b/arch/arm/include/asm/io.h
> index d070741..53637b6 100644
> --- a/arch/arm/include/asm/io.h
> +++ b/arch/arm/include/asm/io.h
> @@ -397,5 +397,9 @@ extern int devmem_is_allowed(unsigned long pfn);
> extern void register_isa_ports(unsigned int mmio, unsigned int io,
> unsigned int io_shift);
>
> +#define __HAVE_ARCH_ATOMIC_IO_MODIFY
> +extern void atomic_io_modify(void __iomem *reg, u32 mask, u32 set);
> +extern void atomic_io_modify_relaxed(void __iomem *reg, u32 mask, u32 set);
> +
> #endif /* __KERNEL__ */
> #endif /* __ASM_ARM_IO_H */
> diff --git a/arch/arm/kernel/io.c b/arch/arm/kernel/io.c
> index dcd5b4d..a8c9c9b 100644
> --- a/arch/arm/kernel/io.c
> +++ b/arch/arm/kernel/io.c
> @@ -1,6 +1,35 @@
> #include <linux/export.h>
> #include <linux/types.h>
> #include <linux/io.h>
> +#include <linux/spinlock.h>
> +
> +static DEFINE_RAW_SPINLOCK(__io_lock);
> +
> +void atomic_io_modify_relaxed(void __iomem *reg, u32 mask, u32 set)
> +{
> + unsigned long flags;
> + u32 value;
> +
> + raw_spin_lock_irqsave(&__io_lock, flags);
> + value = readl_relaxed(reg) & ~mask;
> + value |= (set & mask);
> + writel_relaxed(value, reg);
> + raw_spin_unlock_irqrestore(&__io_lock, flags);
> +}
> +EXPORT_SYMBOL(atomic_io_modify_relaxed);
> +
> +void atomic_io_modify(void __iomem *reg, u32 mask, u32 set)
> +{
> + unsigned long flags;
> + u32 value;
> +
> + raw_spin_lock_irqsave(&__io_lock, flags);
> + value = readl_relaxed(reg) & ~mask;
> + value |= (set & mask);
> + writel(value, reg);
> + raw_spin_unlock_irqrestore(&__io_lock, flags);
> +}
> +EXPORT_SYMBOL(atomic_io_modify);
Is this any different from the generic one introduced in patch 1/4? I
would rather just use the generic definition. Similarly, a generic
atomic_io_modify_relaxed() but guarded with something like
__HAVE_ARCH_RELAXED_IO.
--
Catalin
On Wed, Aug 28, 2013 at 09:53:40AM +0100, Catalin Marinas wrote:
> On Sat, Aug 24, 2013 at 04:35:30PM +0100, Ezequiel Garcia wrote:
> > Implement arch-specific atomic_io_modify and atomic_io_modify_relaxed,
> > which are based on writel/readl_relaxed and writel_relaxed/readl_relaxed,
> > respectively.
> > In both cases, by relaxing the readl, perfomance can be improved.
> >
> > Signed-off-by: Ezequiel Garcia <[email protected]>
> > ---
> > arch/arm/include/asm/io.h | 4 ++++
> > arch/arm/kernel/io.c | 29 +++++++++++++++++++++++++++++
> > 2 files changed, 33 insertions(+)
> >
> > diff --git a/arch/arm/include/asm/io.h b/arch/arm/include/asm/io.h
> > index d070741..53637b6 100644
> > --- a/arch/arm/include/asm/io.h
> > +++ b/arch/arm/include/asm/io.h
> > @@ -397,5 +397,9 @@ extern int devmem_is_allowed(unsigned long pfn);
> > extern void register_isa_ports(unsigned int mmio, unsigned int io,
> > unsigned int io_shift);
> >
> > +#define __HAVE_ARCH_ATOMIC_IO_MODIFY
> > +extern void atomic_io_modify(void __iomem *reg, u32 mask, u32 set);
> > +extern void atomic_io_modify_relaxed(void __iomem *reg, u32 mask, u32 set);
> > +
> > #endif /* __KERNEL__ */
> > #endif /* __ASM_ARM_IO_H */
> > diff --git a/arch/arm/kernel/io.c b/arch/arm/kernel/io.c
> > index dcd5b4d..a8c9c9b 100644
> > --- a/arch/arm/kernel/io.c
> > +++ b/arch/arm/kernel/io.c
> > @@ -1,6 +1,35 @@
> > #include <linux/export.h>
> > #include <linux/types.h>
> > #include <linux/io.h>
> > +#include <linux/spinlock.h>
> > +
> > +static DEFINE_RAW_SPINLOCK(__io_lock);
> > +
> > +void atomic_io_modify_relaxed(void __iomem *reg, u32 mask, u32 set)
> > +{
> > + unsigned long flags;
> > + u32 value;
> > +
> > + raw_spin_lock_irqsave(&__io_lock, flags);
> > + value = readl_relaxed(reg) & ~mask;
> > + value |= (set & mask);
> > + writel_relaxed(value, reg);
> > + raw_spin_unlock_irqrestore(&__io_lock, flags);
> > +}
> > +EXPORT_SYMBOL(atomic_io_modify_relaxed);
> > +
> > +void atomic_io_modify(void __iomem *reg, u32 mask, u32 set)
> > +{
> > + unsigned long flags;
> > + u32 value;
> > +
> > + raw_spin_lock_irqsave(&__io_lock, flags);
> > + value = readl_relaxed(reg) & ~mask;
> > + value |= (set & mask);
> > + writel(value, reg);
> > + raw_spin_unlock_irqrestore(&__io_lock, flags);
> > +}
> > +EXPORT_SYMBOL(atomic_io_modify);
>
> Is this any different from the generic one introduced in patch 1/4? I
> would rather just use the generic definition.
Well, according to Will Deacon (and as documented in the commit log)
we can optimize in ARM by using readl_relaxed instead of readl.
Now, I'm sure you now better than me if that results (or not) in any
significant optimization.
> Similarly, a generic
> atomic_io_modify_relaxed() but guarded with something like
> __HAVE_ARCH_RELAXED_IO.
>
No, that's not possible. As far as I understand, there's no guarantee
of _relaxed variants to be available architecture-wide.
--
Ezequiel García, Free Electrons
Embedded Linux, Kernel and Android Engineering
http://free-electrons.com
Dear Ezequiel Garcia,
On Wed, 28 Aug 2013 06:49:08 -0300, Ezequiel Garcia wrote:
> > Is this any different from the generic one introduced in patch 1/4? I
> > would rather just use the generic definition.
>
> Well, according to Will Deacon (and as documented in the commit log)
> we can optimize in ARM by using readl_relaxed instead of readl.
>
> Now, I'm sure you now better than me if that results (or not) in any
> significant optimization.
>
> > Similarly, a generic
> > atomic_io_modify_relaxed() but guarded with something like
> > __HAVE_ARCH_RELAXED_IO.
> >
>
> No, that's not possible. As far as I understand, there's no guarantee
> of _relaxed variants to be available architecture-wide.
I think what Catalin was suggesting is that atomic_io_modify() should
use readl() and writel() (i.e *not* the relaxed variants), and that a
separate atomic_io_modify_relaxed() could be added on architectures
that define __HAVE_ARCH_RELAXED_IO.
I think you misread Catalin's comment when you say there's no guarantee
of _relaxed variants to be available architecture-wide, since Catalin
precisely suggested to guard that with __HAVE_ARCH_RELAXED_IO, which
indicates that _relaxed variants are available.
Thanks,
Thomas
--
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com
On Tue, Aug 27, 2013 at 01:37:09PM -0700, Andrew Morton wrote:
> On Sat, 24 Aug 2013 12:35:29 -0300 Ezequiel Garcia <[email protected]> wrote:
>
> > Some platforms have MMIO regions that are shared across orthogonal
> > subsystems. This commit implements a possible solution for the
> > thread-safe access of such regions through a spinlock-protected API.
>
> Seem sensible. Perhaps.
>
> It only works if both subsystems agree to use atomic_io_modify(). And
> if they're both capable of doing that, they are both capable of
> implementing an agreed-upon internal locking scheme, so why bother?
>
One of the scenarios where this could be helpful and an agreed-upon
lock seemed difficult to design is this: a watchdog driver that shares
some control register with *two* different clocksource drivers.
So, one first solution is to have a function in the two clocksource
drivers (with matching prototype) and have the watchdog access
the register through it.
However, because of multiplatform builds, both these clocksource drivers
could be built at the same time. Therefore we would have a symbol
collision, doubly-defined, in each driver.
How would that work? What other internal locking scheme could we
implement?
BTW, this is the current use case we are trying to fix.
This atomic function seemed like a simpler (perhaps cleaner?) approach.
[..]
>
> I disagree with the presence of the ifndef. If
> __HAVE_ARCH_ATOMIC_IO_MODIFY is undefined, the architecture must still
> implement the identical function signature. The best way to ensure that
> is to use the same prototype in both cases.
>
I agree, but how can this be done?
I'm looking at examples, such as include/asm-generic/atomic.h or
include/linux/string.h and they all do this sort of trick.
> > #endif /* _LINUX_IO_H */
> > diff --git a/lib/Makefile b/lib/Makefile
> > index 7baccfd..695d6e2 100644
> > --- a/lib/Makefile
> > +++ b/lib/Makefile
> > @@ -13,7 +13,7 @@ lib-y := ctype.o string.o vsprintf.o cmdline.o \
> > sha1.o md5.o irq_regs.o reciprocal_div.o argv_split.o \
> > proportions.o flex_proportions.o prio_heap.o ratelimit.o show_mem.o \
> > is_single_threaded.o plist.o decompress.o kobject_uevent.o \
> > - earlycpio.o percpu-refcount.o
> > + earlycpio.o percpu-refcount.o atomicio.o
> >
> > obj-$(CONFIG_ARCH_HAS_DEBUG_STRICT_USER_COPY_CHECKS) += usercopy.o
> > lib-$(CONFIG_MMU) += ioremap.o
> > diff --git a/lib/atomicio.c b/lib/atomicio.c
> > new file mode 100644
> > index 0000000..1750f9d
> > --- /dev/null
> > +++ b/lib/atomicio.c
> > @@ -0,0 +1,27 @@
> > +#include <linux/io.h>
> > +#include <linux/spinlock.h>
> > +
> > +#ifndef __HAVE_ARCH_ATOMIC_IO_MODIFY
> > +/*
> > + * Generic atomic MMIO modify.
> > + *
> > + * Allows thread-safe access to registers shared by unrelated subsystems.
> > + * The access is protected by a single MMIO-wide lock.
> > + *
> > + * Optimized variants can be implemented on a per-architecture basis.
> > + */
>
> Some kerneldoc would be nice.
>
Ok.
> > +static DEFINE_RAW_SPINLOCK(__io_lock);
>
> This could be local to atomic_io_modify().
>
Ok.
> > +void atomic_io_modify(void __iomem *reg, u32 mask, u32 set)
> > +{
> > + unsigned long flags;
> > + u32 value;
> > +
> > + raw_spin_lock_irqsave(&__io_lock, flags);
> > + value = readl(reg) & ~mask;
> > + value |= (set & mask);
>
> Could just be "value |= set". The code as you have it permits callers
> to set bits in "set" which aren't permitted in "mask", which would be
> pretty darn stupid of them.
>
Right.
> > + writel(value, reg);
> > + raw_spin_unlock_irqrestore(&__io_lock, flags);
> > +
> > +}
> > +EXPORT_SYMBOL(atomic_io_modify);
> > +#endif
>
> What about 8, 16 and 64-bit IO addresses?
>
Right, forgot about these. I guess we can add those if we agree about the approach.
--
Ezequiel García, Free Electrons
Embedded Linux, Kernel and Android Engineering
http://free-electrons.com
Linus,
Andrew suggested you might have opinions on this, so I'm cc'ing you.
Since you'll probably want some better context, here it is:
http://lwn.net/Articles/564709/
On Sat, Aug 24, 2013 at 12:35:29PM -0300, Ezequiel Garcia wrote:
> Some platforms have MMIO regions that are shared across orthogonal
> subsystems. This commit implements a possible solution for the
> thread-safe access of such regions through a spinlock-protected API.
>
> Concurrent access is protected with a single spinlock for the
> entire MMIO address space. While this protects shared-registers,
> it also serializes access to unrelated/unshared registers.
>
> Signed-off-by: Ezequiel Garcia <[email protected]>
> ---
> include/linux/io.h | 5 +++++
> lib/Makefile | 2 +-
> lib/atomicio.c | 27 +++++++++++++++++++++++++++
> 3 files changed, 33 insertions(+), 1 deletion(-)
> create mode 100644 lib/atomicio.c
>
> diff --git a/include/linux/io.h b/include/linux/io.h
> index f4f42fa..c331dcb 100644
> --- a/include/linux/io.h
> +++ b/include/linux/io.h
> @@ -101,4 +101,9 @@ static inline void arch_phys_wc_del(int handle)
> #define arch_phys_wc_add arch_phys_wc_add
> #endif
>
> +#ifndef __HAVE_ARCH_ATOMIC_IO_MODIFY
> +/* Atomic MMIO-wide IO modify */
> +extern void atomic_io_modify(void __iomem *reg, u32 mask, u32 set);
> +#endif
> +
> #endif /* _LINUX_IO_H */
> diff --git a/lib/Makefile b/lib/Makefile
> index 7baccfd..695d6e2 100644
> --- a/lib/Makefile
> +++ b/lib/Makefile
> @@ -13,7 +13,7 @@ lib-y := ctype.o string.o vsprintf.o cmdline.o \
> sha1.o md5.o irq_regs.o reciprocal_div.o argv_split.o \
> proportions.o flex_proportions.o prio_heap.o ratelimit.o show_mem.o \
> is_single_threaded.o plist.o decompress.o kobject_uevent.o \
> - earlycpio.o percpu-refcount.o
> + earlycpio.o percpu-refcount.o atomicio.o
>
> obj-$(CONFIG_ARCH_HAS_DEBUG_STRICT_USER_COPY_CHECKS) += usercopy.o
> lib-$(CONFIG_MMU) += ioremap.o
> diff --git a/lib/atomicio.c b/lib/atomicio.c
> new file mode 100644
> index 0000000..1750f9d
> --- /dev/null
> +++ b/lib/atomicio.c
> @@ -0,0 +1,27 @@
> +#include <linux/io.h>
> +#include <linux/spinlock.h>
> +
> +#ifndef __HAVE_ARCH_ATOMIC_IO_MODIFY
> +/*
> + * Generic atomic MMIO modify.
> + *
> + * Allows thread-safe access to registers shared by unrelated subsystems.
> + * The access is protected by a single MMIO-wide lock.
> + *
> + * Optimized variants can be implemented on a per-architecture basis.
> + */
> +static DEFINE_RAW_SPINLOCK(__io_lock);
> +void atomic_io_modify(void __iomem *reg, u32 mask, u32 set)
> +{
> + unsigned long flags;
> + u32 value;
> +
> + raw_spin_lock_irqsave(&__io_lock, flags);
> + value = readl(reg) & ~mask;
> + value |= (set & mask);
> + writel(value, reg);
> + raw_spin_unlock_irqrestore(&__io_lock, flags);
> +
> +}
> +EXPORT_SYMBOL(atomic_io_modify);
> +#endif
> --
> 1.8.1.5
>
--
Ezequiel García, Free Electrons
Embedded Linux, Kernel and Android Engineering
http://free-electrons.com
On Wed, Aug 28, 2013 at 11:01:22AM +0100, Thomas Petazzoni wrote:
> Dear Ezequiel Garcia,
>
> On Wed, 28 Aug 2013 06:49:08 -0300, Ezequiel Garcia wrote:
>
> > > Is this any different from the generic one introduced in patch 1/4? I
> > > would rather just use the generic definition.
> >
> > Well, according to Will Deacon (and as documented in the commit log)
> > we can optimize in ARM by using readl_relaxed instead of readl.
> >
> > Now, I'm sure you now better than me if that results (or not) in any
> > significant optimization.
> >
> > > Similarly, a generic
> > > atomic_io_modify_relaxed() but guarded with something like
> > > __HAVE_ARCH_RELAXED_IO.
> > >
> >
> > No, that's not possible. As far as I understand, there's no guarantee
> > of _relaxed variants to be available architecture-wide.
>
> I think what Catalin was suggesting is that atomic_io_modify() should
> use readl() and writel() (i.e *not* the relaxed variants), and that a
> separate atomic_io_modify_relaxed() could be added on architectures
> that define __HAVE_ARCH_RELAXED_IO.
>
> I think you misread Catalin's comment when you say there's no guarantee
> of _relaxed variants to be available architecture-wide, since Catalin
> precisely suggested to guard that with __HAVE_ARCH_RELAXED_IO, which
> indicates that _relaxed variants are available.
Indeed, thanks for the translation ;).
--
Catalin
On Wed, Aug 28, 2013 at 3:37 AM, Ezequiel Garcia
<[email protected]> wrote:
> Linus,
>
> Andrew suggested you might have opinions on this, so I'm cc'ing you.
> Since you'll probably want some better context, here it is:
>
> http://lwn.net/Articles/564709/
>
> On Sat, Aug 24, 2013 at 12:35:29PM -0300, Ezequiel Garcia wrote:
>> Some platforms have MMIO regions that are shared across orthogonal
>> subsystems. This commit implements a possible solution for the
>> thread-safe access of such regions through a spinlock-protected API.
>>
>> Concurrent access is protected with a single spinlock for the
>> entire MMIO address space. While this protects shared-registers,
>> it also serializes access to unrelated/unshared registers.
I have nothing against this, except that
"__HAVE_ARCH_ATOMIC_IO_MODIFY" needs to die, along with the #ifdef.
It should be a CONFIG_xyz option that gets set by the architectures
that have it, and then instead of an #ifdef in code, the Makefile
should make the generic target be conditional.
So add a generic
bool GENERIC_ATOMIC_MMIO_MODIFY
depends on !ARCH_ATOMIC_MMIO_MODIFY
default y
to the lib/Kconfig file, and then lib/Makefile just does
obj-$(CONFIG_GENERIC_ATOMIC_MMIO_MODIFY) += atomic_io.o
After that, ARM can then implement some architecture-optimized
version, and in its own Kconfig file just do "select
ARCH_ATOMIC_MMIO_MODIFY" to let the generic code know that it should
disable that generic version.
Linus
On Wed, 28 Aug 2013 07:24:23 -0300 Ezequiel Garcia <[email protected]> wrote:
> On Tue, Aug 27, 2013 at 01:37:09PM -0700, Andrew Morton wrote:
> > On Sat, 24 Aug 2013 12:35:29 -0300 Ezequiel Garcia <[email protected]> wrote:
> >
> > > Some platforms have MMIO regions that are shared across orthogonal
> > > subsystems. This commit implements a possible solution for the
> > > thread-safe access of such regions through a spinlock-protected API.
> >
> > Seem sensible. Perhaps.
> >
> > It only works if both subsystems agree to use atomic_io_modify(). And
> > if they're both capable of doing that, they are both capable of
> > implementing an agreed-upon internal locking scheme, so why bother?
> >
>
> One of the scenarios where this could be helpful and an agreed-upon
> lock seemed difficult to design is this: a watchdog driver that shares
> some control register with *two* different clocksource drivers.
>
> So, one first solution is to have a function in the two clocksource
> drivers (with matching prototype) and have the watchdog access
> the register through it.
>
> However, because of multiplatform builds, both these clocksource drivers
> could be built at the same time. Therefore we would have a symbol
> collision, doubly-defined, in each driver.
>
> How would that work? What other internal locking scheme could we
> implement?
I guess the locking would need to be in a standalone module which the
various driver modules would then depend upon. I'm not really
advocating doing this - I'm just making noise.
> [..]
> >
> > I disagree with the presence of the ifndef. If
> > __HAVE_ARCH_ATOMIC_IO_MODIFY is undefined, the architecture must still
> > implement the identical function signature. The best way to ensure that
> > is to use the same prototype in both cases.
> >
>
> I agree, but how can this be done?
Just remove the ifndefs. Then remove the identical function prototype
from the arm header.
Dear Andrew Morton,
On Wed, 28 Aug 2013 12:33:52 -0700, Andrew Morton wrote:
> > > It only works if both subsystems agree to use atomic_io_modify(). And
> > > if they're both capable of doing that, they are both capable of
> > > implementing an agreed-upon internal locking scheme, so why bother?
> > >
> >
> > One of the scenarios where this could be helpful and an agreed-upon
> > lock seemed difficult to design is this: a watchdog driver that shares
> > some control register with *two* different clocksource drivers.
> >
> > So, one first solution is to have a function in the two clocksource
> > drivers (with matching prototype) and have the watchdog access
> > the register through it.
> >
> > However, because of multiplatform builds, both these clocksource drivers
> > could be built at the same time. Therefore we would have a symbol
> > collision, doubly-defined, in each driver.
> >
> > How would that work? What other internal locking scheme could we
> > implement?
>
> I guess the locking would need to be in a standalone module which the
> various driver modules would then depend upon. I'm not really
> advocating doing this - I'm just making noise.
I think the idea of this "atomic MMIO modify" function was precisely to
solve the situations where one or two "misc" registers need to be
accessed by various unrelated drivers, and having a separate standalone
module to control those one or two "misc" registers would be a bit too
annoying.
This is a situation that we have fairly often at least in some ARM
SoCs: the registers of the various IP blocks are generally nicely
organized in "regions", where all the registers for each UART are
grouped together, for each I2C controller, each SPI controller and so
on. But there are always a bunch of misc, system control registers that
do not really belong to any particular IP block, but some of the device
drivers sometimes need to set/clear a bit in such registers. Of course,
when those "system control" registers control a clock, or pin muxing,
or something well-known, we have existing frameworks in the kernel to
support that. But there is always this bizarre "system control"
feature, that does not fit into an existing kernel framework, and for
which writing an entirely separate driver is really overkill.
The case highlighted by the patches 3/4 and 4/4 of Ezequiel are I
believe a good example. While each timer and the watchdog each have
their own "region" of registers to be controlled, there is also one
single global register to enable/disable the different timers and
watchdog (with one bit per timer or watchdog). So this register needs
to be accessed by both the timer (clocksource) and watchdog drivers,
even though they are otherwise completely unrelated. Writing a separate
driver just to control this register, that is accessed infrequently
(i.e only when the kernel boots essentially), would require a lot of
code for no real benefit.
Does that clarify the intended usage?
Thanks for your feedback,
Thomas
--
Thomas Petazzoni, Free Electrons
Kernel, drivers, real-time and embedded Linux
development, consulting, training and support.
http://free-electrons.com