After another round of review, here is a revised set of MMIO copy patches.
These have switched the name of the copy routine to __iowrite32_copy,
to match the naming convention in include/asm-generic/iomap.h, and at
Andi's request. The name is double-underscored to make it clear that
the routine does not guarantee the order of writes or perform a memory
barrier; the kernel doc also explicitly states this.
These define the generic __iowrite32_copy as a weak symbol, which
arches are free to override. We provide a specialised implementation
for x86_64. In a change from prior versions of these patches, the x86_64
version no longer makes any assumptions about the bus breaking up 64-bit
transactions, and uses 32-bit copies directly.
We also introduce include/linux/io.h, which is tiny now, but a candidate
for later cleanups of all the per-arch asm-*/{io,iomap}.h files.
These patches should apply cleanly against current -git, and have been
tested on i386 and x86_64.
The patch series is as follows:
iomap_copy.patch
Introduce the generic MMIO 32-bit copy routine.
x86_64-iomap_copy.patch
Add a faster __iowrite32_copy routine to x86_64.
Signed-off-by: Bryan O'Sullivan <[email protected]>
This arch-independent routine copies data to a memory-mapped I/O region,
using 32-bit accesses. The naming is double-underscored to make it clear
that it does not guarantee write ordering, nor does it perform a
memory barrier afterwards; the kernel doc also explicitly states this.
This style of access is required by some devices.
This change also introduces include/linux/io.h, at Andrew's suggestion.
It only has one occupant at the moment, but is a logical destination
for oft-replicated contents of include/asm-*/{io,iomap}.h to migrate to.
Signed-off-by: Bryan O'Sullivan <[email protected]>
diff -r c90267e4a29b -r ec2b3675168a lib/Makefile
--- a/lib/Makefile Wed Jan 11 13:31:24 2006 +0800
+++ b/lib/Makefile Thu Jan 12 09:03:37 2006 -0800
@@ -9,7 +9,7 @@
lib-y += kobject.o kref.o kobject_uevent.o klist.o
-obj-y += sort.o parser.o halfmd4.o
+obj-y += sort.o parser.o halfmd4.o iomap_copy.o
ifeq ($(CONFIG_DEBUG_KOBJECT),y)
CFLAGS_kobject.o += -DDEBUG
diff -r c90267e4a29b -r ec2b3675168a include/linux/io.h
--- /dev/null Thu Jan 1 00:00:00 1970 +0000
+++ b/include/linux/io.h Thu Jan 12 09:03:37 2006 -0800
@@ -0,0 +1,25 @@
+/*
+ * Copyright 2006 PathScale, Inc. All Rights Reserved.
+ *
+ * This file is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301, USA.
+ */
+
+#ifndef _LINUX_IO_H
+#define _LINUX_IO_H
+
+#include <asm/io.h>
+
+void __iowrite32_copy(void __iomem *to, const void *from, size_t count);
+
+#endif /* _LINUX_IO_H */
diff -r c90267e4a29b -r ec2b3675168a lib/iomap_copy.c
--- /dev/null Thu Jan 1 00:00:00 1970 +0000
+++ b/lib/iomap_copy.c Thu Jan 12 09:03:37 2006 -0800
@@ -0,0 +1,42 @@
+/*
+ * Copyright 2006 PathScale, Inc. All Rights Reserved.
+ *
+ * This file is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301, USA.
+ */
+
+#include <linux/io.h>
+#include <linux/module.h>
+
+/**
+ * __iowrite32_copy - copy data to MMIO space, in 32-bit units
+ * @to: destination, in MMIO space (must be 32-bit aligned)
+ * @from: source (must be 32-bit aligned)
+ * @count: number of 32-bit quantities to copy
+ *
+ * Copy data from kernel space to MMIO space, in units of 32 bits at a
+ * time. Order of access is not guaranteed, nor is a memory barrier
+ * performed afterwards.
+ */
+void __attribute__((weak)) __iowrite32_copy(void __iomem *to,
+ const void *from,
+ size_t count)
+{
+ u32 __iomem *dst = to;
+ const u32 *src = from;
+ const u32 *end = src + count;
+
+ while (src < end)
+ __raw_writel(*src++, dst++);
+}
+EXPORT_SYMBOL_GPL(__iowrite32_copy);
This assembly version is measurably faster than the generic version
in lib/iomap_copy.c.
Signed-off-by: Bryan O'Sullivan <[email protected]>
diff -r ec2b3675168a -r 30c7112c6e81 arch/x86_64/lib/Makefile
--- a/arch/x86_64/lib/Makefile Thu Jan 12 09:03:37 2006 -0800
+++ b/arch/x86_64/lib/Makefile Thu Jan 12 09:03:37 2006 -0800
@@ -4,7 +4,7 @@
CFLAGS_csum-partial.o := -funroll-loops
-obj-y := io.o
+obj-y := io.o iomap_copy.o
lib-y := csum-partial.o csum-copy.o csum-wrappers.o delay.o \
usercopy.o getuser.o putuser.o \
diff -r ec2b3675168a -r 30c7112c6e81 arch/x86_64/lib/iomap_copy.S
--- /dev/null Thu Jan 1 00:00:00 1970 +0000
+++ b/arch/x86_64/lib/iomap_copy.S Thu Jan 12 09:03:37 2006 -0800
@@ -0,0 +1,26 @@
+/*
+ * Copyright 2006 PathScale, Inc. All Rights Reserved.
+ *
+ * This file is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software Foundation,
+ * Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301, USA.
+ */
+
+/*
+ * override generic version in lib/iomap_copy.c
+ */
+ .globl __iowrite32_copy
+ .p2align 4
+__iowrite32_copy:
+ movl %edx,%ecx
+ rep movsd
+ ret