Paul Jackson
[email protected]
21 April 2004
Bitmap and Cpumask Cleanup
This is the fourth version of my cleanup of cpumasks.
This set of 17 patches applies against a vanilla 2.6.5 kernel.
The actual patches will be posted in another hour or two, as
followups to this initial post.
Primay goals:
The primary goal of this patch set is to simplify the code for
cpumask_t and (later) nodemask_t, make them easier to use, and
reduce code duplication.
Several flavors of cpumask have been reduced to one, with some
local special handling to optimize for that vast majority of
systems which have less than 32 CPUs.
The bitmap operations upon which cpumasks depend have been
optimized a bit more, with a more careful mix of inline and
outofline code.
By simplifying masks to a single file, it should also be
easier to add other such mask types, such as the nodemask
that Matthew Dobson has waiting in the wings, just by
copying cpumask.h and making a few global edits.
Compared to the third version (8 April) of this patch set:
1) Doesn't include Matthew Dobson's nodemask patch.
Matthew will submit that patch separately, after
the fate of this cleanup patch is resolved.
2) Has one typo fixed in a comment.
What's next:
I will seek out the feedback of the architecture maintainers
over the next few days to this patch, and continue local
testing. Then it should be ready to submit to Andrew.
What's new (comparing 2.6.5 to this patch set):
1) Some 27 files matching the pattern include/*/*mask*.h
are replaced with the single file include/linux/cpumask.h
The variety of arch-specific redirect headers for various
cpumask implementation flavors is gone.
2) The bitmap operations (bitmap.h, bitmap.c) are optimized
for systems of less than 32 CPUs.
3) The cpumask operations are now just a thin layer on top
of the bitmask and a few other operations. A cpumask is a
bitmask of exactly NR_CPUS bits, wrapped in a structure.
4) bitmap_complement and cpumask_complement now take two args,
source and target, instead of working in place.
5) Some uses of these macros elsewhere in the kernel were fine
tuned.
6) On ia64, find_first_bit and find_first_zero_bit are
uninlined - saving quite a bit of kernel text space.
The architectures: alpha, parisc, ppc, sh, sparc, sparc64
have this same bit find code, and might also want to uninline.
7) Comments in bitmap.h and cpumask.h list available ops for ease
of browsing.
8) The MASK_ALL macro zeros out unused bits on multiword bitmaps.
9) This patch includes Bill Irwin's very recent rewrite
of the bitmap_shift operators to avoid assuming some
fixed upper bound on bitmap sizes.
10) A few more mask macros have been added, to make it easier to
code mask manipulations correctly and easily. They provide
xor, andnot, intersects and subset operators.
Bug fixes:
1) *_complement macros don't leave unused high bits set
2) MASK_ALL for sizes > 1 word, but not exact word multiple,
doesn't have unused high bits set
3) Explicit, documented semantics for handling these unused high bits.
4) A few missing const attributes in bitmap & bitops added.
5) The (Hamming) cpumask weight macros were using the bitops hweight*()
macros, which don't mask high unused bits - fixed to use the bitmap
weight macro which does this masking.
Do to the rather limited use so far of cpumask macros, I am
not aware of anything that these bugs would actually break in
current mm or linus kernels.
Testing so far:
Kernels have been built for i386 and ia64, and booted and
minimally tested for ia64. Kernel text size has been
compared and found to typically get a little smaller for
these two arch's. Correct function of bit operations has
been tested for a variety of NR_CPUS values in a user level
framework. Gcc compiler versions 2.95.3, 3.2.3 and 3.3.2
have been built successfully.
Risks:
This patch set makes changes in bitmap and cpumask handling
code in several architectures. I could have broken something.
Other affected arch's, need to be tested and reviewed.
The other arch's seeing the most changes were i386, ppc64
and x86_64, but specific changes to alpha, arm, mips, sparc64
and sh aloo need review.
This patch set removes several more optimized variants of
mask, which were targeted for optimum performance on various
architectures, and replaces that with essentially a single
variant. Preliminary analysis on the one architecture I
cared most about (large ia64) shows no noticeable degradation
in the quality of the code generated for mask operations,
but other architectures or more demanding uses may require
further optimization work.
This patch set might have dependencies on specific gcc or
toolchain versions. Portions have worked with both gcc
2.95.3 and 3.3.2 inside a user level scaffolding on i386.
Kernels have been built and booted on ia64 using gcc 3.2.3
and built on i386 using gcc 3.3.2. But non-trivial use is
made of gcc extensions, so certain archs or toolchains might
not build, or might not produce sufficiently optimal code,
or might not even work.
This patch set changes the semantics of cpumasks slightly.
If all mask operations are performed with correct parameters,
and no bits are intentionally set outside the range of bits
covered by such a mask (typically this means outside the
range [0 ... NR_CPUS-1] for cpumask_t), then there should be
no noticeable difference. Code that (improperly) sets bits
outside that valid range may notice differences in when such
bits affect the results of other mask operations.
Patches 13 through 16 (see list of patches, below) in
particular attempt to "cleanup" some cpumask manipulation
code in various sparc and intel arch's. These patches are
not essential - if they get in the way of other work, or are
too much distraction, or break something, any of them can
be dropped. At a minimum, they require review and approval
from the affected arch's.
================ Patch Details ================
Here's what each patch does:
[Patch 1 of 17] cpumask v4 - Document bitmap.c bit model.
Document the bitmap bit model, including handling of unused bits,
and operation preconditions and postconditions.
[Patch 2 of 17] cpumask v4 - Dont generate nonzero unused bits in bitmap
Tighten up bitmap so it does not generate nonzero bits
in the unused tail if it is not given any on input.
[Patch 3 of 17] cpumask v4 - New bitmap operators and two op complement
Add intersects, subset, xor and andnot operators.
Change bitmap_complement to take two operands.
[Patch 4 of 17] cpumask v4 - two missing 'const' qualifiers in bitops/bitmap
Add a couple of missing 'const' qualifiers on
bitops test_bit and bitmap_equal args.
[Patch 5 of 17] cpumask v4 - Optimize and extend bitmap.
This bitmap improvements make it a suitable basis for
fully supporting cpumask_t and nodemask_t. Inline macros
with compile-time checks enable generating tight code on
both small and large systems (large meaning cpumask_t
requires more than one unsigned long's worth of bits).
The existing bitmap_<op> macros in lib/bitmap.c
are renamed to __bitmap_<op>, and wrappers for each
bitmap_<op> are exposed in include/linux/bitmap.h
This patch _includes_ Bill Irwins rewrite of the
bitmap_shift operators to not require a fixed length
intermediate bitmap.
Improved comments list each available operator for easy
browsing.
[Patch 6 of 17] cpumask v4 - Uninline find_next_bit on ia64
Move the page of code (~700 bytes of instructions)
for find_next_bit and find_next_zero_bit from inline
in include/asm-ia64/bitops.h to a real function in
arch/ia64/lib/bitops.c, leaving a declaration and
macro wrapper behind.
The other arch's with almost this same code might want to
also uninline it: alpha, parisc, ppc, sh, sparc, sparc64.
These are too big to inline.
[Patch 7 of 17] cpumask v4 - Rewrite cpumask.h to use bitmap directly.
Major rewrite of cpumask to use a single implementation,
as a struct-wrapped bitmap.
This patch leaves some 26 include/asm-*/cpumask*.h
header files orphaned - to be removed next patch.
Some nine cpumask macros for const variants and to
coerce and promote between an unsigned long and a
cpumask are obsolete. Simple emulation wrappers are
provided in this patch, which can be removed once each
of the 3 archs (i386, ppc64, x86_64) using them are
recoded in follow-on patches to not need them.
The CPU_MASK_ALL macro now avoids leaving possible
garbage one bits in any unused portion of the high word.
An improved comment lists all available operators, for
convenient browsing.
[Patch 8 of 17] cpumask v4 - Remove 26 no longer used cpumask headers.
With the cpumask rewrite in the previous patch, these
various include/asm-*/cpumask*.h headers are no longer used.
[Patch 9 of 17] cpumask v4 - Recode obsolete cpumask macros - arch i386
Remove by recoding all uses of the obsolete cpumask const,
coerce and promote macros.
[Patch 10 of 17] cpumask v4 - Recode obsolete cpumask macros - arch ppc64
Remove by recoding all uses of the obsolete cpumask const,
coerce and promote macros.
[Patch 11 of 17] cpumask v4 - Recode obsolete cpumask macros - arch x86_64
Remove by recoding all uses of the obsolete cpumask const,
coerce and promote macros.
[Patch 12 of 17] cpumask v4 - Remove obsolete emulation from cpumask.h
Now that the emulation of the obsolete cpumask macros is no
longer needed, remove it from cpumask.h
[Patch 13 of 17] cpumask v4 - Simplify some sparc64 cpumask loop code
Make use of for_each_cpu_mask() macro to simplify and optimize
a couple of sparc64 per-CPU loops. This code change has _not_
been tested or reviewed. Feedback welcome. There is non-trivial
risk that I still don't understand the logic here.
[Patch 14 of 17] cpumask v4 - Optimize i386 cpumask macro usage.
Optimize a bit of cpumask code for asm-i386/mach-es7000
Code untested, unreviewed. Feedback welcome.
[Patch 15 of 17] cpumask v4 - Convert physids_complement() to use both args
Provide for specifying distinct source and dest args to the
physids_complement(). No one actually uses this macro yet.
The physid_mask type would be a good candidate to convert to
using this new mask ADT as a base.
[Patch 16 of 17] cpumask v4 - Remove cpumask hack from asm-x86_64/topology.h
This file had the cpumask cpu_online_map as type
unsigned long, instead of type cpumask_t, for no good
reason that I could see. So I changed it. Everywhere
else, cpu_online_map is already of type cpumask_t.
[Patch 17 of 17] cpumask v4 - Cpumask code clarification in kernel/sched.c
Clarify and slightly optimize set_cpus_allowed() cpumask check
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373
mask12-rm-old-cpumask-emul - Remove obsolete cpumask emulation from cpumask.h
Now that the emulation of the obsolete cpumask macros is no
longer needed, remove it from cpumask.h
Index: 2.6.5.bitmap/include/linux/cpumask.h
===================================================================
--- 2.6.5.bitmap.orig/include/linux/cpumask.h 2004-04-07 21:34:10.000000000 -0700
+++ 2.6.5.bitmap/include/linux/cpumask.h 2004-04-07 21:57:34.000000000 -0700
@@ -315,16 +315,4 @@
#define for_each_online_cpu(cpu) \
for_each_cpu_mask(cpu, cpu_online_map)
-/* Begin obsolete cpumask operator emulation */
-#define cpu_isset_const(a,b) cpu_isset(a,b)
-#define cpumask_const_t cpumask_t
-#define cpus_coerce(m) (cpus_addr(m)[0])
-#define cpus_coerce_const cpus_coerce
-#define cpus_promote(x) ({ cpumask_t m; m.bits[0] = x; m; })
-#define cpus_weight_const cpus_weight
-#define first_cpu_const first_cpu
-#define mk_cpumask_const(x) x
-#define next_cpu_const next_cpu
-/* End of obsolete cpumask operator emulation */
-
#endif /* __LINUX_CPUMASK_H */
%diffstat
cpumask.h | 12 ------------
1 files changed, 12 deletions(-)
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373
mask11-cpumask-x86-64-fixup - Remove/recode obsolete cpumask macros from arch x86_64
Remove by recoding all uses of the obsolete cpumask const,
coerce and promote macros.
Diffstat Patch_9_of_23:
arch/x86_64/kernel/io_apic.c | 2 +-
arch/x86_64/kernel/pci-gart.c | 4 ++--
arch/x86_64/kernel/smp.c | 2 +-
include/asm-x86_64/smp.h | 4 ++--
4 files changed, 6 insertions(+), 6 deletions(-)
Index: 2.6.5.mask/arch/x86_64/kernel/io_apic.c
===================================================================
--- 2.6.5.mask.orig/arch/x86_64/kernel/io_apic.c 2004-04-03 23:37:43.000000000 -0800
+++ 2.6.5.mask/arch/x86_64/kernel/io_apic.c 2004-04-03 23:52:03.000000000 -0800
@@ -1368,7 +1368,7 @@
unsigned long flags;
unsigned int dest;
- dest = cpu_mask_to_apicid(mk_cpumask_const(mask));
+ dest = cpu_mask_to_apicid(mask);
/*
* Only the first 8 bits are valid.
Index: 2.6.5.mask/arch/x86_64/kernel/pci-gart.c
===================================================================
--- 2.6.5.mask.orig/arch/x86_64/kernel/pci-gart.c 2004-04-03 23:37:43.000000000 -0800
+++ 2.6.5.mask/arch/x86_64/kernel/pci-gart.c 2004-04-03 23:52:03.000000000 -0800
@@ -149,7 +149,7 @@
{
unsigned long flags;
int bus = dev ? dev->bus->number : -1;
- cpumask_const_t bus_cpumask = pcibus_to_cpumask(bus);
+ cpumask_t bus_cpumask = pcibus_to_cpumask(bus);
int flushed = 0;
int i;
@@ -159,7 +159,7 @@
u32 w;
if (!northbridges[i])
continue;
- if (bus >= 0 && !(cpu_isset_const(i, bus_cpumask)))
+ if (bus >= 0 && !(cpu_isset(i, bus_cpumask)))
continue;
pci_write_config_dword(northbridges[i], 0x9c,
northbridge_flush_word[i] | 1);
Index: 2.6.5.mask/arch/x86_64/kernel/smp.c
===================================================================
--- 2.6.5.mask.orig/arch/x86_64/kernel/smp.c 2004-04-03 23:37:43.000000000 -0800
+++ 2.6.5.mask/arch/x86_64/kernel/smp.c 2004-04-03 23:52:03.000000000 -0800
@@ -94,7 +94,7 @@
static inline void send_IPI_mask(cpumask_t cpumask, int vector)
{
- unsigned long mask = cpus_coerce(cpumask);
+ unsigned long mask = cpus_addr(cpumask)[0];
unsigned long cfg;
unsigned long flags;
Index: 2.6.5.mask/include/asm-x86_64/smp.h
===================================================================
--- 2.6.5.mask.orig/include/asm-x86_64/smp.h 2004-04-03 23:38:16.000000000 -0800
+++ 2.6.5.mask/include/asm-x86_64/smp.h 2004-04-03 23:52:03.000000000 -0800
@@ -95,9 +95,9 @@
#define TARGET_CPUS 1
#ifndef ASSEMBLY
-static inline unsigned int cpu_mask_to_apicid(cpumask_const_t cpumask)
+static inline unsigned int cpu_mask_to_apicid(cpumask_t cpumask)
{
- return cpus_coerce_const(cpumask);
+ return cpus_addr(cpumask)[0];
}
#endif
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373
mask3-bitmap-newops - New bitmap operators and two op complement
Add intersects, subset, xor and andnot operators.
Change bitmap_complement to take two operands.
Index: 2.6.5.bitmap/include/asm-generic/cpumask_array.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-generic/cpumask_array.h 2004-04-07 20:58:38.000000000 -0700
+++ 2.6.5.bitmap/include/asm-generic/cpumask_array.h 2004-04-07 21:14:12.000000000 -0700
@@ -17,7 +17,7 @@
#define cpus_and(dst,src1,src2) bitmap_and((dst).mask,(src1).mask, (src2).mask, NR_CPUS)
#define cpus_or(dst,src1,src2) bitmap_or((dst).mask, (src1).mask, (src2).mask, NR_CPUS)
#define cpus_clear(map) bitmap_clear((map).mask, NR_CPUS)
-#define cpus_complement(map) bitmap_complement((map).mask, NR_CPUS)
+#define cpus_complement(map) bitmap_complement((map).mask, (map).mask, NR_CPUS)
#define cpus_equal(map1, map2) bitmap_equal((map1).mask, (map2).mask, NR_CPUS)
#define cpus_empty(map) bitmap_empty(map.mask, NR_CPUS)
#define cpus_addr(map) ((map).mask)
Index: 2.6.5.bitmap/include/asm-i386/mpspec.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-i386/mpspec.h 2004-04-07 20:58:38.000000000 -0700
+++ 2.6.5.bitmap/include/asm-i386/mpspec.h 2004-04-07 20:58:41.000000000 -0700
@@ -53,7 +53,7 @@
#define physids_and(dst, src1, src2) bitmap_and((dst).mask, (src1).mask, (src2).mask, MAX_APICS)
#define physids_or(dst, src1, src2) bitmap_or((dst).mask, (src1).mask, (src2).mask, MAX_APICS)
#define physids_clear(map) bitmap_clear((map).mask, MAX_APICS)
-#define physids_complement(map) bitmap_complement((map).mask, MAX_APICS)
+#define physids_complement(map) bitmap_complement((map).mask, (map).mask, MAX_APICS)
#define physids_empty(map) bitmap_empty((map).mask, MAX_APICS)
#define physids_equal(map1, map2) bitmap_equal((map1).mask, (map2).mask, MAX_APICS)
#define physids_weight(map) bitmap_weight((map).mask, MAX_APICS)
Index: 2.6.5.bitmap/include/asm-x86_64/mpspec.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-x86_64/mpspec.h 2004-04-07 20:58:38.000000000 -0700
+++ 2.6.5.bitmap/include/asm-x86_64/mpspec.h 2004-04-07 20:58:41.000000000 -0700
@@ -212,7 +212,7 @@
#define physids_and(dst, src1, src2) bitmap_and((dst).mask, (src1).mask, (src2).mask, MAX_APICS)
#define physids_or(dst, src1, src2) bitmap_or((dst).mask, (src1).mask, (src2).mask, MAX_APICS)
#define physids_clear(map) bitmap_clear((map).mask, MAX_APICS)
-#define physids_complement(map) bitmap_complement((map).mask, MAX_APICS)
+#define physids_complement(map) bitmap_complement((map).mask, (map).mask, MAX_APICS)
#define physids_empty(map) bitmap_empty((map).mask, MAX_APICS)
#define physids_equal(map1, map2) bitmap_equal((map1).mask, (map2).mask, MAX_APICS)
#define physids_weight(map) bitmap_weight((map).mask, MAX_APICS)
Index: 2.6.5.bitmap/include/linux/bitmap.h
===================================================================
--- 2.6.5.bitmap.orig/include/linux/bitmap.h 2004-04-07 20:58:38.000000000 -0700
+++ 2.6.5.bitmap/include/linux/bitmap.h 2004-04-07 21:14:15.000000000 -0700
@@ -13,8 +13,8 @@
int bitmap_empty(const unsigned long *bitmap, int bits);
int bitmap_full(const unsigned long *bitmap, int bits);
int bitmap_equal(const unsigned long *bitmap1,
- unsigned long *bitmap2, int bits);
-void bitmap_complement(unsigned long *bitmap, int bits);
+ const unsigned long *bitmap2, int bits);
+void bitmap_complement(unsigned long *dst, const unsigned long *src, int bits);
static inline void bitmap_clear(unsigned long *bitmap, int bits)
{
@@ -40,6 +40,14 @@
const unsigned long *bitmap2, int bits);
void bitmap_or(unsigned long *dst, const unsigned long *bitmap1,
const unsigned long *bitmap2, int bits);
+void bitmap_xor(unsigned long *dst, const unsigned long *bitmap1,
+ const unsigned long *bitmap2, int bits);
+void bitmap_andnot(unsigned long *dst, const unsigned long *bitmap1,
+ const unsigned long *bitmap2, int bits);
+int bitmap_intersects(const unsigned long *bitmap1,
+ const unsigned long *bitmap2, int bits);
+int bitmap_subset(const unsigned long *bitmap1,
+ const unsigned long *bitmap2, int bits);
int bitmap_weight(const unsigned long *bitmap, int bits);
int bitmap_scnprintf(char *buf, unsigned int buflen,
const unsigned long *maskp, int bits);
Index: 2.6.5.bitmap/lib/bitmap.c
===================================================================
--- 2.6.5.bitmap.orig/lib/bitmap.c 2004-04-07 20:58:41.000000000 -0700
+++ 2.6.5.bitmap/lib/bitmap.c 2004-04-07 21:14:18.000000000 -0700
@@ -81,14 +81,14 @@
}
EXPORT_SYMBOL(bitmap_equal);
-void bitmap_complement(unsigned long *bitmap, int bits)
+void bitmap_complement(unsigned long *dst, const unsigned long *src, int bits)
{
int k, lim = bits/BITS_PER_LONG;
for (k = 0; k < lim; ++k)
- bitmap[k] = ~bitmap[k];
+ dst[k] = ~src[k];
if (bits % BITS_PER_LONG)
- bitmap[k] = ~bitmap[k] & ((1UL << (bits % BITS_PER_LONG)) - 1);
+ dst[k] = ~src[k] & ((1UL << (bits % BITS_PER_LONG)) - 1);
}
EXPORT_SYMBOL(bitmap_complement);
@@ -144,6 +144,60 @@
}
EXPORT_SYMBOL(bitmap_or);
+void bitmap_xor(unsigned long *dst, const unsigned long *bitmap1,
+ const unsigned long *bitmap2, int bits)
+{
+ int k;
+ int nr = BITS_TO_LONGS(bits);
+
+ for (k = 0; k < nr; k++)
+ dst[k] = bitmap1[k] ^ bitmap2[k];
+}
+EXPORT_SYMBOL(bitmap_xor);
+
+void bitmap_andnot(unsigned long *dst, const unsigned long *bitmap1,
+ const unsigned long *bitmap2, int bits)
+{
+ int k;
+ int nr = BITS_TO_LONGS(bits);
+
+ for (k = 0; k < nr; k++)
+ dst[k] = bitmap1[k] & ~bitmap2[k];
+}
+EXPORT_SYMBOL(bitmap_andnot);
+
+int bitmap_intersects(const unsigned long *bitmap1,
+ const unsigned long *bitmap2, int bits)
+{
+ int k, lim = bits/BITS_PER_LONG;
+ for (k = 0; k < lim; ++k)
+ if (bitmap1[k] & bitmap2[k])
+ return 1;
+
+ if (bits % BITS_PER_LONG)
+ if ((bitmap1[k] & bitmap2[k]) &
+ ((1UL << (bits % BITS_PER_LONG)) - 1))
+ return 1;
+ return 0;
+}
+EXPORT_SYMBOL(bitmap_intersects);
+
+int bitmap_subset(const unsigned long *bitmap1,
+ const unsigned long *bitmap2, int bits)
+{
+ int k, lim = bits/BITS_PER_LONG;
+ for (k = 0; k < lim; ++k)
+ if (bitmap1[k] & ~bitmap2[k])
+ return 0;
+
+ if (bits % BITS_PER_LONG)
+ if ((bitmap1[k] & ~bitmap2[k]) &
+ ((1UL << (bits % BITS_PER_LONG)) - 1))
+ return 0;
+ return 1;
+}
+EXPORT_SYMBOL(bitmap_subset);
+
#if BITS_PER_LONG == 32
int bitmap_weight(const unsigned long *bitmap, int bits)
{
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373
mask17-cpumask-sched-refine - Cpumask code clarification in kernel/sched.c
Clarify and slightly optimize set_cpus_allowed() cpumask check
Index: 2.6.5.bitmap.v4/kernel/sched.c
===================================================================
--- 2.6.5.bitmap.v4.orig/kernel/sched.c 2004-04-21 16:45:03.000000000 -0700
+++ 2.6.5.bitmap.v4/kernel/sched.c 2004-04-21 16:45:17.000000000 -0700
@@ -2722,7 +2722,7 @@
runqueue_t *rq;
rq = task_rq_lock(p, &flags);
- if (any_online_cpu(new_mask) == NR_CPUS) {
+ if (!cpus_intersects(new_mask, cpu_online_map)) {
ret = -EINVAL;
goto out;
}
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373
mask6-unline-find-next-bit-ia64 - Uninline find_next_bit on ia64
Move the page of code (~700 bytes of instructions)
for find_next_bit and find_next_zero_bit from inline
in include/asm-ia64/bitops.h to a real function in
arch/ia64/lib/bitops.c, leaving a declaration and
macro wrapper behind.
The other arch's with almost this same code might want to
also uninline it: alpha, parisc, ppc, sh, sparc, sparc64.
These are too big to inline.
Index: 2.6.5.bitmap/include/asm-ia64/bitops.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-ia64/bitops.h 2004-04-08 03:02:48.000000000 -0700
+++ 2.6.5.bitmap/include/asm-ia64/bitops.h 2004-04-08 03:03:28.000000000 -0700
@@ -11,7 +11,7 @@
#include <linux/compiler.h>
#include <linux/types.h>
-
+#include <asm/bitops.h>
#include <asm/intrinsics.h>
/**
@@ -236,7 +236,7 @@
}
/**
- * test_and_change_bit - Change a bit and return its new value
+ * test_and_change_bit - Change a bit and return its old value
* @nr: Bit to set
* @addr: Address to count from
*
@@ -359,93 +359,21 @@
#endif /* __KERNEL__ */
-/*
- * Find next zero bit in a bitmap reasonably efficiently..
- */
-static inline int
-find_next_zero_bit (void *addr, unsigned long size, unsigned long offset)
-{
- unsigned long *p = ((unsigned long *) addr) + (offset >> 6);
- unsigned long result = offset & ~63UL;
- unsigned long tmp;
-
- if (offset >= size)
- return size;
- size -= result;
- offset &= 63UL;
- if (offset) {
- tmp = *(p++);
- tmp |= ~0UL >> (64-offset);
- if (size < 64)
- goto found_first;
- if (~tmp)
- goto found_middle;
- size -= 64;
- result += 64;
- }
- while (size & ~63UL) {
- if (~(tmp = *(p++)))
- goto found_middle;
- result += 64;
- size -= 64;
- }
- if (!size)
- return result;
- tmp = *p;
-found_first:
- tmp |= ~0UL << size;
- if (tmp == ~0UL) /* any bits zero? */
- return result + size; /* nope */
-found_middle:
- return result + ffz(tmp);
-}
+extern int __find_next_zero_bit (void *addr, unsigned long size, \
+ unsigned long offset);
+extern int __find_next_bit(const void *addr, unsigned long size, \
+ unsigned long offset);
+
+#define find_next_zero_bit(addr, size, offset) \
+ __find_next_zero_bit((addr), (size), (offset))
+#define find_next_bit(addr, size, offset) \
+ __find_next_bit((addr), (size), (offset))
/*
* The optimizer actually does good code for this case..
*/
#define find_first_zero_bit(addr, size) find_next_zero_bit((addr), (size), 0)
-/*
- * Find next bit in a bitmap reasonably efficiently..
- */
-static inline int
-find_next_bit(const void *addr, unsigned long size, unsigned long offset)
-{
- unsigned long *p = ((unsigned long *) addr) + (offset >> 6);
- unsigned long result = offset & ~63UL;
- unsigned long tmp;
-
- if (offset >= size)
- return size;
- size -= result;
- offset &= 63UL;
- if (offset) {
- tmp = *(p++);
- tmp &= ~0UL << offset;
- if (size < 64)
- goto found_first;
- if (tmp)
- goto found_middle;
- size -= 64;
- result += 64;
- }
- while (size & ~63UL) {
- if ((tmp = *(p++)))
- goto found_middle;
- result += 64;
- size -= 64;
- }
- if (!size)
- return result;
- tmp = *p;
- found_first:
- tmp &= ~0UL >> (64-size);
- if (tmp == 0UL) /* Are any bits set? */
- return result + size; /* Nope. */
- found_middle:
- return result + __ffs(tmp);
-}
-
#define find_first_bit(addr, size) find_next_bit((addr), (size), 0)
#ifdef __KERNEL__
Index: 2.6.5.bitmap/include/asm-cris/bitops.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-cris/bitops.h 2004-04-08 03:02:48.000000000 -0700
+++ 2.6.5.bitmap/include/asm-cris/bitops.h 2004-04-08 03:03:28.000000000 -0700
@@ -169,7 +169,7 @@
return retval;
}
/**
- * test_and_change_bit - Change a bit and return its new value
+ * test_and_change_bit - Change a bit and return its old value
* @nr: Bit to change
* @addr: Address to count from
*
Index: 2.6.5.bitmap/include/asm-i386/bitops.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-i386/bitops.h 2004-04-08 03:02:48.000000000 -0700
+++ 2.6.5.bitmap/include/asm-i386/bitops.h 2004-04-08 03:03:28.000000000 -0700
@@ -212,7 +212,7 @@
}
/**
- * test_and_change_bit - Change a bit and return its new value
+ * test_and_change_bit - Change a bit and return its old value
* @nr: Bit to change
* @addr: Address to count from
*
Index: 2.6.5.bitmap/include/asm-mips/bitops.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-mips/bitops.h 2004-04-08 03:02:48.000000000 -0700
+++ 2.6.5.bitmap/include/asm-mips/bitops.h 2004-04-08 03:03:28.000000000 -0700
@@ -296,7 +296,7 @@
}
/*
- * test_and_change_bit - Change a bit and return its new value
+ * test_and_change_bit - Change a bit and return its old value
* @nr: Bit to change
* @addr: Address to count from
*
@@ -567,7 +567,7 @@
}
/*
- * test_and_change_bit - Change a bit and return its new value
+ * test_and_change_bit - Change a bit and return its old value
* @nr: Bit to change
* @addr: Address to count from
*
Index: 2.6.5.bitmap/include/asm-x86_64/bitops.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-x86_64/bitops.h 2004-04-08 03:02:48.000000000 -0700
+++ 2.6.5.bitmap/include/asm-x86_64/bitops.h 2004-04-08 03:03:28.000000000 -0700
@@ -204,7 +204,7 @@
}
/**
- * test_and_change_bit - Change a bit and return its new value
+ * test_and_change_bit - Change a bit and return its old value
* @nr: Bit to change
* @addr: Address to count from
*
Index: 2.6.5.bitmap/arch/ia64/lib/bitop.c
===================================================================
--- 2.6.5.bitmap.orig/arch/ia64/lib/bitop.c 2004-04-08 03:03:18.000000000 -0700
+++ 2.6.5.bitmap/arch/ia64/lib/bitop.c 2004-04-08 03:03:28.000000000 -0700
@@ -0,0 +1,88 @@
+#include <linux/compiler.h>
+#include <linux/types.h>
+#include <asm/intrinsics.h>
+#include <linux/module.h>
+#include <asm/bitops.h>
+
+/*
+ * Find next zero bit in a bitmap reasonably efficiently..
+ */
+
+int __find_next_zero_bit (void *addr, unsigned long size, unsigned long offset)
+{
+ unsigned long *p = ((unsigned long *) addr) + (offset >> 6);
+ unsigned long result = offset & ~63UL;
+ unsigned long tmp;
+
+ if (offset >= size)
+ return size;
+ size -= result;
+ offset &= 63UL;
+ if (offset) {
+ tmp = *(p++);
+ tmp |= ~0UL >> (64-offset);
+ if (size < 64)
+ goto found_first;
+ if (~tmp)
+ goto found_middle;
+ size -= 64;
+ result += 64;
+ }
+ while (size & ~63UL) {
+ if (~(tmp = *(p++)))
+ goto found_middle;
+ result += 64;
+ size -= 64;
+ }
+ if (!size)
+ return result;
+ tmp = *p;
+found_first:
+ tmp |= ~0UL << size;
+ if (tmp == ~0UL) /* any bits zero? */
+ return result + size; /* nope */
+found_middle:
+ return result + ffz(tmp);
+}
+EXPORT_SYMBOL(__find_next_zero_bit);
+
+/*
+ * Find next bit in a bitmap reasonably efficiently..
+ */
+int __find_next_bit(const void *addr, unsigned long size, unsigned long offset)
+{
+ unsigned long *p = ((unsigned long *) addr) + (offset >> 6);
+ unsigned long result = offset & ~63UL;
+ unsigned long tmp;
+
+ if (offset >= size)
+ return size;
+ size -= result;
+ offset &= 63UL;
+ if (offset) {
+ tmp = *(p++);
+ tmp &= ~0UL << offset;
+ if (size < 64)
+ goto found_first;
+ if (tmp)
+ goto found_middle;
+ size -= 64;
+ result += 64;
+ }
+ while (size & ~63UL) {
+ if ((tmp = *(p++)))
+ goto found_middle;
+ result += 64;
+ size -= 64;
+ }
+ if (!size)
+ return result;
+ tmp = *p;
+ found_first:
+ tmp &= ~0UL >> (64-size);
+ if (tmp == 0UL) /* Are any bits set? */
+ return result + size; /* Nope. */
+ found_middle:
+ return result + __ffs(tmp);
+}
+EXPORT_SYMBOL(__find_next_bit);
Index: 2.6.5.bitmap/arch/ia64/lib/Makefile
===================================================================
--- 2.6.5.bitmap.orig/arch/ia64/lib/Makefile 2004-04-08 03:01:12.000000000 -0700
+++ 2.6.5.bitmap/arch/ia64/lib/Makefile 2004-04-08 03:03:28.000000000 -0700
@@ -6,7 +6,7 @@
lib-y := __divsi3.o __udivsi3.o __modsi3.o __umodsi3.o \
__divdi3.o __udivdi3.o __moddi3.o __umoddi3.o \
- checksum.o clear_page.o csum_partial_copy.o copy_page.o \
+ bitop.o checksum.o clear_page.o csum_partial_copy.o copy_page.o \
clear_user.o strncpy_from_user.o strlen_user.o strnlen_user.o \
flush.o ip_fast_csum.o do_csum.o \
memset.o strlen.o swiotlb.o
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373
mask16-cpumask-x86-64-online - Remove cpumask hack from asm-x86_64/topology.h
This file had the cpumask cpu_online_map as type
unsigned long, instead of type cpumask_t, for no good
reason that I could see. So I changed it. Everywhere
else, cpu_online_map is already of type cpumask_t.
Index: 2.6.5.bitmap/include/asm-x86_64/topology.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-x86_64/topology.h 2004-04-05 02:41:33.000000000 -0700
+++ 2.6.5.bitmap/include/asm-x86_64/topology.h 2004-04-08 04:23:24.000000000 -0700
@@ -10,18 +10,18 @@
/* Map the K8 CPU local memory controllers to a simple 1:1 CPU:NODE topology */
extern int fake_node;
-/* This is actually a cpumask_t, but doesn't matter because we don't have
- >BITS_PER_LONG CPUs */
-extern unsigned long cpu_online_map;
+extern cpumask_t cpu_online_map;
#define cpu_to_node(cpu) (fake_node ? 0 : (cpu))
#define parent_node(node) (node)
#define node_to_first_cpu(node) (fake_node ? 0 : (node))
#define node_to_cpumask(node) (fake_node ? cpu_online_map : (1UL << (node)))
-static inline unsigned long pcibus_to_cpumask(int bus)
+static inline cpumask_t pcibus_to_cpumask(int bus)
{
- return mp_bus_to_cpumask[bus] & cpu_online_map;
+ cpumask_t tmp;
+ cpus_and(tmp, mp_bus_to_cpumask[bus], cpu_online_map);
+ return tmp;
}
#define NODE_BALANCE_RATE 30 /* CHECKME */
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373
mask4-bitmap-const - two missing 'const' qualifiers in bitops/bitmap
Add a couple of missing 'const' qualifiers on
bitops test_bit and bitmap_equal args.
Index: 2.6.5.bitmap/include/asm-generic/bitops.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-generic/bitops.h 2004-04-05 02:41:32.000000000 -0700
+++ 2.6.5.bitmap/include/asm-generic/bitops.h 2004-04-05 03:15:17.000000000 -0700
@@ -42,7 +42,7 @@
return retval;
}
-extern __inline__ int test_bit(int nr, long * addr)
+extern __inline__ int test_bit(int nr, const unsigned long * addr)
{
int mask;
Index: 2.6.5.bitmap/lib/bitmap.c
===================================================================
--- 2.6.5.bitmap.orig/lib/bitmap.c 2004-04-05 03:12:27.000000000 -0700
+++ 2.6.5.bitmap/lib/bitmap.c 2004-04-05 03:15:17.000000000 -0700
@@ -65,7 +65,7 @@
EXPORT_SYMBOL(bitmap_full);
int bitmap_equal(const unsigned long *bitmap1,
- unsigned long *bitmap2, int bits)
+ const unsigned long *bitmap2, int bits)
{
int k, lim = bits/BITS_PER_LONG;
for (k = 0; k < lim; ++k)
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373
mask14-cpumask-i386-simplify - Optimize i386 cpumask macro usage.
Optimize a bit of cpumask code for asm-i386/mach-es7000
Code untested, unreviewed. Feedback welcome.
Index: 2.6.5.bitmap/include/asm-i386/mach-es7000/mach_ipi.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-i386/mach-es7000/mach_ipi.h 2004-04-08 04:19:28.000000000 -0700
+++ 2.6.5.bitmap/include/asm-i386/mach-es7000/mach_ipi.h 2004-04-08 04:19:34.000000000 -0700
@@ -10,9 +10,8 @@
static inline void send_IPI_allbutself(int vector)
{
- cpumask_t mask = cpumask_of_cpu(smp_processor_id());
- cpus_complement(mask);
- cpus_and(mask, mask, cpu_online_map);
+ cpumask_t mask = cpu_online_map;
+ cpu_clear(smp_processor_id(), mask);
if (!cpus_empty(mask))
send_IPI_mask(mask, vector);
}
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373
mask2-bitmap-complement - Dont generate nonzero unused bits in bitmap
Tighten up bitmap so it does not generate nonzero bits
in the unused tail if it is not given any on input.
Index: 2.6.5.bitmap/lib/bitmap.c
===================================================================
--- 2.6.5.bitmap.orig/lib/bitmap.c 2004-04-05 02:50:25.000000000 -0700
+++ 2.6.5.bitmap/lib/bitmap.c 2004-04-05 03:02:39.000000000 -0700
@@ -26,10 +26,10 @@
* carefully filter out these unused bits from impacting their
* results.
*
- * Except for bitmap_complement, these operations hold to a
- * slightly stronger rule: if you don't input any bitmaps to
- * these ops that have some unused bits set, then they won't
- * output any set unused bits in output bitmaps.
+ * These operations actually hold to a slightly stronger rule:
+ * if you don't input any bitmaps to these ops that have some
+ * unused bits set, then they won't output any set unused bits
+ * in output bitmaps.
*/
#define MAX_BITMAP_BITS 512U /* for ia64 NR_CPUS maximum */
@@ -83,11 +83,12 @@
void bitmap_complement(unsigned long *bitmap, int bits)
{
- int k;
- int nr = BITS_TO_LONGS(bits);
-
- for (k = 0; k < nr; ++k)
+ int k, lim = bits/BITS_PER_LONG;
+ for (k = 0; k < lim; ++k)
bitmap[k] = ~bitmap[k];
+
+ if (bits % BITS_PER_LONG)
+ bitmap[k] = ~bitmap[k] & ((1UL << (bits % BITS_PER_LONG)) - 1);
}
EXPORT_SYMBOL(bitmap_complement);
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373
mask5-bitmap-extensions - Optimize and extend bitmap.
This bitmap improvements make it a suitable basis for
fully supporting cpumask_t and nodemask_t. Inline macros
with compile-time checks enable generating tight code on
both small and large systems (large meaning cpumask_t
requires more than one unsigned long's worth of bits).
The existing bitmap_<op> macros in lib/bitmap.c
are renamed to __bitmap_<op>, and wrappers for each
bitmap_<op> are exposed in include/linux/bitmap.h
This patch _includes_ Bill Irwins rewrite of the
bitmap_shift operators to not require a fixed length
intermediate bitmap.
Improved comments list each available operator for easy
browsing.
Index: 2.6.5.bitmap/include/linux/bitmap.h
===================================================================
--- 2.6.5.bitmap.orig/include/linux/bitmap.h 2004-04-07 21:22:32.000000000 -0700
+++ 2.6.5.bitmap/include/linux/bitmap.h 2004-04-07 21:23:52.000000000 -0700
@@ -1,59 +1,250 @@
-#ifndef __LINUX_BITMAP_H
-#define __LINUX_BITMAP_H
+#ifndef __LINUX__bitmap_H
+#define __LINUX__bitmap_H
#ifndef __ASSEMBLY__
-#include <linux/config.h>
-#include <linux/compiler.h>
#include <linux/types.h>
-#include <linux/kernel.h>
#include <linux/bitops.h>
#include <linux/string.h>
-int bitmap_empty(const unsigned long *bitmap, int bits);
-int bitmap_full(const unsigned long *bitmap, int bits);
-int bitmap_equal(const unsigned long *bitmap1,
- const unsigned long *bitmap2, int bits);
-void bitmap_complement(unsigned long *dst, const unsigned long *src, int bits);
+/*
+ * bitmaps provide bit arrays that consume one or more unsigned
+ * longs. The bitmap interface and available operations are listed
+ * here, in bitmap.h
+ *
+ * Function implementations generic to all architectures are in
+ * lib/bitmap.c. Functions implementations that are architecture
+ * specific are in various include/asm-<arch>/bitops.h headers
+ * and other arch/<arch> specific files.
+ *
+ * See lib/bitmap.c for more details.
+ */
+
+/*
+ * The available bitmap operations and their rough meaning in the
+ * case that the bitmap is a single unsigned long are thus:
+ *
+ * bitmap_clear(dst, nbits) *dst = 0UL
+ * bitmap_fill(dst, nbits) *dst = ~0UL
+ * bitmap_copy(dst, src, nbits) *dst = *src
+ * bitmap_and(dst, src1, src2, nbits) *dst = *src1 & *src2
+ * bitmap_or(dst, src1, src2, nbits) *dst = *src1 | *src2
+ * bitmap_xor(dst, src1, src2, nbits) *dst = *src1 ^ *src2
+ * bitmap_andnot(dst, src1, src2, nbits) *dst = *src1 & ~(*src2)
+ * bitmap_complement(dst, src, nbits) *dst = ~(*src)
+ * bitmap_equal(src1, src2, nbits) Are *src1 and *src2 equal?
+ * bitmap_intersects(src1, src2, nbits) Do *src1 and *src2 overlap?
+ * bitmap_subset(src1, src2, nbits) Is *src1 a subset of *src2?
+ * bitmap_empty(src, nbits) Are all bits zero in *src?
+ * bitmap_full(src, nbits) Are all bits set in *src?
+ * bitmap_weight(src, nbits) Hamming Weight: number set bits
+ * bitmap_shift_right(dst, src, n, nbits) *dst = *src >> n
+ * bitmap_shift_left(dst, src, n, nbits) *dst = *src << n
+ * bitmap_scnprintf(buf, len, src, nbits) Print bitmap src to buf
+ * bitmap_parse(ubuf, ulen, dst, nbits) Parse bitmap dst from buf
+ */
+
+/*
+ * Also the following operations in asm/bitops.h apply to bitmaps.
+ *
+ * set_bit(bit, addr) *addr |= bit
+ * clear_bit(bit, addr) *addr &= ~bit
+ * change_bit(bit, addr) *addr ^= bit
+ * test_bit(bit, addr) Is bit set in *addr?
+ * test_and_set_bit(bit, addr) Set bit and return old value
+ * test_and_clear_bit(bit, addr) Clear bit and return old value
+ * test_and_change_bit(bit, addr) Change bit and return old value
+ * find_first_zero_bit(addr, nbits) Position first zero bit in *addr
+ * find_first_bit(addr, nbits) Position first set bit in *addr
+ * find_next_zero_bit(addr, nbits, bit) Position next zero bit in *addr >= bit
+ * find_next_bit(addr, nbits, bit) Position next set bit in *addr >= bit
+ */
+
+/*
+ * The DECLARE_BITMAP(name,bits) macro, in linux/types.h, can be used
+ * to declare an array named 'name' of just enough unsigned longs to
+ * contain all bit positions from 0 to 'bits' - 1.
+ */
+
+/*
+ * lib/bitmap.c provides these functions:
+ */
+
+extern int __bitmap_empty(const unsigned long *bitmap, int bits);
+extern int __bitmap_full(const unsigned long *bitmap, int bits);
+extern int __bitmap_equal(const unsigned long *bitmap1,
+ const unsigned long *bitmap2, int bits);
+extern void __bitmap_complement(unsigned long *dst, const unsigned long *src,
+ int bits);
+extern void __bitmap_shift_right(unsigned long *dst,
+ const unsigned long *src, int shift, int bits);
+extern void __bitmap_shift_left(unsigned long *dst,
+ const unsigned long *src, int shift, int bits);
+extern void __bitmap_and(unsigned long *dst, const unsigned long *bitmap1,
+ const unsigned long *bitmap2, int bits);
+extern void __bitmap_or(unsigned long *dst, const unsigned long *bitmap1,
+ const unsigned long *bitmap2, int bits);
+extern void __bitmap_xor(unsigned long *dst, const unsigned long *bitmap1,
+ const unsigned long *bitmap2, int bits);
+extern void __bitmap_andnot(unsigned long *dst, const unsigned long *bitmap1,
+ const unsigned long *bitmap2, int bits);
+extern int __bitmap_intersects(const unsigned long *bitmap1,
+ const unsigned long *bitmap2, int bits);
+extern int __bitmap_subset(const unsigned long *bitmap1,
+ const unsigned long *bitmap2, int bits);
+extern int __bitmap_weight(const unsigned long *bitmap, int bits);
+
+extern int bitmap_scnprintf(char *buf, unsigned int len,
+ const unsigned long *src, int nbits);
+extern int bitmap_parse(const char __user *ubuf, unsigned int ulen,
+ unsigned long *dst, int nbits);
+
+#define BITMAP_LAST_WORD_MASK(nbits) \
+( \
+ ((nbits) % BITS_PER_LONG) ? \
+ (1UL<<((nbits) % BITS_PER_LONG))-1 : ~0UL \
+)
-static inline void bitmap_clear(unsigned long *bitmap, int bits)
+static inline void bitmap_clear(unsigned long *dst, int nbits)
{
- CLEAR_BITMAP((unsigned long *)bitmap, bits);
+ if (nbits <= BITS_PER_LONG)
+ *dst = 0UL;
+ else
+ CLEAR_BITMAP(dst, nbits);
}
-static inline void bitmap_fill(unsigned long *bitmap, int bits)
+static inline void bitmap_fill(unsigned long *dst, int nbits)
{
- memset(bitmap, 0xff, BITS_TO_LONGS(bits)*sizeof(unsigned long));
+ size_t nlongs = BITS_TO_LONGS(nbits);
+ if (nlongs > 1) {
+ int len = (nlongs - 1) * sizeof(unsigned long);
+ memset(dst, 0xff, len);
+ }
+ dst[nlongs - 1] = BITMAP_LAST_WORD_MASK(nbits);
}
-static inline void bitmap_copy(unsigned long *dst,
- const unsigned long *src, int bits)
+static inline void bitmap_copy(unsigned long *dst, const unsigned long *src,
+ int nbits)
{
- memcpy(dst, src, BITS_TO_LONGS(bits)*sizeof(unsigned long));
+ if (nbits <= BITS_PER_LONG)
+ *dst = *src;
+ else {
+ int len = BITS_TO_LONGS(nbits) * sizeof(unsigned long);
+ memcpy(dst, src, len);
+ }
}
-void bitmap_shift_right(unsigned long *dst,
- const unsigned long *src, int shift, int bits);
-void bitmap_shift_left(unsigned long *dst,
- const unsigned long *src, int shift, int bits);
-void bitmap_and(unsigned long *dst, const unsigned long *bitmap1,
- const unsigned long *bitmap2, int bits);
-void bitmap_or(unsigned long *dst, const unsigned long *bitmap1,
- const unsigned long *bitmap2, int bits);
-void bitmap_xor(unsigned long *dst, const unsigned long *bitmap1,
- const unsigned long *bitmap2, int bits);
-void bitmap_andnot(unsigned long *dst, const unsigned long *bitmap1,
- const unsigned long *bitmap2, int bits);
-int bitmap_intersects(const unsigned long *bitmap1,
- const unsigned long *bitmap2, int bits);
-int bitmap_subset(const unsigned long *bitmap1,
- const unsigned long *bitmap2, int bits);
-int bitmap_weight(const unsigned long *bitmap, int bits);
-int bitmap_scnprintf(char *buf, unsigned int buflen,
- const unsigned long *maskp, int bits);
-int bitmap_parse(const char __user *ubuf, unsigned int ubuflen,
- unsigned long *maskp, int bits);
+static inline void bitmap_and(unsigned long *dst, const unsigned long *src1,
+ const unsigned long *src2, int nbits)
+{
+ if (nbits <= BITS_PER_LONG)
+ *dst = *src1 & *src2;
+ else
+ __bitmap_and(dst, src1, src2, nbits);
+}
+
+static inline void bitmap_or(unsigned long *dst, const unsigned long *src1,
+ const unsigned long *src2, int nbits)
+{
+ if (nbits <= BITS_PER_LONG)
+ *dst = *src1 | *src2;
+ else
+ __bitmap_or(dst, src1, src2, nbits);
+}
+
+static inline void bitmap_xor(unsigned long *dst, const unsigned long *src1,
+ const unsigned long *src2, int nbits)
+{
+ if (nbits <= BITS_PER_LONG)
+ *dst = *src1 ^ *src2;
+ else
+ __bitmap_xor(dst, src1, src2, nbits);
+}
+
+static inline void bitmap_andnot(unsigned long *dst, const unsigned long *src1,
+ const unsigned long *src2, int nbits)
+{
+ if (nbits <= BITS_PER_LONG)
+ *dst = *src1 & ~(*src2);
+ else
+ __bitmap_andnot(dst, src1, src2, nbits);
+}
+
+static inline void bitmap_complement(unsigned long *dst, const unsigned long *src,
+ int nbits)
+{
+ if (nbits <= BITS_PER_LONG)
+ *dst = ~(*src) & BITMAP_LAST_WORD_MASK(nbits);
+ else
+ __bitmap_complement(dst, src, nbits);
+}
+
+static inline int bitmap_equal(const unsigned long *src1,
+ const unsigned long *src2, int nbits)
+{
+ if (nbits <= BITS_PER_LONG)
+ return ! ((*src1 ^ *src2) & BITMAP_LAST_WORD_MASK(nbits));
+ else
+ return __bitmap_equal(src1, src2, nbits);
+}
+
+static inline int bitmap_intersects(const unsigned long *src1,
+ const unsigned long *src2, int nbits)
+{
+ if (nbits <= BITS_PER_LONG)
+ return ((*src1 & *src2) & BITMAP_LAST_WORD_MASK(nbits)) != 0;
+ else
+ return __bitmap_intersects(src1, src2, nbits);
+}
+
+static inline int bitmap_subset(const unsigned long *src1,
+ const unsigned long *src2, int nbits)
+{
+ if (nbits <= BITS_PER_LONG)
+ return ! ((*src1 & ~(*src2)) & BITMAP_LAST_WORD_MASK(nbits));
+ else
+ return __bitmap_subset(src1, src2, nbits);
+}
+
+static inline int bitmap_empty(const unsigned long *src, int nbits)
+{
+ if (nbits <= BITS_PER_LONG)
+ return ! (*src & BITMAP_LAST_WORD_MASK(nbits));
+ else
+ return __bitmap_empty(src, nbits);
+}
+
+static inline int bitmap_full(const unsigned long *src, int nbits)
+{
+ if (nbits <= BITS_PER_LONG)
+ return ! (~(*src) & BITMAP_LAST_WORD_MASK(nbits));
+ else
+ return __bitmap_full(src, nbits);
+}
+
+static inline int bitmap_weight(const unsigned long *src, int nbits)
+{
+ return __bitmap_weight(src, nbits);
+}
+
+static inline void bitmap_shift_right(unsigned long *dst,
+ const unsigned long *src, int n, int nbits)
+{
+ if (nbits <= BITS_PER_LONG)
+ *dst = *src >> n;
+ else
+ __bitmap_shift_right(dst, src, n, nbits);
+}
+
+static inline void bitmap_shift_left(unsigned long *dst,
+ const unsigned long *src, int n, int nbits)
+{
+ if (nbits <= BITS_PER_LONG)
+ *dst = (*src << n) & BITMAP_LAST_WORD_MASK(nbits);
+ else
+ __bitmap_shift_left(dst, src, n, nbits);
+}
#endif /* __ASSEMBLY__ */
-#endif /* __LINUX_BITMAP_H */
+#endif /* __LINUX__bitmap_H */
Index: 2.6.5.bitmap/lib/bitmap.c
===================================================================
--- 2.6.5.bitmap.orig/lib/bitmap.c 2004-04-07 21:22:32.000000000 -0700
+++ 2.6.5.bitmap/lib/bitmap.c 2004-04-07 21:23:52.000000000 -0700
@@ -15,7 +15,8 @@
/*
* bitmaps provide an array of bits, implemented using an an
* array of unsigned longs. The number of valid bits in a
- * given bitmap need not be an exact multiple of BITS_PER_LONG.
+ * given bitmap does _not_ need to be an exact multiple of
+ * BITS_PER_LONG.
*
* The possible unused bits in the last, partially used word
* of a bitmap are 'don't care'. The implementation makes
@@ -30,11 +31,14 @@
* if you don't input any bitmaps to these ops that have some
* unused bits set, then they won't output any set unused bits
* in output bitmaps.
+ *
+ * The byte ordering of bitmaps is more natural on little
+ * endian architectures. See the big-endian headers
+ * include/asm-ppc64/bitops.h and include/asm-s390/bitops.h
+ * for the best explanations of this ordering.
*/
-#define MAX_BITMAP_BITS 512U /* for ia64 NR_CPUS maximum */
-
-int bitmap_empty(const unsigned long *bitmap, int bits)
+int __bitmap_empty(const unsigned long *bitmap, int bits)
{
int k, lim = bits/BITS_PER_LONG;
for (k = 0; k < lim; ++k)
@@ -42,14 +46,14 @@
return 0;
if (bits % BITS_PER_LONG)
- if (bitmap[k] & ((1UL << (bits % BITS_PER_LONG)) - 1))
+ if (bitmap[k] & BITMAP_LAST_WORD_MASK(bits))
return 0;
return 1;
}
-EXPORT_SYMBOL(bitmap_empty);
+EXPORT_SYMBOL(__bitmap_empty);
-int bitmap_full(const unsigned long *bitmap, int bits)
+int __bitmap_full(const unsigned long *bitmap, int bits)
{
int k, lim = bits/BITS_PER_LONG;
for (k = 0; k < lim; ++k)
@@ -57,14 +61,14 @@
return 0;
if (bits % BITS_PER_LONG)
- if (~bitmap[k] & ((1UL << (bits % BITS_PER_LONG)) - 1))
+ if (~bitmap[k] & BITMAP_LAST_WORD_MASK(bits))
return 0;
return 1;
}
-EXPORT_SYMBOL(bitmap_full);
+EXPORT_SYMBOL(__bitmap_full);
-int bitmap_equal(const unsigned long *bitmap1,
+int __bitmap_equal(const unsigned long *bitmap1,
const unsigned long *bitmap2, int bits)
{
int k, lim = bits/BITS_PER_LONG;
@@ -73,56 +77,85 @@
return 0;
if (bits % BITS_PER_LONG)
- if ((bitmap1[k] ^ bitmap2[k]) &
- ((1UL << (bits % BITS_PER_LONG)) - 1))
+ if ((bitmap1[k] ^ bitmap2[k]) & BITMAP_LAST_WORD_MASK(bits))
return 0;
return 1;
}
-EXPORT_SYMBOL(bitmap_equal);
+EXPORT_SYMBOL(__bitmap_equal);
-void bitmap_complement(unsigned long *dst, const unsigned long *src, int bits)
+void __bitmap_complement(unsigned long *dst, const unsigned long *src, int bits)
{
int k, lim = bits/BITS_PER_LONG;
for (k = 0; k < lim; ++k)
dst[k] = ~src[k];
if (bits % BITS_PER_LONG)
- dst[k] = ~src[k] & ((1UL << (bits % BITS_PER_LONG)) - 1);
+ dst[k] = ~src[k] & BITMAP_LAST_WORD_MASK(bits);
}
-EXPORT_SYMBOL(bitmap_complement);
+EXPORT_SYMBOL(__bitmap_complement);
-void bitmap_shift_right(unsigned long *dst,
+void __bitmap_shift_right(unsigned long *dst,
const unsigned long *src, int shift, int bits)
{
- int k;
- DECLARE_BITMAP(__shr_tmp, MAX_BITMAP_BITS);
-
- BUG_ON(bits > MAX_BITMAP_BITS);
- bitmap_clear(__shr_tmp, bits);
- for (k = 0; k < bits - shift; ++k)
- if (test_bit(k + shift, src))
- set_bit(k, __shr_tmp);
- bitmap_copy(dst, __shr_tmp, bits);
+ int k, lim = BITS_TO_LONGS(bits), left = bits % BITS_PER_LONG;
+ int off = shift/BITS_PER_LONG, rem = shift % BITS_PER_LONG;
+ unsigned long mask = (1UL << left) - 1;
+ for (k = 0; off + k < lim; ++k) {
+ unsigned long upper, lower;
+
+ /*
+ * If shift is not word aligned, take lower rem bits of
+ * word above and make them the top rem bits of result.
+ */
+ if (!rem || off + k + 1 >= lim)
+ upper = 0;
+ else {
+ upper = src[off + k + 1];
+ if (off + k + 1 == lim - 1 && left)
+ upper &= mask;
+ }
+ lower = src[off + k];
+ if (left && off + k == lim - 1)
+ lower &= mask;
+ dst[k] = upper << (BITS_PER_LONG - rem) | lower >> rem;
+ if (left && k == lim - 1)
+ dst[k] &= mask;
+ }
+ if (off)
+ memset(&dst[lim - off], 0, off*sizeof(unsigned long));
}
-EXPORT_SYMBOL(bitmap_shift_right);
+EXPORT_SYMBOL(__bitmap_shift_right);
-void bitmap_shift_left(unsigned long *dst,
+void __bitmap_shift_left(unsigned long *dst,
const unsigned long *src, int shift, int bits)
{
- int k;
- DECLARE_BITMAP(__shl_tmp, MAX_BITMAP_BITS);
-
- BUG_ON(bits > MAX_BITMAP_BITS);
- bitmap_clear(__shl_tmp, bits);
- for (k = bits; k >= shift; --k)
- if (test_bit(k - shift, src))
- set_bit(k, __shl_tmp);
- bitmap_copy(dst, __shl_tmp, bits);
+ int k, lim = BITS_TO_LONGS(bits), left = bits % BITS_PER_LONG;
+ int off = shift/BITS_PER_LONG, rem = shift % BITS_PER_LONG;
+ for (k = lim - off - 1; k >= 0; --k) {
+ unsigned long upper, lower;
+
+ /*
+ * If shift is not word aligned, take upper rem bits of
+ * word below and make them the bottom rem bits of result.
+ */
+ if (rem && k > 0)
+ lower = src[k - 1];
+ else
+ lower = 0;
+ upper = src[k];
+ if (left && k == lim - 1)
+ upper &= (1UL << left) - 1;
+ dst[k + off] = lower >> (BITS_PER_LONG - rem) | upper << rem;
+ if (left && k + off == lim - 1)
+ dst[k + off] &= (1UL << left) - 1;
+ }
+ if (off)
+ memset(dst, 0, off*sizeof(unsigned long));
}
-EXPORT_SYMBOL(bitmap_shift_left);
+EXPORT_SYMBOL(__bitmap_shift_left);
-void bitmap_and(unsigned long *dst, const unsigned long *bitmap1,
+void __bitmap_and(unsigned long *dst, const unsigned long *bitmap1,
const unsigned long *bitmap2, int bits)
{
int k;
@@ -131,9 +164,9 @@
for (k = 0; k < nr; k++)
dst[k] = bitmap1[k] & bitmap2[k];
}
-EXPORT_SYMBOL(bitmap_and);
+EXPORT_SYMBOL(__bitmap_and);
-void bitmap_or(unsigned long *dst, const unsigned long *bitmap1,
+void __bitmap_or(unsigned long *dst, const unsigned long *bitmap1,
const unsigned long *bitmap2, int bits)
{
int k;
@@ -142,9 +175,9 @@
for (k = 0; k < nr; k++)
dst[k] = bitmap1[k] | bitmap2[k];
}
-EXPORT_SYMBOL(bitmap_or);
+EXPORT_SYMBOL(__bitmap_or);
-void bitmap_xor(unsigned long *dst, const unsigned long *bitmap1,
+void __bitmap_xor(unsigned long *dst, const unsigned long *bitmap1,
const unsigned long *bitmap2, int bits)
{
int k;
@@ -153,9 +186,9 @@
for (k = 0; k < nr; k++)
dst[k] = bitmap1[k] ^ bitmap2[k];
}
-EXPORT_SYMBOL(bitmap_xor);
+EXPORT_SYMBOL(__bitmap_xor);
-void bitmap_andnot(unsigned long *dst, const unsigned long *bitmap1,
+void __bitmap_andnot(unsigned long *dst, const unsigned long *bitmap1,
const unsigned long *bitmap2, int bits)
{
int k;
@@ -164,9 +197,9 @@
for (k = 0; k < nr; k++)
dst[k] = bitmap1[k] & ~bitmap2[k];
}
-EXPORT_SYMBOL(bitmap_andnot);
+EXPORT_SYMBOL(__bitmap_andnot);
-int bitmap_intersects(const unsigned long *bitmap1,
+int __bitmap_intersects(const unsigned long *bitmap1,
const unsigned long *bitmap2, int bits)
{
int k, lim = bits/BITS_PER_LONG;
@@ -175,14 +208,13 @@
return 1;
if (bits % BITS_PER_LONG)
- if ((bitmap1[k] & bitmap2[k]) &
- ((1UL << (bits % BITS_PER_LONG)) - 1))
+ if ((bitmap1[k] & bitmap2[k]) & BITMAP_LAST_WORD_MASK(bits))
return 1;
return 0;
}
-EXPORT_SYMBOL(bitmap_intersects);
+EXPORT_SYMBOL(__bitmap_intersects);
-int bitmap_subset(const unsigned long *bitmap1,
+int __bitmap_subset(const unsigned long *bitmap1,
const unsigned long *bitmap2, int bits)
{
int k, lim = bits/BITS_PER_LONG;
@@ -191,15 +223,14 @@
return 0;
if (bits % BITS_PER_LONG)
- if ((bitmap1[k] & ~bitmap2[k]) &
- ((1UL << (bits % BITS_PER_LONG)) - 1))
+ if ((bitmap1[k] & ~bitmap2[k]) & BITMAP_LAST_WORD_MASK(bits))
return 0;
return 1;
}
-EXPORT_SYMBOL(bitmap_subset);
+EXPORT_SYMBOL(__bitmap_subset);
#if BITS_PER_LONG == 32
-int bitmap_weight(const unsigned long *bitmap, int bits)
+int __bitmap_weight(const unsigned long *bitmap, int bits)
{
int k, w = 0, lim = bits/BITS_PER_LONG;
@@ -207,13 +238,12 @@
w += hweight32(bitmap[k]);
if (bits % BITS_PER_LONG)
- w += hweight32(bitmap[k] &
- ((1UL << (bits % BITS_PER_LONG)) - 1));
+ w += hweight32(bitmap[k] & BITMAP_LAST_WORD_MASK(bits));
return w;
}
#else
-int bitmap_weight(const unsigned long *bitmap, int bits)
+int __bitmap_weight(const unsigned long *bitmap, int bits)
{
int k, w = 0, lim = bits/BITS_PER_LONG;
@@ -221,13 +251,12 @@
w += hweight64(bitmap[k]);
if (bits % BITS_PER_LONG)
- w += hweight64(bitmap[k] &
- ((1UL << (bits % BITS_PER_LONG)) - 1));
+ w += hweight64(bitmap[k] & BITMAP_LAST_WORD_MASK(bits));
return w;
}
#endif
-EXPORT_SYMBOL(bitmap_weight);
+EXPORT_SYMBOL(__bitmap_weight);
/*
* Bitmap printing & parsing functions: first version by Bill Irwin,
@@ -344,7 +373,7 @@
if (nchunks == 0 && chunk == 0)
continue;
- bitmap_shift_right(maskp, maskp, CHUNKSZ, nmaskbits);
+ __bitmap_shift_right(maskp, maskp, CHUNKSZ, nmaskbits);
*maskp |= chunk;
nchunks++;
nbits += (nchunks == 1) ? nbits_to_hold_value(chunk) : CHUNKSZ;
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373
mask15-cpumask-physids-complement - Convert physids_complement() to really use both args
Provide for specifying distinct source and dest args to the
physids_complement(). No one actually uses this macro yet.
The physid_mask type would be a good candidate to convert to
using this new mask ADT as a base.
Index: 2.6.5.bitmap/include/asm-i386/mpspec.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-i386/mpspec.h 2004-04-08 03:50:33.000000000 -0700
+++ 2.6.5.bitmap/include/asm-i386/mpspec.h 2004-04-08 04:21:18.000000000 -0700
@@ -53,7 +53,7 @@
#define physids_and(dst, src1, src2) bitmap_and((dst).mask, (src1).mask, (src2).mask, MAX_APICS)
#define physids_or(dst, src1, src2) bitmap_or((dst).mask, (src1).mask, (src2).mask, MAX_APICS)
#define physids_clear(map) bitmap_clear((map).mask, MAX_APICS)
-#define physids_complement(map) bitmap_complement((map).mask, (map).mask, MAX_APICS)
+#define physids_complement(dst, src) bitmap_complement((dst).mask, (src).mask, MAX_APICS)
#define physids_empty(map) bitmap_empty((map).mask, MAX_APICS)
#define physids_equal(map1, map2) bitmap_equal((map1).mask, (map2).mask, MAX_APICS)
#define physids_weight(map) bitmap_weight((map).mask, MAX_APICS)
Index: 2.6.5.bitmap/include/asm-x86_64/mpspec.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-x86_64/mpspec.h 2004-04-08 03:50:33.000000000 -0700
+++ 2.6.5.bitmap/include/asm-x86_64/mpspec.h 2004-04-08 04:21:18.000000000 -0700
@@ -212,7 +212,7 @@
#define physids_and(dst, src1, src2) bitmap_and((dst).mask, (src1).mask, (src2).mask, MAX_APICS)
#define physids_or(dst, src1, src2) bitmap_or((dst).mask, (src1).mask, (src2).mask, MAX_APICS)
#define physids_clear(map) bitmap_clear((map).mask, MAX_APICS)
-#define physids_complement(map) bitmap_complement((map).mask, (map).mask, MAX_APICS)
+#define physids_complement(dst, src) bitmap_complement((dst).mask, (src).mask, MAX_APICS)
#define physids_empty(map) bitmap_empty((map).mask, MAX_APICS)
#define physids_equal(map1, map2) bitmap_equal((map1).mask, (map2).mask, MAX_APICS)
#define physids_weight(map) bitmap_weight((map).mask, MAX_APICS)
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373
mask9-cpumask-i386-fixup - Remove/recode obsolete cpumask macros from arch i386
Remove by recoding all uses of the obsolete cpumask const,
coerce and promote macros.
Diffstat Patch_7_of_23:
arch/i386/kernel/io_apic.c | 2 +-
arch/i386/kernel/smp.c | 2 +-
arch/i386/mach-voyager/voyager_smp.c | 30 +++++++++++++++---------------
include/asm-i386/genapic.h | 2 +-
include/asm-i386/mach-bigsmp/mach_apic.h | 8 ++++----
include/asm-i386/mach-default/mach_apic.h | 10 +++++-----
include/asm-i386/mach-es7000/mach_apic.h | 10 +++++-----
include/asm-i386/mach-numaq/mach_apic.h | 2 +-
include/asm-i386/mach-summit/mach_apic.h | 8 ++++----
include/asm-i386/mach-visws/mach_apic.h | 4 ++--
10 files changed, 39 insertions(+), 39 deletions(-)
Index: 2.6.5.mask/arch/i386/kernel/io_apic.c
===================================================================
--- 2.6.5.mask.orig/arch/i386/kernel/io_apic.c 2004-04-03 23:37:35.000000000 -0800
+++ 2.6.5.mask/arch/i386/kernel/io_apic.c 2004-04-03 23:51:51.000000000 -0800
@@ -264,7 +264,7 @@
struct irq_pin_list *entry = irq_2_pin + irq;
unsigned int apicid_value;
- apicid_value = cpu_mask_to_apicid(mk_cpumask_const(cpumask));
+ apicid_value = cpu_mask_to_apicid(cpumask);
/* Prepare to do the io_apic_write */
apicid_value = apicid_value << 24;
spin_lock_irqsave(&ioapic_lock, flags);
Index: 2.6.5.mask/arch/i386/kernel/smp.c
===================================================================
--- 2.6.5.mask.orig/arch/i386/kernel/smp.c 2004-04-03 23:37:35.000000000 -0800
+++ 2.6.5.mask/arch/i386/kernel/smp.c 2004-04-03 23:51:51.000000000 -0800
@@ -160,7 +160,7 @@
*/
inline void send_IPI_mask_bitmask(cpumask_t cpumask, int vector)
{
- unsigned long mask = cpus_coerce(cpumask);
+ unsigned long mask = cpus_addr(cpumask)[0];
unsigned long cfg;
unsigned long flags;
Index: 2.6.5.mask/arch/i386/mach-voyager/voyager_smp.c
===================================================================
--- 2.6.5.mask.orig/arch/i386/mach-voyager/voyager_smp.c 2004-04-03 23:37:36.000000000 -0800
+++ 2.6.5.mask/arch/i386/mach-voyager/voyager_smp.c 2004-04-03 23:51:51.000000000 -0800
@@ -154,7 +154,7 @@
send_CPI_allbutself(__u8 cpi)
{
__u8 cpu = smp_processor_id();
- __u32 mask = cpus_coerce(cpu_online_map) & ~(1 << cpu);
+ __u32 mask = cpus_addr(cpu_online_map)[0] & ~(1 << cpu);
send_CPI(mask, cpi);
}
@@ -403,11 +403,11 @@
/* set up everything for just this CPU, we can alter
* this as we start the other CPUs later */
/* now get the CPU disposition from the extended CMOS */
- phys_cpu_present_map = cpus_promote(voyager_extended_cmos_read(VOYAGER_PROCESSOR_PRESENT_MASK));
- cpus_coerce(phys_cpu_present_map) |= voyager_extended_cmos_read(VOYAGER_PROCESSOR_PRESENT_MASK + 1) << 8;
- cpus_coerce(phys_cpu_present_map) |= voyager_extended_cmos_read(VOYAGER_PROCESSOR_PRESENT_MASK + 2) << 16;
- cpus_coerce(phys_cpu_present_map) |= voyager_extended_cmos_read(VOYAGER_PROCESSOR_PRESENT_MASK + 3) << 24;
- printk("VOYAGER SMP: phys_cpu_present_map = 0x%lx\n", cpus_coerce(phys_cpu_present_map));
+ cpus_addr(phys_cpu_present_map)[0] = voyager_extended_cmos_read(VOYAGER_PROCESSOR_PRESENT_MASK);
+ cpus_addr(phys_cpu_present_map)[0] |= voyager_extended_cmos_read(VOYAGER_PROCESSOR_PRESENT_MASK + 1) << 8;
+ cpus_addr(phys_cpu_present_map)[0] |= voyager_extended_cmos_read(VOYAGER_PROCESSOR_PRESENT_MASK + 2) << 16;
+ cpus_addr(phys_cpu_present_map)[0] |= voyager_extended_cmos_read(VOYAGER_PROCESSOR_PRESENT_MASK + 3) << 24;
+ printk("VOYAGER SMP: phys_cpu_present_map = 0x%lx\n", cpus_addr(phys_cpu_present_map)[0]);
/* Here we set up the VIC to enable SMP */
/* enable the CPIs by writing the base vector to their register */
outb(VIC_DEFAULT_CPI_BASE, VIC_CPI_BASE_REGISTER);
@@ -709,12 +709,12 @@
/* now that the cat has probed the Voyager System Bus, sanity
* check the cpu map */
if( ((voyager_quad_processors | voyager_extended_vic_processors)
- & cpus_coerce(phys_cpu_present_map)) != cpus_coerce(phys_cpu_present_map)) {
+ & cpus_addr(phys_cpu_present_map)[0]) != cpus_addr(phys_cpu_present_map)[0]) {
/* should panic */
printk("\n\n***WARNING*** Sanity check of CPU present map FAILED\n");
}
} else if(voyager_level == 4)
- voyager_extended_vic_processors = cpus_coerce(phys_cpu_present_map);
+ voyager_extended_vic_processors = cpus_addr(phys_cpu_present_map)[0];
/* this sets up the idle task to run on the current cpu */
voyager_extended_cpus = 1;
@@ -912,7 +912,7 @@
if (!cpumask)
BUG();
- if ((cpumask & cpus_coerce(cpu_online_map)) != cpumask)
+ if ((cpumask & cpus_addr(cpu_online_map)[0]) != cpumask)
BUG();
if (cpumask & (1 << smp_processor_id()))
BUG();
@@ -955,7 +955,7 @@
preempt_disable();
- cpu_mask = cpus_coerce(mm->cpu_vm_mask) & ~(1 << smp_processor_id());
+ cpu_mask = cpus_addr(mm->cpu_vm_mask)[0] & ~(1 << smp_processor_id());
local_flush_tlb();
if (cpu_mask)
flush_tlb_others(cpu_mask, mm, FLUSH_ALL);
@@ -971,7 +971,7 @@
preempt_disable();
- cpu_mask = cpus_coerce(mm->cpu_vm_mask) & ~(1 << smp_processor_id());
+ cpu_mask = cpus_addr(mm->cpu_vm_mask)[0] & ~(1 << smp_processor_id());
if (current->active_mm == mm) {
if (current->mm)
@@ -992,7 +992,7 @@
preempt_disable();
- cpu_mask = cpus_coerce(mm->cpu_vm_mask) & ~(1 << smp_processor_id());
+ cpu_mask = cpus_addr(mm->cpu_vm_mask)[0] & ~(1 << smp_processor_id());
if (current->active_mm == mm) {
if(current->mm)
__flush_tlb_one(va);
@@ -1101,7 +1101,7 @@
int wait)
{
struct call_data_struct data;
- __u32 mask = cpus_coerce(cpu_online_map);
+ __u32 mask = cpus_addr(cpu_online_map)[0];
mask &= ~(1<<smp_processor_id());
@@ -1789,9 +1789,9 @@
unsigned long irq_mask = 1 << irq;
int cpu;
- real_mask = cpus_coerce(mask) & voyager_extended_vic_processors;
+ real_mask = cpus_addr(mask)[0] & voyager_extended_vic_processors;
- if(cpus_coerce(mask) == 0)
+ if(cpus_addr(mask)[0] == 0)
/* can't have no cpu's to accept the interrupt -- extremely
* bad things will happen */
return;
Index: 2.6.5.mask/include/asm-i386/genapic.h
===================================================================
--- 2.6.5.mask.orig/include/asm-i386/genapic.h 2004-04-03 23:38:10.000000000 -0800
+++ 2.6.5.mask/include/asm-i386/genapic.h 2004-04-03 23:51:51.000000000 -0800
@@ -62,7 +62,7 @@
unsigned (*get_apic_id)(unsigned long x);
unsigned long apic_id_mask;
- unsigned int (*cpu_mask_to_apicid)(cpumask_const_t cpumask);
+ unsigned int (*cpu_mask_to_apicid)(cpumask_t cpumask);
/* ipi */
void (*send_IPI_mask)(cpumask_t mask, int vector);
Index: 2.6.5.mask/include/asm-i386/mach-bigsmp/mach_apic.h
===================================================================
--- 2.6.5.mask.orig/include/asm-i386/mach-bigsmp/mach_apic.h 2004-04-03 23:38:10.000000000 -0800
+++ 2.6.5.mask/include/asm-i386/mach-bigsmp/mach_apic.h 2004-04-03 23:51:51.000000000 -0800
@@ -140,14 +140,14 @@
return (1);
}
-static inline unsigned int cpu_mask_to_apicid(cpumask_const_t cpumask)
+static inline unsigned int cpu_mask_to_apicid(cpumask_t cpumask)
{
int num_bits_set;
int cpus_found = 0;
int cpu;
int apicid;
- num_bits_set = cpus_weight_const(cpumask);
+ num_bits_set = cpus_weight(cpumask);
/* Return id to all */
if (num_bits_set == NR_CPUS)
return (int) 0xFF;
@@ -155,10 +155,10 @@
* The cpus in the mask must all be on the apic cluster. If are not
* on the same apicid cluster return default value of TARGET_CPUS.
*/
- cpu = first_cpu_const(cpumask);
+ cpu = first_cpu(cpumask);
apicid = cpu_to_logical_apicid(cpu);
while (cpus_found < num_bits_set) {
- if (cpu_isset_const(cpu, cpumask)) {
+ if (cpu_isset(cpu, cpumask)) {
int new_apicid = cpu_to_logical_apicid(cpu);
if (apicid_cluster(apicid) !=
apicid_cluster(new_apicid)){
Index: 2.6.5.mask/include/asm-i386/mach-default/mach_apic.h
===================================================================
--- 2.6.5.mask.orig/include/asm-i386/mach-default/mach_apic.h 2004-04-03 23:38:10.000000000 -0800
+++ 2.6.5.mask/include/asm-i386/mach-default/mach_apic.h 2004-04-03 23:51:51.000000000 -0800
@@ -5,12 +5,12 @@
#define APIC_DFR_VALUE (APIC_DFR_FLAT)
-static inline cpumask_const_t target_cpus(void)
+static inline cpumask_t target_cpus(void)
{
#ifdef CONFIG_SMP
- return mk_cpumask_const(cpu_online_map);
+ return cpu_online_map;
#else
- return mk_cpumask_const(cpumask_of_cpu(0));
+ return cpumask_of_cpu(0);
#endif
}
#define TARGET_CPUS (target_cpus())
@@ -118,9 +118,9 @@
return physid_isset(GET_APIC_ID(apic_read(APIC_ID)), phys_cpu_present_map);
}
-static inline unsigned int cpu_mask_to_apicid(cpumask_const_t cpumask)
+static inline unsigned int cpu_mask_to_apicid(cpumask_t cpumask)
{
- return cpus_coerce_const(cpumask);
+ return cpus_addr(cpumask)[0];
}
static inline void enable_apic_mode(void)
Index: 2.6.5.mask/include/asm-i386/mach-es7000/mach_apic.h
===================================================================
--- 2.6.5.mask.orig/include/asm-i386/mach-es7000/mach_apic.h 2004-04-03 23:38:10.000000000 -0800
+++ 2.6.5.mask/include/asm-i386/mach-es7000/mach_apic.h 2004-04-03 23:51:51.000000000 -0800
@@ -89,7 +89,7 @@
int apic = bios_cpu_apicid[smp_processor_id()];
printk("Enabling APIC mode: %s. Using %d I/O APICs, target cpus %lx\n",
(apic_version[apic] == 0x14) ?
- "Physical Cluster" : "Logical Cluster", nr_ioapics, cpus_coerce(TARGET_CPUS));
+ "Physical Cluster" : "Logical Cluster", nr_ioapics, cpus_addr(TARGET_CPUS)[0]);
}
static inline int multi_timer_check(int apic, int irq)
@@ -159,14 +159,14 @@
return (1);
}
-static inline unsigned int cpu_mask_to_apicid(cpumask_const_t cpumask)
+static inline unsigned int cpu_mask_to_apicid(cpumask_t cpumask)
{
int num_bits_set;
int cpus_found = 0;
int cpu;
int apicid;
- num_bits_set = cpus_weight_const(cpumask);
+ num_bits_set = cpus_weight(cpumask);
/* Return id to all */
if (num_bits_set == NR_CPUS)
return 0xFF;
@@ -174,10 +174,10 @@
* The cpus in the mask must all be on the apic cluster. If are not
* on the same apicid cluster return default value of TARGET_CPUS.
*/
- cpu = first_cpu_const(cpumask);
+ cpu = first_cpu(cpumask);
apicid = cpu_to_logical_apicid(cpu);
while (cpus_found < num_bits_set) {
- if (cpu_isset_const(cpu, cpumask)) {
+ if (cpu_isset(cpu, cpumask)) {
int new_apicid = cpu_to_logical_apicid(cpu);
if (apicid_cluster(apicid) !=
apicid_cluster(new_apicid)){
Index: 2.6.5.mask/include/asm-i386/mach-numaq/mach_apic.h
===================================================================
--- 2.6.5.mask.orig/include/asm-i386/mach-numaq/mach_apic.h 2004-04-03 23:38:10.000000000 -0800
+++ 2.6.5.mask/include/asm-i386/mach-numaq/mach_apic.h 2004-04-03 23:51:51.000000000 -0800
@@ -136,7 +136,7 @@
* We use physical apicids here, not logical, so just return the default
* physical broadcast to stop people from breaking us
*/
-static inline unsigned int cpu_mask_to_apicid(cpumask_const_t cpumask)
+static inline unsigned int cpu_mask_to_apicid(cpumask_t cpumask)
{
return (int) 0xF;
}
Index: 2.6.5.mask/include/asm-i386/mach-summit/mach_apic.h
===================================================================
--- 2.6.5.mask.orig/include/asm-i386/mach-summit/mach_apic.h 2004-04-03 23:38:10.000000000 -0800
+++ 2.6.5.mask/include/asm-i386/mach-summit/mach_apic.h 2004-04-03 23:51:51.000000000 -0800
@@ -140,14 +140,14 @@
{
}
-static inline unsigned int cpu_mask_to_apicid(cpumask_const_t cpumask)
+static inline unsigned int cpu_mask_to_apicid(cpumask_t cpumask)
{
int num_bits_set;
int cpus_found = 0;
int cpu;
int apicid;
- num_bits_set = cpus_weight_const(cpumask);
+ num_bits_set = cpus_weight(cpumask);
/* Return id to all */
if (num_bits_set == NR_CPUS)
return (int) 0xFF;
@@ -155,10 +155,10 @@
* The cpus in the mask must all be on the apic cluster. If are not
* on the same apicid cluster return default value of TARGET_CPUS.
*/
- cpu = first_cpu_const(cpumask);
+ cpu = first_cpu(cpumask);
apicid = cpu_to_logical_apicid(cpu);
while (cpus_found < num_bits_set) {
- if (cpu_isset_const(cpu, cpumask)) {
+ if (cpu_isset(cpu, cpumask)) {
int new_apicid = cpu_to_logical_apicid(cpu);
if (apicid_cluster(apicid) !=
apicid_cluster(new_apicid)){
Index: 2.6.5.mask/include/asm-i386/mach-visws/mach_apic.h
===================================================================
--- 2.6.5.mask.orig/include/asm-i386/mach-visws/mach_apic.h 2004-04-03 23:38:10.000000000 -0800
+++ 2.6.5.mask/include/asm-i386/mach-visws/mach_apic.h 2004-04-03 23:51:51.000000000 -0800
@@ -84,9 +84,9 @@
return physid_isset(boot_cpu_physical_apicid, phys_cpu_present_map);
}
-static inline unsigned int cpu_mask_to_apicid(cpumask_const_t cpumask)
+static inline unsigned int cpu_mask_to_apicid(cpumask_t cpumask)
{
- return cpus_coerce_const(cpumask);
+ return cpus_addr(cpumask)[0];
}
static inline u32 phys_pkg_id(u32 cpuid_apic, int index_msb)
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373
mask7-new-cpumask-h - Rewrite cpumask.h to use bitmap directly.
Major rewrite of cpumask to use a single implementation,
as a struct-wrapped bitmap.
This patch leaves some 26 include/asm-*/cpumask*.h
header files orphaned - to be removed next patch.
Some nine cpumask macros for const variants and to
coerce and promote between an unsigned long and a
cpumask are obsolete. Simple emulation wrappers are
provided in this patch, which can be removed once each
of the 3 archs (i386, ppc64, x86_64) using them are
recoded in follow-on patches to not need them.
The CPU_MASK_ALL macro now avoids leaving possible
garbage one bits in any unused portion of the high word.
An improved comment lists all available operators, for
convenient browsing.
Index: 2.6.5.bitmap.v4/include/linux/cpumask.h
===================================================================
--- 2.6.5.bitmap.v4.orig/include/linux/cpumask.h 2004-04-21 06:50:41.000000000 -0700
+++ 2.6.5.bitmap.v4/include/linux/cpumask.h 2004-04-21 16:45:03.000000000 -0700
@@ -1,44 +1,330 @@
#ifndef __LINUX_CPUMASK_H
#define __LINUX_CPUMASK_H
+/*
+ * Cpumasks provide a bitmap suitable for representing the
+ * set of CPU's in a system, one bit position per CPU number.
+ *
+ * See detailed comments in the file linux/bitmap.h describing the
+ * data type on which these cpumasks are based.
+ *
+ * For details of cpumask_scnprintf() and cpumask_parse(),
+ * see bitmap_scnprintf() and bitmap_parse() in lib/bitmap.c.
+ *
+ * The available cpumask operations are:
+ *
+ * void cpu_set(cpu, mask) turn on bit 'cpu' in mask
+ * void cpu_clear(cpu, mask) turn off bit 'cpu' in mask
+ * void cpus_setall(mask) set all bits
+ * void cpus_clear(mask) clear all bits
+ * int cpu_isset(cpu, mask) true iff bit 'cpu' set in mask
+ * int cpu_test_and_set(cpu, mask) test and set bit 'cpu' in mask
+ *
+ * void cpus_and(dst, src1, src2) dst = src1 & src2 [intersection]
+ * void cpus_or(dst, src1, src2) dst = src1 | src2 [union]
+ * void cpus_xor(dst, src1, src2) dst = src1 ^ src2
+ * void cpus_andnot(dst, src1, src2) dst = src1 & ~src2
+ * void cpus_complement(dst, src) dst = ~src
+ *
+ * int cpus_equal(mask1, mask2) Does mask1 == mask2?
+ * int cpus_intersects(mask1, mask2) Do mask1 and mask2 intersect?
+ * int cpus_subset(mask1, mask2) Is mask1 a subset of mask2?
+ * int cpus_empty(mask) Is mask empty (no bits sets)?
+ * int cpus_full(mask) Is mask full (all bits sets)?
+ * int cpus_weight(mask) Hamming weigh - number of set bits
+ *
+ * void cpus_shift_right(dst, src, n) Shift right
+ * void cpus_shift_left(dst, src, n) Shift left
+ *
+ * int first_cpu(mask) Number lowest set bit, or NR_CPUS
+ * int next_cpu(cpu, mask) Next cpu past 'cpu', or NR_CPUS
+ *
+ * cpumask_t cpumask_of_cpu(cpu) Return cpumask with bit 'cpu' set
+ * CPU_MASK_ALL Initializer - all bits set
+ * CPU_MASK_NONE Initializer - no bits set
+ * unsigned long *cpus_addr(mask) Array of unsigned long's in mask
+ *
+ * int cpumask_scnprintf(buf, len, mask) Format cpumask for printing
+ * int cpumask_parse(ubuf, ulen, mask) Parse ascii string as cpumask
+ *
+ * int num_online_cpus() Number of online CPUs
+ * int num_possible_cpus() Number of all possible CPUs
+ * int cpu_online(cpu) Is some cpu online?
+ * int cpu_possible(cpu) Is some cpu possible?
+ * void cpu_set_online(cpu) set cpu in cpu_online_map
+ * void cpu_set_offline(cpu) clear cpu in cpu_online_map
+ * int any_online_cpu(mask) First online cpu in mask
+ *
+ * for_each_cpu_mask(cpu, mask) for-loop cpu over mask
+ * for_each_cpu(cpu) for-loop cpu over cpu_possible_map
+ * for_each_online_cpu(cpu) for-loop cpu over cpu_online_map
+ */
+
#include <linux/threads.h>
#include <linux/bitmap.h>
-#include <asm/cpumask.h>
#include <asm/bug.h>
-#ifdef CONFIG_SMP
+typedef struct { DECLARE_BITMAP(bits, NR_CPUS); } cpumask_t;
+extern cpumask_t _unused_cpumask_arg_;
-extern cpumask_t cpu_online_map;
-extern cpumask_t cpu_possible_map;
+#define cpu_set(cpu, dst) __cpu_set((cpu), &(dst))
+static inline void __cpu_set(int cpu, volatile cpumask_t *dstp)
+{
+ if (cpu < NR_CPUS)
+ set_bit(cpu, dstp->bits);
+}
+
+#define cpu_clear(cpu, dst) __cpu_clear((cpu), &(dst))
+static inline void __cpu_clear(int cpu, volatile cpumask_t *dstp)
+{
+ clear_bit(cpu, dstp->bits);
+}
+
+#define cpus_setall(dst) __cpus_setall(&(dst), NR_CPUS)
+static inline void __cpus_setall(cpumask_t *dstp, int nbits)
+{
+ bitmap_fill(dstp->bits, nbits);
+}
+
+#define cpus_clear(dst) __cpus_clear(&(dst), NR_CPUS)
+static inline void __cpus_clear(cpumask_t *dstp, int nbits)
+{
+ bitmap_clear(dstp->bits, nbits);
+}
+
+#define cpu_isset(cpu, cpumask) __cpu_isset((cpu), &(cpumask))
+static inline int __cpu_isset(int cpu, const volatile cpumask_t *addr)
+{
+ return test_bit(cpu, addr->bits);
+}
+
+#define cpu_test_and_set(cpu, cpumask) __cpu_test_and_set((cpu), &(cpumask))
+static inline int __cpu_test_and_set(int cpu, cpumask_t *addr)
+{
+ if (cpu < NR_CPUS)
+ return test_and_set_bit(cpu, addr->bits);
+ else
+ return 0;
+}
+
+#define cpus_and(dst, src1, src2) __cpus_and(&(dst), &(src1), &(src2), NR_CPUS)
+static inline void __cpus_and(cpumask_t *dstp, cpumask_t *src1p,
+ cpumask_t *src2p, int nbits)
+{
+ bitmap_and(dstp->bits, src1p->bits, src2p->bits, nbits);
+}
+
+#define cpus_or(dst, src1, src2) __cpus_or(&(dst), &(src1), &(src2), NR_CPUS)
+static inline void __cpus_or(cpumask_t *dstp, cpumask_t *src1p,
+ cpumask_t *src2p, int nbits)
+{
+ bitmap_or(dstp->bits, src1p->bits, src2p->bits, nbits);
+}
+
+#define cpus_xor(dst, src1, src2) __cpus_xor(&(dst), &(src1), &(src2), NR_CPUS)
+static inline void __cpus_xor(cpumask_t *dstp, cpumask_t *src1p,
+ cpumask_t *src2p, int nbits)
+{
+ bitmap_xor(dstp->bits, src1p->bits, src2p->bits, nbits);
+}
+
+#define cpus_andnot(dst, src1, src2) \
+ __cpus_andnot(&(dst), &(src1), &(src2), NR_CPUS)
+static inline void __cpus_andnot(cpumask_t *dstp, cpumask_t *src1p,
+ cpumask_t *src2p, int nbits)
+{
+ bitmap_andnot(dstp->bits, src1p->bits, src2p->bits, nbits);
+}
+
+#define cpus_complement(dst, src) __cpus_complement(&(dst), &(src), NR_CPUS)
+static inline void __cpus_complement(cpumask_t *dstp,
+ cpumask_t *srcp, int nbits)
+{
+ bitmap_complement(dstp->bits, srcp->bits, nbits);
+}
+
+#define cpus_equal(src1, src2) __cpus_equal(&(src1), &(src2), NR_CPUS)
+static inline int __cpus_equal(cpumask_t *src1p,
+ cpumask_t *src2p, int nbits)
+{
+ return bitmap_equal(src1p->bits, src2p->bits, nbits);
+}
+
+#define cpus_intersects(src1, src2) __cpus_intersects(&(src1), &(src2), NR_CPUS)
+static inline int __cpus_intersects(cpumask_t *src1p,
+ cpumask_t *src2p, int nbits)
+{
+ return bitmap_intersects(src1p->bits, src2p->bits, nbits);
+}
+
+#define cpus_subset(src1, src2) __cpus_subset(&(src1), &(src2), NR_CPUS)
+static inline int __cpus_subset(cpumask_t *src1p,
+ cpumask_t *src2p, int nbits)
+{
+ return bitmap_subset(src1p->bits, src2p->bits, nbits);
+}
+
+#define cpus_empty(src) __cpus_empty(&(src), NR_CPUS)
+static inline int __cpus_empty(cpumask_t *srcp, int nbits)
+{
+ return bitmap_empty(srcp->bits, nbits);
+}
+
+#define cpus_full(cpumask) __cpus_full(&(cpumask), NR_CPUS)
+static inline int __cpus_full(cpumask_t *srcp, int nbits)
+{
+ return bitmap_full(srcp->bits, nbits);
+}
+
+#define cpus_weight(cpumask) __cpus_weight(&(cpumask), NR_CPUS)
+static inline int __cpus_weight(cpumask_t *srcp, int nbits)
+{
+ return bitmap_weight(srcp->bits, nbits);
+}
+
+#define cpus_shift_right(dst, src, n) \
+ __cpus_shift_right(&(dst), &(src), (n), NR_CPUS)
+static inline void __cpus_shift_right(cpumask_t *dstp,
+ cpumask_t *srcp, int n, int nbits)
+{
+ bitmap_shift_right(dstp->bits, srcp->bits, n, nbits);
+}
+
+#define cpus_shift_left(dst, src, n) \
+ __cpus_shift_left(&(dst), &(src), (n), NR_CPUS)
+static inline void __cpus_shift_left(cpumask_t *dstp,
+ cpumask_t *srcp, int n, int nbits)
+{
+ bitmap_shift_left(dstp->bits, srcp->bits, n, nbits);
+}
+
+#define first_cpu(src) __first_cpu(&(src), NR_CPUS)
+static inline int __first_cpu(cpumask_t *srcp, int nbits)
+{
+ return find_first_bit(srcp->bits, nbits);
+}
-#define num_online_cpus() cpus_weight(cpu_online_map)
-#define num_possible_cpus() cpus_weight(cpu_possible_map)
-#define cpu_online(cpu) cpu_isset(cpu, cpu_online_map)
-#define cpu_possible(cpu) cpu_isset(cpu, cpu_possible_map)
-
-#define for_each_cpu_mask(cpu, mask) \
- for (cpu = first_cpu_const(mk_cpumask_const(mask)); \
- cpu < NR_CPUS; \
- cpu = next_cpu_const(cpu, mk_cpumask_const(mask)))
+#define next_cpu(n, src) __next_cpu((n), &(src), NR_CPUS)
+static inline int __next_cpu(int n, cpumask_t *srcp, int nbits)
+{
+ return find_next_bit(srcp->bits, nbits, n+1);
+}
+
+#define cpumask_of_cpu(cpu) \
+({ \
+ typeof(_unused_cpumask_arg_) m; \
+ int c = cpu; \
+ if (sizeof(m) == sizeof(unsigned long)) { \
+ if (c < NR_CPUS) \
+ m.bits[0] = 1UL<<c; \
+ } else { \
+ cpus_clear(m); \
+ cpu_set(c, m); \
+ } \
+ m; \
+})
+
+#define CPU_MASK_LAST_WORD BITMAP_LAST_WORD_MASK(NR_CPUS)
+
+#if NR_CPUS <= BITS_PER_LONG
+
+#define CPU_MASK_ALL \
+{ { \
+ [BITS_TO_LONGS(NR_CPUS)-1] = CPU_MASK_LAST_WORD \
+} }
-#define for_each_cpu(cpu) for_each_cpu_mask(cpu, cpu_possible_map)
-#define for_each_online_cpu(cpu) for_each_cpu_mask(cpu, cpu_online_map)
#else
-#define cpu_online_map cpumask_of_cpu(0)
-#define cpu_possible_map cpumask_of_cpu(0)
-#define num_online_cpus() 1
-#define num_possible_cpus() 1
-#define cpu_online(cpu) ({ BUG_ON((cpu) != 0); 1; })
-#define cpu_possible(cpu) ({ BUG_ON((cpu) != 0); 1; })
-#define for_each_cpu(cpu) for (cpu = 0; cpu < 1; cpu++)
-#define for_each_online_cpu(cpu) for (cpu = 0; cpu < 1; cpu++)
+#define CPU_MASK_ALL \
+{ { \
+ [0 ... BITS_TO_LONGS(NR_CPUS)-2] = ~0UL, \
+ [BITS_TO_LONGS(NR_CPUS)-1] = CPU_MASK_LAST_WORD \
+} }
+
#endif
-#define cpumask_scnprintf(buf, buflen, map) \
- bitmap_scnprintf(buf, buflen, cpus_addr(map), NR_CPUS)
+#define CPU_MASK_NONE \
+{ { \
+ [0 ... BITS_TO_LONGS(NR_CPUS)-1] = 0UL \
+} }
+
+#define cpus_addr(src) ((src).bits)
+
+#define cpumask_scnprintf(buf, len, src) \
+ __cpumask_scnprintf((buf), (len), &(src), NR_CPUS)
+static inline int __cpumask_scnprintf(char *buf, int len,
+ cpumask_t *srcp, int nbits)
+{
+ return bitmap_scnprintf(buf, len, srcp->bits, nbits);
+}
+
+#define cpumask_parse(ubuf, ulen, src) \
+ __cpumask_parse((ubuf), (ulen), &(src), NR_CPUS)
+static inline int __cpumask_parse(const char __user *buf, int len,
+ cpumask_t *srcp, int nbits)
+{
+ return bitmap_parse(buf, len, srcp->bits, nbits);
+}
+
+/*
+ * The following particular system cpumasks and operations
+ * on them manage all (possible) and online cpus.
+ */
+
+extern cpumask_t cpu_online_map;
+extern cpumask_t cpu_possible_map;
+
+#ifdef CONFIG_SMP
+
+#define num_online_cpus() cpus_weight(cpu_online_map)
+#define num_possible_cpus() cpus_weight(cpu_possible_map)
+#define cpu_online(cpu) cpu_isset((cpu), cpu_online_map)
+#define cpu_possible(cpu) cpu_isset((cpu), cpu_possible_map)
+#define cpu_set_online(cpu) cpu_set((cpu), cpu_online_map)
+#define cpu_set_offline(cpu) cpu_clear((cpu), cpu_online_map)
+
+#define any_online_cpu(mask) \
+({ \
+ cpumask_t m; \
+ cpus_and(m, mask, cpu_online_map); \
+ first_cpu(m); \
+})
+
+#define for_each_cpu_mask(cpu, mask) \
+ for (cpu = first_cpu(mask); \
+ cpu < NR_CPUS; \
+ cpu = next_cpu(cpu, mask))
+
+#else /* !CONFIG_SMP */
+
+#define num_online_cpus() 1
+#define num_possible_cpus() 1
+#define cpu_online(cpu) ({ BUG_ON((cpu) != 0); 1; })
+#define cpu_possible(cpu) ({ BUG_ON((cpu) != 0); 1; })
+#define cpu_set_online(cpu) ({ BUG_ON((cpu) != 0); })
+#define cpu_set_offline(cpu) ({ BUG(); })
+
+#define any_online_cpu(mask) 0
+
+#define for_each_cpu_mask(cpu, mask) for (cpu = 0; cpu < 1; cpu++)
+
+#endif /* CONFIG_SMP */
+
+#define for_each_cpu(cpu) \
+ for_each_cpu_mask(cpu, cpu_possible_map)
+#define for_each_online_cpu(cpu) \
+ for_each_cpu_mask(cpu, cpu_online_map)
-#define cpumask_parse(buf, buflen, map) \
- bitmap_parse(buf, buflen, cpus_addr(map), NR_CPUS)
+/* Begin obsolete cpumask operator emulation */
+#define cpu_isset_const(a,b) cpu_isset(a,b)
+#define cpumask_const_t cpumask_t
+#define cpus_coerce(m) (cpus_addr(m)[0])
+#define cpus_coerce_const cpus_coerce
+#define cpus_promote(x) ({ cpumask_t m; m.bits[0] = x; m; })
+#define cpus_weight_const cpus_weight
+#define first_cpu_const first_cpu
+#define mk_cpumask_const(x) x
+#define next_cpu_const next_cpu
+/* End of obsolete cpumask operator emulation */
#endif /* __LINUX_CPUMASK_H */
Index: 2.6.5.bitmap.v4/kernel/sched.c
===================================================================
--- 2.6.5.bitmap.v4.orig/kernel/sched.c 2004-04-21 06:50:45.000000000 -0700
+++ 2.6.5.bitmap.v4/kernel/sched.c 2004-04-21 16:45:03.000000000 -0700
@@ -2349,6 +2349,11 @@
return retval;
}
+#ifndef CONFIG_SMP
+cpumask_t cpu_online_map = CPU_MASK_ALL;
+cpumask_t cpu_possible_map = CPU_MASK_ALL;
+#endif
+
/**
* sys_sched_getaffinity - get the cpu affinity of a process
* @pid: pid of the process
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373
mask1-bitmap-comment - Document bitmap.c bit model.
Document the bitmap bit model, including handling of unused bits,
and operation preconditions and postconditions.
Index: 2.6.5.bitmap/lib/bitmap.c
===================================================================
--- 2.6.5.bitmap.orig/lib/bitmap.c 2004-04-05 02:00:15.000000000 -0700
+++ 2.6.5.bitmap/lib/bitmap.c 2004-04-05 02:50:25.000000000 -0700
@@ -12,6 +12,26 @@
#include <asm/bitops.h>
#include <asm/uaccess.h>
+/*
+ * bitmaps provide an array of bits, implemented using an an
+ * array of unsigned longs. The number of valid bits in a
+ * given bitmap need not be an exact multiple of BITS_PER_LONG.
+ *
+ * The possible unused bits in the last, partially used word
+ * of a bitmap are 'don't care'. The implementation makes
+ * no particular effort to keep them zero. It ensures that
+ * their value will not affect the results of any operation.
+ * The bitmap operations that return Boolean (bitmap_empty,
+ * for example) or scalar (bitmap_weight, for example) results
+ * carefully filter out these unused bits from impacting their
+ * results.
+ *
+ * Except for bitmap_complement, these operations hold to a
+ * slightly stronger rule: if you don't input any bitmaps to
+ * these ops that have some unused bits set, then they won't
+ * output any set unused bits in output bitmaps.
+ */
+
#define MAX_BITMAP_BITS 512U /* for ia64 NR_CPUS maximum */
int bitmap_empty(const unsigned long *bitmap, int bits)
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373
mask8-remove-old-cpumask-files - Remove 26 no longer used cpumask headers.
With the cpumask rewrite in the previous patch, these
various include/asm-*/cpumask*.h headers are no longer used.
Index: 2.6.5.bitmap/include/asm-alpha/cpumask.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-alpha/cpumask.h 2004-04-08 01:08:17.000000000 -0700
+++ 2.6.5.bitmap/include/asm-alpha/cpumask.h 1969-12-31 16:00:00.000000000 -0800
@@ -1,6 +0,0 @@
-#ifndef _ASM_ALPHA_CPUMASK_H
-#define _ASM_ALPHA_CPUMASK_H
-
-#include <asm-generic/cpumask.h>
-
-#endif /* _ASM_ALPHA_CPUMASK_H */
Index: 2.6.5.bitmap/include/asm-arm/cpumask.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-arm/cpumask.h 2004-04-08 01:08:17.000000000 -0700
+++ 2.6.5.bitmap/include/asm-arm/cpumask.h 1969-12-31 16:00:00.000000000 -0800
@@ -1,6 +0,0 @@
-#ifndef _ASM_ARM_CPUMASK_H
-#define _ASM_ARM_CPUMASK_H
-
-#include <asm-generic/cpumask.h>
-
-#endif /* _ASM_ARM_CPUMASK_H */
Index: 2.6.5.bitmap/include/asm-arm26/cpumask.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-arm26/cpumask.h 2004-04-08 01:08:17.000000000 -0700
+++ 2.6.5.bitmap/include/asm-arm26/cpumask.h 1969-12-31 16:00:00.000000000 -0800
@@ -1,6 +0,0 @@
-#ifndef _ASM_ARM26_CPUMASK_H
-#define _ASM_ARM26_CPUMASK_H
-
-#include <asm-generic/cpumask.h>
-
-#endif /* _ASM_ARM26_CPUMASK_H */
Index: 2.6.5.bitmap/include/asm-cris/cpumask.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-cris/cpumask.h 2004-04-08 01:08:17.000000000 -0700
+++ 2.6.5.bitmap/include/asm-cris/cpumask.h 1969-12-31 16:00:00.000000000 -0800
@@ -1,6 +0,0 @@
-#ifndef _ASM_CRIS_CPUMASK_H
-#define _ASM_CRIS_CPUMASK_H
-
-#include <asm-generic/cpumask.h>
-
-#endif /* _ASM_CRIS_CPUMASK_H */
Index: 2.6.5.bitmap/include/asm-generic/cpumask.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-generic/cpumask.h 2004-04-08 01:08:17.000000000 -0700
+++ 2.6.5.bitmap/include/asm-generic/cpumask.h 1969-12-31 16:00:00.000000000 -0800
@@ -1,40 +0,0 @@
-#ifndef __ASM_GENERIC_CPUMASK_H
-#define __ASM_GENERIC_CPUMASK_H
-
-#include <linux/config.h>
-#include <linux/kernel.h>
-#include <linux/threads.h>
-#include <linux/types.h>
-#include <linux/bitmap.h>
-
-#if NR_CPUS > BITS_PER_LONG && NR_CPUS != 1
-#define CPU_ARRAY_SIZE BITS_TO_LONGS(NR_CPUS)
-
-struct cpumask
-{
- unsigned long mask[CPU_ARRAY_SIZE];
-};
-
-typedef struct cpumask cpumask_t;
-
-#else
-typedef unsigned long cpumask_t;
-#endif
-
-#ifdef CONFIG_SMP
-#if NR_CPUS > BITS_PER_LONG
-#include <asm-generic/cpumask_array.h>
-#else
-#include <asm-generic/cpumask_arith.h>
-#endif
-#else
-#include <asm-generic/cpumask_up.h>
-#endif
-
-#if NR_CPUS <= 4*BITS_PER_LONG
-#include <asm-generic/cpumask_const_value.h>
-#else
-#include <asm-generic/cpumask_const_reference.h>
-#endif
-
-#endif /* __ASM_GENERIC_CPUMASK_H */
Index: 2.6.5.bitmap/include/asm-generic/cpumask_arith.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-generic/cpumask_arith.h 2004-04-08 01:08:17.000000000 -0700
+++ 2.6.5.bitmap/include/asm-generic/cpumask_arith.h 1969-12-31 16:00:00.000000000 -0800
@@ -1,49 +0,0 @@
-#ifndef __ASM_GENERIC_CPUMASK_ARITH_H
-#define __ASM_GENERIC_CPUMASK_ARITH_H
-
-/*
- * Arithmetic type -based cpu bitmaps. A single unsigned long is used
- * to contain the whole cpu bitmap.
- */
-
-#define cpu_set(cpu, map) set_bit(cpu, &(map))
-#define cpu_clear(cpu, map) clear_bit(cpu, &(map))
-#define cpu_isset(cpu, map) test_bit(cpu, &(map))
-#define cpu_test_and_set(cpu, map) test_and_set_bit(cpu, &(map))
-
-#define cpus_and(dst,src1,src2) do { dst = (src1) & (src2); } while (0)
-#define cpus_or(dst,src1,src2) do { dst = (src1) | (src2); } while (0)
-#define cpus_clear(map) do { map = 0; } while (0)
-#define cpus_complement(map) do { map = ~(map); } while (0)
-#define cpus_equal(map1, map2) ((map1) == (map2))
-#define cpus_empty(map) ((map) == 0)
-#define cpus_addr(map) (&(map))
-
-#if BITS_PER_LONG == 32
-#define cpus_weight(map) hweight32(map)
-#elif BITS_PER_LONG == 64
-#define cpus_weight(map) hweight64(map)
-#endif
-
-#define cpus_shift_right(dst, src, n) do { dst = (src) >> (n); } while (0)
-#define cpus_shift_left(dst, src, n) do { dst = (src) << (n); } while (0)
-
-#define any_online_cpu(map) \
-({ \
- cpumask_t __tmp__; \
- cpus_and(__tmp__, map, cpu_online_map); \
- __tmp__ ? first_cpu(__tmp__) : NR_CPUS; \
-})
-
-#define CPU_MASK_ALL (~((cpumask_t)0) >> (8*sizeof(cpumask_t) - NR_CPUS))
-#define CPU_MASK_NONE ((cpumask_t)0)
-
-/* only ever use this for things that are _never_ used on large boxen */
-#define cpus_coerce(map) ((unsigned long)(map))
-#define cpus_promote(map) ({ map; })
-#define cpumask_of_cpu(cpu) ({ ((cpumask_t)1) << (cpu); })
-
-#define first_cpu(map) __ffs(map)
-#define next_cpu(cpu, map) find_next_bit(&(map), NR_CPUS, cpu + 1)
-
-#endif /* __ASM_GENERIC_CPUMASK_ARITH_H */
Index: 2.6.5.bitmap/include/asm-generic/cpumask_array.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-generic/cpumask_array.h 2004-04-08 01:08:17.000000000 -0700
+++ 2.6.5.bitmap/include/asm-generic/cpumask_array.h 1969-12-31 16:00:00.000000000 -0800
@@ -1,54 +0,0 @@
-#ifndef __ASM_GENERIC_CPUMASK_ARRAY_H
-#define __ASM_GENERIC_CPUMASK_ARRAY_H
-
-/*
- * Array-based cpu bitmaps. An array of unsigned longs is used to contain
- * the bitmap, and then contained in a structure so it may be passed by
- * value.
- */
-
-#define CPU_ARRAY_SIZE BITS_TO_LONGS(NR_CPUS)
-
-#define cpu_set(cpu, map) set_bit(cpu, (map).mask)
-#define cpu_clear(cpu, map) clear_bit(cpu, (map).mask)
-#define cpu_isset(cpu, map) test_bit(cpu, (map).mask)
-#define cpu_test_and_set(cpu, map) test_and_set_bit(cpu, (map).mask)
-
-#define cpus_and(dst,src1,src2) bitmap_and((dst).mask,(src1).mask, (src2).mask, NR_CPUS)
-#define cpus_or(dst,src1,src2) bitmap_or((dst).mask, (src1).mask, (src2).mask, NR_CPUS)
-#define cpus_clear(map) bitmap_clear((map).mask, NR_CPUS)
-#define cpus_complement(map) bitmap_complement((map).mask, (map).mask, NR_CPUS)
-#define cpus_equal(map1, map2) bitmap_equal((map1).mask, (map2).mask, NR_CPUS)
-#define cpus_empty(map) bitmap_empty(map.mask, NR_CPUS)
-#define cpus_addr(map) ((map).mask)
-#define cpus_weight(map) bitmap_weight((map).mask, NR_CPUS)
-#define cpus_shift_right(d, s, n) bitmap_shift_right((d).mask, (s).mask, n, NR_CPUS)
-#define cpus_shift_left(d, s, n) bitmap_shift_left((d).mask, (s).mask, n, NR_CPUS)
-#define first_cpu(map) find_first_bit((map).mask, NR_CPUS)
-#define next_cpu(cpu, map) find_next_bit((map).mask, NR_CPUS, cpu + 1)
-
-/* only ever use this for things that are _never_ used on large boxen */
-#define cpus_coerce(map) ((map).mask[0])
-#define cpus_promote(map) ({ cpumask_t __cpu_mask = CPU_MASK_NONE;\
- __cpu_mask.mask[0] = map; \
- __cpu_mask; \
- })
-#define cpumask_of_cpu(cpu) ({ cpumask_t __cpu_mask = CPU_MASK_NONE;\
- cpu_set(cpu, __cpu_mask); \
- __cpu_mask; \
- })
-#define any_online_cpu(map) \
-({ \
- cpumask_t __tmp__; \
- cpus_and(__tmp__, map, cpu_online_map); \
- find_first_bit(__tmp__.mask, NR_CPUS); \
-})
-
-
-/*
- * um, these need to be usable as static initializers
- */
-#define CPU_MASK_ALL { {[0 ... CPU_ARRAY_SIZE-1] = ~0UL} }
-#define CPU_MASK_NONE { {[0 ... CPU_ARRAY_SIZE-1] = 0UL} }
-
-#endif /* __ASM_GENERIC_CPUMASK_ARRAY_H */
Index: 2.6.5.bitmap/include/asm-generic/cpumask_const_reference.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-generic/cpumask_const_reference.h 2004-04-08 01:08:17.000000000 -0700
+++ 2.6.5.bitmap/include/asm-generic/cpumask_const_reference.h 1969-12-31 16:00:00.000000000 -0800
@@ -1,29 +0,0 @@
-#ifndef __ASM_GENERIC_CPUMASK_CONST_REFERENCE_H
-#define __ASM_GENERIC_CPUMASK_CONST_REFERENCE_H
-
-struct cpumask_ref {
- const cpumask_t *val;
-};
-
-typedef const struct cpumask_ref cpumask_const_t;
-
-#define mk_cpumask_const(map) ((cpumask_const_t){ &(map) })
-#define cpu_isset_const(cpu, map) cpu_isset(cpu, *(map).val)
-
-#define cpus_and_const(dst,src1,src2) cpus_and(dst,*(src1).val,*(src2).val)
-#define cpus_or_const(dst,src1,src2) cpus_or(dst,*(src1).val,*(src2).val)
-
-#define cpus_equal_const(map1, map2) cpus_equal(*(map1).val, *(map2).val)
-
-#define cpus_copy_const(map1, map2) bitmap_copy((map1).mask, (map2).val->mask, NR_CPUS)
-
-#define cpus_empty_const(map) cpus_empty(*(map).val)
-#define cpus_weight_const(map) cpus_weight(*(map).val)
-#define first_cpu_const(map) first_cpu(*(map).val)
-#define next_cpu_const(cpu, map) next_cpu(cpu, *(map).val)
-
-/* only ever use this for things that are _never_ used on large boxen */
-#define cpus_coerce_const(map) cpus_coerce(*(map).val)
-#define any_online_cpu_const(map) any_online_cpu(*(map).val)
-
-#endif /* __ASM_GENERIC_CPUMASK_CONST_REFERENCE_H */
Index: 2.6.5.bitmap/include/asm-generic/cpumask_const_value.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-generic/cpumask_const_value.h 2004-04-08 01:08:17.000000000 -0700
+++ 2.6.5.bitmap/include/asm-generic/cpumask_const_value.h 1969-12-31 16:00:00.000000000 -0800
@@ -1,21 +0,0 @@
-#ifndef __ASM_GENERIC_CPUMASK_CONST_VALUE_H
-#define __ASM_GENERIC_CPUMASK_CONST_VALUE_H
-
-typedef const cpumask_t cpumask_const_t;
-
-#define mk_cpumask_const(map) (map)
-#define cpu_isset_const(cpu, map) cpu_isset(cpu, map)
-#define cpus_and_const(dst,src1,src2) cpus_and(dst, src1, src2)
-#define cpus_or_const(dst,src1,src2) cpus_or(dst, src1, src2)
-#define cpus_equal_const(map1, map2) cpus_equal(map1, map2)
-#define cpus_empty_const(map) cpus_empty(map)
-#define cpus_copy_const(map1, map2) do { map1 = (cpumask_t)map2; } while (0)
-#define cpus_weight_const(map) cpus_weight(map)
-#define first_cpu_const(map) first_cpu(map)
-#define next_cpu_const(cpu, map) next_cpu(cpu, map)
-
-/* only ever use this for things that are _never_ used on large boxen */
-#define cpus_coerce_const(map) cpus_coerce(map)
-#define any_online_cpu_const(map) any_online_cpu(map)
-
-#endif /* __ASM_GENERIC_CPUMASK_CONST_VALUE_H */
Index: 2.6.5.bitmap/include/asm-generic/cpumask_up.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-generic/cpumask_up.h 2004-04-08 01:08:17.000000000 -0700
+++ 2.6.5.bitmap/include/asm-generic/cpumask_up.h 1969-12-31 16:00:00.000000000 -0800
@@ -1,59 +0,0 @@
-#ifndef __ASM_GENERIC_CPUMASK_UP_H
-#define __ASM_GENERIC_CPUMASK_UP_H
-
-#define cpus_coerce(map) (map)
-
-#define cpu_set(cpu, map) do { (void)(cpu); cpus_coerce(map) = 1UL; } while (0)
-#define cpu_clear(cpu, map) do { (void)(cpu); cpus_coerce(map) = 0UL; } while (0)
-#define cpu_isset(cpu, map) ((void)(cpu), cpus_coerce(map) != 0UL)
-#define cpu_test_and_set(cpu, map) ((void)(cpu), test_and_set_bit(0, &(map)))
-
-#define cpus_and(dst, src1, src2) \
- do { \
- if (cpus_coerce(src1) && cpus_coerce(src2)) \
- cpus_coerce(dst) = 1UL; \
- else \
- cpus_coerce(dst) = 0UL; \
- } while (0)
-
-#define cpus_or(dst, src1, src2) \
- do { \
- if (cpus_coerce(src1) || cpus_coerce(src2)) \
- cpus_coerce(dst) = 1UL; \
- else \
- cpus_coerce(dst) = 0UL; \
- } while (0)
-
-#define cpus_clear(map) do { cpus_coerce(map) = 0UL; } while (0)
-
-#define cpus_complement(map) \
- do { \
- cpus_coerce(map) = !cpus_coerce(map); \
- } while (0)
-
-#define cpus_equal(map1, map2) (cpus_coerce(map1) == cpus_coerce(map2))
-#define cpus_empty(map) (cpus_coerce(map) == 0UL)
-#define cpus_addr(map) (&(map))
-#define cpus_weight(map) (cpus_coerce(map) ? 1UL : 0UL)
-#define cpus_shift_right(d, s, n) do { cpus_coerce(d) = 0UL; } while (0)
-#define cpus_shift_left(d, s, n) do { cpus_coerce(d) = 0UL; } while (0)
-#define first_cpu(map) (cpus_coerce(map) ? 0 : 1)
-#define next_cpu(cpu, map) 1
-
-/* only ever use this for things that are _never_ used on large boxen */
-#define cpus_promote(map) \
- ({ \
- cpumask_t __tmp__; \
- cpus_coerce(__tmp__) = map; \
- __tmp__; \
- })
-#define cpumask_of_cpu(cpu) ((void)(cpu), cpus_promote(1))
-#define any_online_cpu(map) (cpus_coerce(map) ? 0 : 1)
-
-/*
- * um, these need to be usable as static initializers
- */
-#define CPU_MASK_ALL 1UL
-#define CPU_MASK_NONE 0UL
-
-#endif /* __ASM_GENERIC_CPUMASK_UP_H */
Index: 2.6.5.bitmap/include/asm-h8300/cpumask.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-h8300/cpumask.h 2004-04-08 01:08:17.000000000 -0700
+++ 2.6.5.bitmap/include/asm-h8300/cpumask.h 1969-12-31 16:00:00.000000000 -0800
@@ -1,6 +0,0 @@
-#ifndef _ASM_H8300_CPUMASK_H
-#define _ASM_H8300_CPUMASK_H
-
-#include <asm-generic/cpumask.h>
-
-#endif /* _ASM_H8300_CPUMASK_H */
Index: 2.6.5.bitmap/include/asm-i386/cpumask.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-i386/cpumask.h 2004-04-08 01:08:17.000000000 -0700
+++ 2.6.5.bitmap/include/asm-i386/cpumask.h 1969-12-31 16:00:00.000000000 -0800
@@ -1,6 +0,0 @@
-#ifndef _ASM_I386_CPUMASK_H
-#define _ASM_I386_CPUMASK_H
-
-#include <asm-generic/cpumask.h>
-
-#endif /* _ASM_I386_CPUMASK_H */
Index: 2.6.5.bitmap/include/asm-m68k/cpumask.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-m68k/cpumask.h 2004-04-08 01:08:17.000000000 -0700
+++ 2.6.5.bitmap/include/asm-m68k/cpumask.h 1969-12-31 16:00:00.000000000 -0800
@@ -1,6 +0,0 @@
-#ifndef _ASM_M68K_CPUMASK_H
-#define _ASM_M68K_CPUMASK_H
-
-#include <asm-generic/cpumask.h>
-
-#endif /* _ASM_M68K_CPUMASK_H */
Index: 2.6.5.bitmap/include/asm-m68knommu/cpumask.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-m68knommu/cpumask.h 2004-04-08 01:08:17.000000000 -0700
+++ 2.6.5.bitmap/include/asm-m68knommu/cpumask.h 1969-12-31 16:00:00.000000000 -0800
@@ -1,6 +0,0 @@
-#ifndef _ASM_M68KNOMMU_CPUMASK_H
-#define _ASM_M68KNOMMU_CPUMASK_H
-
-#include <asm-generic/cpumask.h>
-
-#endif /* _ASM_M68KNOMMU_CPUMASK_H */
Index: 2.6.5.bitmap/include/asm-mips/cpumask.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-mips/cpumask.h 2004-04-08 01:08:17.000000000 -0700
+++ 2.6.5.bitmap/include/asm-mips/cpumask.h 1969-12-31 16:00:00.000000000 -0800
@@ -1,6 +0,0 @@
-#ifndef _ASM_MIPS_CPUMASK_H
-#define _ASM_MIPS_CPUMASK_H
-
-#include <asm-generic/cpumask.h>
-
-#endif /* _ASM_MIPS_CPUMASK_H */
Index: 2.6.5.bitmap/include/asm-parisc/cpumask.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-parisc/cpumask.h 2004-04-08 01:08:17.000000000 -0700
+++ 2.6.5.bitmap/include/asm-parisc/cpumask.h 1969-12-31 16:00:00.000000000 -0800
@@ -1,6 +0,0 @@
-#ifndef _ASM_PARISC_CPUMASK_H
-#define _ASM_PARISC_CPUMASK_H
-
-#include <asm-generic/cpumask.h>
-
-#endif /* _ASM_PARISC_CPUMASK_H */
Index: 2.6.5.bitmap/include/asm-ppc/cpumask.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-ppc/cpumask.h 2004-04-08 01:08:17.000000000 -0700
+++ 2.6.5.bitmap/include/asm-ppc/cpumask.h 1969-12-31 16:00:00.000000000 -0800
@@ -1,6 +0,0 @@
-#ifndef _ASM_PPC_CPUMASK_H
-#define _ASM_PPC_CPUMASK_H
-
-#include <asm-generic/cpumask.h>
-
-#endif /* _ASM_PPC_CPUMASK_H */
Index: 2.6.5.bitmap/include/asm-ppc64/cpumask.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-ppc64/cpumask.h 2004-04-08 01:08:17.000000000 -0700
+++ 2.6.5.bitmap/include/asm-ppc64/cpumask.h 1969-12-31 16:00:00.000000000 -0800
@@ -1,6 +0,0 @@
-#ifndef _ASM_PPC64_CPUMASK_H
-#define _ASM_PPC64_CPUMASK_H
-
-#include <asm-generic/cpumask.h>
-
-#endif /* _ASM_PPC64_CPUMASK_H */
Index: 2.6.5.bitmap/include/asm-s390/cpumask.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-s390/cpumask.h 2004-04-08 01:08:17.000000000 -0700
+++ 2.6.5.bitmap/include/asm-s390/cpumask.h 1969-12-31 16:00:00.000000000 -0800
@@ -1,6 +0,0 @@
-#ifndef _ASM_S390_CPUMASK_H
-#define _ASM_S390_CPUMASK_H
-
-#include <asm-generic/cpumask.h>
-
-#endif /* _ASM_S390_CPUMASK_H */
Index: 2.6.5.bitmap/include/asm-sh/cpumask.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-sh/cpumask.h 2004-04-08 01:08:17.000000000 -0700
+++ 2.6.5.bitmap/include/asm-sh/cpumask.h 1969-12-31 16:00:00.000000000 -0800
@@ -1,6 +0,0 @@
-#ifndef _ASM_SH_CPUMASK_H
-#define _ASM_SH_CPUMASK_H
-
-#include <asm-generic/cpumask.h>
-
-#endif /* _ASM_SH_CPUMASK_H */
Index: 2.6.5.bitmap/include/asm-sparc/cpumask.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-sparc/cpumask.h 2004-04-08 01:08:17.000000000 -0700
+++ 2.6.5.bitmap/include/asm-sparc/cpumask.h 1969-12-31 16:00:00.000000000 -0800
@@ -1,6 +0,0 @@
-#ifndef _ASM_SPARC_CPUMASK_H
-#define _ASM_SPARC_CPUMASK_H
-
-#include <asm-generic/cpumask.h>
-
-#endif /* _ASM_SPARC_CPUMASK_H */
Index: 2.6.5.bitmap/include/asm-sparc64/cpumask.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-sparc64/cpumask.h 2004-04-08 01:08:17.000000000 -0700
+++ 2.6.5.bitmap/include/asm-sparc64/cpumask.h 1969-12-31 16:00:00.000000000 -0800
@@ -1,6 +0,0 @@
-#ifndef _ASM_SPARC64_CPUMASK_H
-#define _ASM_SPARC64_CPUMASK_H
-
-#include <asm-generic/cpumask.h>
-
-#endif /* _ASM_SPARC64_CPUMASK_H */
Index: 2.6.5.bitmap/include/asm-um/cpumask.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-um/cpumask.h 2004-04-08 01:08:17.000000000 -0700
+++ 2.6.5.bitmap/include/asm-um/cpumask.h 1969-12-31 16:00:00.000000000 -0800
@@ -1,6 +0,0 @@
-#ifndef _ASM_UM_CPUMASK_H
-#define _ASM_UM_CPUMASK_H
-
-#include <asm-generic/cpumask.h>
-
-#endif /* _ASM_UM_CPUMASK_H */
Index: 2.6.5.bitmap/include/asm-v850/cpumask.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-v850/cpumask.h 2004-04-08 01:08:17.000000000 -0700
+++ 2.6.5.bitmap/include/asm-v850/cpumask.h 1969-12-31 16:00:00.000000000 -0800
@@ -1,6 +0,0 @@
-#ifndef _ASM_V850_CPUMASK_H
-#define _ASM_V850_CPUMASK_H
-
-#include <asm-generic/cpumask.h>
-
-#endif /* _ASM_V850_CPUMASK_H */
Index: 2.6.5.bitmap/include/asm-x86_64/cpumask.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-x86_64/cpumask.h 2004-04-08 01:08:17.000000000 -0700
+++ 2.6.5.bitmap/include/asm-x86_64/cpumask.h 1969-12-31 16:00:00.000000000 -0800
@@ -1,6 +0,0 @@
-#ifndef _ASM_X86_64_CPUMASK_H
-#define _ASM_X86_64_CPUMASK_H
-
-#include <asm-generic/cpumask.h>
-
-#endif /* _ASM_X86_64_CPUMASK_H */
Index: 2.6.5.bitmap/include/asm-ia64/cpumask.h
===================================================================
--- 2.6.5.bitmap.orig/include/asm-ia64/cpumask.h 2004-04-08 01:08:17.000000000 -0700
+++ 2.6.5.bitmap/include/asm-ia64/cpumask.h 1969-12-31 16:00:00.000000000 -0800
@@ -1,6 +0,0 @@
-#ifndef _ASM_IA64_CPUMASK_H
-#define _ASM_IA64_CPUMASK_H
-
-#include <asm-generic/cpumask.h>
-
-#endif /* _ASM_IA64_CPUMASK_H */
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373
mask10-cpumask-ppc64-fixup - Remove/recode obsolete cpumask macros from arch ppc64
Remove by recoding all uses of the obsolete cpumask const,
coerce and promote macros.
Diffstat Patch_8_of_23:
open_pic.c | 8 ++++----
rtasd.c | 6 +++---
2 files changed, 7 insertions(+), 7 deletions(-)
Index: 2.6.5.mask/arch/ppc64/kernel/open_pic.c
===================================================================
--- 2.6.5.mask.orig/arch/ppc64/kernel/open_pic.c 2004-04-03 23:37:42.000000000 -0800
+++ 2.6.5.mask/arch/ppc64/kernel/open_pic.c 2004-04-03 23:51:59.000000000 -0800
@@ -592,7 +592,7 @@
void openpic_init_processor(u_int cpumask)
{
openpic_write(&OpenPIC->Global.Processor_Initialization,
- physmask(cpumask & cpus_coerce(cpu_online_map)));
+ physmask(cpumask & cpus_addr(cpu_online_map)[0]));
}
#ifdef CONFIG_SMP
@@ -626,7 +626,7 @@
CHECK_THIS_CPU;
check_arg_ipi(ipi);
openpic_write(&OpenPIC->THIS_CPU.IPI_Dispatch(ipi),
- physmask(cpumask & cpus_coerce(cpu_online_map)));
+ physmask(cpumask & cpus_addr(cpu_online_map)[0]));
}
void openpic_request_IPIs(void)
@@ -712,7 +712,7 @@
{
check_arg_timer(timer);
openpic_write(&OpenPIC->Global.Timer[timer].Destination,
- physmask(cpumask & cpus_coerce(cpu_online_map)));
+ physmask(cpumask & cpus_addr(cpu_online_map)[0]));
}
@@ -837,7 +837,7 @@
cpumask_t tmp;
cpus_and(tmp, cpumask, cpu_online_map);
- openpic_mapirq(irq_nr - open_pic_irq_offset, physmask(cpus_coerce(tmp)));
+ openpic_mapirq(irq_nr - open_pic_irq_offset, physmask(cpus_addr(tmp)[0]));
}
#ifdef CONFIG_SMP
Index: 2.6.5.mask/arch/ppc64/kernel/rtasd.c
===================================================================
--- 2.6.5.mask.orig/arch/ppc64/kernel/rtasd.c 2004-04-03 23:37:42.000000000 -0800
+++ 2.6.5.mask/arch/ppc64/kernel/rtasd.c 2004-04-03 23:51:59.000000000 -0800
@@ -411,7 +411,7 @@
}
lock_cpu_hotplug();
- cpu = first_cpu_const(mk_cpumask_const(cpu_online_map));
+ cpu = first_cpu(cpu_online_map);
for (;;) {
set_cpus_allowed(current, cpumask_of_cpu(cpu));
do_event_scan(event_scan);
@@ -425,9 +425,9 @@
schedule_timeout((HZ*60/rtas_event_scan_rate) / 2);
lock_cpu_hotplug();
- cpu = next_cpu_const(cpu, mk_cpumask_const(cpu_online_map));
+ cpu = next_cpu(cpu, cpu_online_map);
if (cpu == NR_CPUS)
- cpu = first_cpu_const(mk_cpumask_const(cpu_online_map));
+ cpu = first_cpu(cpu_online_map);
}
error_vfree:
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373
mask13-cpumask-sparc-simplify - Simplify some sparc64 cpumask loop code
Make use of for_each_cpu_mask() macro to simplify and optimize
a couple of sparc64 per-CPU loops. This code change has _not_
been tested or reviewed. Feedback welcome. There is non-trivial
risk that I still don't understand the logic here.
Index: 2.6.5.bitmap/arch/sparc64/kernel/smp.c
===================================================================
--- 2.6.5.bitmap.orig/arch/sparc64/kernel/smp.c 2004-04-05 02:41:32.000000000 -0700
+++ 2.6.5.bitmap/arch/sparc64/kernel/smp.c 2004-04-08 04:18:02.000000000 -0700
@@ -406,14 +406,8 @@
int i;
__asm__ __volatile__("rdpr %%pstate, %0" : "=r" (pstate));
- for (i = 0; i < NR_CPUS; i++) {
- if (cpu_isset(i, mask)) {
- spitfire_xcall_helper(data0, data1, data2, pstate, i);
- cpu_clear(i, mask);
- if (cpus_empty(mask))
- break;
- }
- }
+ for_each_cpu_mask(i, mask)
+ spitfire_xcall_helper(data0, data1, data2, pstate, i);
}
/* Cheetah now allows to send the whole 64-bytes of data in the interrupt
@@ -456,25 +450,19 @@
nack_busy_id = 0;
{
- cpumask_t work_mask = mask;
int i;
- for (i = 0; i < NR_CPUS; i++) {
- if (cpu_isset(i, work_mask)) {
- u64 target = (i << 14) | 0x70;
-
- if (!is_jalapeno)
- target |= (nack_busy_id << 24);
- __asm__ __volatile__(
- "stxa %%g0, [%0] %1\n\t"
- "membar #Sync\n\t"
- : /* no outputs */
- : "r" (target), "i" (ASI_INTR_W));
- nack_busy_id++;
- cpu_clear(i, work_mask);
- if (cpus_empty(work_mask))
- break;
- }
+ for_each_cpu_mask(i, mask) {
+ u64 target = (i << 14) | 0x70;
+
+ if (!is_jalapeno)
+ target |= (nack_busy_id << 24);
+ __asm__ __volatile__(
+ "stxa %%g0, [%0] %1\n\t"
+ "membar #Sync\n\t"
+ : /* no outputs */
+ : "r" (target), "i" (ASI_INTR_W));
+ nack_busy_id++;
}
}
@@ -507,7 +495,6 @@
printk("CPU[%d]: mondo stuckage result[%016lx]\n",
smp_processor_id(), dispatch_stat);
} else {
- cpumask_t work_mask = mask;
int i, this_busy_nack = 0;
/* Delay some random time with interrupts enabled
@@ -518,22 +505,17 @@
/* Clear out the mask bits for cpus which did not
* NACK us.
*/
- for (i = 0; i < NR_CPUS; i++) {
- if (cpu_isset(i, work_mask)) {
- u64 check_mask;
-
- if (is_jalapeno)
- check_mask = (0x2UL << (2*i));
- else
- check_mask = (0x2UL <<
- this_busy_nack);
- if ((dispatch_stat & check_mask) == 0)
- cpu_clear(i, mask);
- this_busy_nack += 2;
- cpu_clear(i, work_mask);
- if (cpus_empty(work_mask))
- break;
- }
+ for_each_cpu_mask(i, mask) {
+ u64 check_mask;
+
+ if (is_jalapeno)
+ check_mask = (0x2UL << (2*i));
+ else
+ check_mask = (0x2UL <<
+ this_busy_nack);
+ if ((dispatch_stat & check_mask) == 0)
+ cpu_clear(i, mask);
+ this_busy_nack += 2;
}
goto retry;
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373
Hi Paul,
This patch gets the new bitmap stuff to compile and pass a boot-test
on Opteron.
I especially like the cleaned-up, orthogonal API and your addition of
the missing bitmap_andnot() function. Good work!
Regards,
Joe
"Money can buy bandwidth, but latency is forever" -- John Mashey
diff -ura 2.6.5-pj/arch/x86_64/kernel/smpboot.c 2.6.5-pj2/arch/x86_64/kernel/smpboot.c
--- 2.6.5-pj/arch/x86_64/kernel/smpboot.c 2004-04-03 22:38:26.000000000 -0500
+++ 2.6.5-pj2/arch/x86_64/kernel/smpboot.c 2004-04-22 12:06:02.209264270 -0400
@@ -827,7 +827,7 @@
if (apicid == boot_cpu_id)
continue;
- if (!cpu_isset(apicid, phys_cpu_present_map))
+ if (!physid_isset(apicid, phys_cpu_present_map))
continue;
if ((max_cpus >= 0) && (max_cpus <= cpucount+1))
continue;
diff -ura 2.6.5-pj/include/asm-x86_64/smp.h 2.6.5-pj2/include/asm-x86_64/smp.h
--- 2.6.5-pj/include/asm-x86_64/smp.h 2004-04-22 12:19:58.665010667 -0400
+++ 2.6.5-pj2/include/asm-x86_64/smp.h 2004-04-22 11:43:49.190316603 -0400
@@ -60,7 +60,6 @@
extern cpumask_t cpu_callout_map;
#define cpu_possible_map cpu_callout_map
-#define cpu_online(cpu) cpu_isset(cpu, cpu_online_map)
static inline int num_booting_cpus(void)
{
@@ -85,7 +84,6 @@
#define safe_smp_processor_id() (disable_apic ? 0 : x86_apicid_to_cpu[hard_smp_processor_id()])
-#define cpu_online(cpu) cpu_isset(cpu, cpu_online_map)
#endif /* !ASSEMBLY */
#define NO_PROC_ID 0xFF /* No processor magic marker */
> This patch gets the new bitmap stuff to compile and pass a boot-test
> on Opteron.
Ok - thanks for testing it. I'll add that patch. And thanks for the
favorable feedback.
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <[email protected]> 1.650.933.1373