We are pleased to announce v2 of the unified arch/x86 project we are
working on.
To recap: the core idea behind our project is simple to describe: we
introduce a new arch/x86/ and include/asm-x86/ file hierarchy that
includes all the existing 32-bit and 64-bit x86 code and allows the
building of either a 32-bit (i386) kernel or a 64-bit (x86_64) kernel.
No code is lost and almost no code is modified. We want the window
for regressions and tree incompatibilities to be zero. (The full initial
announcement can be found at http://lkml.org/lkml/2007/7/20/447)
The main technical request for improvement we received was the lack of a
patch-series, and this v2 release does that, in form of a 600+ commits
git tree:
git://git.kernel.org/pub/scm/linux/kernel/git/tglx/linux-2.6-x86.git
One (large!) combo patch is available at:
http://kernel.org/pub/linux/kernel/people/tglx/linux-x86.2.6.23-rc3-git-cb00e99c-combo.patch.bz2
The tree is fully bisectable and it does a finegrained, gradual
move-over from arch/i386, arch/x86_64 into the unified arch/x86 tree,
with a total effect of:
1735 files changed, 167230 insertions(+), 170368 deletions(-)
On file level:
mainline arch/x86 delta
Total 905 842 - 63 files
(see the full stats further below)
The tree (and the combo patch) is against commit
cb00e99c0abd844b884c64c6b54aa3b7d345ebb1 of Linus's git tree.
More details about v2:
The patch series contains 9 steps of modifications:
1) Preparatory patches, which simplify the scripted conversion
(Makefile cleanups, Kbuild infrastructure changes)
2) Scripted analysis of the i386 and the x86_64 source tree to
identify source files, which exist in both trees with the same
name. These files are checked, whether they are identical or simply
include the corresponding source file from the other tree. If the
files are different and do not include the other file, the file is
renamed with a _32 resp. _64 extension and the Makefiles are fixed
up. Makefiles in corresponding directories are renamed as well and
a stub Makefile which selects the include of the _32 or the _64
Makefile variants depending on the config option CONFIG_X86_32 is
created.
3) Scripted move of the files on a per directory basis from arch/i386
and arch/x86_64 to arch/x86. The files which are just providing an
include of the corresponding source file in the other source tree
are eliminated. In each step a fixup of the Makefiles is done to
keep the tree intact.
The only remaining files in arch/i386 and arch/x86_64 are the main
Makefiles, Kconfig files and the defconfigs.
4) Scripted analysis of the exported headers in include/asm-i386 and
include/asm-x86_64. This is similar to step 2). The identical
header files are identified along with those, which just include
the corresponding header file from the other include path. The
header files which have the same name in both directories and which
are different are renamed with a _32 resp. _64 extension and a stub
header file which includes the 32 or the 64 bit version depending
on the __i386__ define is created. The Kbuild files are updated.
5) Scripted analysis of the not exported headers in include/asm-i386
and include/asm-x86_64. This is mostly the same as step 4). The
stub file include switch depends on the config option
CONFIG_X86_32.
6) Scripted move of all header files in include/asm-i386 and
include/asm/x86_64 to include/asm-x86. Makefile references are
fixed up automatically, the Kbuild file is unified and a patch to
the build system is applied which fixes the header check and
exports. After this step the header export is not longer a bi-arch
export. All headers along with the stub files are exported in one
go.
7) Scripted fixup of Makefile references. During the conversion cross
tree references like "../../i386/kernel/" have been replaced by
"../../x86/kernel". This script makes these references relative to
the new unified source path.
8) Scripted analysis of _32 and _64 header files in include/asm-x86.
The script replaces the defines and ifdefs, which protect from
recursive includes, by a unified automatically generated version
and removes white space damage from the files. After this the files
are compared. If they are identical the newly created file is
written instead of the _32 and _64 files. Otherwise the original
files are left intact. After this commit a second commit is done
which replaces the stub header file by the new unified variant and
removes the _32 and _64 variants.
9) Semi scripted unification of _32 and _64 file variants. The script
identified files which are not really different, e.g. white space
damage, comment changes, small modifications. The result of the
script is manually fixed up and patches are generated. These steps
have been carefully verified. The unified files are header files,
source files and Makefiles.
All 600+ bisection points have been build-tested (and a number of
boot-tests were done too), the end-result was build and boot tested on
x86-32 and x86-64 as well, and passes allyesconfig, allmodconfig and
allnoconfig builds.
Some statistical information:
arch/i386 unique files: 255
arch/x86_64 unique files: 76
arch/x86_64 reused files: 65 (Makefile ref to arch/i386/..)
equal files: 0
included files: 6
same name files: 124
include/asm-i386 unique headers: 138
include/asm-x86_64 unique headers: 45
include/asm-x86_64 orphaned headers: 1 (deleted)
equal headers: 7
included headers: 14
same name headers: 254
Unified headerfiles by script: 21
Unified Makefiles (script aided) 4
Unified header files (script aided) 59
Unified source files (script aided) 3
mainline arch/x86 delta
--------------------------------------------------
Source i386 335 6 -329 files
Source x86_64 153 4 -149 files
Source x86 0 465 +465 files
Source total 488 475 - 13 files
Header i386 243 0 -243 files
Header x86_64 174 0 -174 files
Header x86 0 367 +367 files
Header total 417 367 - 50 files
--------------------------------------------------
Total 905 842 - 63 files
Source i386 97918 3420 - 94498 lines
Source x86_64 41309 2394 - 38915 lines
Source x86 0 133062 +133062 lines
Source total 139227 138876 - 351 lines
Header i386 22779 0 - 22779 lines
Header x86_64 14105 0 - 14105 lines
Header x86 0 34092 + 34092 lines
Header total 36884 34092 - 2792 lines
--------------------------------------------------
Total 176111 172968 - 3143 lines
As usual, comments and suggestions are welcome!
Thomas, Ingo
On Wed, Aug 22, 2007 at 11:56:22PM +0200, Thomas Gleixner wrote:
> We are pleased to announce v2 of the unified arch/x86 project we are
> working on.
>
> To recap: the core idea behind our project is simple to describe: we
> introduce a new arch/x86/ and include/asm-x86/ file hierarchy that
> includes all the existing 32-bit and 64-bit x86 code and allows the
> building of either a 32-bit (i386) kernel or a 64-bit (x86_64) kernel.
> No code is lost and almost no code is modified. We want the window
> for regressions and tree incompatibilities to be zero. (The full initial
> announcement can be found at http://lkml.org/lkml/2007/7/20/447)
>
> The main technical request for improvement we received was the lack of a
> patch-series, and this v2 release does that, in form of a 600+ commits
> git tree:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/tglx/linux-2.6-x86.git
Nice work. cpufreq bits look sane from a quick eyeball.
Dave
--
http://www.codemonkey.org.uk
On Wed, Aug 22, 2007 at 11:56:22PM +0200, Thomas Gleixner wrote:
> We are pleased to announce v2 of the unified arch/x86 project we are
> working on.
After having recently inadvertently broken i386 while making x86-64
changes, thumbs up from me!
Cheers,
Muli
On Wed, 22 Aug 2007 23:56:22 +0200
Thomas Gleixner <[email protected]> wrote:
> We are pleased to announce v2 of the unified arch/x86 project we are
> working on.
>
> To recap: the core idea behind our project is simple to describe: we
> introduce a new arch/x86/ and include/asm-x86/ file hierarchy that
> includes all the existing 32-bit and 64-bit x86 code and allows the
> building of either a 32-bit (i386) kernel or a 64-bit (x86_64)
> kernel. No code is lost and almost no code is modified. We want the
> window for regressions and tree incompatibilities to be zero. (The
> full initial announcement can be found at
> http://lkml.org/lkml/2007/7/20/447)
>
as said before I like this a lot ;)
there's still some low hanging fruit left:
[arjan@laptopd505 x86]$ for i in */*_32.c */*/*_32.c ; do perl same_function.pl $i ; done
Identical function: crypto/aes_32.c - aes_fini
Identical function: crypto/aes_32.c - aes_encrypt
Identical function: crypto/aes_32.c - aes_init
Identical function: crypto/aes_32.c - aes_decrypt
Identical function: kernel/apic_32.c - init_lapic_sysfs
Identical function: kernel/hpet_32.c - read_hpet
Identical function: kernel/io_apic_32.c - io_apic_base
Identical function: kernel/io_apic_32.c - irq_polarity
Identical function: kernel/io_apic_32.c - ioapic_read_entry
Identical function: kernel/io_apic_32.c - ioapic_init_sysfs
Identical function: kernel/io_apic_32.c - irq_trigger
Identical function: kernel/io_apic_32.c - ioapic_mask_entry
Identical function: kernel/io_apic_32.c - find_irq_entry
Identical function: kernel/io_apic_32.c - io_apic_write
Identical function: kernel/io_apic_32.c - io_apic_read
Identical function: kernel/io_apic_32.c - startup_ioapic_irq
Identical function: kernel/io_apic_32.c - ioapic_write_entry
Identical function: kernel/kprobes_32.c - arch_disarm_kprobe
Identical function: kernel/kprobes_32.c - arch_arm_kprobe
Identical function: kernel/ldt_32.c - read_ldt
Identical function: kernel/ldt_32.c - copy_ldt
Identical function: kernel/ldt_32.c - flush_ldt
Identical function: kernel/mpparse_32.c - mpf_checksum
Identical function: kernel/nmi_32.c - lapic_nmi_resume
Identical function: kernel/nmi_32.c - lapic_nmi_suspend
Identical function: kernel/nmi_32.c - __acpi_nmi_disable
Identical function: kernel/pci-dma_32.c - via_no_dac
Identical function: kernel/process_32.c - play_dead
Identical function: kernel/setup_32.c - copy_edd
Identical function: mm/fault_32.c - notify_page_fault
Identical function: mm/pageattr_32.c - flush_map
Identical function: boot/compressed/misc_32.c - putstr
Identical function: boot/compressed/misc_32.c - scroll
Identical function: boot/compressed/misc_32.c - memcpy
Identical function: boot/compressed/misc_32.c - free
Identical function: boot/compressed/misc_32.c - memset
Identical function: boot/compressed/misc_32.c - error
Identical function: boot/compressed/misc_32.c - flush_window
Identical function: boot/compressed/misc_32.c - fill_inbuf
Identical function: boot/compressed/misc_32.c - gzip_mark
Identical function: kernel/acpi/sleep_32.c - acpi_sleep_setup
I'll get someone in my team to investigate merging these in the next few days and send patches to you for them.