Almost every architecture generates Assembler values to map the offsets
of fields in C structures, about the only exception is i386 and that is
because its offsets are hard coded into entry.S. Every arch has done
it differently, none of them have got it exactly right.
As part of kbuild 2.5 I am standardizing on one method for generating
Assembler offsets. This change is required for kbuild 2.5 but it can
be added to 2.4 without disturbing the current kbuild, I want to do
this gradually now instead of a single massive change in kernel 2.5. I
will be issuing per architecture changes for generating Assembler
offsets against 2.4.
The kbuild 2.5 method for generating Assembler offsets satisfies these
requirements:
* No manual intervention required. Many architectures rely on users
running make dep after changing config options that affect the
Assembler offsets. If the user forgets to run make dep then the C
and Assembler code is out of sync - totally unacceptable. This is
completely fixed in kbuild 2.5; I cannot do a complete fix in kbuild
2.4 but it is still better than the existing manual system.
* Standard name for the related files. There are 6+ different names
for the files used to generate Assembler offsets, kbuild 2.5 uses
asm-offsets.[csh] on all architectures.
* Allows for multiple parallel compiles from the same source tree.
Writing the generated asm-offsets.h to include/asm is not an option,
it prevents concurrent compiles.
* The method must work in native and cross compile mode and give
exactly the same results. Some 2.4 code only works in native mode,
some architectures have different methods for native and cross
compile with different output formats. Yeuch!
* Standard scripts for generating the output. Every arch does it
differently in 2.4, standards are good!
* Correct dependency trees. Because 2.4 make dep does not scan .S
files, there is little or no dependency information. Even if the
offsets are regenerated, the affected Assembler code does not always
get rebuilt. kbuild 2.5 handles dependencies for Assembler as well
as C; I cannot get kbuild 2.4 perfect but I can improve on the
existing (or non-existent) 2.4 dependencies.
All architectures will define arch/$(ARCH)/asm-offsets.c. This has a
standard prologue for the macros that convert offsets to Assembler,
followed by arch specific field references.
arch/$(ARCH)/asm-offsets.s is generated from arch/$(ARCH)/asm-offsets.c
using standard rules, although kbuild 2.4 needs some tweaking.
arch/$(ARCH)/asm-offsets.h is generated from arch/$(ARCH)/asm-offsets.s
by a semi-standard script. Most of the script is common to all
architectures but the precise format of the Assembler output is arch
specific.
The final result is included in *only* the Assembler programs that need
it, as #include "asm-offsets.h" with -I arch/$(ARCH) in the relevant
Makefiles. Hard coding relative paths in source files is a pet hate,
use #include "localname.h" and -I instead. Including the generated
file in C code is not allowed, it severly pollutes the dependency
chain, to the extent that any config change can force a complete
recompile, unacceptable.
Example from i386:
arch/i386/asm-offsets.c
/*
* Generate definitions needed by assembly language modules.
* This code generates raw asm output which is post-processed to extract
* and format the required data.
*/
#include <linux/types.h>
#include <linux/stddef.h>
#include <linux/sched.h>
/* Use marker if you need to separate the values later */
#define DEFINE(sym, val, marker) \
asm volatile("\n-> " #sym " %0 " #val " " #marker : : "i" (val))
#define BLANK() asm volatile("\n->" : : )
int
main(void)
{
DEFINE(state, offsetof(struct task_struct, state),);
DEFINE(flags, offsetof(struct task_struct, flags),);
DEFINE(sigpending, offsetof(struct task_struct, sigpending),);
DEFINE(addr_limit, offsetof(struct task_struct, addr_limit),);
DEFINE(exec_domain, offsetof(struct task_struct, exec_domain),);
DEFINE(need_resched, offsetof(struct task_struct, need_resched),);
DEFINE(tsk_ptrace, offsetof(struct task_struct, ptrace),);
DEFINE(processor, offsetof(struct task_struct, processor),);
BLANK();
DEFINE(ENOSYS, ENOSYS,);
return 0;
}
asm-offsets.s to asm-offsets.h.
# Convert raw asm offsets into something that can be included as
# assembler definitions. It converts
# -> symbol $value source
# into
# symbol = value /* 0xvalue source */
echo '#ifndef __ASM_OFFSETS_H__'
echo '#define __ASM_OFFSETS_H__'
echo '/*'
echo ' * DO NOT MODIFY'
echo ' *'
echo " * This file was generated by arch/$(ARCH)/Makefile.in."
echo ' *'
echo ' */'
echo ''
awk '
/^->$/{printf("\n")}
/^-> /{
sym = $2;
val = $3;
sub(/^\$/, "", val);
$1 = "";
$2 = "";
$3 = "";
printf("%-20s = %3d\t/* 0x%x\t%s */\n", sym, val, val, $0)
}
'
echo '#endif'
Generated arch/i386/asm-offsets.h
#ifndef __ASM_OFFSETS_H__
#define __ASM_OFFSETS_H__
/*
* DO NOT MODIFY
*
* This file was generated by arch/i386/Makefile.in.
*
*/
state = 0 /* 0x0 offsetof(struct task_struct, state) */
flags = 4 /* 0x4 offsetof(struct task_struct, flags) */
sigpending = 8 /* 0x8 offsetof(struct task_struct, sigpending) */
addr_limit = 12 /* 0xc offsetof(struct task_struct, addr_limit) */
exec_domain = 16 /* 0x10 offsetof(struct task_struct, exec_domain) */
need_resched = 20 /* 0x14 offsetof(struct task_struct, need_resched) */
tsk_ptrace = 24 /* 0x18 offsetof(struct task_struct, ptrace) */
processor = 52 /* 0x34 offsetof(struct task_struct, processor) */
ENOSYS = 38 /* 0x26 ENOSYS */
#endif
This sound great. It has been a struggle for me for some time.
A couple of comments:
The symbol name IMHO should contain both the member name and the
structure name. Otherwise there may be a problem if two structures use
the same member name (flags comes to mind).
This is way down on the list, but is it possible to generate a separate
file for each *.S AND put the "required symbols" in the *.S.
When I last did this we put a comment in the *.S that had a special
format and changed the make rule for *.S->*.o to first build the
required file. The *.S then used #include "sym*.h" to get the symbols.
I think the standard make rule even removed the sys*.h file after the
*.o was built. The actual section in the *.S that defined the required
symbols was written as cpp macros which were stripped with a sed
script. The result (with a standard header and tail) was compiled and
run to generate the sym*.h file. Even the includes needed to generate
the symbols were in the *.S so a dep script can generate the needed
stuff for the *.S.
The nice thing about this method is that it is easy to keep track of the
required symbols and the make dependencies are simple (and easily
derived).
George
Keith Owens wrote:
>
> Almost every architecture generates Assembler values to map the offsets
> of fields in C structures, about the only exception is i386 and that is
> because its offsets are hard coded into entry.S. Every arch has done
> it differently, none of them have got it exactly right.
>
> As part of kbuild 2.5 I am standardizing on one method for generating
> Assembler offsets. This change is required for kbuild 2.5 but it can
> be added to 2.4 without disturbing the current kbuild, I want to do
> this gradually now instead of a single massive change in kernel 2.5. I
> will be issuing per architecture changes for generating Assembler
> offsets against 2.4.
>
> The kbuild 2.5 method for generating Assembler offsets satisfies these
> requirements:
>
> * No manual intervention required. Many architectures rely on users
> running make dep after changing config options that affect the
> Assembler offsets. If the user forgets to run make dep then the C
> and Assembler code is out of sync - totally unacceptable. This is
> completely fixed in kbuild 2.5; I cannot do a complete fix in kbuild
> 2.4 but it is still better than the existing manual system.
>
> * Standard name for the related files. There are 6+ different names
> for the files used to generate Assembler offsets, kbuild 2.5 uses
> asm-offsets.[csh] on all architectures.
>
> * Allows for multiple parallel compiles from the same source tree.
> Writing the generated asm-offsets.h to include/asm is not an option,
> it prevents concurrent compiles.
>
> * The method must work in native and cross compile mode and give
> exactly the same results. Some 2.4 code only works in native mode,
> some architectures have different methods for native and cross
> compile with different output formats. Yeuch!
>
> * Standard scripts for generating the output. Every arch does it
> differently in 2.4, standards are good!
>
> * Correct dependency trees. Because 2.4 make dep does not scan .S
> files, there is little or no dependency information. Even if the
> offsets are regenerated, the affected Assembler code does not always
> get rebuilt. kbuild 2.5 handles dependencies for Assembler as well
> as C; I cannot get kbuild 2.4 perfect but I can improve on the
> existing (or non-existent) 2.4 dependencies.
>
> All architectures will define arch/$(ARCH)/asm-offsets.c. This has a
> standard prologue for the macros that convert offsets to Assembler,
> followed by arch specific field references.
>
> arch/$(ARCH)/asm-offsets.s is generated from arch/$(ARCH)/asm-offsets.c
> using standard rules, although kbuild 2.4 needs some tweaking.
>
> arch/$(ARCH)/asm-offsets.h is generated from arch/$(ARCH)/asm-offsets.s
> by a semi-standard script. Most of the script is common to all
> architectures but the precise format of the Assembler output is arch
> specific.
>
> The final result is included in *only* the Assembler programs that need
> it, as #include "asm-offsets.h" with -I arch/$(ARCH) in the relevant
> Makefiles. Hard coding relative paths in source files is a pet hate,
> use #include "localname.h" and -I instead. Including the generated
> file in C code is not allowed, it severly pollutes the dependency
> chain, to the extent that any config change can force a complete
> recompile, unacceptable.
>
> Example from i386:
>
> arch/i386/asm-offsets.c
>
> /*
> * Generate definitions needed by assembly language modules.
> * This code generates raw asm output which is post-processed to extract
> * and format the required data.
> */
>
> #include <linux/types.h>
> #include <linux/stddef.h>
> #include <linux/sched.h>
>
> /* Use marker if you need to separate the values later */
>
> #define DEFINE(sym, val, marker) \
> asm volatile("\n-> " #sym " %0 " #val " " #marker : : "i" (val))
>
> #define BLANK() asm volatile("\n->" : : )
>
> int
> main(void)
> {
> DEFINE(state, offsetof(struct task_struct, state),);
> DEFINE(flags, offsetof(struct task_struct, flags),);
> DEFINE(sigpending, offsetof(struct task_struct, sigpending),);
> DEFINE(addr_limit, offsetof(struct task_struct, addr_limit),);
> DEFINE(exec_domain, offsetof(struct task_struct, exec_domain),);
> DEFINE(need_resched, offsetof(struct task_struct, need_resched),);
> DEFINE(tsk_ptrace, offsetof(struct task_struct, ptrace),);
> DEFINE(processor, offsetof(struct task_struct, processor),);
> BLANK();
> DEFINE(ENOSYS, ENOSYS,);
> return 0;
> }
>
> asm-offsets.s to asm-offsets.h.
>
> # Convert raw asm offsets into something that can be included as
> # assembler definitions. It converts
> # -> symbol $value source
> # into
> # symbol = value /* 0xvalue source */
>
> echo '#ifndef __ASM_OFFSETS_H__'
> echo '#define __ASM_OFFSETS_H__'
> echo '/*'
> echo ' * DO NOT MODIFY'
> echo ' *'
> echo " * This file was generated by arch/$(ARCH)/Makefile.in."
> echo ' *'
> echo ' */'
> echo ''
> awk '
> /^->$/{printf("\n")}
> /^-> /{
> sym = $2;
> val = $3;
> sub(/^\$/, "", val);
> $1 = "";
> $2 = "";
> $3 = "";
> printf("%-20s = %3d\t/* 0x%x\t%s */\n", sym, val, val, $0)
> }
> '
> echo '#endif'
>
> Generated arch/i386/asm-offsets.h
>
> #ifndef __ASM_OFFSETS_H__
> #define __ASM_OFFSETS_H__
> /*
> * DO NOT MODIFY
> *
> * This file was generated by arch/i386/Makefile.in.
> *
> */
>
> state = 0 /* 0x0 offsetof(struct task_struct, state) */
> flags = 4 /* 0x4 offsetof(struct task_struct, flags) */
> sigpending = 8 /* 0x8 offsetof(struct task_struct, sigpending) */
> addr_limit = 12 /* 0xc offsetof(struct task_struct, addr_limit) */
> exec_domain = 16 /* 0x10 offsetof(struct task_struct, exec_domain) */
> need_resched = 20 /* 0x14 offsetof(struct task_struct, need_resched) */
> tsk_ptrace = 24 /* 0x18 offsetof(struct task_struct, ptrace) */
> processor = 52 /* 0x34 offsetof(struct task_struct, processor) */
>
> ENOSYS = 38 /* 0x26 ENOSYS */
> #endif
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
On Thu, 04 Oct 2001 08:36:54 -0700,
george anzinger <[email protected]> wrote:
>The symbol name IMHO should contain both the member name and the
>structure name. Otherwise there may be a problem if two structures use
>the same member name (flags comes to mind).
The asm symbol name is arch defined, I am defining the standard method,
not the asm names. It is up to the arch maintainers to pick suitable
names, e.g. ia64 does
DEFINE(IA64_SWITCH_STACK_AR_BSPSTORE_OFFSET, offsetof(struct switch_stack, ar_bspstore),);
>This is way down on the list, but is it possible to generate a separate
>file for each *.S AND put the "required symbols" in the *.S.
That works for a small number of mappings but not when there are a
large number that are required in several places. Take a look at
arch/ia64/tools/print_offsets.c, 130+ mappings used by 5 or 6 different
asm sources. There are also technical reasons (to do with the kernel
CONFIG system) why a single asm-offsets file is easier to maintain.
First patch to clean up Assembler offsets, as described in
http://marc.theaimsgroup.com/?l=linux-kernel&m=100219616028442&w=2
This patch is larger than normal because my testing highlighted a long
standing race in the top level kernel Makefile.
i386 patch. Yes, I know that doing asm-offsets.c for i386 is overkill,
it only needs a few values that "never" change so they can be hard
coded. However I need a reference implementation, plus the race needs
to be mitigated.
BTW, this is *much* cleaner in kbuild 2.5. Bits of the patch are
workarounds for recursive make problems and to handle mkdep not
processing Assembler dependencies :(.
Index: 11-pre4.1/Makefile
--- 11-pre4.1/Makefile Fri, 05 Oct 2001 15:05:09 +1000 kaos (linux-2.4/T/c/50_Makefile 1.1.2.15.1.2.2.25.2.2.1.17.1.4.1.29 644)
+++ 11-pre4.1(w)/Makefile Sat, 06 Oct 2001 15:07:25 +1000 kaos (linux-2.4/T/c/50_Makefile 1.1.2.15.1.2.2.25.2.2.1.17.1.4.1.29 644)
@@ -205,7 +205,8 @@ CLEAN_FILES = \
drivers/scsi/53c700-mem.c \
net/khttpd/make_times_h \
net/khttpd/times.h \
- submenu*
+ submenu* \
+ arch/$(ARCH)/asm-offsets.[sh]
# directories removed with 'make clean'
CLEAN_DIRS = \
modules
@@ -245,9 +246,9 @@ export CPPFLAGS CFLAGS AFLAGS
export NETWORKS DRIVERS LIBS HEAD LDFLAGS LINKFLAGS MAKEBOOT ASFLAGS
.S.s:
- $(CPP) $(AFLAGS) -traditional -o $*.s $<
+ $(CPP) $(AFLAGS) $(AFLAGS_$@) -traditional -o $@ $<
.S.o:
- $(CC) $(AFLAGS) -traditional -c -o $*.o $<
+ $(CC) $(AFLAGS) $(AFLAGS_$@) -traditional -c -o $@ $<
Version: dummy
@rm -f include/linux/compile.h
@@ -273,29 +274,63 @@ symlinks:
mkdir include/linux/modules; \
fi
-oldconfig: symlinks
+# There is a race here against anything built by the top level Makefile or by
+# arch/$(ARCH)/Makefile (directly included by the top level Makefile).
+
+# split-include changes timestamps on config settings without telling make.
+# Because make is not aware that config files have been changed, the timestamp
+# checks in .hdepend (see Rules.make) use the old times for config files.
+# Therefore anything built at the same time that split-include is running gets
+# wrong timestamps for config settings and for the include files that depend on
+# config settings.
+
+# I cannot completely fix this in kbuild 2.4 without moving most of the top
+# level Makefile into a second file and doing make -C on that file after
+# split-include has run. Explicitly running split-include after *config instead
+# of delaying until make dep mitigates the race, although it does not completely
+# remove it.
+
+# The race is not a problem for SUBDIRS processed from the top level Makefile
+# because split-include has completed before they are entered. However it is a
+# problem for top level builds and for directories invoked from
+# arch/$(ARCH)/Makefile without using SUBDIRS.
+
+# kbuild 2.5 has a clean separation between configuration (top level and
+# scripts/Makefile) and the rest of the kernel build, so it does not suffer from
+# this race. Also kbuild 2.5 splits arch/$(ARCH)/Makefile into definitions
+# (Makefile.defs.{no}config) and kernel building commands (Makefile.in).
+# KAO
+
+split_include := scripts/split-include include/linux/autoconf.h include/config && touch include/config/MARKER
+
+oldconfig: symlinks scripts/split-include
$(CONFIG_SHELL) scripts/Configure -d arch/$(ARCH)/config.in
+ @$(split_include)
-xconfig: symlinks
+xconfig: symlinks scripts/split-include
$(MAKE) -C scripts kconfig.tk
wish -f scripts/kconfig.tk
+ @$(split_include)
-menuconfig: include/linux/version.h symlinks
+menuconfig: include/linux/version.h symlinks scripts/split-include
$(MAKE) -C scripts/lxdialog all
$(CONFIG_SHELL) scripts/Menuconfig arch/$(ARCH)/config.in
+ @$(split_include)
-config: symlinks
+config: symlinks scripts/split-include
$(CONFIG_SHELL) scripts/Configure arch/$(ARCH)/config.in
+ @$(split_include)
include/config/MARKER: scripts/split-include include/linux/autoconf.h
- scripts/split-include include/linux/autoconf.h include/config
- @ touch include/config/MARKER
+ @$(split_include)
linuxsubdirs: $(patsubst %, _dir_%, $(SUBDIRS))
-$(patsubst %, _dir_%, $(SUBDIRS)) : dummy include/linux/version.h include/config/MARKER
+$(patsubst %, _dir_%, $(SUBDIRS)) : dummy include/linux/version.h include/config/MARKER before_subdirs
$(MAKE) CFLAGS="$(CFLAGS) $(CFLAGS_KERNEL)" -C $(patsubst _dir_%, %, $@)
+.PHONY: before_subdirs
+
$(TOPDIR)/include/linux/version.h: include/linux/version.h
$(TOPDIR)/include/linux/compile.h: include/linux/compile.h
@@ -356,7 +391,7 @@ endif
modules: $(patsubst %, _mod_%, $(SUBDIRS))
.PHONY: $(patsubst %, _mod_%, $(SUBDIRS))
-$(patsubst %, _mod_%, $(SUBDIRS)) : include/linux/version.h include/config/MARKER
+$(patsubst %, _mod_%, $(SUBDIRS)) : include/linux/version.h include/config/MARKER before_subdirs
$(MAKE) -C $(patsubst _mod_%, %, $@) CFLAGS="$(CFLAGS) $(MODFLAGS)" MAKING_MODULES=1 modules
.PHONY: modules_install
Index: 11-pre4.1/arch/i386/asm-offsets.c
--- 11-pre4.1/arch/i386/asm-offsets.c Sat, 06 Oct 2001 15:07:38 +1000 kaos ()
+++ 11-pre4.1(w)/arch/i386/asm-offsets.c Fri, 05 Oct 2001 21:53:18 +1000 kaos (linux-2.4/q/f/50_asm-offset 644)
@@ -0,0 +1,32 @@
+/*
+ * Generate definitions needed by assembly language modules.
+ * This code generates raw asm output which is post-processed to extract
+ * and format the required data.
+ */
+
+#include <linux/types.h>
+#include <linux/stddef.h>
+#include <linux/sched.h>
+
+/* Use marker if you need to separate the values later */
+
+#define DEFINE(sym, val, marker) \
+ asm volatile("\n-> " #sym " %0 " #val " " #marker : : "i" (val))
+
+#define BLANK() asm volatile("\n->" : : )
+
+int
+main(void)
+{
+ DEFINE(state, offsetof(struct task_struct, state),);
+ DEFINE(flags, offsetof(struct task_struct, flags),);
+ DEFINE(sigpending, offsetof(struct task_struct, sigpending),);
+ DEFINE(addr_limit, offsetof(struct task_struct, addr_limit),);
+ DEFINE(exec_domain, offsetof(struct task_struct, exec_domain),);
+ DEFINE(need_resched, offsetof(struct task_struct, need_resched),);
+ DEFINE(tsk_ptrace, offsetof(struct task_struct, ptrace),);
+ DEFINE(processor, offsetof(struct task_struct, processor),);
+ BLANK();
+ DEFINE(ENOSYS, ENOSYS,);
+ return 0;
+}
Index: 11-pre4.1/arch/i386/kernel/entry.S
--- 11-pre4.1/arch/i386/kernel/entry.S Mon, 01 Oct 2001 12:23:40 +1000 kaos (linux-2.4/S/c/24_entry.S 1.1.5.5 644)
+++ 11-pre4.1(w)/arch/i386/kernel/entry.S Sat, 06 Oct 2001 13:04:03 +1000 kaos (linux-2.4/S/c/24_entry.S 1.1.5.5 644)
@@ -68,20 +68,7 @@ IF_MASK = 0x00000200
NT_MASK = 0x00004000
VM_MASK = 0x00020000
-/*
- * these are offsets into the task-struct.
- */
-state = 0
-flags = 4
-sigpending = 8
-addr_limit = 12
-exec_domain = 16
-need_resched = 20
-tsk_ptrace = 24
-processor = 52
-
-ENOSYS = 38
-
+#include "asm-offsets.h"
#define SAVE_ALL \
cld; \
Index: 11-pre4.1/arch/i386/kernel/Makefile
--- 11-pre4.1/arch/i386/kernel/Makefile Wed, 19 Sep 2001 14:59:20 +1000 kaos (linux-2.4/S/c/21_Makefile 1.1.2.1 644)
+++ 11-pre4.1(w)/arch/i386/kernel/Makefile Fri, 05 Oct 2001 22:37:10 +1000 kaos (linux-2.4/S/c/21_Makefile 1.1.2.1 644)
@@ -8,7 +8,7 @@
# Note 2! The CFLAGS definitions are now in the main makefile...
.S.o:
- $(CC) $(AFLAGS) -traditional -c $< -o $*.o
+ $(CC) $(AFLAGS) $(EXTRA_AFLAGS) $(AFLAGS_$@) -traditional -c $< -o $@
all: kernel.o head.o init_task.o
@@ -40,5 +40,8 @@ obj-$(CONFIG_SMP) += smp.o smpboot.o tr
obj-$(CONFIG_X86_LOCAL_APIC) += mpparse.o apic.o nmi.o
obj-$(CONFIG_X86_IO_APIC) += io_apic.o
obj-$(CONFIG_X86_VISWS_APIC) += visws_apic.o
+
+AFLAGS_entry.o += -I $(TOPDIR)/arch/$(ARCH)
+entry.o: $(TOPDIR)/arch/$(ARCH)/asm-offsets.h
include $(TOPDIR)/Rules.make
Index: 11-pre4.1/arch/i386/Makefile
--- 11-pre4.1/arch/i386/Makefile Wed, 18 Apr 2001 11:00:10 +1000 kaos (linux-2.4/T/c/37_Makefile 1.1.2.1.2.1 644)
+++ 11-pre4.1(w)/arch/i386/Makefile Sat, 06 Oct 2001 15:07:31 +1000 kaos (linux-2.4/T/c/37_Makefile 1.1.2.1.2.1 644)
@@ -145,3 +145,45 @@ archmrproper:
archdep:
@$(MAKEBOOT) dep
+
+# Convert raw asm offsets into something that can be included as
+# assembler definitions. It converts
+# -> symbol $value source
+# into
+# symbol = value /* 0xvalue source */
+
+arch/$(ARCH)/asm-offsets.h: arch/$(ARCH)/asm-offsets.s
+ @echo Creating $@
+ @set -e; \
+ (echo "#ifndef __ASM_OFFSETS_H__"; \
+ echo "#define __ASM_OFFSETS_H__"; \
+ echo "/*"; \
+ echo " * DO NOT MODIFY"; \
+ echo " *"; \
+ echo " * This file was generated by arch/$(ARCH)/Makefile."; \
+ echo " *"; \
+ echo " */"; \
+ echo ""; \
+ awk "/^->\$$/{printf(\"\\n\");} \
+ /^-> /{ \
+ sym = \$$2; \
+ val = \$$3; \
+ sub(/^\\\$$/, \"\", val); \
+ \$$1 = \"\"; \
+ \$$2 = \"\"; \
+ \$$3 = \"\"; \
+ printf(\"%-20s = %3d\\t\\t\\t/* 0x%x\\t%s */\\n\", \
+ sym, val, val, \$$0) \
+ }"; \
+ echo ""; \
+ echo "#endif"; \
+ ) < $< > $@;
+
+arch/$(ARCH)/asm-offsets.s: include/config/MARKER \
+ include/linux/sched.h \
+ include/linux/types.h \
+ include/linux/stddef.h
+
+# Kludge to prevent subdirs from being processed before asm-offsets.h is created.
+# I hate recursive make! KAO.
+before_subdirs: arch/$(ARCH)/asm-offsets.h
Hi there.
If anyone is interested I have already made a perl
script that produces assembler offsets from structure
members.
It doesn't need to run native since it reads the
header files, extract the structures and by using
objdump calculates the offsets automatically.
Maybe it needs some more work for what you describe,
but it's exactly what you describe.
If you're interested please email me directly for
more information.
Regards
From: Pantelis Antoniou <[email protected]>
Date: Mon, 08 Oct 2001 12:35:27 +0300
If anyone is interested I have already made a perl
script that produces assembler offsets from structure
members.
It doesn't need to run native since it reads the
header files, extract the structures and by using
objdump calculates the offsets automatically.
BTW, I assume you have already taken a look at how we
do this on Sparc64. See arch/sparc64/kernel/check_asm.sh
and the "check_asm" target in arch/sparc64/kernel/Makefile
It also works in all cross-compilation etc. environments.
And I bet it would work on every platform with very minimal
changes, if any.
Franks a lot,
David S. Miller
[email protected]
Pantelis Antoniou wrote:
>
> Hi there.
>
> If anyone is interested I have already made a perl
> script that produces assembler offsets from structure
> members.
>
> It doesn't need to run native since it reads the
> header files, extract the structures and by using
> objdump calculates the offsets automatically.
>
> Maybe it needs some more work for what you describe,
> but it's exactly what you describe.
>
> If you're interested please email me directly for
> more information.
>
One of the problems with this sort of thing is that it has a hard time
getting the CPP macros right. The best way to do this sort of thing is
to actually compile the header file with all the CONFIG defines and a
set of tools (read macros) that produce the required offsets. This way
you get what you want and don't have to reinvent the CPP stuff. It also
allows production of #define constants and other constructs that folks
push into CPP, in a very simple and straight forward manner.
Been there, done that.
George
At the risk of sticking my foot in it, is there something wrong with the
ANSI C offsetof() macro, defined in <stddef.h>?
--Georg
Georg Nikodym wrote:
>
> At the risk of sticking my foot in it, is there something wrong with the
> ANSI C offsetof() macro, defined in <stddef.h>?
>
> --Georg
No, and it could have been (and was) written prio to ANSI C defining
it. Something like:
#define offsetof(x, instruct) &((struct instruct *)0)->x
The issues that CPP resolves have to deal with the following sort of
structure:
struct instruct {
struct foo * bar;
#ifdef CONFIG_OPTION_DIDDLE
int diddle_flag;
int diddle_array[CONFIG_DIDDLE_SIZE];
#endif
int x;
}
Or for the simple need for a constant:
#define Const (CONFIG_DIDDLE_SIZE * sizeof(int))
Of course you could have any number of constant operators in the
expression. Note also, that the array in the structure above is defined
by a CONFIG symbol. This could also involve math, i.e.:
#define CONFIG_DIDDLE_SIZE CLOCK_RATE / HZ
and so on. All in all, it best to let CPP do what it does best and
scarf up the result:
#define GENERATE_CONSTANT(name,c) printf(#name " equ %d\n",c)
then:
GENERATE_CONSTANT(diddle_size,CONFIG_DIDDLE_SIZE);
In the code we did, we put all the GENERATE macros in a .h file. The
the code looked like:
#include.... all the headers needed....
#include <generate.h>
GENERATE.... all the generate macro calls...
} // all done (assumes that the "main(){" is in the generate.h file)
This whole mess was included as comments in the asm file. The make rule
then used a sed script to extract it, compile and run it to produce a
new header file which the asm source included outside of the above
stuff.
George
"David S. Miller" wrote:
>
> From: Pantelis Antoniou <[email protected]>
> Date: Mon, 08 Oct 2001 12:35:27 +0300
>
> If anyone is interested I have already made a perl
> script that produces assembler offsets from structure
> members.
>
> It doesn't need to run native since it reads the
> header files, extract the structures and by using
> objdump calculates the offsets automatically.
>
> BTW, I assume you have already taken a look at how we
> do this on Sparc64. See arch/sparc64/kernel/check_asm.sh
> and the "check_asm" target in arch/sparc64/kernel/Makefile
>
> It also works in all cross-compilation etc. environments.
> And I bet it would work on every platform with very minimal
> changes, if any.
>
> Franks a lot,
> David S. Miller
> [email protected]
I've look at your script and it kinda flew over my head.
Would you mind explain this a bit?
Thanks
#ifndef _TST2_INC
#define _TST2_INC
/****************************************************************
Simple defines list
****************************************************************/
#define TST2_H
#endif
I think your work is way over-engineered and that you really
need to look at the very simple method by which we do this
on sparc64.
Franks a lot,
David S. Miller
[email protected]
From: Pantelis Antoniou <[email protected]>
Date: Tue, 09 Oct 2001 10:25:19 +0300
I've look at your script and it kinda flew over my head.
Would you mind explain this a bit?
We generate the offsets as data items in an assembler file, then we
parse out those data section entries and spit them into the header.
It allows cross compilation setups to work, ie. even in cases when you
cannot generate and run a binary.
Franks a lot,
David S. Miller
[email protected]
On Mon, 08 Oct 2001 02:49:46 -0700 (PDT),
"David S. Miller" <[email protected]> wrote:
>BTW, I assume you have already taken a look at how we
>do this on Sparc64. See arch/sparc64/kernel/check_asm.sh
>and the "check_asm" target in arch/sparc64/kernel/Makefile
>
>It also works in all cross-compilation etc. environments.
>And I bet it would work on every platform with very minimal
>changes, if any.
I finally had time to look at sparc64 check_asm. The code suffers from
several problems:
* It is only built at make dep time so any patches after make dep that
affect the asm offsets are not picked up.
* The code has to know about all the config options that affect the
structures, it manually tests for SMP and SPIN_LOCK_DEBUG. That is
bad enough but other architectures are much worse, several config
settings can change the asm offsets, each config setting would have
to be manually defined.
* The asm output lists all fields in the input structures. It is
arguable if this is a good or bad thing, I prefer to explicitly
define just the symbols required by asm so I view this as a bad
feature.
* check_asm can only generate offsetof and sizeof fields. So
calculated fields like AOFF_task_fpregs have to be done elsewhere.
* The code is complex. I thought my asm-offsets.c->.s->.h conversion
was complicated but asm_check takes the cake.
Here is a first cut at doing sparc64 asm offsets the same way that i386
and ia64 are done in kbuild 2.5.
* It uses .c->.s->h so it plugs directly into the kbuild 2.5 dependency
handling, any patch or config changes that affect asm-offsets.h will
automatically rebuild asm-offsets.h.
* Only the fields actually used by asm are listed. I may have missed a
field or two but gcc will tell us that.
* asm-offsets.c DEFINE() can do any calculation. So AOFF_task_fpregs
is now calculated here.
* Less complex :).
* The addition of comments against each define makes the result more
readable.
==== arch/sparc64/asm-offsets.c ====
/*
* Generate definitions needed by assembly language modules.
* This code generates raw asm output which is post-processed to extract
* and format the required data.
*/
#include <linux/types.h>
#include <linux/stddef.h>
#include <linux/sched.h>
/* Use marker if you need to separate the values later */
#define DEFINE(sym, val, marker) \
asm volatile("\n-> " #sym " %0 " #val " " #marker : : "i" (val))
#define BLANK() asm volatile("\n->" : : )
int
main(void)
{
DEFINE(AOFF_mm_context, offsetof(struct mm_struct, context),);
BLANK();
DEFINE(AOFF_task_blocked, offsetof(struct task_struct, blocked),);
DEFINE(AOFF_task_egid, offsetof(struct task_struct, egid),);
DEFINE(ASIZ_task_egid, sizeof(((struct task_struct *)NULL)->egid),);
DEFINE(AOFF_task_euid, offsetof(struct task_struct, euid),);
DEFINE(ASIZ_task_euid, sizeof(((struct task_struct *)NULL)->euid),);
DEFINE(AOFF_task_flags, offsetof(struct task_struct, flags),);
DEFINE(AOFF_task_fpregs, (sizeof(struct task_struct) + (64 - 1)) & ~(64 - 1),);
DEFINE(AOFF_task_gid, offsetof(struct task_struct, gid),);
DEFINE(ASIZ_task_gid, sizeof(((struct task_struct *)NULL)->gid),);
DEFINE(AOFF_task_need_resched, offsetof(struct task_struct, need_resched),);
DEFINE(AOFF_task_personality, offsetof(struct task_struct, personality),);
DEFINE(ASIZ_task_personality, sizeof(((struct task_struct *)NULL)->personality),);
DEFINE(AOFF_task_pid, offsetof(struct task_struct, pid),);
DEFINE(AOFF_task_p_opptr, offsetof(struct task_struct, p_opptr),);
DEFINE(AOFF_task_processor, offsetof(struct task_struct, processor),);
DEFINE(AOFF_task_ptrace, offsetof(struct task_struct, ptrace),);
DEFINE(AOFF_task_sigpending, offsetof(struct task_struct, sigpending),);
DEFINE(AOFF_task_thread, offsetof(struct task_struct, thread),);
DEFINE(AOFF_task_uid, offsetof(struct task_struct, uid),);
DEFINE(ASIZ_task_uid, sizeof(((struct task_struct *)NULL)->uid),);
BLANK();
DEFINE(AOFF_thread_current_ds, offsetof(struct thread_struct, current_ds),);
DEFINE(AOFF_thread_fault_address, offsetof(struct thread_struct, fault_address),);
DEFINE(AOFF_thread_fault_code, offsetof(struct thread_struct, fault_code),);
DEFINE(AOFF_thread_flags, offsetof(struct thread_struct, flags),);
DEFINE(AOFF_thread_fork_kpsr, offsetof(struct thread_struct, fork_kpsr),);
DEFINE(AOFF_thread_fpdepth, offsetof(struct thread_struct, fpdepth),);
DEFINE(AOFF_thread_fpsaved, offsetof(struct thread_struct, fpsaved),);
DEFINE(AOFF_thread_gsr, offsetof(struct thread_struct, gsr),);
DEFINE(AOFF_thread_kernel_cntd0, offsetof(struct thread_struct, kernel_cntd0),);
DEFINE(AOFF_thread_pcr_reg, offsetof(struct thread_struct, pcr_reg),);
DEFINE(AOFF_thread_reg_window, offsetof(struct thread_struct, reg_window),);
DEFINE(AOFF_thread_rwbuf_stkptrs, offsetof(struct thread_struct, rwbuf_stkptrs),);
DEFINE(AOFF_thread_use_blkcommit, offsetof(struct thread_struct, use_blkcommit),);
DEFINE(AOFF_thread_utraps, offsetof(struct thread_struct, utraps),);
DEFINE(AOFF_thread_uwinmask, offsetof(struct thread_struct, uwinmask),);
DEFINE(AOFF_thread_w_saved, offsetof(struct thread_struct, w_saved),);
DEFINE(AOFF_thread_xfsr, offsetof(struct thread_struct, xfsr),);
return 0;
}
==== arch/sparc64/Makefile.in ====
# Convert raw asm offsets into something that can be included as
# assembler definitions. It converts
# -> symbol $value source
# into
# #define symbol value /* 0xvalue source */
user_command(asm-offsets.h
($(objfile asm-offsets.s))
(set -e;
(echo "#ifndef __ASM_OFFSETS_H__";
echo "#define __ASM_OFFSETS_H__";
echo "/*";
echo " * DO NOT MODIFY";
echo " *";
echo " * This file was generated by arch/$(ARCH)/Makefile.in.";
echo " *";
echo " */";
echo "";
awk "/^->\$$/{printf(\"\\n\");}
/^-> /{
sym = \$$2;
val = \$$3;
sub(/^\\\$$/, \"\", val);
\$$1 = \"\";
\$$2 = \"\";
\$$3 = \"\";
printf(\"#define %-24s %4d\\t\\t\\t/* 0x%x\\t%s */\\n\",
sym, val, val, \$$0)
}";
echo "";
echo "#endif";
) < $< > $@;
)
()
)
==== arch/sparc64/asm-offsets.h output ====
This is partial output and was actually generated on i386 so the
numbers are wrong but it shows what the output will look like. I do
not have access to a machine for compiling 2.4 sparc64 kernels so none
of the thread values are in this sample.
#ifndef __ASM_OFFSETS_H__
#define __ASM_OFFSETS_H__
/*
* DO NOT MODIFY
*
* This file was generated by arch/sparc64/Makefile.in.
*
*/
#define AOFF_mm_context 128 /* 0x80 offsetof(struct mm_struct, context) */
#define AOFF_task_blocked 1632 /* 0x660 offsetof(struct task_struct, blocked) */
#define AOFF_task_egid 572 /* 0x23c offsetof(struct task_struct, egid) */
#define ASIZ_task_egid 4 /* 0x4 sizeof(((struct task_struct *)NULL)->egid) */
#define AOFF_task_euid 556 /* 0x22c offsetof(struct task_struct, euid) */
#define ASIZ_task_euid 4 /* 0x4 sizeof(((struct task_struct *)NULL)->euid) */
#define AOFF_task_flags 4 /* 0x4 offsetof(struct task_struct, flags) */
#define AOFF_task_fpregs 1728 /* 0x6c0 (sizeof(struct task_struct) + (64 - 1)) & ~(64 - 1) */
#define AOFF_task_gid 568 /* 0x238 offsetof(struct task_struct, gid) */
#define ASIZ_task_gid 4 /* 0x4 sizeof(((struct task_struct *)NULL)->gid) */
#define AOFF_task_need_resched 20 /* 0x14 offsetof(struct task_struct, need_resched) */
#define AOFF_task_personality 116 /* 0x74 offsetof(struct task_struct, personality) */
#define ASIZ_task_personality 4 /* 0x4 sizeof(((struct task_struct *)NULL)->personality) */
#define AOFF_task_pid 124 /* 0x7c offsetof(struct task_struct, pid) */
#define AOFF_task_p_opptr 148 /* 0x94 offsetof(struct task_struct, p_opptr) */
#define AOFF_task_processor 52 /* 0x34 offsetof(struct task_struct, processor) */
#define AOFF_task_ptrace 24 /* 0x18 offsetof(struct task_struct, ptrace) */
#define AOFF_task_sigpending 8 /* 0x8 offsetof(struct task_struct, sigpending) */
#define AOFF_task_thread 880 /* 0x370 offsetof(struct task_struct, thread) */
#define AOFF_task_uid 552 /* 0x228 offsetof(struct task_struct, uid) */
#define ASIZ_task_uid 4 /* 0x4 sizeof(((struct task_struct *)NULL)->uid) */
/* Thread values would be here if I could compile for sparc64. */
#endif