KBUILD_BUILD_VERSION is currently not documented but it is
needed when rebuilding a kernel that should result in the identical
binary. This is a brief documentation of KBUILD_BUILD_VERSION.
Signed-off-by: Nicholas Mc Guire <[email protected]>
---
Patch is against 4.0-rc6 (localversion-next is -next-20150402)
Documentation/kbuild/kbuild.txt | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/Documentation/kbuild/kbuild.txt b/Documentation/kbuild/kbuild.txt
index 6466704..7ad01d9 100644
--- a/Documentation/kbuild/kbuild.txt
+++ b/Documentation/kbuild/kbuild.txt
@@ -213,6 +213,14 @@ UTS_VERSION definition (uname -v in the running kernel). The value has to
be a string that can be passed to date -d. The default value
is the output of the date command at one point during build.
+KBUILD_BUILD_VERSION
+--------------------------------------------------
+Setting this to an integer overrides the build version number used in the
+UTS_VERSION definition (uname -v in the running kernel shown as e.g. #1).
+The default value is set to 0 if .version does not yet exist and is
+generated by incrementing the current value in .version by one if it
+already exists.
+
KBUILD_BUILD_USER, KBUILD_BUILD_HOST
--------------------------------------------------
These two variables allow to override the user@host string displayed during
--
1.7.10.4
On Sun, 5 Apr 2015 08:44:28 +0200
Nicholas Mc Guire <[email protected]> wrote:
> KBUILD_BUILD_VERSION is currently not documented but it is
> needed when rebuilding a kernel that should result in the identical
> binary. This is a brief documentation of KBUILD_BUILD_VERSION.
Can we add something like the above to the document itself so that
readers have an idea of why they might want to tweak this?
Either way, I can take it in the docs tree if that's best..Michal?
Thanks,
jon
On Sat, 11 Apr 2015, Jonathan Corbet wrote:
> On Sun, 5 Apr 2015 08:44:28 +0200
> Nicholas Mc Guire <[email protected]> wrote:
>
> > KBUILD_BUILD_VERSION is currently not documented but it is
> > needed when rebuilding a kernel that should result in the identical
> > binary. This is a brief documentation of KBUILD_BUILD_VERSION.
>
> Can we add something like the above to the document itself so that
> readers have an idea of why they might want to tweak this?
>
> Either way, I can take it in the docs tree if that's best..Michal?
>
I thought of that but it would be inconsistent as all other descriptions
here are only the function not the use. I did not find a file where
the problem of identical rebuild would really fit.
thx!
hofrat
Dne 11.4.2015 v 15:20 Nicholas Mc Guire napsal(a):
> On Sat, 11 Apr 2015, Jonathan Corbet wrote:
>
>> On Sun, 5 Apr 2015 08:44:28 +0200
>> Nicholas Mc Guire <[email protected]> wrote:
>>
>>> KBUILD_BUILD_VERSION is currently not documented but it is
>>> needed when rebuilding a kernel that should result in the identical
>>> binary. This is a brief documentation of KBUILD_BUILD_VERSION.
>>
>> Can we add something like the above to the document itself so that
>> readers have an idea of why they might want to tweak this?
>>
>> Either way, I can take it in the docs tree if that's best..Michal?
>>
> I thought of that but it would be inconsistent as all other descriptions
> here are only the function not the use.
Most of the entries in this file predate efforts at deterministic
builds, so I'd prefer usefulness over consistency here :-).
> I did not find a file where
> the problem of identical rebuild would really fit.
... unless, of course, you want to start a new file covering this topic.
Because it's not just the few override variables, but also some options
have to be turned off (I remember CONFIG_GCOV_KERNEL) and the paths must
be the same, or relative paths must be used.
Michal
On Sat, 11 Apr 2015, Michal Marek wrote:
> Dne 11.4.2015 v 15:20 Nicholas Mc Guire napsal(a):
> > On Sat, 11 Apr 2015, Jonathan Corbet wrote:
> >
> >> On Sun, 5 Apr 2015 08:44:28 +0200
> >> Nicholas Mc Guire <[email protected]> wrote:
> >>
> >>> KBUILD_BUILD_VERSION is currently not documented but it is
> >>> needed when rebuilding a kernel that should result in the identical
> >>> binary. This is a brief documentation of KBUILD_BUILD_VERSION.
> >>
> >> Can we add something like the above to the document itself so that
> >> readers have an idea of why they might want to tweak this?
> >>
> >> Either way, I can take it in the docs tree if that's best..Michal?
> >>
> > I thought of that but it would be inconsistent as all other descriptions
> > here are only the function not the use.
>
> Most of the entries in this file predate efforts at deterministic
> builds, so I'd prefer usefulness over consistency here :-).
>
>
> > I did not find a file where
> > the problem of identical rebuild would really fit.
>
> ... unless, of course, you want to start a new file covering this topic.
> Because it's not just the few override variables, but also some options
> have to be turned off (I remember CONFIG_GCOV_KERNEL) and the paths must
> be the same, or relative paths must be used.
>
The identical path really only should be needed for GCOV as it compiles
path info into the binary - are you aware of any other cnfig option that
would need the same path ?
It could be in a short document but it could also just be placed right
under the descriptions of the relevant variables - something like:
Use of KBUILD_BUILD_{TIMESTAMP,VERSION,USER,HOST}
--------------------------------------------------
When the kernel is built, Kbuild stores the build timestamp, version, user
and hostname in include/generated/compile.h. By default this information
is extracted from the build-environment on every build. To override this
behavior and rebuild an identical binary kernel, given a running kernel,
the original compile time information needs to be passed to Kbuild.
Prerequisites:
* Original kernel config available (e.g extract it from /proc/config.gz)
* Identical source tree
* Identical toolchain
* Same source path if building with CONFIG_GCOV_KERNEL
* Module signature (CONFIG_MODULE_SIG) must be disabled
The build process for rebuilding the identical binary kernel can extract the
remaining necessary information from dmesg and set the Kbuild and make env:
from the boot log or dmesg in the running system:
[ 0.000000] Linux version 4.0.0-rc6-sil2 (hofrat@debian) (gcc version 4.7.2
(Debian 4.7.2-5) ) #0 SMP Mon Apr 6 01:49:56 EDT 2015
^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^
KBUILD_BUILD_VERSION---' |
KBUILD_BUILD_TIMESTAMP---------------'
$ export KBUILD_BUILD_VERSION=0
$ export KBUILD_BUILD_TIMESTAMP="Mon Apr 6 01:49:56 EDT 2015"
$ export KBUILD_BUILD_USER=hofrat
$ export KBUILD_BUILD_HOST=debian
$ git reset --hard v4.0-rc6
$ zcat /proc/config.gz > .config
$ make oldconfig
$ make -j 8 EXTRAVERSION=-rc6-sil2
probably not quite sufficient yet - but atleast for
defconfigs and a few other trials it worked for me.
If that makes sense to cleanup I'll try to figure out what
other config options could cause problems and submit it
as a patch to kbuild.txt
thx!
hofrat
On 2015-04-12 06:08, Nicholas Mc Guire wrote:
> On Sat, 11 Apr 2015, Michal Marek wrote:
>
>> Dne 11.4.2015 v 15:20 Nicholas Mc Guire napsal(a):
>>> On Sat, 11 Apr 2015, Jonathan Corbet wrote:
>>>
>>>> On Sun, 5 Apr 2015 08:44:28 +0200
>>>> Nicholas Mc Guire <[email protected]> wrote:
>>>>
>>>>> KBUILD_BUILD_VERSION is currently not documented but it is
>>>>> needed when rebuilding a kernel that should result in the identical
>>>>> binary. This is a brief documentation of KBUILD_BUILD_VERSION.
>>>>
>>>> Can we add something like the above to the document itself so that
>>>> readers have an idea of why they might want to tweak this?
>>>>
>>>> Either way, I can take it in the docs tree if that's best..Michal?
>>>>
>>> I thought of that but it would be inconsistent as all other descriptions
>>> here are only the function not the use.
>>
>> Most of the entries in this file predate efforts at deterministic
>> builds, so I'd prefer usefulness over consistency here :-).
>>
>>
>>> I did not find a file where
>>> the problem of identical rebuild would really fit.
>>
>> ... unless, of course, you want to start a new file covering this topic.
>> Because it's not just the few override variables, but also some options
>> have to be turned off (I remember CONFIG_GCOV_KERNEL) and the paths must
>> be the same, or relative paths must be used.
>>
> The identical path really only should be needed for GCOV as it compiles
> path info into the binary - are you aware of any other cnfig option that
> would need the same path ?
The paths appear in BUG() messages if you are using O=<path>. This can
be solver by either not using O= or using O=<subdir>, in which case
kbuild uses '..' as path to the srctree.
IIRC the problem with GCOV was that gcc was adding a randomly named
symbol to each object file.
> It could be in a short document but it could also just be placed right
> under the descriptions of the relevant variables - something like:
>
>
> Use of KBUILD_BUILD_{TIMESTAMP,VERSION,USER,HOST}
> --------------------------------------------------
>
> When the kernel is built, Kbuild stores the build timestamp, version, user
> and hostname in include/generated/compile.h. By default this information
> is extracted from the build-environment on every build. To override this
> behavior and rebuild an identical binary kernel, given a running kernel,
> the original compile time information needs to be passed to Kbuild.
>
> Prerequisites:
> * Original kernel config available (e.g extract it from /proc/config.gz)
> * Identical source tree
> * Identical toolchain
> * Same source path if building with CONFIG_GCOV_KERNEL
> * Module signature (CONFIG_MODULE_SIG) must be disabled
I think that only MODULE_SIG_ALL is problematic, the signature checking
code itself is OK.
> If that makes sense to cleanup I'll try to figure out what
> other config options could cause problems and submit it
> as a patch to kbuild.txt
Michal
On Mon, 13 Apr 2015, Michal Marek wrote:
> On 2015-04-12 06:08, Nicholas Mc Guire wrote:
> > On Sat, 11 Apr 2015, Michal Marek wrote:
> >
> >> Dne 11.4.2015 v 15:20 Nicholas Mc Guire napsal(a):
> >>> On Sat, 11 Apr 2015, Jonathan Corbet wrote:
> >>>
> >>>> On Sun, 5 Apr 2015 08:44:28 +0200
> >>>> Nicholas Mc Guire <[email protected]> wrote:
> >>>>
> >>>>> KBUILD_BUILD_VERSION is currently not documented but it is
> >>>>> needed when rebuilding a kernel that should result in the identical
> >>>>> binary. This is a brief documentation of KBUILD_BUILD_VERSION.
> >>>>
> >>>> Can we add something like the above to the document itself so that
> >>>> readers have an idea of why they might want to tweak this?
> >>>>
> >>>> Either way, I can take it in the docs tree if that's best..Michal?
> >>>>
> >>> I thought of that but it would be inconsistent as all other descriptions
> >>> here are only the function not the use.
> >>
> >> Most of the entries in this file predate efforts at deterministic
> >> builds, so I'd prefer usefulness over consistency here :-).
> >>
> >>
> >>> I did not find a file where
> >>> the problem of identical rebuild would really fit.
> >>
> >> ... unless, of course, you want to start a new file covering this topic.
> >> Because it's not just the few override variables, but also some options
> >> have to be turned off (I remember CONFIG_GCOV_KERNEL) and the paths must
> >> be the same, or relative paths must be used.
> >>
> > The identical path really only should be needed for GCOV as it compiles
> > path info into the binary - are you aware of any other cnfig option that
> > would need the same path ?
>
> The paths appear in BUG() messages if you are using O=<path>. This can
> be solver by either not using O= or using O=<subdir>, in which case
> kbuild uses '..' as path to the srctree.
>
> IIRC the problem with GCOV was that gcc was adding a randomly named
> symbol to each object file.
>
well it also simply compiles the absolute path into the binary
so that is in the checksum.
>
> > It could be in a short document but it could also just be placed right
> > under the descriptions of the relevant variables - something like:
> >
> >
> > Use of KBUILD_BUILD_{TIMESTAMP,VERSION,USER,HOST}
> > --------------------------------------------------
> >
> > When the kernel is built, Kbuild stores the build timestamp, version, user
> > and hostname in include/generated/compile.h. By default this information
> > is extracted from the build-environment on every build. To override this
> > behavior and rebuild an identical binary kernel, given a running kernel,
> > the original compile time information needs to be passed to Kbuild.
> >
> > Prerequisites:
> > * Original kernel config available (e.g extract it from /proc/config.gz)
> > * Identical source tree
> > * Identical toolchain
> > * Same source path if building with CONFIG_GCOV_KERNEL
> > * Module signature (CONFIG_MODULE_SIG) must be disabled
>
> I think that only MODULE_SIG_ALL is problematic, the signature checking
> code itself is OK.
>
thanks - will check that - not sure if I had _SIG AND _ALL on while testing
thanks for the notes - running a randbuild script to see if this holds or not
should give atleast some level of confidence that the
exeptions are complete.
thx!
hofrat