Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Sender: eberman=codeaurora.org@mg.codeaurora.org
DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org 14DE2C43217
From:   Elliot Berman <eberman@codeaurora.org>
To:     Masahiro Yamada <masahiroy@kernel.org>,
        Michal Marek <michal.lkml@markovi.net>
Cc:     Elliot Berman <eberman@codeaurora.org>,
        linux-kbuild@vger.kernel.org, linux-kernel@vger.kernel.org,
        Matthias Maennich <maennich@google.com>,
        Trilok Soni <tsoni@codeaurora.org>
Subject: [RESEND v2] Kbuild: Update config_data.gz only if KCONFIG_CONFIG materially changed
Date:   Fri, 23 Apr 2021 10:00:35 -0700
Message-Id: <1619197235-13860-1-git-send-email-eberman@codeaurora.org>
In-Reply-To: 1617317072-26770-1-git-send-email-eberman@codeaurora.org
Precedence: bulk

If you update the timestamp of KCONFIG_CONFIG without actually changing
anything, config_data.gz is re-generated and causes vmlinux to re-link.
When Link Time Optimization is enabled, unnecessary re-linking of
vmlinux is highly desirable since it adds several minutes to build time.

Avoid touching config_data.gz by using a script to compare the existing
config_data.gz, KCONFIG_CONFIG, or script itself to update only if any
is mismatched.  The script follows gen_kheaders.sh approach for
determing in update is needed. The script intentionally avoids
re-compressing KCONFIG_CONFIG.

The .config can be touched, for instance, by a build script which
installs the default defconfig and then applies a defconfig fragment on
top.

For a simple example on my x86 machine, I modified x86 default defconfig to set
CONFIG_IKCONFIG=y and run:
  make -j50 defconfig tiny.config vmlinux
  make -j50 defconfig tiny.config vmlinux
With this patch, vmlinux is not re-built as a result of config_data.gz
change.

Changes in v2:
 - Use md5 checksum to compare .config instead of gzip'ing again

Signed-off-by: Elliot Berman <eberman@codeaurora.org>
---
 kernel/.gitignore         |  1 +
 kernel/Makefile           |  4 +++-
 kernel/gen_config_data.sh | 31 +++++++++++++++++++++++++++++++
 3 files changed, 35 insertions(+), 1 deletion(-)
 create mode 100755 kernel/gen_config_data.sh

diff --git a/kernel/.gitignore b/kernel/.gitignore
index 78701ea..a191136 100644
--- a/kernel/.gitignore
+++ b/kernel/.gitignore
@@ -1,4 +1,5 @@
 # SPDX-License-Identifier: GPL-2.0-only
+config_data.gz.md5
 kheaders.md5
 timeconst.h
 hz.bc
diff --git a/kernel/Makefile b/kernel/Makefile
index 320f1f3..0784bf3d 100644
--- a/kernel/Makefile
+++ b/kernel/Makefile
@@ -139,8 +139,10 @@ obj-$(CONFIG_SCF_TORTURE_TEST) += scftorture.o
 $(obj)/configs.o: $(obj)/config_data.gz
 
 targets += config_data.gz
+quiet_cmd_genicfg = CHK     $(obj)/config_data.gz
+      cmd_genicfg = $(CONFIG_SHELL) $(srctree)/kernel/gen_config_data.sh $@ $<
 $(obj)/config_data.gz: $(KCONFIG_CONFIG) FORCE
-	$(call if_changed,gzip)
+	$(call cmd,genicfg)
 
 $(obj)/kheaders.o: $(obj)/kheaders_data.tar.xz
 
diff --git a/kernel/gen_config_data.sh b/kernel/gen_config_data.sh
new file mode 100755
index 00000000..e9ff193
--- /dev/null
+++ b/kernel/gen_config_data.sh
@@ -0,0 +1,31 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+
+# This script generates a compressed version of .config, if its checksum has changed
+set -e
+
+this_file="$(readlink -f "$0")"
+outfile=$1
+infile=$2
+
+config_md5="$(md5sum $infile | cut -d ' ' -f1)"
+# Any changes to this script will also cause a rebuild of config_data.
+this_file_md5="$(md5sum $sfile | cut -d ' ' -f1)"
+if [ -f $outfile ]; then outfile_md5="$(md5sum $outfile | cut -d ' ' -f1)"; fi
+
+if [ -f $outfile.md5 ] &&
+	[ "$(head -n 1 $outfile.md5)" = "$config_md5" ] &&
+	[ "$(head -n 2 $outfile.md5 | tail -n 1)" = "$this_file_md5" ] &&
+	[ "$(tail -n 1 $outfile.md5)" = "$outfile_md5" ]; then
+		exit
+fi
+
+if [ "${quiet}" != "silent_" ]; then
+	echo "  GEN     $outfile"
+fi
+
+${KGZIP} -c -n -f -9 $infile > $outfile
+
+echo "$config_md5" > $outfile.md5
+echo "$this_file_md5" >> $outfile.md5
+echo "$(md5sum $outfile | cut -d ' ' -f1)" >> $outfile.md5
-- 
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project