Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Nick Alcock <nick.alcock@oracle.com>
To:     mcgrof@kernel.org, masahiroy@kernel.org
Cc:     linux-modules@vger.kernel.org, linux-trace-kernel@vger.kernel.org,
        linux-kernel@vger.kernel.org, arnd@arndb.de,
        akpm@linux-foundation.org, eugene.loh@oracle.com,
        kris.van.hees@oracle.com
Subject: [PATCH v10 07/13] kbuild: generate an address ranges map at vmlinux link time
Date:   Mon,  5 Dec 2022 16:31:51 +0000
Message-Id: <20221205163157.269335-8-nick.alcock@oracle.com>
In-Reply-To: <20221205163157.269335-1-nick.alcock@oracle.com>
References: <20221205163157.269335-1-nick.alcock@oracle.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?rA9hNnko6xWb27amdR92SjFgEAyxn/gZRZINFkh3kNsrVmynYc2ztqLhlLsY?=
 =?us-ascii?Q?unrgLDr1LmhpvPvkJMNjtZ2M8aDD4piJh3C3UeOdg7gnt/UDif9pGSc1Egtj?=
 =?us-ascii?Q?GtxDyK4NVUK0uxFBDbBSVwWLpnH6MFGOaX7Ni+5Alx4mvw+FO55qsH+5+ZAz?=
 =?us-ascii?Q?LePWk//IzyZPDNTyf378UkdFzn7xxm0Y72j8aS67d7tYFhPddYBnPY1BelKn?=
 =?us-ascii?Q?5e+kla6R54eOsANPkewJamoTiWzB0WivNfjbFFR9Q/lsIn9rMDOn2VcRlT8r?=
 =?us-ascii?Q?8CLVHu6fvszeJdDcp3Y2sp/ofHWbBF8eNnbEQ/cYjzKIghpHqJZEhX3kG0u0?=
 =?us-ascii?Q?03Sa6YvJtkx+YfxyLN0YNewiyexO9DjXZKUzPC7m8jtZ3cSYFa8jPV074naI?=
 =?us-ascii?Q?7tUZsYuXD+ZULQB0lJDzHvZV4Kfty6ugDjTChBjAqKbj65hcA0mUU/ZSiP0g?=
 =?us-ascii?Q?rH4O9ogaJYMz+M7Okl5Z6eYRu0lgY/rPJ4j7rSd4q1yGTijM1O/koV4rmqC3?=
 =?us-ascii?Q?Ppl4U1FuApH1rQ2opcYBfIeHZdMZqq3Ha+OWY211z/CmvE0kXMsZ7sBm58CN?=
 =?us-ascii?Q?xDxTUpntd0J+7684ixKACb+s+04vKlIQckI4pz2447cOxFZYYs9wX/D+kz9r?=
 =?us-ascii?Q?n/QDpt6VhBIDcBHSNf/9IU3ewqGqyF+A6IHH1BqMeIqMW1Kt5ITt5tQx6O5F?=
 =?us-ascii?Q?IdZpYQcgEPe5Aflqg/+MpQmSMK7UPkFwN8gsF0fe6Zl/EycwqCB1NoOHYfyJ?=
 =?us-ascii?Q?przNioqCU5E3fX/Lwn5HMiefpV5WyP6kiGvf+IYgUUfzV5Yt/vhySmSLZ8Qh?=
 =?us-ascii?Q?Klay3KEYxBDaw0UkMSyFDq7wsl5/EEFhstluLmj3aywynL7W26xFUpcZkn/H?=
 =?us-ascii?Q?KHZVby0vjSrNZTggxfdVmEUMm0+nUKRsesPKyq/U8fJAfboi2gIqqUG2MAye?=
 =?us-ascii?Q?oo/AQhrh6RwwlAz1IOhD4iGn0jw9B/DB+7QKHizv3H2bn3LdvX0yzM9CDG2X?=
 =?us-ascii?Q?5NNALKC1RYOSfVPRD7THGEYqEzKIZfD39ZyiiLJDgwBaZ1VhisTgLjg61oBu?=
 =?us-ascii?Q?fYAdXmTA3U6WMb66lXe0++O0actiYjHHc2VYwyeAUypxoNp+M8azbSFL2ztA?=
 =?us-ascii?Q?zc5zfJCo2YpUhrfWXp/4kwZA1Rr55B4dQscaD5hYzhD7D/xceBI3XMySjck6?=
 =?us-ascii?Q?078Wmfzqgu+BPS+boeGrWQRE3QQKkgOEWebEoeX0LbETgM14/s0S1IDboXZ0?=
 =?us-ascii?Q?MfM3ZIQhQXHXJDtEw3S4YxuKjKC3NWm9h94FQfxYaaI7Z1mUTWoeOwtGCpz3?=
 =?us-ascii?Q?UiAk9Sm5MIBkF0oPCkSZ2gGCMXSA96kTHLcML/fcJoUBwx0iPVBFj4Ike7Ct?=
 =?us-ascii?Q?yQ4ONVFlsCtXpGeEeD87RuIhvZV41PhKvONfjrKOrzDUXvmR4tZJ1NyLFBxI?=
 =?us-ascii?Q?NUIta/vqAszcByQTosQi7q6fWqYmEDoCLKdEw6CptDfU0TaB+9kxD4lrJWlz?=
 =?us-ascii?Q?4LnTviQJnM7xXEgM97K7yNggUGvWPM/SWvSMWhIPRzxKeBhJMPoLWU+dsbua?=
 =?us-ascii?Q?Ohh1c7nZFb3gf30dD5Lex4/8rfRXUbMOSOg4d6JfPVDJmzI1Kji/XKWcqkHa?=
 =?us-ascii?Q?2A=3D=3D?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: =?us-ascii?Q?0DLIgnR8bSFRuP7jRBwu1AeMArmnDV/akCRHneBWrB/UXYynx/lFTy52JW3Z?=
 =?us-ascii?Q?jUkKIqMyhMgImqsrp7ftBKkzdb1KprrlPlQOikITzWHA+J1drJ9Gy3qDiRFH?=
 =?us-ascii?Q?5WHQVyjinvAlj1i7+/DTagI0dke7B267KNdzQq5tDKKNDw19v9DbteclQoxj?=
 =?us-ascii?Q?0lcgTg1MSgtMOj+82m8BGvozawdHuOuLMKRbwRdjucTsyvYGzFh9QPWdn3bO?=
 =?us-ascii?Q?mD+aa9pDYebf0WXKiDzKAtSnhnpihKhgau59ZhcuXFnsh6E43VxFURxBQYtx?=
 =?us-ascii?Q?Yp0vX1+leS9vp8F9UfwZJSGVj7sUIG4zYRjo8BSRXKJ0J4MNzZ8R2P45OiAi?=
 =?us-ascii?Q?bi8fkf97Ib3e1B4uqwywJ0UujGPLyHO6jATlTwbFwdmVaGevRbGnLwO7V652?=
 =?us-ascii?Q?+KLf3r6Y15zOFeXzSJUBDWT/DY60YI/AaQjjKszxUf7zAFJP+9sXvmy4OWHu?=
 =?us-ascii?Q?9vNpdl/rjJLvXXfPmpy0jdlY0gzsEa/+y6tmeCTmetJ4F/pZXnY4uwRUxqcx?=
 =?us-ascii?Q?H/ZoFx130K9PsJN5VYam1X2PHMZ5hIic0S0DYul5vM6ve6Frk6ymfSrGT9p7?=
 =?us-ascii?Q?L9c7ZMxsQfjLmSVN39njejGuJMl0p4oiaDLUJBwsqnf+RgZNwPuCIKjmVD5i?=
 =?us-ascii?Q?j0maYjU0e/aoH3GAr84nhyi8HCZSC6NLenm49HFk75O8vv5LbBO5yKBfk9Vl?=
 =?us-ascii?Q?vAaygkveE10SBsaVn5BcLiJJG54RPL45ObnvDDsFsLu8Fsj3kSKrW4iWpCsq?=
 =?us-ascii?Q?K4lQhFBuC1VzIPKbYTKOG6vW3D/AKuXQ19OcT+qiuSRTQ7W4VQcNSVXmw8wA?=
 =?us-ascii?Q?n5MLPYCStAr4cpY9T4JfnaTqp/DCrhfbI8agLsuPCD9nm6L9FhhyElBHcdxz?=
 =?us-ascii?Q?uHmDKy+OCqkpbmxj7Q9Wugv0+ZevHvCK4zPQB4uO73g3Ud5+lH53DER2TZ86?=
 =?us-ascii?Q?qZhJvTL+n5eQpss0fdfsnPedKZl2WDm6jg0vWCuG2GI=3D?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 3dd7cb37-bd56-4e76-7107-08dad6de5583
X-MS-Exchange-CrossTenant-AuthSource: DS0PR10MB6798.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Dec 2022 16:32:43.3826
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: BZxXhgOnBtkNLchqQyBFoAHJLgBkfj4R6m6L6lZlxLCGKt3LOtcCLgb7KSF/do3gCesXYF5ihU0+M7/1vUZfCw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR10MB7324
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1
 definitions=2022-12-05_01,2022-12-05_01,2022-06-22_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 mlxlogscore=999 bulkscore=0
 malwarescore=0 mlxscore=0 suspectscore=0 adultscore=0 phishscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000
 definitions=main-2212050137
X-Proofpoint-GUID: ADxBPUqObwMMh_bwklZw5xDHmzc_nhAs
X-Proofpoint-ORIG-GUID: ADxBPUqObwMMh_bwklZw5xDHmzc_nhAs
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

This emits a new file, .tmp_vmlinux.ranges, which maps address
range/size pairs in vmlinux to the object files which make them up,
e.g., in part:

0x0000000000000000 0x30 arch/x86/kernel/cpu/common.o
0x0000000000001000 0x1000 arch/x86/events/intel/ds.o
0x0000000000002000 0x4000 arch/x86/kernel/irq_64.o
0x0000000000006000 0x5000 arch/x86/kernel/process.o
0x000000000000b000 0x1000 arch/x86/kernel/cpu/common.o
0x000000000000c000 0x5000 arch/x86/mm/cpu_entry_area.o
0x0000000000011000 0x10 arch/x86/kernel/espfix_64.o
0x0000000000011010 0x2 arch/x86/kernel/cpu/common.o
[...]

This will be used by kallmodsyms to let us associate symbols with
built-in modules at address-range granularity (i.e., space-efficiently),
and to let us disambiguate symbols with identical name and module by
annotating them with the translation unit they come from.

In my simple tests this seems to work with clang too, but if I'm not
sure how stable the format of clang's linker mapfiles is: if it turns
out not to work in some versions, the mapfile-massaging awk script added
here might need some adjustment.

CONFIG_LTO_CLANG by default optimizes by working from the LTOed
vmlinux.o file: but this doesn't work for kallmodsyms, as the resulting
mapfile lists vmlinux.o as the source object file in all cases, ruining
its attempt to disambiguate symbols using object file names.  So we
suppress that optimization in this case.

We also suppress it when IBT is enabled, but IBT *requires* use of the
intermediate vmlinux.o, since that's the .o that objtool has run over.
We lift this restriction in the next commit.

(There are similar problems with everything else that uses ld -r: since
this amounts, in total, to a few parts of KVM on aarch64, I haven't
implemented a general ld -r fix: the fix in the next commit is specific
to vmlinux.o.)

Signed-off-by: Nick Alcock <nick.alcock@oracle.com>
Reviewed-by: Kris Van Hees <kris.van.hees@oracle.com>
---

Notes:
    v6: use ${wl} where appropriate to avoid failure on UML
    v10: mention ultimate use in kallmodsyms, conflicts with IBT and ld -r

 scripts/link-vmlinux.sh | 24 +++++++++++++++++++++---
 1 file changed, 21 insertions(+), 3 deletions(-)

diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh
index 32e573943cf0..a40d372b1289 100755
--- a/scripts/link-vmlinux.sh
+++ b/scripts/link-vmlinux.sh
@@ -60,7 +60,10 @@ vmlinux_link()
 	# skip output file argument
 	shift
 
-	if is_enabled CONFIG_LTO_CLANG || is_enabled CONFIG_X86_KERNEL_IBT; then
+	# kallmodsyms needs a linker mapfile that contains original object
+	# file names, so cannot use this optimization.
+	if { is_enabled CONFIG_LTO_CLANG || is_enabled CONFIG_X86_KERNEL_IBT; } && \
+	   ! is_enabled CONFIG_KALLMODSYMS; then
 		# Use vmlinux.o instead of performing the slow LTO link again.
 		objs=vmlinux.o
 		libs=
@@ -94,7 +97,7 @@ vmlinux_link()
 		ldflags="${ldflags} ${wl}--strip-debug"
 	fi
 
-	if is_enabled CONFIG_VMLINUX_MAP; then
+	if is_enabled CONFIG_VMLINUX_MAP || is_enabled CONFIG_KALLMODSYMS; then
 		ldflags="${ldflags} ${wl}-Map=${output}.map"
 	fi
 
@@ -144,6 +147,21 @@ kallsyms()
 {
 	local kallsymopt;
 
+	# read the linker map to identify ranges of addresses:
+	#   - for each *.o file, report address, size, pathname
+	#       - most such lines will have four fields
+	#       - but sometimes there is a line break after the first field
+	#   - start reading at "Linker script and memory map"
+	#   - stop reading at ".brk"
+	if is_enabled CONFIG_KALLMODSYMS; then
+		${AWK} '
+		    /\.o$/ && start==1 { print $(NF-2), $(NF-1), $NF }
+		    /^Linker script and memory map/ { start = 1 }
+		    /^\.brk/ { exit(0) }
+		' ${3} | sort > .tmp_vmlinux.ranges
+	fi
+
+	# get kallsyms options
 	if is_enabled CONFIG_KALLSYMS_ALL; then
 		kallsymopt="${kallsymopt} --all-symbols"
 	fi
@@ -175,7 +193,7 @@ kallsyms_step()
 
 	vmlinux_link ${kallsyms_vmlinux} "${kallsymso_prev}" ${btf_vmlinux_bin_o}
 	mksysmap ${kallsyms_vmlinux} ${kallsyms_vmlinux}.syms
-	kallsyms ${kallsyms_vmlinux}.syms ${kallsyms_S}
+	kallsyms ${kallsyms_vmlinux}.syms ${kallsyms_S} ${kallsyms_vmlinux}.map
 
 	info AS ${kallsyms_S}
 	${CC} ${NOSTDINC_FLAGS} ${LINUXINCLUDE} ${KBUILD_CPPFLAGS} \
-- 
2.38.0.266.g481848f278