Received-SPF: pass (google.com: domain of linux-kernel+bounces-180169-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Date: Wed, 15 May 2024 12:50:11 -0400
From: Kris Van Hees <kris.van.hees@oracle.com>
To: Masahiro Yamada <masahiroy@kernel.org>
Cc: Kris Van Hees <kris.van.hees@oracle.com>, linux-kernel@vger.kernel.org,
        linux-kbuild@vger.kernel.org, linux-modules@vger.kernel.org,
        linux-trace-kernel@vger.kernel.org,
        Steven Rostedt <rostedt@goodmis.org>,
        Luis Chamberlain <mcgrof@kernel.org>,
        Masami Hiramatsu <mhiramat@kernel.org>,
        Nick Desaulniers <ndesaulniers@google.com>,
        Jiri Olsa <olsajiri@gmail.com>,
        Elena Zannoni <elena.zannoni@oracle.com>
Subject: Re: [PATCH v2 0/6] Generate address range data for built-in modules
Message-ID: <ZkTnwwyTF0WSMmqI@oracle.com>
References: <20240511224035.27775-1-kris.van.hees@oracle.com>
 <CAK7LNATwSDyAWR2FqccF5RFLpw5CYFyndR0N814nC7G7EaL2Tw@mail.gmail.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAK7LNATwSDyAWR2FqccF5RFLpw5CYFyndR0N814nC7G7EaL2Tw@mail.gmail.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?us-ascii?Q?E6YBDXRQilAMBslu7mYe+2EoHzQ/9ckrLY73EWYP9MXFuc67YXQqderLVWP8?=
 =?us-ascii?Q?O9XTUoB/qu49zBM+cVjrff5WMIqQo8LNs/OlE4y2hg8r1s/P4D3CXr0pCviP?=
 =?us-ascii?Q?lWHyquj1ErEGUXmwKM8xiLWv8DERBh7C7CT9YSya2wTybFuW/nPQWtPVZqS6?=
 =?us-ascii?Q?9g4U/2Mzll5ZOtEyUYCA3R3ReA7wujj/TPzlMUj9H9f3OSmQHk6SDHw8vzAq?=
 =?us-ascii?Q?EezqGGLGWus9Hk/EVAKzXr8SYb4bIyvXsMoIQWG69CIVmtjr9ikNygV4z59T?=
 =?us-ascii?Q?VZcqk3bM6uaUjR+Ffnq2pyEA8DMXmh1W/76ZiSVB8EDogeOmXJnIOIma8+6n?=
 =?us-ascii?Q?PDkQYkNaU0zNwu/+QvXyUNzZ4WaSmft9mTuZbnKW1fXRXIR/nOJPVcXb879S?=
 =?us-ascii?Q?NDVrKW7cE4H8hxNs25OgsRw0M4aAqULePkA83iuiVOGfOb71Dus7Y+x4vE95?=
 =?us-ascii?Q?RHcs4ET6b6y+auyzo3DlLwX7P4+j5letw0345keAMdg/3cPSwS8bWWgfjTQJ?=
 =?us-ascii?Q?Sc0D5oBgiDUZglQdSQIHmZF5nMj+to58UI6GtI1KlurV0JqbFImYzoSgtnQX?=
 =?us-ascii?Q?U5d+ETQ53qhbL6j3Rje/H8vLSUnMObnNxI/eSi0fDKuFjpeB0uvtDisgNCU4?=
 =?us-ascii?Q?ewG9jI2OWSIdb7p41lgMN8W4fLUaH3IK3kQzw0Ssy6IegsWBci0VypNnq/If?=
 =?us-ascii?Q?5oa8+9z2FXAq5g8/B2KjfSkjwd0mrV+MlTw80E9oKlIYh6UlSw2bXb74ApfT?=
 =?us-ascii?Q?L2SsxfrNIov0Vk3hyzBAZbwx5YBxsyAbFOtzWJ4oKGnExihqi6OD6sCZ2O12?=
 =?us-ascii?Q?2u5obRJgrfNGoRB/2p46YmLtvONYDYr2lYfb3ctPr4HclhCSzdALqTn8rQws?=
 =?us-ascii?Q?Lqu/GnVlfQxqMRoBr3xcVJ9seU0iCByzVFljFmMrEUlgOABpq7tOpUh6cmot?=
 =?us-ascii?Q?Q6Xs6LRZd4XtQPLgn9Q9qT5/AhOpXFSnHgq7G8W8qwv0HNi+JS9a5zxzxr8u?=
 =?us-ascii?Q?6lAsx6YeKSgpqX6bk7aFCItH3Q3t3qfUfqq26R0sixcbC9i656GsbjwoMS1/?=
 =?us-ascii?Q?hx3K18LPhqfgt+B9ENrx1gPA9qWn25luftzzB0uIoQ13xXyKqhjW+VQWZu3N?=
 =?us-ascii?Q?C4DHaYzAfko1fyUXhyq0qHPQr4OpLtUNPtAkKYumOZIMenLS/aQ4WrOIcNCl?=
 =?us-ascii?Q?vTsk+ugiqMCTr/JpwXS8XNLEh2SEpjP3p4ABzEYYiHSYGwoyo4y8LUUnSyiW?=
 =?us-ascii?Q?rz7XX0gGq2LGeTk1r3+IJSA2OEozq4ckIHVzV7wi5eQ30mQve7/Os5N0LHGW?=
 =?us-ascii?Q?peoBFt5Lok4lqmmxZUNnkgDy5pP99pb+ijlol7PikapPhQkjyPvL6/cs1e8n?=
 =?us-ascii?Q?y23245Lt0Cea0wteUaMbmTc0V0RZc+tB3x+AY2ewCEg9Er3RhGhqFfc8kdO6?=
 =?us-ascii?Q?f/XZ5TdHeBUIfiBSGUDQwYwSnB4FhRW5nt7j78cSOxEk7C/GaHFv+Sh6SiTx?=
 =?us-ascii?Q?fkq4CsmXYoThjEueI76Dhy+x+PFq8qMaQeM01rqz9QEXyUZzatxOYLiX1L0W?=
 =?us-ascii?Q?V+K3+ZEU8oc2fHM/EBx+VSlzW38INvCABH7Qg5zWPOaMRpz8D00JfomA9zWf?=
 =?us-ascii?Q?AA=3D=3D?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	QFcebcwYRl5LM9h8Bv1IlftDA2f/4Ud0yQPVSenylKeL+8NC6xgDmrdNJ7cJgOsZvzvW77Sw/EAcku9WOENCYVWPyhxfqQF5trD2a83hshImFxqdo8Q9flulrnZY5dNwLYBGJpKrw5CsDwua5QjMTFzWYdK+riivj7wIrKEo/DuenO7fOLJ5kVGYBwWun//k3zFCyts5+8IusbphX8hBB0wsqDelXDXqYmBoFQrxMumwmMDwDVMQHoQwkkaJdPYr7XyzoOAHz0nrCdCW2erfiNKN70C1Nq2mcu3k6BZI8uQdWenticEwdmjhZyHybNxPt1n4zH4+ot5mwxNsASeJbvs2fed27CQj00Qb5MiSFwhQgiAbYilmAEZg3FiZfNujx/Y+LfmGGRiMsybZMc7iJVAYAH40bqGnO0OTe4XpFmfl0ihspvfp8Nx35BDigInlES7gNyvqb45G4wsp9v8ZozdXU97d5+CwVK6Mfp/cFwSVnfJSeJ47QcLFN55BWDddjqRklGEg1yG2V695wpFMxrR2QQ3CxyNkSzpoqzt/VFkabVTZS6OCumFs05XTtY8xTQ+KCg1vnYXbSrOExuErhCPeJ6g/BwqvlNs411KUPb4=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c7771cd4-329f-43b9-44c5-08dc74ff17b5
X-MS-Exchange-CrossTenant-AuthSource: SN7PR10MB6287.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 May 2024 16:50:14.3705
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: n492pvsz9DqNcdkVe8cpM2W4JY/W6nkA1ASuemS2LNgnEjVabtOPOL7D0mgqifTbMWLELbYrif3TmJETkuJ3UKEJSaUGiyiLf2pr6FkQjm4=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR10MB5198
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26
 definitions=2024-05-15_10,2024-05-15_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 bulkscore=0
 mlxlogscore=999 spamscore=0 suspectscore=0 mlxscore=0 malwarescore=0
 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2405010000 definitions=main-2405150119
X-Proofpoint-ORIG-GUID: vzFI6vPEyGzqD8TLOaMSbDOSpSDHHoyw
X-Proofpoint-GUID: vzFI6vPEyGzqD8TLOaMSbDOSpSDHHoyw

On Mon, May 13, 2024 at 01:43:15PM +0900, Masahiro Yamada wrote:
> On Sun, May 12, 2024 at 7:42???AM Kris Van Hees <kris.van.hees@oracle.com> wrote:
> >
> > Especially for tracing applications, it is convenient to be able to
> > refer to a symbol using a <module name, symbol name> pair and to be able
> > to translate an address into a <nodule mname, symbol name> pair.  But
> > that does not work if the module is built into the kernel because the
> > object files that comprise the built-in module implementation are simply
> > linked into the kernel image along with all other kernel object files.
> >
> > This is especially visible when providing tracing scripts for support
> > purposes, where the developer of the script targets a particular kernel
> > version, but does not have control over whether the target system has
> > a particular module as loadable module or built-in module.  When tracing
> > symbols within a module, referring them by <module name, symbol name>
> > pairs is both convenient and aids symbol lookup.  But that naming will
> > not work if the module name information is lost if the module is built
> > into the kernel on the target system.
> >
> > Earlier work addressing this loss of information for built-in modules
> > involved adding module name information to the kallsyms data, but that
> > required more invasive code in the kernel proper.  This work never did
> > get merged into the kernel tree.
> >
> > All that is really needed is knowing whether a given address belongs to
> > a particular module (or multiple modules if they share an object file).
> > Or in other words, whether that address falls within an address range
> > that is associated with one or more modules.
> >
> > This patch series is baaed on Luis Chamberlain's patch to generate
> > modules.builtin.objs, associating built-in modules with their object
> > files.  Using this data, vmlinux.o.map and vmlinux.map can be parsed in
> > a single pass to generate a modules.buitin.ranges file with offset range
> > information (relative to the base address of the associated section) for
> > built-in modules.  The file gets installed along with the other
> > modules.builtin.* files.
> 
> 
> 
> I still do not want to see modules.builtin.objs.
> 
> 
> During the vmlinux.o.map parse, every time an object path
> is encountered, you can open the corresponding .cmd file.
> 
> 
> 
> Let's say, you have the following in vmlinux.o.map:
> 
> .text          0x00000000007d4fe0     0x46c8 drivers/i2c/i2c-core-base.o
> 
> 
> 
> You can check drivers/i2c/.i2c-core-base.o.cmd
> 
> 
> $ cat drivers/i2c/.i2c-core-base.o.cmd | tr ' ' '\n' | grep KBUILD_MODFILE
> -DKBUILD_MODFILE='"drivers/i2c/i2c-core"'
> 
> 
> Now you know this object is part of drivers/i2c/i2c-core
> (that is, its modname is "i2c-core")
> 
> 
> 
> 
> Next, you will get the following:
> 
>  .text          0x00000000007dc550     0x13c4 drivers/i2c/i2c-core-acpi.o
> 
> 
> $ cat drivers/i2c/.i2c-core-acpi.o.cmd | tr ' ' '\n' | grep KBUILD_MODFILE
> -DKBUILD_MODFILE='"drivers/i2c/i2c-core"'
> 
> 
> This one is also a part of drivers/i2c/i2c-core
>
> 
> You will get the address range of "i2c-core" without changing Makefiles.

Thank you for this suggestion.  I have this approach now implemented, making
use of both KBUILD_MODFILE and KBUILD_MODNAME (both are needed to conclusively
determine that an object belongs to a module).

However, this is not catching objects that are compiled from assembler source,
because modfile_flags and modname_flags are not added to the assembler flags,
and thus KBUILD_MODFILE and KBUILD_MODNAME are not present in the .cmd file
for those objects.

It would seem that it is harmless to add those flags to assembler flags, so
would that be an acceptable solution?  It definitely would provide consistency
with non-asm objects.  And we already pass modfile and modname flags to the
non-asm builds for objects that most certainly do not belong in modules amnyway,
e.g.

$ cat arch/x86/boot/.cmdline.o.cmd| tr ' ' '\n' | grep -- -DKBUILD_MOD
-DKBUILD_MODFILE='"arch/x86/boot/cmdline"'
-DKBUILD_MODNAME='"cmdline"'

> You still need to modify scripts/Makefile.vmlinux(_o)
> but you can implement everything else in your script,
> although I did not fully understand the gawk script.
> 
> 
> Now, you can use Python if you like:
> 
>   https://lore.kernel.org/lkml/20240512-python-version-v2-1-382870a1fa1d@linaro.org/
> 
> Presumably, python code will be more readable for many people.
> 
> 
> GNU awk is not documented in Documentation/process/changes.rst
> If you insist on using gawk, you need to add it to the doc.
> 
> 
> 
> 
> 
> Having said that, I often hope to filter traced functions
> by an object path instead of a modname because modname
> filtering is only useful tristate code.
> For example, filter by "path:drivers/i2c/" or "path:drivers/i2c/i2c-core*"
> rather than "mod:i2c-core"
> 
> <object path, symbol name> reference will be useful for always-builtin code.
> 
> 
> 
> 
> >
> > The impact on the kernel build is minimal because everything is done
> > using a single-pass AWK script.  The generated data size is minimal as
> > well, (depending on the exact kernel configuration) usually in the range
> > of 500-700 lines, with a file size of 20-40KB.
> >
> > Changes since v1:
> >  - Renamed CONFIG_BUILTIN_RANGES to CONFIG_BUILTIN_MODULE_RANGES
> >  - Moved the config option to the tracers section
> >  - 2nd arg to generate_builtin_ranges.awk should be vmlinux.map
> >
> > Kris Van Hees (5):
> >   trace: add CONFIG_BUILTIN_MODULE_RANGES option
> >   kbuild: generate a linker map for vmlinux.o
> >   module: script to generate offset ranges for builtin modules
> >   kbuild: generate modules.builtin.ranges when linking the kernel
> >   module: add install target for modules.builtin.ranges
> >
> > Luis Chamberlain (1):
> >   kbuild: add modules.builtin.objs
> >
> >  .gitignore                          |   2 +-
> >  Documentation/dontdiff              |   2 +-
> >  Documentation/kbuild/kbuild.rst     |   5 ++
> >  Makefile                            |   8 +-
> >  include/linux/module.h              |   4 +-
> >  kernel/trace/Kconfig                |  17 ++++
> >  scripts/Makefile.lib                |   5 +-
> >  scripts/Makefile.modinst            |  11 ++-
> >  scripts/Makefile.vmlinux            |  17 ++++
> >  scripts/Makefile.vmlinux_o          |  18 ++++-
> >  scripts/generate_builtin_ranges.awk | 149 ++++++++++++++++++++++++++++++++++++
> >  11 files changed, 228 insertions(+), 10 deletions(-)
> >  create mode 100755 scripts/generate_builtin_ranges.awk
> >
> >
> > base-commit: dd5a440a31fae6e459c0d6271dddd62825505361
> > --
> > 2.42.0
> >
> >
> 
> 
> -- 
> Best Regards
> Masahiro Yamada