Received-SPF: pass (google.com: domain of linux-kernel+bounces-4517-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Date: Tue, 19 Dec 2023 00:23:18 +0000
From: Justin Stitt <justinstitt@google.com>
To: Carlos Llamas <cmllamas@google.com>
Cc: Nathan Chancellor <nathan@kernel.org>,
	Nick Desaulniers <ndesaulniers@google.com>,
	Tom Rix <trix@redhat.com>, linux-kernel@vger.kernel.org,
	kernel-team@android.com, Will Deacon <will@kernel.org>,
	John Stultz <jstultz@google.com>,
	Masahiro Yamada <masahiroy@kernel.org>
Subject: Re: [PATCH] scripts/decode_stacktrace.sh: optionally use LLVM
 utilities
Message-ID: <20231219002318.p7yciyr4ld62l4ad@google.com>
References: <20230929034836.403735-1-cmllamas@google.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20230929034836.403735-1-cmllamas@google.com>

Hi,

On Fri, Sep 29, 2023 at 03:48:17AM +0000, Carlos Llamas wrote:
> GNU's addr2line can have problems parsing a vmlinux built with LLVM,
> particularly when LTO was used. In order to decode the traces correctly
> this patch adds the ability to switch to LLVM's utilities readelf and
> addr2line. The same approach is followed by Will in [1].
>
> Before:
>   $ scripts/decode_stacktrace.sh vmlinux < kernel.log
>   [17716.240635] Call trace:
>   [17716.240646] skb_cow_data (??:?)
>   [17716.240654] esp6_input (ld-temp.o:?)
>   [17716.240666] xfrm_input (ld-temp.o:?)
>   [17716.240674] xfrm6_rcv (??:?)
>   [...]

Perhaps it is due to where I am dumping the stack, but I am getting good
stack traces on mainline (with a LLVM-built kernel) without this patch.

With that being said, applying this patch still has good stack traces
and it makes sense if it helps others.

>
> After:
>   $ LLVM=1 scripts/decode_stacktrace.sh vmlinux < kernel.log
>   [17716.240635] Call trace:
>   [17716.240646] skb_cow_data (include/linux/skbuff.h:2172 net/core/skbuff.c:4503)
>   [17716.240654] esp6_input (net/ipv6/esp6.c:977)
>   [17716.240666] xfrm_input (net/xfrm/xfrm_input.c:659)
>   [17716.240674] xfrm6_rcv (net/ipv6/xfrm6_input.c:172)
>   [...]
>
> Note that one could set CROSS_COMPILE=llvm- instead to hack around this
> issue. However, doing so can break the decodecode routine as it will
> force the selection of other LLVM utilities down the line e.g. llvm-as.
>
> [1] https://lore.kernel.org/all/20230914131225.13415-3-will@kernel.org/
>
> Cc: Will Deacon <will@kernel.org>
> Cc: John Stultz <jstultz@google.com>
> Cc: Masahiro Yamada <masahiroy@kernel.org>
> Signed-off-by: Carlos Llamas <cmllamas@google.com>

Tested-by: Justin Stitt <justinstitt@google.com>

Note that this patch is similar: https://lore.kernel.org/all/20231215-llvm-decode-stacktrace-v1-1-201cb86f4879@quicinc.com/

> ---
>  scripts/decode_stacktrace.sh | 19 +++++++++++++++++--
>  1 file changed, 17 insertions(+), 2 deletions(-)
>
> diff --git a/scripts/decode_stacktrace.sh b/scripts/decode_stacktrace.sh
> index 564c5632e1a2..bfe5a4082d8e 100755
> --- a/scripts/decode_stacktrace.sh
> +++ b/scripts/decode_stacktrace.sh
> @@ -16,6 +16,21 @@ elif type c++filt >/dev/null 2>&1 ; then
>  	cppfilt_opts=-i
>  fi
>
> +UTIL_SUFFIX=
> +if [[ -z ${LLVM:-} ]]; then
> +	UTIL_PREFIX=${CROSS_COMPILE:-}
> +else
> +	UTIL_PREFIX=llvm-
> +	if [[ ${LLVM} == */ ]]; then
> +		UTIL_PREFIX=${LLVM}${UTIL_PREFIX}
> +	elif [[ ${LLVM} == -* ]]; then
> +		UTIL_SUFFIX=${LLVM}
> +	fi
> +fi
> +
> +READELF=${UTIL_PREFIX}readelf${UTIL_SUFFIX}
> +ADDR2LINE=${UTIL_PREFIX}addr2line${UTIL_SUFFIX}
> +
>  if [[ $1 == "-r" ]] ; then
>  	vmlinux=""
>  	basepath="auto"
> @@ -75,7 +90,7 @@ find_module() {
>
>  	if [[ "$modpath" != "" ]] ; then
>  		for fn in $(find "$modpath" -name "${module//_/[-_]}.ko*") ; do
> -			if readelf -WS "$fn" | grep -qwF .debug_line ; then
> +			if ${READELF} -WS "$fn" | grep -qwF .debug_line ; then
>  				echo $fn
>  				return
>  			fi
> @@ -169,7 +184,7 @@ parse_symbol() {
>  	if [[ $aarray_support == true && "${cache[$module,$address]+isset}" == "isset" ]]; then
>  		local code=${cache[$module,$address]}
>  	else
> -		local code=$(${CROSS_COMPILE}addr2line -i -e "$objfile" "$address" 2>/dev/null)
> +		local code=$(${ADDR2LINE} -i -e "$objfile" "$address" 2>/dev/null)
>  		if [[ $aarray_support == true ]]; then
>  			cache[$module,$address]=$code
>  		fi
> --
> 2.42.0.582.g8ccd20d70d-goog
>

Thanks
Justin