Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753406AbdFMPz0 (ORCPT ); Tue, 13 Jun 2017 11:55:26 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:52987 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751826AbdFMPzY (ORCPT ); Tue, 13 Jun 2017 11:55:24 -0400 Subject: Re: [PATCH v2] perf: libdw support for powerpc [ping] To: Mark Wielaard References: <1496312681-20133-1-git-send-email-pbonzini@redhat.com> <593E81ED.8060606@linux.vnet.ibm.com> <1497354282.3755.201.camel@klomp.org> Cc: Paolo Bonzini , linux-kernel@vger.kernel.org, acme@kernel.org, "Naveen N. Rao" , linuxppc-dev@lists.ozlabs.org, Ravi Bangoria From: Ravi Bangoria Date: Tue, 13 Jun 2017 21:25:09 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.5.1 MIME-Version: 1.0 In-Reply-To: <1497354282.3755.201.camel@klomp.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-TM-AS-GCONF: 00 x-cbid: 17061315-0052-0000-0000-00000221205D X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00007225; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000212; SDB=6.00874235; UDB=6.00435148; IPR=6.00654323; BA=6.00005419; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00015808; XFM=3.00000015; UTC=2017-06-13 15:55:23 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17061315-0053-0000-0000-000050F25B63 Message-Id: <59400ADD.4090709@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-06-13_08:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1706130274 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 5441 Lines: 104 Hi Mark, On Tuesday 13 June 2017 05:14 PM, Mark Wielaard wrote: > I see the same on very short runs. But when doing a slightly longer run, > even just using ls -lahR, which does some more work, then I do see user > backtraces. They are still missing for some of the early samples though. > It is as if there is a stack/memory address mismatch when the probe is > "too early" in ld.so. > > Could you do a test run on some program that does some more work to see > if you never get any user stack traces, or if you only not get them for > some specific probes? Thanks for checking. I tried a proper workload this time, but I still don't see any userspace callchain getting unwound. $ ./perf record --call-graph=dwarf -- zip -q -r temp.zip . [ perf record: Woken up 2891 times to write data ] [ perf record: Captured and wrote 723.290 MB perf.data (87934 samples) ] With libdw: $ LD_LIBRARY_PATH=/home/ravi/elfutils-git/usr/local/lib:\ /home/ravi/elfutils-git/usr/local/lib/elfutils/:$LD_LIBRARY_PATH\ ./perf script zip 16699 6857.354633: 37371 cycles:u: ecedc xmon_core (/usr/lib/debug/lib/modules/4.11.0-3.el7.ppc64le/vmlinux) 8c4fc __hash_page_64K (/usr/lib/debug/lib/modules/4.11.0-3.el7.ppc64le/vmlinux) 83450 hash_preload (/usr/lib/debug/lib/modules/4.11.0-3.el7.ppc64le/vmlinux) 7cc34 update_mmu_cache (/usr/lib/debug/lib/modules/4.11.0-3.el7.ppc64le/vmlinux) 330064 alloc_set_pte (/usr/lib/debug/lib/modules/4.11.0-3.el7.ppc64le/vmlinux) 330efc do_fault (/usr/lib/debug/lib/modules/4.11.0-3.el7.ppc64le/vmlinux) 334580 __handle_mm_fault (/usr/lib/debug/lib/modules/4.11.0-3.el7.ppc64le/vmlinux) 335040 handle_mm_fault (/usr/lib/debug/lib/modules/4.11.0-3.el7.ppc64le/vmlinux) 7bf94 do_page_fault (/usr/lib/debug/lib/modules/4.11.0-3.el7.ppc64le/vmlinux) 7bec4 do_page_fault (/usr/lib/debug/lib/modules/4.11.0-3.el7.ppc64le/vmlinux) 7be78 do_page_fault (/usr/lib/debug/lib/modules/4.11.0-3.el7.ppc64le/vmlinux) 1a4f8 handle_page_fault (/usr/lib/debug/lib/modules/4.11.0-3.el7.ppc64le/vmlinux) zip 16699 6857.354663: 300677 cycles:u: zip 16699 6857.354895: 584131 cycles:u: zip 16699 6857.355312: 589687 cycles:u: zip 16699 6857.355606: 560142 cycles:u: With libunwind: $ ./perf script zip 16699 6857.354633: 37371 cycles:u: ecedc xmon_core (/usr/lib/debug/lib/modules/4.11.0-3.el7.ppc64le/vmlinux) 8c4fc __hash_page_64K (/usr/lib/debug/lib/modules/4.11.0-3.el7.ppc64le/vmlinux) 83450 hash_preload (/usr/lib/debug/lib/modules/4.11.0-3.el7.ppc64le/vmlinux) 7cc34 update_mmu_cache (/usr/lib/debug/lib/modules/4.11.0-3.el7.ppc64le/vmlinux) 330064 alloc_set_pte (/usr/lib/debug/lib/modules/4.11.0-3.el7.ppc64le/vmlinux) 330efc do_fault (/usr/lib/debug/lib/modules/4.11.0-3.el7.ppc64le/vmlinux) 334580 __handle_mm_fault (/usr/lib/debug/lib/modules/4.11.0-3.el7.ppc64le/vmlinux) 335040 handle_mm_fault (/usr/lib/debug/lib/modules/4.11.0-3.el7.ppc64le/vmlinux) 7bf94 do_page_fault (/usr/lib/debug/lib/modules/4.11.0-3.el7.ppc64le/vmlinux) 7bec4 do_page_fault (/usr/lib/debug/lib/modules/4.11.0-3.el7.ppc64le/vmlinux) 7be78 do_page_fault (/usr/lib/debug/lib/modules/4.11.0-3.el7.ppc64le/vmlinux) 1a4f8 handle_page_fault (/usr/lib/debug/lib/modules/4.11.0-3.el7.ppc64le/vmlinux) 1920 _start (/usr/lib64/ld-2.17.so) zip 16699 6857.354663: 300677 cycles:u: fa38 _dl_new_object (/usr/lib64/ld-2.17.so) 3073 dl_main (/usr/lib64/ld-2.17.so) 2045b _dl_sysdep_start (/usr/lib64/ld-2.17.so) 1c7f _dl_start_final (/usr/lib64/ld-2.17.so) 5ce7 _dl_start (/usr/lib64/ld-2.17.so) 1937 _start (/usr/lib64/ld-2.17.so) zip 16699 6857.354895: 584131 cycles:u: 103d0 _dl_relocate_object (/usr/lib64/ld-2.17.so) zip 16699 6857.355312: 589687 cycles:u: df68 do_lookup_x (/usr/lib64/ld-2.17.so) e8d7 _dl_lookup_symbol_x (/usr/lib64/ld-2.17.so) 14bb3 _dl_fixup (/usr/lib64/ld-2.17.so) 1ef37 _dl_runtime_resolve (/usr/lib64/ld-2.17.so) 20bf7 copy_args (/usr/bin/zip) 286f main (/usr/bin/zip) 2497f generic_start_main.isra.0 (/usr/lib64/libc-2.17.so) 24b73 __libc_start_main (/usr/lib64/libc-2.17.so) zip 16699 6857.355606: 560142 cycles:u: 84764 _IO_getc (/usr/lib64/libc-2.17.so) c4d7 find_next_signature (/usr/bin/zip) 15c8b readzipfile (/usr/bin/zip) 3477 main (/usr/bin/zip) 2497f generic_start_main.isra.0 (/usr/lib64/libc-2.17.so) 24b73 __libc_start_main (/usr/lib64/libc-2.17.so) Right now, I'm busy with few other things so I'm not able to dig further. But I can look into this next week or so. Thanks, Ravi