Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S965508AbdIZGNA (ORCPT ); Tue, 26 Sep 2017 02:13:00 -0400 Received: from mga05.intel.com ([192.55.52.43]:18881 "EHLO mga05.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S965299AbdIZGM6 (ORCPT ); Tue, 26 Sep 2017 02:12:58 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.42,439,1500966000"; d="asc'?scan'208";a="153349681" Date: Tue, 26 Sep 2017 14:06:30 +0800 From: "Du, Changbin" To: "Du, Changbin" Cc: Arnaldo Carvalho de Melo , peterz@infradead.org, mingo@redhat.com, alexander.shishkin@linux.intel.com, linux-kernel@vger.kernel.org Subject: Re: Does perf-annotate work correctly? Message-ID: <20170926060629.GB8868@intel.com> References: <20170912101035.GA21638@intel.com> <20170912143350.GA3452@kernel.org> <20170913015425.GA607@intel.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="EuxKj2iCbKjpUGkD" Content-Disposition: inline In-Reply-To: <20170913015425.GA607@intel.com> User-Agent: Mutt/1.5.24 (2015-08-30) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 11485 Lines: 261 --EuxKj2iCbKjpUGkD Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi Arnaldo and all, Is there someone looking at this issue? I can provide more info if you need. This issue makes it so inconvenient to use perf. Thanks. On Wed, Sep 13, 2017 at 09:54:25AM +0800, Du, Changbin wrote: > On Tue, Sep 12, 2017 at 11:33:50AM -0300, Arnaldo Carvalho de Melo wrote: > > Em Tue, Sep 12, 2017 at 06:10:35PM +0800, Du, Changbin escreveu: > > > When a annotate a symbol, I find the annotated C source code doesn't = match assembly code. > > > So I cannot determine which line of C code has much overhead withou g= db's help. > > >=20 > > > Here is a example result of function apic_has_interrupt_for_ppr() in = kvm module. > >=20 > > Ok, was this using the module .ko file or /proc/kcore? You forgot to > > cut'n'paste the first line on the screen. > >=20 > It is arch/x86/kvm/kvm.ko. >=20 > > Also, how did you use gdb? > >=20 > $ gdb arch/x86/kvm/kvm.ko > $ (gdb) disassemble /s apic_has_interrupt_for_ppr >=20 > > perf uses objdump to do the disassembly, and depending on how it is used > > (live system, post processing on a different machine, permissions) it > > may use different files to do the disassembly. > >=20 > But objdump has same out as gdb. (Always on same machine, and no binary c= hanged.) >=20 > $ objdump -d -S arch/x86/kvm/kvm.o > ... > static int apic_has_interrupt_for_ppr(struct kvm_lapic *apic, u32 ppr) > { > 3b4a0: e8 00 00 00 00 callq 3b4a5 > 3b4a5: 55 push %rbp > 3b4a6: 48 89 e5 mov %rsp,%rbp > 3b4a9: 48 83 ec 08 sub $0x8,%rsp > int highest_irr; > if (kvm_x86_ops->sync_pir_to_irr && apic->vcpu->arch.apicv_active) > 3b4ad: 48 8b 05 00 00 00 00 mov 0x0(%rip),%rax # 3b4b4 > 3b4b4: 48 8b 80 38 02 00 00 mov 0x238(%rax),%rax > 3b4bb: 48 85 c0 test %rax,%rax > 3b4be: 74 10 je 3b4d0 > 3b4c0: 48 8b 97 88 00 00 00 mov 0x88(%rdi),%rdx > 3b4c7: 80 ba 28 03 00 00 00 cmpb $0x0,0x328(%rdx) > 3b4ce: 75 3a jne 3b50a >=20 > /* > * Note that irr_pending is just a hint. It will be always > * true with virtual interrupt delivery enabled. > */ > if (!apic->irr_pending) > 3b4d0: 80 bf 91 00 00 00 00 cmpb $0x0,0x91(%rdi) > 3b4d7: 74 2a je 3b503 > 3b4d9: 48 8b 8f a0 00 00 00 mov 0xa0(%rdi),%rcx > static int find_highest_vector(void *bitmap) > { > int vec; > u32 *reg; >=20 > for (vec =3D MAX_APIC_VECTOR - APIC_VECTORS_PER_REG; > 3b4e0: b8 e0 00 00 00 mov $0xe0,%eax > vec >=3D 0; vec -=3D APIC_VECTORS_PER_REG) { > reg =3D bitmap + REG_POS(vec); > if (*reg) > 3b4e5: 89 c2 mov %eax,%edx > 3b4e7: c1 fa 05 sar $0x5,%edx > 3b4ea: c1 e2 04 shl $0x4,%edx > 3b4ed: 48 63 d2 movslq %edx,%rdx > 3b4f0: 8b 94 11 00 02 00 00 mov 0x200(%rcx,%rdx,1),%edx > 3b4f7: 85 d2 test %edx,%edx > 3b4f9: 75 2d jne 3b528 >=20 >=20 >=20 > > Please provide more detailed information on the exact command line > > arguments and usage scenario. > > =20 > > - Arnaldo >=20 > >=20 > > > =E2=94=82580 __clear_bit(KVM_APIC_PV_EOI_PENDING, &vcp= u->arch.apic_attention); =E2=96=92 > > > =E2=94=82581 } = =E2=96=92 > > > =E2=94=82 = =E2=96=92 > > > =E2=94=82583 static int apic_has_interrupt_for_ppr(struct kvm_= lapic *apic, u32 ppr) =E2=96=92 > > > =E2=94=82584 { = =E2=96=92 > > > 0.88 =E2=94=8230: cmpb $0x0,0x91(%rdi) = =E2=96=92 > > > 2.54 =E2=94=82 =E2=86=93 je 63 = =E2=96=92 > > > 0.20 =E2=94=82 mov 0xa0(%rdi),%rcx = =E2=96=92 > > > =E2=94=82581 int highest_irr; = =E2=96=92 > > > =E2=94=82582 if (kvm_x86_ops->sync_pir_to_irr && apic-= >vcpu->arch.apicv_active) =E2=96=92 > > > 4.91 =E2=94=82 mov $0xe0,%eax x = =E2=96=92 > > > 1.46 =E2=94=8245: mov %eax,%edx x = =E2=96=92 > > > 0.02 =E2=94=82 sar $0x5,%edx x = =E2=96=92 > > > 3.57 =E2=94=82 shl $0x4,%edx x = =E2=96=92 > > > 3.34 =E2=94=82 movslq %edx,%rdx x = =E2=96=92 > > > 1.25 =E2=94=82 mov 0x200(%rcx,%rdx,1),%edx x = =E2=96=92 > > > 42.44 =E2=94=82 test %edx,%edx x = =E2=96=92 > > > 0.01 =E2=94=82 =E2=94=8C=E2=94=80=E2=94=80jne 88 = x =E2=96=92 > > > 3.48 =E2=94=82 =E2=94=82 sub $0x20,%eax = x =E2=96=92 > > > 2.24 =E2=94=82 =E2=94=82 cmp $0xffffffe0,%eax = x =E2=96=92 > > > =E2=94=82586=E2=94=82apic_find_highest_irr(): = =E2=96=92 > > > =E2=94=82 =E2=94=82 = =E2=96=92 > > > =E2=94=82407=E2=94=82 /* = =E2=96=92 > > > =E2=94=82408=E2=94=82 * Note that irr_pending is just = a hint. It will be always =E2=96=92 > > > =E2=94=82409=E2=94=82 * true with virtual interrupt de= livery enabled. =E2=96=92 > > > =E2=94=82410=E2=94=82 */ = =E2=96=92 > > > =E2=94=82411=E2=94=82 if (!apic->irr_pending) = =E2=96=92 > > > =E2=94=82 =E2=94=82=E2=86=91 jne 45 = =E2=96=92 > > > 0.62 =E2=94=8263:=E2=94=82 mov $0xffffffff,%eax = =E2=97=86 > > > 0.83 =E2=94=82 =E2=94=82 leaveq = =E2=96=92 > > > 13.52 =E2=94=82 =E2=94=82=E2=86=90 retq = =E2=96=92 > > > =E2=94=826a:=E2=94=82 mov %esi,-0x4(%rbp) = =E2=96=92 > > > =E2=94=82 =E2=94=82 mov %rdx,%rdi = =E2=96=92 > > > =E2=94=82418=E2=94=82find_highest_vector(): = =E2=96=92 > > > =E2=94=82340=E2=94=82static int find_highest_vector(void *bitm= ap) =E2=96=92 > > > =E2=94=82341=E2=94=82{ = =E2=96=92 > > > =E2=94=82342=E2=94=82 int vec; = =E2=96=92 > > > =E2=94=82343=E2=94=82 u32 *reg; = =E2=96=92 > > > =E2=94=82 =E2=94=82 = =E2=96=92 > > > =E2=94=82345=E2=94=82 for (vec =3D MAX_APIC_VECTOR - AP= IC_VECTORS_PER_REG; =E2=96=92 > > > =E2=94=82 =E2=94=82=E2=86=92 callq *%rax = =E2=96=92 > > > =E2=94=82 =E2=94=82 mov -0x4(%rbp),%esi = =E2=96=92 > > > =E2=94=82343=E2=94=82 vec >=3D 0; vec -=3D APIC_VE= CTORS_PER_REG) { =E2=96=92 > > > =E2=94=82344=E2=94=82 reg =3D bitmap + REG_POS(= vec); =E2=96=92 > > > =E2=94=82345=E2=94=82 if (*reg) = =E2=96=92 > > > 0.05 =E2=94=8275:=E2=94=82 cmp $0xffffffff,%eax = =E2=96=92 > > > =E2=94=82 =E2=94=82=E2=86=91 je 63 = =E2=96=92 > > > 1.95 =E2=94=82 =E2=94=82 mov %eax,%edx = =E2=96=92 > > > 1.45 =E2=94=82 =E2=94=82 and $0xf0,%edx = =20 > > >=20 > > >=20 > > > Look at the assembly code block where I have put a 'x' on the right. = Apparently the > > > assembly code doesn't match the C source code arrounded. Let's look t= he correct disassemble > > > result from gdb: > > >=20 > > > 340 for (vec =3D MAX_APIC_VECTOR - APIC_VECTORS_PER_REG; > > > 0x000000000003b4e0 <+64>: mov $0xe0,%eax > > >=20 > > > 342 reg =3D bitmap + REG_POS(vec); > > > 343 if (*reg) > > > 0x000000000003b4e5 <+69>: mov %eax,%edx > > > 0x000000000003b4e7 <+71>: sar $0x5,%edx > > > 0x000000000003b4ea <+74>: shl $0x4,%edx > > > 0x000000000003b4ed <+77>: movslq %edx,%rdx > > > 0x000000000003b4f0 <+80>: mov 0x200(%rcx,%rdx,1),%edx > > > 0x000000000003b4f7 <+87>: test %edx,%edx > > > 0x000000000003b4f9 <+89>: jne 0x3b528 > > >=20 > > > 341 vec >=3D 0; vec -=3D APIC_VECTORS_PER_REG) { > > > 0x000000000003b4fb <+91>: sub $0x20,%eax > > >=20 > > > 340 for (vec =3D MAX_APIC_VECTOR - APIC_VECTORS_PER_REG; > > > 0x000000000003b4fe <+94>: cmp $0xffffffe0,%eax > > > 0x000000000003b501 <+97>: jne 0x3b4e5 > > >=20 > > >=20 > > > Compared to gdb, perf-annoate has messed up. is it a bug or just perf= is not as perfect as gdb? > > >=20 > > > --=20 > > > Thanks, > > > Changbin Du > >=20 > >=20 >=20 > --=20 > Thanks, > Changbin Du --=20 Thanks, Changbin Du --EuxKj2iCbKjpUGkD Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJZye5lAAoJEAanuZwLnPNUn58H/0hJpKjEaw5B+xAOlAN29M9q zyWj6h69ng+Nq8+aXkRCGg6XvXNXuDGtutlAEhAEc+vAA7k2HSB8ymveWlBAG8DE UTSnjHlDbFV1qaD7meK05ODqf2Ah1zZaEvXRHJbjqGXREw9jc3kbkKaw8EpBj9Mm 5aFClnuvjtjOQa4X9weho1fFfE8p7kpRYZfYs9pNIc1LU8m8Yo3MEEcilKwhD08R 73YoqEYSqdwmdLEIt0WA4lqnLFLIp4uFWRSln0MbnQVbx0rF17CYA/8G2nS3P932 U4XN2tM2QC8h8YvIW3VEE+/jBfarXNLu2o/W9joFKaYjujEjMZDXlpOK4c857Uo= =LNO/ -----END PGP SIGNATURE----- --EuxKj2iCbKjpUGkD--