Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
From:   Song Liu <songliubraving@fb.com>
To:     Peter Zijlstra <peterz@infradead.org>
CC:     "kasong@redhat.com" <kasong@redhat.com>,
        lkml <linux-kernel@vger.kernel.org>,
        Kernel Team <Kernel-team@fb.com>,
        "Josh Poimboeuf" <jpoimboe@redhat.com>
Subject: Re: Getting empty callchain from perf_callchain_kernel()
Thread-Topic: Getting empty callchain from perf_callchain_kernel()
Thread-Index: AQHVDEJXbyjc1nknl06FlgTGlvLrzKZu8J8AgAAG+ICAAIw1AA==
Date:   Fri, 17 May 2019 16:32:47 +0000
Message-ID: <8EF9E8F7-0B89-43E2-B767-AECD86E627B5@fb.com>
References: <3CD3EE63-0CD2-404A-A403-E11DCF2DF8D9@fb.com>
 <20190517074600.GJ2623@hirez.programming.kicks-ass.net>
 <20190517081057.GQ2650@hirez.programming.kicks-ass.net>
In-Reply-To: <20190517081057.GQ2650@hirez.programming.kicks-ass.net>
Accept-Language: en-US
Content-Language: en-US
received-spf: None (protection.outlook.com: fb.com does not designate
 permitted sender hosts)
Content-Type: text/plain; charset="us-ascii"
Content-ID: <D3DCF5FA2E41BC4390A1C0C59C671E04@namprd15.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: 6cd2a4e7-9385-40a5-27d5-08d6dae54c05
X-MS-Exchange-CrossTenant-originalarrivaltime: 17 May 2019 16:32:47.5644
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR15MB1615
X-OriginatorOrg: fb.com
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:,, definitions=2019-05-17_09:,,
 signatures=0
X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 priorityscore=1501
 malwarescore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0
 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0
 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx
 scancount=1 engine=8.0.1-1810050000 definitions=main-1905170099
X-FB-Internal: deliver
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


> On May 17, 2019, at 1:10 AM, Peter Zijlstra <peterz@infradead.org> wrote:
>=20
> On Fri, May 17, 2019 at 09:46:00AM +0200, Peter Zijlstra wrote:
>> On Thu, May 16, 2019 at 11:51:55PM +0000, Song Liu wrote:
>>> Hi,=20
>>>=20
>>> We found a failure with selftests/bpf/tests_prog in test_stacktrace_map=
 (on bpf/master
>>> branch).=20
>>>=20
>>> After digging into the code, we found that perf_callchain_kernel() is g=
iving empty
>>> callchain for tracepoint sched/sched_switch. And it seems related to co=
mmit
>>>=20
>>> d15d356887e770c5f2dcf963b52c7cb510c9e42d
>>> ("perf/x86: Make perf callchains work without CONFIG_FRAME_POINTER")
>>>=20
>>> Before this commit, perf_callchain_kernel() returns callchain with regs=
->ip. With
>>> this commit, regs->ip is not sent for !perf_hw_regs(regs) case.=20
>>=20
>> So while I think the below is indeed right; we should store regs->ip
>> regardless of the unwind path chosen, I still think there's something
>> fishy if this results in just the 1 entry.
>>=20
>> The sched/sched_switch event really should have a non-trivial stack.
>>=20
>> Let me see if I can reproduce with just perf.
>=20
> $ perf record -g -e "sched:sched_switch" -- make clean
> $ perf report -D
>=20
> 12 904071759467 0x1790 [0xd0]: PERF_RECORD_SAMPLE(IP, 0x1): 7236/7236: 0x=
ffffffff81c29562 period: 1 addr: 0
> ... FP chain: nr:10
> .....  0: ffffffffffffff80
> .....  1: ffffffff81c29562
> .....  2: ffffffff81c29933
> .....  3: ffffffff8111f688
> .....  4: ffffffff81120b9d
> .....  5: ffffffff81120ce5
> .....  6: ffffffff8100254a
> .....  7: ffffffff81e0007d
> .....  8: fffffffffffffe00
> .....  9: 00007f9b6cd9682a
> ... thread: sh:7236
> ...... dso: /lib/modules/5.1.0-12177-g41bbb9129767/build/vmlinux
>=20

Hmm... I also get this FP chain even with CONFIG_UNWINDER_ORC=3Dy.=20
I guess it is actually from ORC?=20

Thanks,
Song=20