Received: by 2002:a25:1506:0:0:0:0:0 with SMTP id 6csp1900259ybv; Fri, 21 Feb 2020 05:22:49 -0800 (PST) X-Google-Smtp-Source: APXvYqwRD2I8CDKR70/cAxBEA8+zgxfOAXL3dKN55mZ0M4gaT9R69VI2xqBoRNuDKSIhE+MlQ9aY X-Received: by 2002:a54:448b:: with SMTP id v11mr1940749oiv.74.1582291369506; Fri, 21 Feb 2020 05:22:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1582291369; cv=none; d=google.com; s=arc-20160816; b=D9z3QjUiRR9hnfW5d9jkvVFy84d9YpdJI8sCogBUU/qmZ7AqAReSTGJL/8gNpe9hht +tH9DwvWt00cX/ifUsikLzBIyyaV/s6j5BZ/k9RwDv+7tVkNvUy8h59beBv7gyaAyStx eVIlitNUYQnzQBog6tQwWfaKAtnZ7/1HBQEbdVLDyksFfOuh+QTOO9aGEnCDPsV+tl4T szFm9GenUewVJNVjv7QPaOL/41X5vSbgF466T5dNSACgVtyXHn2PN1s1nh4IItxZia1E Cam9S2C9w1E3Cv3MSKptz1sETGprOewfw3yqiNXQVGq5pTk6wL+NMEWz1LaOukXbkv4M KG+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:in-reply-to:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :dkim-signature; bh=K4ms2CgVns2BaGnINjY8fAnvcQ0WnyDlOpXKLBLyMwo=; b=rgpi+ASzpQbS47baQRUoVYXmgj/rU+zBehgf/6/66/Y0RtImWx2tse/o0eeer010sN FDQggKP8et9nvxDBOmo+cEjvBLWW+fkR6ZT9HHSTRZRvtEapXb3GVY8iAVRRNWMuaugG 3ZuqNP6HXRbG314SWRG9ycYj17x877dI+L8eDQsucMB2YNsAQR/7DMyew1/pzaoIP5Zo SjIlhiDXBM6VAoMX6lm8k+A6ggI83hR9Wa5bU8WdSPz17MjkdHydEGLkxwLOc9z7yuEh tIj6x8mA0Z4T4pG3i60G63eF3yMFRhULwYdp4TtduNgWIlVVHS6v7wghkRkB6fd1PIf0 t7yg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=JSRizBa+; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id d3si711462oia.236.2020.02.21.05.22.37; Fri, 21 Feb 2020 05:22:49 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=JSRizBa+; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728526AbgBUNVE (ORCPT + 99 others); Fri, 21 Feb 2020 08:21:04 -0500 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:36970 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727699AbgBUNVE (ORCPT ); Fri, 21 Feb 2020 08:21:04 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1582291263; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=K4ms2CgVns2BaGnINjY8fAnvcQ0WnyDlOpXKLBLyMwo=; b=JSRizBa+LIXeP5hmVH2CE3VUhNBCclNWwE0Pfj5Iy9gnoExuPF8OXkvikYsTVvNODzcOMJ k2JOM0aKf43v+3S2Q5RKes4ER2PWuj2I5TLngNwhLxTEClEc11F5Nklk/a1dJ2u2EDcTYz AhMAX10Oo6HVKC1U6gydOh1/3tvGnR8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-29-wrBPfRNlOUmJH_P1DeT36w-1; Fri, 21 Feb 2020 08:20:57 -0500 X-MC-Unique: wrBPfRNlOUmJH_P1DeT36w-1 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D4AF7107ACC5; Fri, 21 Feb 2020 13:20:54 +0000 (UTC) Received: from krava (unknown [10.43.17.9]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 1FE0427180; Fri, 21 Feb 2020 13:20:50 +0000 (UTC) Date: Fri, 21 Feb 2020 14:20:48 +0100 From: Jiri Olsa To: Feng Tang Cc: Peter Zijlstra , kernel test robot , Ingo Molnar , Vince Weaver , Jiri Olsa , Alexander Shishkin , Arnaldo Carvalho de Melo , Arnaldo Carvalho de Melo , Linus Torvalds , "Naveen N. Rao" , Ravi Bangoria , Stephane Eranian , Thomas Gleixner , LKML , lkp@lists.01.org, andi.kleen@intel.com, ying.huang@intel.com Subject: Re: [LKP] Re: [perf/x86] 81ec3f3c4c: will-it-scale.per_process_ops -5.5% regression Message-ID: <20200221132048.GE652992@krava> References: <20200205123216.GO12867@shao2-debian> <20200205125804.GM14879@hirez.programming.kicks-ass.net> <20200221080325.GA67807@shbuild999.sh.intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200221080325.GA67807@shbuild999.sh.intel.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Feb 21, 2020 at 04:03:25PM +0800, Feng Tang wrote: > > On Wed, Feb 05, 2020 at 01:58:04PM +0100, Peter Zijlstra wrote: > > On Wed, Feb 05, 2020 at 08:32:16PM +0800, kernel test robot wrote: > > > Greeting, > > > > > > FYI, we noticed a -5.5% regression of will-it-scale.per_process_ops due to commit: > > > > > > > > > commit: 81ec3f3c4c4d78f2d3b6689c9816bfbdf7417dbb ("perf/x86: Add check_period PMU callback") > > > https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master > > > > > > > I'm fairly sure this bisect/result is bogus. > > > Hi Peter, > > Some updates: > > We checked more on this. We run 14 times test for it, and the > results are consistent about the 5.5% degradation, and we > run the same test on several other platforms, whose test results > are also consistent, though there are no such -5.5% seen. > > We are also curious that the commit seems to be completely not > relative to this scalability test of signal, which starts a task > for each online CPU, and keeps calling raise(), and calculating > the run numbers. > > One experiment we did is checking which part of the commit > really affects the test, and it turned out to be the change of > "struct pmu". Effectively, applying this patch upon 5.0-rc6 > which triggers the same regression. > > diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h > index 1d5c551..e1a0517 100644 > --- a/include/linux/perf_event.h > +++ b/include/linux/perf_event.h > @@ -447,6 +447,11 @@ struct pmu { > * Filter events for PMU-specific reasons. > */ > int (*filter_match) (struct perf_event *event); /* optional */ > + > + /* > + * Check period value for PERF_EVENT_IOC_PERIOD ioctl. > + */ > + int (*check_period) (struct perf_event *event, u64 value); /* optional */ > }; > > So likely, this commit changes the layout of the kernel text > and data, which may trigger some cacheline level change. From > the system map of the 2 kernels, a big trunk of symbol's address > changes which follow the global "pmu", nice, I wonder we could see that in perf c2c output ;-) I'll try to run and check thanks, jirka > > 5.0-rc6-systemap: > > ffffffff8221d000 d pmu > ffffffff8221d100 d pmc_reserve_mutex > ffffffff8221d120 d amd_f15_PMC53 > ffffffff8221d160 d amd_f15_PMC50 > > 5.0-rc6+pmu-change-systemap: > > ffffffff8221d000 d pmu > ffffffff8221d120 d pmc_reserve_mutex > ffffffff8221d140 d amd_f15_PMC53 > ffffffff8221d180 d amd_f15_PMC50 > > But we can hardly identify which exact symbol is responsible > for the change, as too many symbols are offseted. > > btw, we've seen similar case that an irrelevant commit changes > the benchmark, like a hugetlb patch improves pagefault test on > a platform that never uses hugetlb https://lkml.org/lkml/2020/1/14/150 > > Thanks, > Feng > > > _______________________________________________ > > LKP mailing list -- lkp@lists.01.org > > To unsubscribe send an email to lkp-leave@lists.01.org >