Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
IronPort-SDR: I+KjmFhqnqChobdF0J/4eLyiKgLlO/4Mt9vIrIJYoFZzDTWi4z6vpIGI8IR1EwCFDxeQpjTHib
 RIYUMTbut+Zw==
IronPort-SDR: bywuvKU1786MClS10ZZLwShMJM+68apVAuEAph5z9aduHixWIjk4bwKZsdL4ObFh2DQWOYYOyQ
 YKQBgTLOQg2Q==
Date:   Wed, 26 Aug 2020 09:45:58 +0800
From:   Feng Tang <feng.tang@intel.com>
To:     "Luck, Tony" <tony.luck@intel.com>
Cc:     Borislav Petkov <bp@suse.de>,
        "Chen, Rong A" <rong.a.chen@intel.com>,
        LKML <linux-kernel@vger.kernel.org>,
        "lkp@lists.01.org" <lkp@lists.01.org>,
        Mel Gorman <mgorman@suse.com>
Subject: Re: [LKP] Re: [x86/mce] 1de08dccd3: will-it-scale.per_process_ops
 -14.1% regression
Message-ID: <20200826014558.GC83850@shbuild999.sh.intel.com>
References: <20200425130136.GA28245@zn.tnic>
 <20200818082943.GA65567@shbuild999.sh.intel.com>
 <20200818200654.GA21494@agluck-desk2.amr.corp.intel.com>
 <20200819020437.GA2605@shbuild999.sh.intel.com>
 <20200821020259.GA90000@shbuild999.sh.intel.com>
 <20200824151425.GF4794@zn.tnic>
 <20200824153300.GA56944@shbuild999.sh.intel.com>
 <20200824161238.GI4794@zn.tnic>
 <20200825062305.GA83850@shbuild999.sh.intel.com>
 <b704f3c6298f46abbc59105ec0d52d14@intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <b704f3c6298f46abbc59105ec0d52d14@intel.com>
User-Agent: Mutt/1.5.24 (2015-08-30)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Wed, Aug 26, 2020 at 12:44:37AM +0800, Luck, Tony wrote:
> > These 2 variables are accessed in 2 hot call stacks (for this 288 CPU
> > Xeon Phi platform):
> 
> This might be the key element of "weirdness" for this system. It
> has 288 CPUs ... cache alignment problems are often not too bad
> on "small" systems. The as you scale up to bigger machines you
> suddenly hit some critical point and performance drops dramatically.
> 
> It's good that you are picking up tips on how to bisect these and diagnose
> the underlying problem. Number of cores is going to keep increasing, so
> we will keep finding new issues like this.

Yes, now we have one more bullet for shooting this kind of strange
performance changes :)

Thanks,
Feng

> -Tony