Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [fs] a0918006f9: netperf.Throughput_tps -11.6% regression
To:     Kees Cook <keescook@chromium.org>,
        kernel test robot <oliver.sang@intel.com>
CC:     =?UTF-8?Q?Micka=c3=abl_Sala=c3=bcn?= <mic@digikod.net>,
        <lkp@lists.01.org>, <lkp@intel.com>, <ying.huang@intel.com>,
        <feng.tang@intel.com>, <zhengjun.xing@linux.intel.com>,
        Al Viro <viro@zeniv.linux.org.uk>,
        Andrew Morton <akpm@linux-foundation.org>,
        "Aleksa Sarai" <cyphar@cyphar.com>,
        Andy Lutomirski <luto@kernel.org>,
        Arnd Bergmann <arnd@arndb.de>,
        Casey Schaufler <casey@schaufler-ca.com>,
        Christian Brauner <christian.brauner@ubuntu.com>,
        Christian Heimes <christian@python.org>,
        Deven Bowers <deven.desai@linux.microsoft.com>,
        Dmitry Vyukov <dvyukov@google.com>,
        Eric Biggers <ebiggers@kernel.org>,
        Eric Chiang <ericchiang@google.com>,
        Florian Weimer <fweimer@redhat.com>,
        "Geert Uytterhoeven" <geert@linux-m68k.org>,
        James Morris <jmorris@namei.org>, "Jan Kara" <jack@suse.cz>,
        Jann Horn <jannh@google.com>, Jonathan Corbet <corbet@lwn.net>,
        Lakshmi Ramasubramanian <nramas@linux.microsoft.com>,
        "Madhavan T . Venkataraman" <madvenka@linux.microsoft.com>,
        Matthew Garrett <mjg59@google.com>,
        Matthew Wilcox <willy@infradead.org>,
        Miklos Szeredi <mszeredi@redhat.com>,
        Mimi Zohar <zohar@linux.ibm.com>,
        Paul Moore <paul@paul-moore.com>,
        =?UTF-8?Q?Philippe_Tr=c3=a9buchet?= 
        <philippe.trebuchet@ssi.gouv.fr>,
        Scott Shell <scottsh@microsoft.com>,
        "Shuah Khan" <shuah@kernel.org>,
        Steve Dower <steve.dower@python.org>,
        Steve Grubb <sgrubb@redhat.com>,
        Thibaut Sautereau <thibaut.sautereau@ssi.gouv.fr>,
        Vincent Strubel <vincent.strubel@ssi.gouv.fr>,
        <kernel-hardening@lists.openwall.com>, <linux-api@vger.kernel.org>,
        <linux-fsdevel@vger.kernel.org>, <linux-integrity@vger.kernel.org>,
        <linux-kernel@vger.kernel.org>,
        <linux-security-module@vger.kernel.org>,
        =?UTF-8?Q?Micka=c3=abl_Sala=c3=bcn?= <mic@linux.microsoft.com>
References: <20211012192410.2356090-2-mic@digikod.net>
 <20211105064159.GB17949@xsang-OptiPlex-9020>
 <202111090920.4958E610D1@keescook>
From:   Yin Fengwei <fengwei.yin@intel.com>
Message-ID: <129a2714-bad3-6f91-d841-51121bcc6e8d@intel.com>
Date:   Wed, 10 Nov 2021 09:54:27 +0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Firefox/78.0 Thunderbird/78.14.0
In-Reply-To: <202111090920.4958E610D1@keescook>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?UkxUNUlVZ2ZLbnBLSWxpVzBYa3ZScXhjKzQ0T3dicXpFRnVjT2V5ejI4U0pn?=
 =?utf-8?B?WG5UM0hlRHVmT2hKSGhyVWpnYlZsTE9IWUlGWFpjQjd1dU82anVOR083dnNE?=
 =?utf-8?B?Qi8vMEpaaHZEdm1WSmJXNlpxRHVSTlRqRXhLM0s2TFRaOFB2eW5DTTlTVjZZ?=
 =?utf-8?B?MXhNNVJ1L09uYzJUejdxTGVyM3R0aEk5M2EwbFZWK1orTTlnNHE0NWtvY0to?=
 =?utf-8?B?UEx2OEQwRk51c1V2Q3hobzNPYTJsaUc4anNxbDJtWHRrUndQYXRGSWUrV2VX?=
 =?utf-8?B?Mmg3ZkJtZ2UwdG5lcWNJa1ZoajVKaHcvb2RIcjJnMEpKUUpuamtNaDlpMmlF?=
 =?utf-8?B?QzF3RGNGM3BZQW1XYThkWU5YVE9YZGc0R3FYaEhydGwrbEVuNjU0T2lKZHJT?=
 =?utf-8?B?ckhrWHY4Rkh4c3U1QlNuTzlNOWFNSGJVa1Nza1cyUno4bGFMMUdjRUxkNnNM?=
 =?utf-8?B?NW42TStYSVZCMm53MkdjVTl2bVRFaDAvRjRQVEQ3SDhybU03OWptU3phSlBB?=
 =?utf-8?B?bENwV1p3RUtWbTE1NjhOODdHK0pCN0dlNG5TTlVrU2Qzd05GV2pZQzdoS3l1?=
 =?utf-8?B?aVFHNlFZN09oUmF1dXo5ZHVtdUNJS2gxY2lNN2VmSStxYnhCSUZEQTViV1N1?=
 =?utf-8?B?UFlQOURkc3V3Z2NyL1NTYWdISStVY3V2dDJZbGIvaEUra3QrRHQxcVo5Q0RB?=
 =?utf-8?B?YjlWd1FkV3NjWkdST1dIOTVNWkMvcjV0eW9xQmthdG9nRUlYYkR5NWN4VUZp?=
 =?utf-8?B?ek5XOFZ1Zi9mZG1KOGk5QXNDd0dKMWZNVUlTRzlHbVlBdm0wMXMzWG8wbzNX?=
 =?utf-8?B?QzgxL0ZaaGN4WXgrZ0d0ODNTTnhmSGcyTm52K3lZVGxvM3luZi9YcU9hYk5X?=
 =?utf-8?B?OUEyRk1oTkNFMTJZYVhWb1k5OTZNZ29CeGhOS1FhbVkveWoyQXp5RFpiaWFY?=
 =?utf-8?B?YnRCc1Z4TXZqc3ROSC9RaEJEcDhRek1DNVVJSHJ6bnVLKy9RZ3J4TFFuRXNw?=
 =?utf-8?B?S1ZRU0JQZE1OTk5kT1hqamxDRzVOZlFMakcyU0prQ0NCbjEwSk5pT2txeXpU?=
 =?utf-8?B?ZUZlcUxLMWRkKzVsU3FMZHRqVnVYNVRGK2MwMExQL2dpekE5K3puaDlSUm9K?=
 =?utf-8?B?ZEhwcFJSUExvbkNpUWFtbW1CUUMraXI0d1pzZWtBNDdlOFpLUHcrcGhsZWtv?=
 =?utf-8?B?NlVobGI5dDF2djN1b1IvMG9SYmxHTElvYnVOdVk3T2ZDMTlVSm9ianFENWJ0?=
 =?utf-8?B?amU4WEtSL3FWUnI4YUt3OGNNWjc2bFozYVprbnNIMHkyNnIrY0JMS285Q3V5?=
 =?utf-8?B?aXo4ZGJRMlRNQ3VSYU85TWM4SFF3V2VyUTcxcTJnbnJyNlJ3UE5JYTUweDBt?=
 =?utf-8?B?QnlHekZ6SWtJTUE5OGhjNi9mOWhmblZndjI1SmhUZmxpcGhDVktiaGlYY1U3?=
 =?utf-8?B?YkxsYUNpdDJvWHFXU2NZalV4TDZDbnM3VUVOdVBINTZmVWkyanVha3JzS3Vz?=
 =?utf-8?B?eVYwemlsZm12L0ovV29Fa2s3UkdtOEsyVnNjd0NncnJPVS9Gbm05Vyt4KzFI?=
 =?utf-8?B?N2lZdHl6VlFKUGdGd3JzTSswTkpHSUduWC9vc3BJVW9ONnZEMEtJNS9GM3I3?=
 =?utf-8?B?dUdhWkpPOGlzVVRvWVhRczVKd2hRbkpvbVZ4MmF3dThKTTBsaWpIemE4S0F3?=
 =?utf-8?B?bTRsbnVWMlJYbUxta1FXRzVEQnZiUTR5SFhMTmtOQkdIV0MrQm1yWWo1V3Vj?=
 =?utf-8?B?Mlo3RThra1BEeHJzc0xkaUhzc3BHa095bGw5THQ5V28yRnFIWjlZWDBwWnVo?=
 =?utf-8?B?MktHQXQxTWUwa3FTNkZwRFdLYWxSUHdLbkVSVFV6VlBHZTIrdldOSEtzTWRI?=
 =?utf-8?B?L0FDd011MEtSK2ZlcGJEVUhid2tsRjBVaDhNWVo2RWowVlBFbWxkR0dicWJ5?=
 =?utf-8?B?MURaYVB2VS9EbWhVKzA3V0FHOVV0MzRtdDlPU1BJQ1Nka0d6M29lWDltb2Nn?=
 =?utf-8?B?VHo5dVpuRTJTZlJiaGR3TzZ0WmJNUGhIeFVQUEVROEdBWkZBdHprbHB5Y0di?=
 =?utf-8?B?cWZTUzFtQ0gyTXFSb1dwNXFvVVFoREdENm1RWG0xb2ZmZFNHVkZFUE50cjNK?=
 =?utf-8?B?bllpNWEveTBqcmV2azJFamNUQTVESWVMSnpmZlhsdEdYb1hwOVNZWlI4M0h3?=
 =?utf-8?Q?oR1rRnt8WEX7ddXzyRzLOn0=3D?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 938f4f3a-f4b1-4121-72f1-08d9a3ed1196
X-MS-Exchange-CrossTenant-AuthSource: CO1PR11MB4820.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Nov 2021 01:54:45.2890
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: j3Jh1BornAHvLfcEGSzOil5/EDZUGq567DvMmF4M2Ih/xr3AL3FPnYQL4uwCAJ+L8gVE6Zk6BRxRg5iFLzka3w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR11MB5912
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Kees,

On 11/10/21 1:21 AM, Kees Cook wrote:
>>     555600            -0.1%     555305        netperf.Throughput_total_tps
>>      34725            -0.1%      34706        netperf.Throughput_tps
>>
>>
>> Fengwei also helped review these results and commented:
>> I suppose these three CPUs have different cache policy. It also could be
>> related with netperf throughput testing.
> Does moving the syscall implementation somewhere else change things?
> That's a _huge_ performance change for something that isn't even called.
> What's going on here?

We just tried to do trick change to make sure the new function doesn't
make other kernel function address changed. But didn't try to move around
the new function even it's not called. We could try to move it around to
see the impact to netperf throughput.


We tried the original patch (without change to make sure no kernel function
address patch) on other box. As report, the regression are different on
different CPUs like:
       -11.6% vs -5.7% vs 0.1%

So my guess is that the different CPUs on these test box have different
cache policies which cause different performance impact when kernel
functions/data address are different. 

Yes. This is strange. We don't know exact reason. Need deeper investigation
on this.


Regards
Yin, Fengwei

> 
> -Kees
>