Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <bc18bc42-3df8-30c9-10ff-8a4ba3d3d9f6@amd.com>
Date:   Fri, 16 Jun 2023 12:32:40 +0530
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Thunderbird/102.10.0
From:   Ravi Bangoria <ravi.bangoria@amd.com>
Subject: Re: [PATCH v3 1/3] perf test: Add metric value validation test
To:     Weilin Wang <weilin.wang@intel.com>
Cc:     Kan Liang <kan.liang@linux.intel.com>,
        Samantha Alt <samantha.alt@intel.com>,
        Perry Taylor <perry.taylor@intel.com>,
        Caleb Biggers <caleb.biggers@intel.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Ingo Molnar <mingo@redhat.com>,
        Arnaldo Carvalho de Melo <acme@kernel.org>,
        Jiri Olsa <jolsa@kernel.org>,
        Namhyung Kim <namhyung@kernel.org>,
        Adrian Hunter <adrian.hunter@intel.com>,
        Ian Rogers <irogers@google.com>,
        linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org,
        Ravi Bangoria <ravi.bangoria@amd.com>
References: <CAP-5=fVixzBLLiWnim8x6qcDOKH7uVfEWojTCP_H531Fhy7j-g@mail.gmail.com>
 <20230614203824.2895333-1-weilin.wang@intel.com>
 <20230614203824.2895333-2-weilin.wang@intel.com>
Content-Language: en-US
In-Reply-To: <20230614203824.2895333-2-weilin.wang@intel.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?MENWTTFZZmhOZjFVUStxQ0dwNWtmRjVaU1VUcFZPeGVLZUJ5L3lKbHBhZTdP?=
 =?utf-8?B?bWNkQnBYaWJBajh0R1JZWWRIY0REc1hIaU5aOHNRUksxS2VwOGw4T1lYamp4?=
 =?utf-8?B?MGdodmZ2Y1FkQ1AxdWNNeFJCQlpRcFg3SjgzUWJwby9VTnRrVVVCRjA5MzZh?=
 =?utf-8?B?dXpVMW5ISkxUUkwvcXRORnh6RW5ET1plRXZBYUNoOTRLOTM2YTlGTEU1Zkpi?=
 =?utf-8?B?ZU1SbTh5YUxxcHFGZEU3SUZGQVlsVDBKR0xQMW5DTnpVelNsUkNUd05nR1lG?=
 =?utf-8?B?QU1tc0swSW9Jb3pzcEE3RTJlUlZKWG10eG15N2hocXhOaGZBQmdRVnNuV0RR?=
 =?utf-8?B?eWlZNXhOWW81YUhJUGhieVd6WC9JdU40bW5WVHJYc25Fc0dHMzRpU3Fkb09Z?=
 =?utf-8?B?WDY5SktKR0tiZnd0emYrdDJHTWNscE5LUkNIcGt6ZGFtUHg3N0ExdUk1UlYy?=
 =?utf-8?B?TE1EUVNuMVdJUXREWmRWNVNQenc5VUNOVTA1TlVJSmQybERZY2ZtMFJoQzVw?=
 =?utf-8?B?a3NaWTJudjhSckE0RzdoMjh0dkI2aVczSkZMYlhqTTJwNStXakF6Q3NQUmpq?=
 =?utf-8?B?K0NyZFpzamQ3OWtwTmJEd3ltbmExVUU0STJMOUlSSUcxRC84ZCsyRkZldlN6?=
 =?utf-8?B?ZEhYMy9iTUpyR21EQ3pVZEdzQjArQWdwL3hsNi93NUpZdW1HQ2UrSGlNemQ2?=
 =?utf-8?B?SForMU5TejVhUWJ0WXhzVnlHVmRNMjhQUjQwOEk3QnhZOTZPNFZqeUZza1Yw?=
 =?utf-8?B?Mi9rWjY3QmFtc1VWa1hCbyttbEtPeEdYUGZpNitES0Q4c1RHandiWGpVOGJO?=
 =?utf-8?B?V29kREJjSjZLRHNLdlJnRHlYWUd6cUcxTGxGbSs3OXl1cmRwb09PSVRyWjBO?=
 =?utf-8?B?SGRidGlnVW5yUGRpU2ZJUWR3QUtwZDE3YUQ3ZzlMbnVyM0VKWTNMVFB0TE9T?=
 =?utf-8?B?OWpHMVZuUTZVOGVLVGpyRjVTZHdkblpXLzh6VXJXbk8zSEROcTJ2bjJUWEs5?=
 =?utf-8?B?Slpwc1JnNUJ6OWErSVRzMEZPUlI2RzB5c1VKVnQxTDRFZ3JlcE83a2RTdTNR?=
 =?utf-8?B?cUR1WGthSnRFV1JqbUczNEx0cXQxL0huY0hHZFdreFQ2eXJEdmhkYXhZWDY0?=
 =?utf-8?B?MzZDYWJhL1p4Lys3cGd4YWd2c0diOG5NRnFCeXF1Q055UVRVSmJXM2k3a3lO?=
 =?utf-8?B?Uk5VUkpaQlZIQ1A3OWRQZ2FZSlBEeGNOSnE0ZkdqV2NjbjRIVXVUTis4cmpL?=
 =?utf-8?B?ZlZSUzEySzY2NFVyT2dDajkrckpqMjlPbTdKRFRaS0lkU2IwRjZTV25TeVVB?=
 =?utf-8?B?TTkrbU1JSStFdzV2VFo3U3dFSEZLZGk4Q2YvWXBvcEUybW5pdHZkdkJ1d0Nr?=
 =?utf-8?B?RkNDU216a1VRRVZjVVA1Nk5TVlRGTVJqdjczb2F6d2hvVHRERTF1dDlYa3Fp?=
 =?utf-8?B?TmUyWnZHMDJoZ2ZBNVNoNUZFN3l0NzcxRStoVUs2Qm1Mc0ZXMWthVTN4TFVU?=
 =?utf-8?B?NU9yK3FSeklVc1VpRk5lSnh4VEpldWpQQ0tvZ09NQzJ5Z2FGZVpPQjNkOVNO?=
 =?utf-8?B?MEtsbzlvYTA5eEhaemphVTZSOUg1UXMxaWU0QTlBNmEvcnhXUDIzRitiMGF0?=
 =?utf-8?B?dDd0VFVRMkQ4dGV3TmNqRXNYcU03czlNWEpWa2JnVWNuczl1R2xZczY5K3lO?=
 =?utf-8?B?RXMvZWlHRWlLR0Z3NVUyT2J4VDJMM2dBVFVtc1Fwa2N1R1EzNU5seCtYb0dD?=
 =?utf-8?B?Wlpkalc3Q0VPUGQ1Q1hyb21RRDk2cXV2cFJQTUkyOUNNcG1LU2szdkhOays3?=
 =?utf-8?B?d0ltN0phcTJzaEhhWWhjamtRMDh0eHh4bEpqRmViQkltWUJGQWRUTFRPV25N?=
 =?utf-8?B?U0FycVFKaU1nK3lOZWVyaDEvcDBlT1FDbUVGekREb2pubzd4aXRaT0NEc0ZP?=
 =?utf-8?B?WWNCemNGS1VyTUxTeTZFVG5IVXlCZFpDRVJqUWpWcXpYaDdQdC9Za2lDQXlP?=
 =?utf-8?B?bVBSQjJnVW1UcHpDV3crbGYwaWpEdFlYVm44MUtLbWM0NFZCMmJtb0NOdjIy?=
 =?utf-8?B?OE5aa0w3a3RwKzMvYzFXbGZrcXVUaTlycG1mTHpTamN3SGJmN3FSTElXK3hi?=
 =?utf-8?Q?r90ZJESM8wtp8s/rwYhS0jX8+?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2e2af3c8-0d4f-4d0e-36cf-08db6e37b663
X-MS-Exchange-CrossTenant-AuthSource: PH7PR12MB6588.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jun 2023 07:02:56.9143
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: O21bXYO8hUCq2lroRccGSfZ2/VbxMZ9e+EgKn7POrtGfXqVWE5l6On97jJqM4+XKYkAJZo1rkFis2naNUnSQ/Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB8778
X-Spam-Status: No, score=-1.2 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,NICE_REPLY_A,
        SPF_HELO_PASS,SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=no
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi,

> diff --git a/tools/perf/tests/shell/stat_metrics_values.sh b/tools/perf/tests/shell/stat_metrics_values.sh
> new file mode 100755
> index 000000000000..65a15c65eea7
> --- /dev/null
> +++ b/tools/perf/tests/shell/stat_metrics_values.sh
> @@ -0,0 +1,30 @@
> +#!/bin/bash
> +# perf metrics value validation
> +# SPDX-License-Identifier: GPL-2.0
> +if [ "x$PYTHON" == "x" ]
> +then
> +	if which python3 > /dev/null
> +	then
> +		PYTHON=python3
> +	else
> +		echo Skipping test, python3 not detected please set environment variable PYTHON.
> +		exit 2
> +	fi
> +fi
> +
> +grep -q Intel /proc/cpuinfo || (echo Skipping non-Intel; exit 2)

This check doesn't seem to be working. On my Zen3 AMD machine:

  $ sudo ./perf test -vvv 105
  105: perf metrics value validation                                   :
  --- start ---
  test child forked, pid 518035
  Skipping non-Intel
  Launch python validation script ./tests/shell/lib/perf_metric_validation.py
  Output will be stored in: /tmp/__perf_test.program.tnPoW
  Starting perf collection
  ...

Interestingly, it passes :)

  ...
  Test validation finished. Final report:
  [
      {
          "Workload": "perf bench futex hash -r 2 -s",
          "Report": {
              "Metric Validation Statistics": {
                  "Total Rule Count": 2,
                  "Passed Rule Count": 2
              },
              "Tests in Category": {
                  "PositiveValueTest": {
                      "Total Tests": 12,
                      "Passed Tests": 12,
                      "Failed Tests": []
                  },
                  "RelationshipTest": {
                      "Total Tests": 0,
                      "Passed Tests": 0,
                      "Failed Tests": []
                  },
                  "SingleMetricTest": {
                      "Total Tests": 2,
                      "Passed Tests": 2,
                      "Failed Tests": []
                  }
              },
              "Errors": []
          }
      }
  ]
  test child finished with 0
  ---- end ----
  perf metrics value validation: Ok

I haven't yet investigated whether it's genuine or false positive.

Thanks,
Ravi