Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 5 Dec 2022 17:18:24 +0800
From:   Huang Rui <ray.huang@amd.com>
To:     "Meng, Li (Jassmine)" <Li.Meng@amd.com>
Cc:     Shuah Khan <skhan@linuxfoundation.org>,
        "linux-kselftest@vger.kernel.org" <linux-kselftest@vger.kernel.org>,
        "Rafael J . Wysocki" <rafael.j.wysocki@intel.com>,
        "Fontenot, Nathan" <Nathan.Fontenot@amd.com>,
        "Sharma, Deepak" <Deepak.Sharma@amd.com>,
        "Deucher, Alexander" <Alexander.Deucher@amd.com>,
        "Limonciello, Mario" <Mario.Limonciello@amd.com>,
        "Huang, Shimmer" <Shimmer.Huang@amd.com>,
        "Yuan, Perry" <Perry.Yuan@amd.com>,
        "Du, Xiaojian" <Xiaojian.Du@amd.com>,
        Viresh Kumar <viresh.kumar@linaro.org>,
        Borislav Petkov <bp@alien8.de>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [RESEND PATCH V2 2/2] Documentation: amd-pstate: Add speedometer
 test introduction
Message-ID: <Y423YIb8bxIe5xY8@amd.com>
References: <20221107011127.1818705-1-li.meng@amd.com>
 <20221107011127.1818705-3-li.meng@amd.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20221107011127.1818705-3-li.meng@amd.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?h5tcHRaBqmMBoeg31cFft7pl8aOyoiIuW0pb3Jpr/WLlOx2kPfUdCv5fOoGf?=
 =?us-ascii?Q?NmP4vAgouRNHjwShsk//9HhZJGCqWcCV21Vmsy6WktylYyfVPMHjmcQ13pG0?=
 =?us-ascii?Q?weeH/EVOF0GxELM1zFYCkrvu5xuZoTvsFWNnPsRc50W+gLzrVCRlJVJTxC+L?=
 =?us-ascii?Q?hl+dQvV3J+23pB7ddU6JkVHfF2H0DDHIPA7eTCh7TD4Mi51uzS/jA7IJ2rxZ?=
 =?us-ascii?Q?HHlWcpsn7lTwEFI25b0brlxs4CHTzRTpBDeMh/XHmaQN8F8TJCcLeE9sJTSs?=
 =?us-ascii?Q?1pm35uoYZAE2FHUExOmPP83wmLsNpXa7vTTSvhVfwVizyOEoJAKOP8lCWvoR?=
 =?us-ascii?Q?APKwKxqdVVL6i2+fhMp99ZYDmwj7QALyWGjArxP2XhtG5k15dpsJ6EdLviqM?=
 =?us-ascii?Q?K3RmCKTccLR0Cr0cHF5e5yMA9a2s+/7XDZD6A2O6pVdoy+6+yg6K6E2EV4M5?=
 =?us-ascii?Q?0HBCRFCzoJWip+1OvAN6xFI3fwHRLkN+rdKWCd52qIu3mQN6HXZXuSrAYLuI?=
 =?us-ascii?Q?wCU1N7TP1B7FPipleP8Es9Xs0AJPVYhMc3Bo+gqZOcmzBcIQP0tdhR/OijzS?=
 =?us-ascii?Q?zsUdjQ5XU/6fbLxeyYwF3FhWV4OMU6nO+l0z4YyqEO7pQ6/xHJfPmMSG6vsZ?=
 =?us-ascii?Q?1Q2sjQXleSLP/h4lFEu23P1uweuFqzhqjwo5TVhKg5Zg1IvMBgJ7A0EdB0cJ?=
 =?us-ascii?Q?D8BhfrDteYzEwonBI/WNfzR5VlSzapKLJmJWmFYEGXv+CMaRVFoGLl/NKPyX?=
 =?us-ascii?Q?uYhSNyt48HzQMD+IJ8j5SMwMGdtWKdnC0pnDL2RW3s8r8UHaMol4SdDnfKhD?=
 =?us-ascii?Q?65cAWsmQBUmH15Qv7n/h102hjNzL77dxP8kf5xRQh0oPmKp40MrTYUqSr0hg?=
 =?us-ascii?Q?+Mv1ChDBqP1v+mgbPqcJAhJQvZqPcMGKVrW8Xh7zrmqQwLuWSnnW2Sha8GPv?=
 =?us-ascii?Q?Ye4zlnWepiOHjsNEv0CujI2RZHSQvHTnK22mpsfzOVypQj7Vz4ZA4s3f3yCg?=
 =?us-ascii?Q?ke8yTlndB7fhPhBRUz5HIr0p7Irkdwqyn423fvhg+Mae9k7PT+uNB1cybAyL?=
 =?us-ascii?Q?Joz7bi0YcVbjJ9JH8NddwWI5Du/2/XBz8g45+0tE89TjH8PeqJOJgCSaV3Wa?=
 =?us-ascii?Q?dBwDxPrRikW6HP7wL/p9azQDzXABfUZcNRnxGfSKRP2jiUYh78dn2hNp/lmG?=
 =?us-ascii?Q?EgYztq4d6z1AodIs1Ku7PsRyDnExlP8zxa8meUQUSgGj55cVgqJ+fY7kDhU7?=
 =?us-ascii?Q?bhyrjOibISf++02RmllPh1301yyPtKjBE4CyMqI0fFmIx/gmLj+seN4rm+p1?=
 =?us-ascii?Q?FNfWA7gBsv+v/6xq31zpNohv0uI2TIYBznNJ/EKQAZQ6M+Z5QkgrrN6hGuhJ?=
 =?us-ascii?Q?xsmsy3kFdG0Q68+H6Vr97UWX6sWhKdi9/wYPLq31s9PLD7/w4S/RoSVT+f1G?=
 =?us-ascii?Q?zBftKWu3or39MXW9yzmzbwcwIeVUYhJ+ZTz3pcDtbi9xADEl/ciwXmZnR6TE?=
 =?us-ascii?Q?iOc6JKSeY/YFeyvGQa3/hlDzSkCDPjPCBhKfzKi2zG2iA0NCxoGO8hMAT80M?=
 =?us-ascii?Q?xCaTWou5ojfO/XrjsY5b1T7MS6tKdDBQQkI48BFB?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0ea897e9-9448-41c3-2e36-08dad6a1b7c8
X-MS-Exchange-CrossTenant-AuthSource: DM5PR12MB2504.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Dec 2022 09:18:48.9024
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: nuI5NS4MbP6CIUEOpv+nQwAsyJeGy2IBFJuboqWtwjwnck4mB42Y5KWJTm5Wv38At6Xdlkmrv9R/MZ2wlpRYnQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8351
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Mon, Nov 07, 2022 at 09:11:27AM +0800, Meng, Li (Jassmine) wrote:
> Introduce speedometer test cases design and implementation.
> Monitor cpus changes about performance and power consumption etc.
> 
> Signed-off-by: Meng Li <li.meng@amd.com>
> ---
>  Documentation/admin-guide/pm/amd-pstate.rst | 75 ++++++++++++++++++++-
>  1 file changed, 74 insertions(+), 1 deletion(-)
> 
> diff --git a/Documentation/admin-guide/pm/amd-pstate.rst b/Documentation/admin-guide/pm/amd-pstate.rst
> index 29c50e96eb95..0163d1231aa7 100644
> --- a/Documentation/admin-guide/pm/amd-pstate.rst
> +++ b/Documentation/admin-guide/pm/amd-pstate.rst
> @@ -451,6 +451,23 @@ Unit Tests for amd-pstate
>          The specified governor is ondemand or schedutil.
>          Gitsource can also be tested on the ``acpi-cpufreq`` kernel driver for comparison.
>  
> +    4). Speedometer test
> +
> +        Test and monitor the cpu changes when running speedometer benchmark under the specified governor.
> +        These changes include desire performance, frequency, load, time, energy etc.
> +        The specified governor is ondemand or schedutil.
> +        Speedometer can also be tested on the ``acpi-cpufreq`` kernel driver for comparison.
> +
> +#. Preparations before tests
> +
> +    1). Speedometer
> +
> +     + Python version 3.0.x or higher
> +     + Install chromium-browser
> +     + Install chromium-chromedriver
> +     + Install selenium on the client
> +     + Install selenium-server on the server
> +
>  #. How to execute the tests
>  
>     We use test module in the kselftest frameworks to implement it.
> @@ -487,6 +504,8 @@ Unit Tests for amd-pstate
>          $ sudo ./run.sh -t tbench -m acpi-cpufreq
>          $ sudo ./run.sh -t gitsource
>          $ sudo ./run.sh -t gitsource -m acpi-cpufreq
> +        $ sudo ./run.sh -t speedometer
> +        $ sudo ./run.sh -t speedometer -m acpi-cpufreq

While I run the above scripts: sudo ./run.sh -t speedometer

I found Tbench test result will be reported as well, but it reported as a
runtime error.

Tbench test result for ondemand (loops:3)
--------------------------------------------------
Tbench-ondemand avg des perf: -nan
Tbench-ondemand avg freq: -nan
Tbench-ondemand avg load: -nan
Tbench-ondemand total throughput(MB/s): 0
Tbench-ondemand avg throughput(MB/s): 0
Tbench-ondemand total power consumption(J): 0
Tbench-ondemand avg power consumption(J): 0
Runtime error (func=(main), adr=14): Divide by zero
Tbench-ondemand performance per watt(MB/J):

I think we may have more detailed steps to tell every one how to execute the tests easily.

Thanks,
Ray

>          $ ./run.sh --help
>          ./run.sh: illegal option -- -
>          Usage: ./run.sh [OPTION...]
> @@ -495,7 +514,8 @@ Unit Tests for amd-pstate
>                  [-c <all: All testing,
>                       basic: Basic testing,
>                       tbench: Tbench testing,
> -                     gitsource: Gitsource testing.>]
> +                     gitsource: Gitsource testing,
> +                     speedometer: Speedometer testing.>]
>                  [-t <tbench time limit>]
>                  [-p <tbench process number>]
>                  [-l <loop times for tbench>]
> @@ -621,6 +641,59 @@ Unit Tests for amd-pstate
>           + acpi-cpufreq-schedutil VS amd-pstate-schedutil  | Comprison(%) |          |          |          | 2.1115      | 4.2873  | -4.1110              |
>           +-------------------------------------------------+--------------+----------+----------+----------+-------------+---------+----------------------+
>  
> +        + speedometer
> +
> +         When you finish test, you will get selftest.speedometer.csv and png images.
> +         The selftest.speedometer.csv file contains the raw data and the drop of the comparative test.
> +         The png images shows the goal, time, energy and performan per watt of each test.
> +         Open selftest.speedometer.csv :
> +
> +         +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
> +         + Governor                                        | Round        | Des-perf | Freq     | Load     | Goal        | Time        | Energy  | Performance Per Watt |
> +         +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
> +         + Unit                                            |              |          | GHz      |          | Runs/Minute | s           | J       | Runs/w               |
> +         +=================================================+==============+==========+==========+==========+=============+=============+=========+======================+
> +         + acpi-cpufreq-ondemand                           | 1            |          |          |          | 209         | 51          | 731.84  | 0.2427               |
> +         +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
> +         + acpi-cpufreq-ondemand                           | 2            |          |          |          | 205         | 51          | 759.03  | 0.2295               |
> +         +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
> +         + acpi-cpufreq-ondemand                           | 3            |          |          |          | 206         | 51          | 755.15  | 0.2318               |
> +         +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
> +         + acpi-cpufreq-ondemand                           | Average      |          |          |          | 206.667     | 51          | 748.673 | 0.2346               |
> +         +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
> +         + acpi-cpufreq-schedutil                          | 1            |          |          |          | 206         | 56          | 775.08  | 0.248                |
> +         +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
> +         + acpi-cpufreq-schedutil                          | 2            |          |          |          | 204         | 51          | 762.06  | 0.2275               |
> +         +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
> +         + acpi-cpufreq-schedutil                          | 3            |          |          |          | 207         | 56          | 776.35  | 0.2488               |
> +         +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
> +         + acpi-cpufreq-schedutil                          | Average      |          |          |          | 205.667     | 54.3333     | 771.163 | 0.2415               |
> +         +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
> +         + amd-pstate-ondemand                             | 1            | 24.7974  | 1.73142  | 7.71728  | 195         | 64          | 756.6   | 0.2749               |
> +         +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
> +         + amd-pstate-ondemand                             | 2            | 26.1653  | 1.91492  | 8.5525   | 195         | 51          | 705.67  | 0.2348               |
> +         +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
> +         + amd-pstate-ondemand                             | 3            | 24.1789  | 1.69516  | 7.41152  | 196         | 65          | 758.98  | 0.2797               |
> +         +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
> +         + amd-pstate-ondemand                             | Average      | 25.0472  | 1.7805   | 7.89377  | 195.333     | 60          | 740.417 | 0.2638               |
> +         +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
> +         + amd-pstate-schedutil                            | 1            | 67.0214  | 2.76691  | 17.1314  | 197         | 51          | 737.52  | 0.227                |
> +         +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
> +         + amd-pstate-schedutil                            | 2            | 64.3032  | 2.75981  | 16.1196  | 198         | 55          | 763.57  | 0.2376               |
> +         +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
> +         + amd-pstate-schedutil                            | 3            | 65.5175  | 2.59423  | 17.0067  | 201         | 51          | 735.96  | 0.2321               |
> +         +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
> +         + amd-pstate-schedutil                            | Average      | 65.614   | 2.70698  | 16.7526  | 198.667     | 52.3333     | 745.683 | 0.2323               |
> +         +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
> +         + acpi-cpufreq-ondemand VS acpi-cpufreq-schedutil | Comprison(%) |          |          |          | -0.4838     | 6.5358      | 3.0039  | 2.9411               |
> +         +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
> +         + amd-pstate-ondemand VS amd-pstate-schedutil     | Comprison(%) | 161.9614 | 52.0348  | 112.2255 | 1.7068      | -12.7778    | 0.7112  | -11.9408             |
> +         +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
> +         + acpi-cpufreq-ondemand VS amd-pstate-ondemand    | Comprison(%) |          |          |          | -5.4841     | 17.647      | -1.1027 | 12.4467              |
> +         +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
> +         + acpi-cpufreq-schedutil VS amd-pstate-schedutil  | Comprison(%) |          |          |          | -3.4035     | -3.6809     | -3.3041 | -3.8095              |
> +         +-------------------------------------------------+--------------+----------+----------+----------+-------------+-------------+---------+----------------------+
> +
>  Reference
>  ===========
>  
> -- 
> 2.34.1
>