Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 16 Jan 2023 09:43:38 +0800
From:   Chen Yu <yu.c.chen@intel.com>
To:     Daniel Bristot de Oliveira <bristot@kernel.org>
CC:     <linux-kernel@vger.kernel.org>, Ingo Molnar <mingo@redhat.com>,
        "Peter Zijlstra" <peterz@infradead.org>,
        Juri Lelli <juri.lelli@redhat.com>,
        "Vincent Guittot" <vincent.guittot@linaro.org>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
        "Daniel Bristot de Oliveira" <bristot@redhat.com>,
        Valentin Schneider <vschneid@redhat.com>,
        "Joe Mario" <jmario@redhat.com>
Subject: Re: [PATCH] sched/idle: Make idle poll dynamic per-cpu
Message-ID: <Y8Sryq96epb17Dkj@chenyu5-mobl1>
References: <20230112162426.217522-1-bristot@kernel.org>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20230112162426.217522-1-bristot@kernel.org>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?aR+jFr4rLJ9/HQDJ5x3kXyNn15Oc3FL8OuwuZPzTkxfg8J/kLuzQGGiW4F4v?=
 =?us-ascii?Q?hAOcygtdUaQ4dNk/QXVJ4UuengITcqdvcGa0hC1JOO7HsEDxDJZjfwRJoV6I?=
 =?us-ascii?Q?rKMqdwGDCpKNP7jmKSud4H6Gyit56JTdsfmgqUg3Ktuxhu/38/e/ndwM0YYt?=
 =?us-ascii?Q?Ro4BdnodYjbaw86PtgPybjy9t/ctg84aMLch4sloFpJh9cfbC+KlLN4Q57uR?=
 =?us-ascii?Q?Bcww8Pz3UeynANq0Hb1vaX2B6wd5z11zBPwzzR0DvtUyukdfrbVwk8gsvZEN?=
 =?us-ascii?Q?Pwhdzbq8ZNxJ1eGNhcKDv1cpbW51HpI90DAoArdYlLob5BZ0ULEPxIs18GAo?=
 =?us-ascii?Q?IX5Lvk4Rk5UkoUatzRtUOZDIZ57+9vAraWav7rwOScIlqiiWg0nYqLgTt3Vf?=
 =?us-ascii?Q?rUNLYoIUPd2vIYZBWR6/S0QJTzN7s/A7C0DkiPA9OGLHrunmRby6dAy7bF85?=
 =?us-ascii?Q?UQwiWppVPImkS2QoXgP9d/WSvd7XL+wnAqU+mwcMURJZgzpzmkl2WlCvRqep?=
 =?us-ascii?Q?yY+fH9ygFVved8srSFp2zcOH497bGUPjjYIfoj8Qq8pVxVbxOdq96M4hpzf1?=
 =?us-ascii?Q?lneJSD9A8Qy9HBSMepsjrEPtwMKwsBohLFQN9SFu/n1egwpJK0OV6TGG7kMa?=
 =?us-ascii?Q?MzBbSfN1GprJ4N4F+4QsyzhdVW0XOEIihbrC7Dq5n40dWHm9vYeGbzXo/jmU?=
 =?us-ascii?Q?kSZOlL+OGVzHk/maT+7c9ltLGqPS79FGa2YkpgJ9kWHZd9ehvYSE12rgh+nl?=
 =?us-ascii?Q?QOmHgzXST7Tbjlj8A9prkuwtx8cmdPvnlMQIAYqP0stg4+XMey71blaB1g47?=
 =?us-ascii?Q?9+1GDwJ02xDAMt02wpvDTi1aQB/7cBLPzRFtxYulDwGdojxxszOyJ/tosUVc?=
 =?us-ascii?Q?zZ3JEbh7ZMXqWRznD4Rn/Egxuhx3G2qiMJalynvc4stsF2L/I5rrXfI4mVcl?=
 =?us-ascii?Q?WQXSehD5aDeNmQat/K4DFsizyi+8FElnC01ikIjiPqF12Tu2S17YVOQHjoGy?=
 =?us-ascii?Q?10azFd7rBOMTR0hg08zThlNiAnhXQk9CXEtJsTVae/Lm0BkIyR6h24pZtG23?=
 =?us-ascii?Q?YcXnDwIVGnCcEVrhY67HKK2Yt0TbZsKbo2w81sey6pFfmkQXPWYcGiQu43FA?=
 =?us-ascii?Q?qpU13w80CYJ8EX2ja5ec/92m9gF6vccBg3yQG0AV0jGSfzJeQefWypQrVjny?=
 =?us-ascii?Q?UySJS7IEdcZb2UXPOj/4jb5/M/50YLBUWxnsWPnrm4qgqN7tOXfnD1H8Bv7x?=
 =?us-ascii?Q?U+R2f2wxtHkdhKYKaC5UjChmOutviQdqW5OmeUfPBY2usty17TINSBLbhsuD?=
 =?us-ascii?Q?5VTKZYetjwr5zTVara3ailDqYVDZPN0UEHn3ejJA2jY5mSGjtZRbEZtfZ8Ey?=
 =?us-ascii?Q?2Szr5rlxunM6ekUEMscS95aOlgkVKN3juGQ/I3ahA8Rw25fem7JckJh7UNia?=
 =?us-ascii?Q?C/2uH/uXxjCECK6RNiekYpk5E8UIoB3OwdyDXABj0QhK1OOctGSJQ7CjgSrb?=
 =?us-ascii?Q?WypvoZiRULGULouUEw41ui2R8NpULGoOwUSGMU2TYLkhxhcXuVSYuMrbTgSH?=
 =?us-ascii?Q?br7KBDs1DMEfwgNNlhkzi0jlhVVd+zpVZXcbNpl5?=
X-MS-Exchange-CrossTenant-Network-Message-Id: b8c2b141-0124-4c4d-563b-08daf7632187
X-MS-Exchange-CrossTenant-AuthSource: MN0PR11MB6206.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jan 2023 01:43:56.5342
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 8MADrAFPzAzjd7LVJ0qE0L5YmGxttXGu0wBY997pIAvo94W/AGynkNUzX1vshwobJKG35WblsUoHrAlol9M/8Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR11MB5053
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Daniel,
On 2023-01-12 at 17:24:26 +0100, Daniel Bristot de Oliveira wrote:
> idle=poll is frequently used on ultra-low-latency systems. Examples of
> such systems are high-performance trading and 5G NVRAM. The performance
> gain is given by avoiding the idle driver machinery and by keeping the
> CPU is always in an active state - avoiding (odd) hardware heuristics that
> are out of the control of the OS.
> 
> Currently, idle=poll is an all-or-nothing static option defined at
> boot time. The motivation for creating this option dynamic and per-cpu
> are two:
> 
>   1) Reduce the power usage/heat by allowing only selected CPUs to
>      do idle polling;
>   2) Allow multi-tenant systems (e.g., Kubernetes) to enable idle
>      polling only when ultra-low-latency applications are present
>      on specific CPUs.
> 
> Joe Mario did some experiments with this option enabled, and the results
> were significant. For example, by using dynamic idle polling on
> selected CPUs, cyclictest performance is optimal (like when using
> idle=poll), but cpu power consumption drops from 381 to 233 watts.
> 
> Also, limiting idle=poll to the set of CPUs that benefits from
> it allows other CPUs to benefit from frequency boosts. Joe also
> shows that the results can be in the order of 80nsec round trip
> improvement when system-wide idle=poll was not used.
> 
> The user can enable idle polling with this command:
>   # echo 1 > /sys/devices/system/cpu/cpu{CPU_ID}/idle_poll
> 
> And disable it via:
>   # echo 0 > /sys/devices/system/cpu/cpu{CPU_ID}/idle_poll
>
Maybe I understood it incorrectly, is above command intended to
put specific CPU only in poll mode? Can the c-state sysfs
do this?

grep . /sys/devices/system/cpu/cpu0/cpuidle/state*/name 
/sys/devices/system/cpu/cpu0/cpuidle/state0/name:POLL
/sys/devices/system/cpu/cpu0/cpuidle/state1/name:C1
/sys/devices/system/cpu/cpu0/cpuidle/state2/name:C1E
/sys/devices/system/cpu/cpu0/cpuidle/state3/name:C6

grep . /sys/devices/system/cpu/cpu0/cpuidle/state*/disable
/sys/devices/system/cpu/cpu0/cpuidle/state0/disable:0
/sys/devices/system/cpu/cpu0/cpuidle/state1/disable:1
/sys/devices/system/cpu/cpu0/cpuidle/state2/disable:1
/sys/devices/system/cpu/cpu0/cpuidle/state3/disable:1
 
thanks,
Chenyu