Received-SPF: pass (google.com: domain of linux-kernel+bounces-93733-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Message-ID: <bf7b4e37-51cf-50d6-dc8e-626f29b5bdd8@amd.com>
Date: Wed, 6 Mar 2024 15:48:00 +0530
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.13.1
Subject: Re: [RFC PATCH 00/14] Introducing TIF_NOTIFY_IPI flag
Content-Language: en-US
To: Vincent Guittot <vincent.guittot@linaro.org>
Cc: linux-kernel@vger.kernel.org, "Gautham R. Shenoy"
 <gautham.shenoy@amd.com>, Richard Henderson <richard.henderson@linaro.org>,
 Ivan Kokshaysky <ink@jurassic.park.msu.ru>, Matt Turner
 <mattst88@gmail.com>, Russell King <linux@armlinux.org.uk>,
 Guo Ren <guoren@kernel.org>, Michal Simek <monstr@monstr.eu>,
 Dinh Nguyen <dinguyen@kernel.org>, Jonas Bonn <jonas@southpole.se>,
 Stefan Kristiansson <stefan.kristiansson@saunalahti.fi>,
 Stafford Horne <shorne@gmail.com>,
 "James E.J. Bottomley" <James.Bottomley@hansenpartnership.com>,
 Helge Deller <deller@gmx.de>, Michael Ellerman <mpe@ellerman.id.au>,
 Nicholas Piggin <npiggin@gmail.com>,
 Christophe Leroy <christophe.leroy@csgroup.eu>,
 "Aneesh Kumar K.V" <aneesh.kumar@kernel.org>,
 "Naveen N. Rao" <naveen.n.rao@linux.ibm.com>,
 Yoshinori Sato <ysato@users.sourceforge.jp>, Rich Felker <dalias@libc.org>,
 John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de>,
 "David S. Miller" <davem@davemloft.net>, Thomas Gleixner
 <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>,
 Borislav Petkov <bp@alien8.de>, Dave Hansen <dave.hansen@linux.intel.com>,
 "H. Peter Anvin" <hpa@zytor.com>, "Rafael J. Wysocki" <rafael@kernel.org>,
 Daniel Lezcano <daniel.lezcano@linaro.org>,
 Peter Zijlstra <peterz@infradead.org>, Juri Lelli <juri.lelli@redhat.com>,
 Dietmar Eggemann <dietmar.eggemann@arm.com>,
 Steven Rostedt <rostedt@goodmis.org>, Ben Segall <bsegall@google.com>,
 Mel Gorman <mgorman@suse.de>, Daniel Bristot de Oliveira
 <bristot@redhat.com>, Valentin Schneider <vschneid@redhat.com>,
 Al Viro <viro@zeniv.linux.org.uk>, Linus Walleij <linus.walleij@linaro.org>,
 Ard Biesheuvel <ardb@kernel.org>, Andrew Donnellan <ajd@linux.ibm.com>,
 Nicholas Miehlbradt <nicholas@linux.ibm.com>,
 Andrew Morton <akpm@linux-foundation.org>, Arnd Bergmann <arnd@arndb.de>,
 Josh Poimboeuf <jpoimboe@kernel.org>,
 "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>,
 Rick Edgecombe <rick.p.edgecombe@intel.com>,
 Tony Battersby <tonyb@cybernetics.com>, Brian Gerst <brgerst@gmail.com>,
 Tim Chen <tim.c.chen@linux.intel.com>, David Vernet <void@manifault.com>,
 x86@kernel.org, linux-alpha@vger.kernel.org,
 linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org,
 linux-openrisc@vger.kernel.org, linux-parisc@vger.kernel.org,
 linuxppc-dev@lists.ozlabs.org, linux-sh@vger.kernel.org,
 sparclinux@vger.kernel.org, linux-pm@vger.kernel.org,
 Julia Lawall <julia.lawall@inria.fr>
References: <20240220171457.703-1-kprateek.nayak@amd.com>
 <CAKfTPtBqPVQ5bo8HTZ=sPCUTYr48qtH61A8Z1dwCT434O7cSyQ@mail.gmail.com>
From: K Prateek Nayak <kprateek.nayak@amd.com>
In-Reply-To: <CAKfTPtBqPVQ5bo8HTZ=sPCUTYr48qtH61A8Z1dwCT434O7cSyQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Z3FBK1lmZHNCcDlqTFhha3ZjdHVSdzhaLzlqaE9seFJjdHNzeWZVK0RkTGwx?=
 =?utf-8?B?MkdQaFA3NVpXSkN1OGJOeVpwR09wYzNyQi9zRFVMNUtNVjdwTU50YzZrS1Nw?=
 =?utf-8?B?UlYzcytxTDR2QlpJc0hScFM3R0dHcW00c002am0zejF6bFBVWE5rWlZUR3VY?=
 =?utf-8?B?MlE0MHIvZTZ1TisybXR3cnZiV2xsRlcvWS9IN2gyaCtyc296cHMrUEM4c21F?=
 =?utf-8?B?OFpLdHpZVzlpcDlvTmRrUkhkN01ncmg0Nmh1NFQzdGVvUWhtd0NFYjBZS0Fw?=
 =?utf-8?B?SStyVHpMbXlNUjFyZDNrUmtvU3RWRXFSTk1ISlYvSHFqZVZuejBPK2N1anRW?=
 =?utf-8?B?eVA0d3dpaERLY2NyZWVERlppZTFEMElEQTBBSUtFb2dCSzBtV0J5dmgxZ05S?=
 =?utf-8?B?S2ZBaVFPSzhKTkJhRllRSjdzc2s2bDBrSGMyMkFILy9iZkRGeE1VWDRnU2RV?=
 =?utf-8?B?VEkrNm8velBPblFOV25XajFjUGsreXQxYWZ2b1Rza3RtK3E3SzNtSlM3Zks3?=
 =?utf-8?B?SjdFY3M1Z0lEL3pXbkxhOERtazV0TWcxYmhGVVl2aXE3YW81WlcybkhGNk95?=
 =?utf-8?B?YzVKUS8xK05UbHRnNDRxN2I2QzZTVWlmTkJmeEl1dnJZYmpMMmhVSWlQbHJk?=
 =?utf-8?B?dXB4RFdYQlZNRzdiWWFsdThBSWVteTFhYVB5L040RzcwZnNMTWo0YmJiR0FP?=
 =?utf-8?B?UHc2TlhhT3QzVUhHVVR6Rml4bnE2QWpBcjZINjdLeDhFdzA2TjNpVWV3RXJl?=
 =?utf-8?B?eE5jb0p3YXByTkpaL3NaenhhSG1EbDc5Q1JlZXN5S2crMDFUS1lEWmMwdmV0?=
 =?utf-8?B?Tjd0dGlFd3dxc0x4czdJU3NKSTA0Z3NKZmxwSncxeDBuYmhmcFEzRVU5eUZz?=
 =?utf-8?B?bHh3WmZDQ3FJd3RpMnE5TVh2Ri9rcVBpWStSUFQ5Ti93ajdSb2F0WjdEUWpi?=
 =?utf-8?B?M2NmQnRwekRzRXlaY2gwRWdtMEN5eXNIOVgrTDZxMVlSRlJqaERxbW40R3ov?=
 =?utf-8?B?QXFML2lMVVRTN3NoVk1RZFdYSTJDT3pXRlI3eG12djIvS3hXczh0L1JZcWor?=
 =?utf-8?B?Rm9zZ0dhcFlndHVyL3VEWU1VNHhScmZsRUFEL2JJSHZTN0ovREdVWkFKdE1i?=
 =?utf-8?B?WHd2Z29VQkFoVFJuNlVDS28wMXhadTBObkNqK1JUNnRFWDZxcHE4Wm1NU0Jh?=
 =?utf-8?B?aUh6dkdmUXpGR01neVp1SDFvVVQrZVBUbUdkZkxYS25PMVlheE1FekhIVENp?=
 =?utf-8?B?Mzk4RWF4bVA1aFNoQmNydVlQRTFmenRZMTMvbzRYVnRjWDd4QUlaTjlaRGtx?=
 =?utf-8?B?RUVGbG5ESHRYeVhTVkw0VE1GQ3RwUzFSaEYvQ3BCbTBmRVV0S3VwdXArcTdr?=
 =?utf-8?B?TGNxbU55dysvQmZ4ejZ3MEc0ZncvUmsyWnpNNnBFVWpKTDVSclh0WTBKYWdG?=
 =?utf-8?B?cEVITC81bFpyaW5MMkxsajdZRTJtSlExSHVURHhzTDdqLzluRW96NmNZaHVo?=
 =?utf-8?B?RzVDOUZFa3c2dlY4TFVTV3BOaEVpVUVTUDJySmcrTFhadTViZkxzVnZERGdM?=
 =?utf-8?B?ZWNxWElxNWVYd1NDbUpCV2tYUG14SWI0dG1VcGVmQmticDRDblhvZVVRK2RE?=
 =?utf-8?B?cWdqYnByeXVTYlBrTTJTQXRleTI4dnhEaHpUZSthbStzSWU0ckNYK0xYdVpv?=
 =?utf-8?B?dTAwRERodWpGZlY2UzNqWW1LejdpTVhyRFBCQ0FUQjJaUy9seUc5RCtrLzgw?=
 =?utf-8?B?bENaVCtOV285cGRFakkxVXR1NlRrWUVXZFlUbWdQaFZpbXNzTHk3VjNWcGMr?=
 =?utf-8?B?YjMwZmlBYkFSb0NuUmdxSVhFa1BJckJPWWNwR2w4cGREaHBzZFkzVFo5NENH?=
 =?utf-8?B?NDloSTkrR3BlbXpINzhyc0plR2hNOWg0bUVvdllQL1RlMUlSV3VXWlZobC9i?=
 =?utf-8?B?SERwNEI3dmVaaXErREFWL2tXcU9XZjJyYnhHNE9zN2FqbVlsamRmdUdzQktL?=
 =?utf-8?B?NUlsbll3bUNkY1JvSU4zcTBJRnVqRGV0RjZSTDJkQUFBWFZYM3I5TVVTaTBo?=
 =?utf-8?B?ZzVYeGtLTmVoZmUvcHNMdEJxQXA1MHVDcDlnUi9Qamg0N2NhYm5tbXBFcTZL?=
 =?utf-8?Q?GSYAXrRfmKXHExTDyYI5tZPko?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 404a825b-adf7-4eaf-2334-08dc3dc6c0e3
X-MS-Exchange-CrossTenant-AuthSource: MW2PR12MB2379.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Mar 2024 10:18:23.1766
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: c+MbiJnl6k0arshKYqL5YUo5amx69vThIYAgOi+z6rR54wa3qW1yFPzYt4Jch8A+HpehTn+GiXM8swO87ahncQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ1PR12MB6146

Hello Vincent,

Thank you for taking a look at the series.

On 3/6/2024 3:29 PM, Vincent Guittot wrote:
> Hi Prateek,
> 
> Adding Julia who could be interested in this patchset. Your patchset
> should trigger idle load balance instead of newly idle load balance
> now when the polling is used. This was one reason for not migrating
> task in idle CPU

Thank you.

> 
> On Tue, 20 Feb 2024 at 18:15, K Prateek Nayak <kprateek.nayak@amd.com> wrote:
>>
>> Hello everyone,
>>
>> [..snip..]
>>
>>
>> Skipping newidle_balance()
>> ==========================
>>
>> In an earlier attempt to solve the challenge of the long IRQ disabled
>> section, newidle_balance() was skipped when a CPU waking up from idle
>> was found to have no runnable tasks, and was transitioning back to
>> idle [2]. Tim [3] and David [4] had pointed out that newidle_balance()
>> may be viable for CPUs that are idling with tick enabled, where the
>> newidle_balance() has the opportunity to pull tasks onto the idle CPU.
>>
>> Vincent [5] pointed out a case where the idle load kick will fail to
>> run on an idle CPU since the IPI handler launching the ILB will check
>> for need_resched(). In such cases, the idle CPU relies on
>> newidle_balance() to pull tasks towards itself.
> 
> Calling newidle_balance() instead of the normal idle load balance
> prevents the CPU to pull tasks from other groups

Thank you for the correction.

> 
>>
>> Using an alternate flag instead of NEED_RESCHED to indicate a pending
>> IPI was suggested as the correct approach to solve this problem on the
>> same thread.
>>
>>
>> Proposed solution: TIF_NOTIFY_IPI
>> =================================
>>
>> Instead of reusing TIF_NEED_RESCHED bit to pull an TIF_POLLING CPU out
>> of idle, TIF_NOTIFY_IPI is a newly introduced flag that
>> call_function_single_prep_ipi() sets on a target TIF_POLLING CPU to
>> indicate a pending IPI, which the idle CPU promises to process soon.
>>
>> On architectures that do not support the TIF_NOTIFY_IPI flag (this
>> series only adds support for x86 and ARM processors for now),
> 
> I'm surprised that you are mentioning ARM processors because they
> don't use TIF_POLLING.

Yup I just realised that after Linus Walleij pointed it out on the
thread.

> 
>> call_function_single_prep_ipi() will fallback to setting
>> TIF_NEED_RESCHED bit to pull the TIF_POLLING CPU out of idle.
>>
>> Since the pending IPI handlers are processed before the call to
>> schedule_idle() in do_idle(), schedule_idle() will only be called if the
>> IPI handler have woken / migrated a new task on the idle CPU and has set
>> TIF_NEED_RESCHED bit to indicate the same. This avoids running into the
>> long IRQ disabled section in schedule_idle() unnecessarily, and any
>> need_resched() check within a call function will accurately notify if a
>> task is waiting for CPU time on the CPU handling the IPI.
>>
>> Following is the crude visualization of how the situation changes with
>> the newly introduced TIF_NOTIFY_IPI flag:
>> --
>> CPU0                                                    CPU1
>> ====                                                    ====
>>                                                         do_idle() {
>>                                                                 __current_set_polling();
>>                                                                 ...
>>                                                                 monitor(addr);
>>                                                                 if (!need_resched_or_ipi())
>>                                                                         mwait() {
>>                                                                         /* Waiting */
>> smp_call_function_single(CPU1, func, wait = 1) {                                ...
>>         ...                                                                     ...
>>         set_nr_if_polling(CPU1) {                                               ...
>>                 /* Realizes CPU1 is polling */                                  ...
>>                 try_cmpxchg(addr,                                               ...
>>                             &val,                                               ...
>>                             val | _TIF_NOTIFY_IPI);                             ...
>>         } /* Does not send an IPI */                                            ...
>>         ...                                                             } /* mwait exit due to write at addr */
>>         csd_lock_wait() {                                       ...
>>         /* Waiting */                                           preempt_fold_need_resched(); /* fold if NEED_RESCHED */
>>                 ...                                             __current_clr_polling();
>>                 ...                                             flush_smp_call_function_queue() {
>>                 ...                                                     func(); /* Will set NEED_RESCHED if sched_ttwu_pending() */
>>         } /* End of wait */                                     }
>> }                                                               if (need_resched()) {
>>                                                                         schedule_idle();
>> smp_call_function_single(CPU1, func, wait = 1) {                }
>>         ...                                                     ... /* IRQs remain enabled */
>>         arch_send_call_function_single_ipi(CPU1); ----------->  /* Processes the IPI */
>> --
>>
>> Results
>> =======
>>
>> With the TIF_NOTIFY_IPI, the time taken to complete a fixed set of IPIs
>> using ipistorm improves drastically. Following are the numbers from the
>> same dual socket 3rd Generation EPYC system (2 x 64C/128T) (boost on,
>> C2 disabled) running ipistorm between CPU8 and CPU16:
>>
>> cmdline: insmod ipistorm.ko numipi=100000 single=1 offset=8 cpulist=8 wait=1
>>
>>   ==================================================================
>>   Test          : ipistorm (modified)
>>   Units         : Normalized runtime
>>   Interpretation: Lower is better
>>   Statistic     : AMean
>>   ==================================================================
>>   kernel:                               time [pct imp]
>>   tip:sched/core                        1.00 [0.00]
>>   tip:sched/core + revert               0.81 [19.36]
>>   tip:sched/core + TIF_NOTIFY_IPI       0.20 [80.99]
>>
>> Same experiment was repeated on an dual socket ARM server (2 x 64C)
>> which too saw a significant improvement in the ipistorm performance:
> 
> Could you share more details about this ARM server ? Could it be an Arm64 one ?
> I was not expecting any change for arm/arm64 which are not using TIF_POLLING

I looked at the lscpu output and it said It was an "aarch64" server with
model name "Neoverse-N1". Let me go back and test it once again just to
be sure I did not catch a one off behavior (Might be a while since I
have limited access to this machine) I'll also add a debug
WARN_ON_ONCE() to see if "TIF_NOTIF_IPI" is being set.

> 
> 
>>
>>   ==================================================================
>>   Test          : ipistorm (modified)
>>   Units         : Normalized runtime
>>   Interpretation: Lower is better
>>   Statistic     : AMean
>>   ==================================================================
>>   kernel:                               time [pct imp]
>>   tip:sched/core                        1.00 [0.00]
>>   tip:sched/core + TIF_NOTIFY_IPI       0.41 [59.29]
>>
>> netperf and tbench results with the patch match the results on tip on
>> the dual socket 3rd Generation AMD system (2 x 64C/128T). Additionally,
>> hackbench, stream, and schbench too were tested, with results from the
>> patched kernel matching that of the tip.
>>
>>
>> Future Work
>> ===========
>>
>> Evaluate impact of newidle_balance() when scheduler tick hits an idle
>> CPU. The call to newidle_balance() will be skipped with the
> 
> But it should call the normal idle load balance instead

Yup, but the frequency of normal idle balance will be lower than the
frequency at which a newidle balance is being triggered currently if
tick is not disabled right? Please correct me if I'm wrong. 

> 
>> TIF_NOTIFY_IPI solution similar to [2]. Counter argument for the case is
>> that if the idle state did not set the TIF_POLLING bit, the idle CPU
>> would not have called schedule_idle() unless the IPI handler set the
>> NEED_RESCHED bit.
>>
>>
>> Links
>> =====
>>
>> [1] https://github.com/antonblanchard/ipistorm
>> [2] https://lore.kernel.org/lkml/20240119084548.2788-1-kprateek.nayak@amd.com/
>> [3] https://lore.kernel.org/lkml/b4f5ac150685456cf45a342e3bb1f28cdd557a53.camel@linux.intel.com/
>> [4] https://lore.kernel.org/lkml/20240123211756.GA221793@maniforge/
>> [5] https://lore.kernel.org/lkml/CAKfTPtC446Lo9CATPp7PExdkLhHQFoBuY-JMGC7agOHY4hs-Pw@mail.gmail.com/
>>
>> This series is based on tip:sched/core at tag "sched-core-2024-01-08".
>> [..snip..]
>>
 
--
Thanks and Regards,
Prateek