Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <5b4bd044-ba88-649b-9b85-e08e175691f9@fb.com>
Date:   Mon, 16 May 2022 19:54:37 -0700
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0)
 Gecko/20100101 Thunderbird/91.9.0
Subject: Re: [PATCH bpf-next 1/2] cpuidle/rcu: Making arch_cpu_idle and
 rcu_idle_exit noinstr
Content-Language: en-US
To:     Jiri Olsa <jolsa@kernel.org>, Alexei Starovoitov <ast@kernel.org>,
        Daniel Borkmann <daniel@iogearbox.net>,
        Andrii Nakryiko <andrii@kernel.org>,
        Masami Hiramatsu <mhiramat@kernel.org>,
        "Paul E. McKenney" <paulmck@kernel.org>
Cc:     netdev@vger.kernel.org, bpf@vger.kernel.org,
        lkml <linux-kernel@vger.kernel.org>,
        Martin KaFai Lau <kafai@fb.com>,
        Song Liu <songliubraving@fb.com>,
        John Fastabend <john.fastabend@gmail.com>,
        KP Singh <kpsingh@chromium.org>,
        Steven Rostedt <rostedt@goodmis.org>
References: <20220515203653.4039075-1-jolsa@kernel.org>
From:   Yonghong Song <yhs@fb.com>
In-Reply-To: <20220515203653.4039075-1-jolsa@kernel.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Znl6aVVHdDRac0RSbmE4QTkwWWhQVlROZzNuMG9CODJvV3dlZWJ4OEVWaHgx?=
 =?utf-8?B?OUJVMjlFK1EzbkFEZkYyNTRBT1E1dEREYU9YTHBuQWk2QTVtK1Z6d0xEdlpZ?=
 =?utf-8?B?dmJ6R3Y2eXNSeDIyYXZvRSsvOUI5NzVScHFyR2VCWUIwaVgxR0prS1dvcUw4?=
 =?utf-8?B?NnVRdi9BQ0pWRnhpc1NXeEVqV1hTdVZJR1dUaGdsS0tySStIUHAvUlc3Q210?=
 =?utf-8?B?bmFkY1NYcC85bG9HZE9LM004ZlRyUlQ2MkQrb2N0NlMxWHhxWTVCSEsrUXNv?=
 =?utf-8?B?Z29ORFlxRlRxUmo5dE9Oa0F3VEFOM0VQMmR3SjlmNE1iQlpjVWllZi9YeFdB?=
 =?utf-8?B?YjcwZElrY0ZteHAybEpUM2o3SDl6S1NXZUdWNUNrV2gvTStEc0FsME5mcnI0?=
 =?utf-8?B?VE11bnlTSWJid2FDbkUvVVJGREVMSHUwai9uRXVTUTZaRHg0SzZLb3VxdGxs?=
 =?utf-8?B?L2lwOGhTN1dFUkxvTTU0c0tjZjd3WVZqakQwamhwaGpPaDdqTzdCNUxJUU0z?=
 =?utf-8?B?SWtQbHF0M2lHRnVwNG5Za3puL1BTRUs1N3BubFhGMjc1aENlN1RjYjIzeWor?=
 =?utf-8?B?dndWL0t1RENZSDRXbEw2REFRN0dVbU9RWGtiM0dWWHJ2eWU3aGd3ZlVLL3Fn?=
 =?utf-8?B?eGRWaXZMZDFSQ0JRSm9LWUZya3FQbk1jVzdWdFlaTEo2WWlGZzVSbFZtMXBU?=
 =?utf-8?B?NjhUeEswUHl5d1hHRkNGMDVZTk1XVkliNTR5c3NvdmRvVEE1TFlWaDlCRU4y?=
 =?utf-8?B?WWxuR01PckI3bkxzeGkwMmVwV1c3SlBkTHBuSVEvU0lHNlpSUTNvQ0RCVWV1?=
 =?utf-8?B?OWV3b3E1TU9IbXBWYmpvZnFuS3djUWYrQjlLekFwdCtKeG1RZ01uZTJGUlJq?=
 =?utf-8?B?RHhXbXBML0FqMERRU0l4T0RQbHYrU3R6S09MWGszazZYd2RCQXhJYU5GczdU?=
 =?utf-8?B?ZmVCMnFONXByNVNnYzNCcmVkMVNya1BrRTdLQ3hLQ0pBOFl1SERvOTl6bEl6?=
 =?utf-8?B?dHNoaHdQdDllUTlmWXk4Y2JHODE2ZDJDWFdINytGamlsN0dHMWVrVUF1WDRj?=
 =?utf-8?B?VUY3ZWxCZ2hIcnQ5aERodlNOcERxcVV1bHQ3cDcyT1RyVTdDNWtsaitHRkl5?=
 =?utf-8?B?bG5aNGoybVpTVjhkaXZ1TWFpd2ZzNE01OER3V2laUjE3NkFrVHhYOFRWUThh?=
 =?utf-8?B?UUpmYUNCQmYvU3NXYUtCZitUcjA1K2NnQlZySnQ2N2s2Qk1Vay9mc3loRVhp?=
 =?utf-8?B?SlBHWWw5RTFWQnIvcU80Tm9pU05xVFlOZmhGdVdnVnNwU2ppRmROMkkxWEln?=
 =?utf-8?B?NVhBZTVsZlJTNnIvQjFaaG1WVWhyblJjb2RXVDFSKzNxUTQ0T1BST3crUjdD?=
 =?utf-8?B?YW1ITHUzcFVvWjRKZyt0L2NVRFRhd21QbmFVbTVXeitWRHNHVHlYa01vT2Vz?=
 =?utf-8?B?ekhCQWFwQklQWEFZL200VVVkUjlpLzdUYjZhQjJzc2tFODJnc1VjRW5sYnZq?=
 =?utf-8?B?dGdBdXBVUE9OWE1UNGhxcGhCN3djQVRnQzBGOVVmYVdrbDZqd0YwU1hjMGJs?=
 =?utf-8?B?Z1haMHFCN1lhd0k0TkREWjE0VXVtMU5ROWwvcjJuWkE3S2ZwSURXTnZ1cGVQ?=
 =?utf-8?B?MmJpcjZJbGtGVHdNWmNKK2t5MDdNaDhucjREYXVGUmVaOFlpUFFjOU9VSnJW?=
 =?utf-8?B?M2pGeDJrUkdhbnlOejZ5RmNjaWhTdzZ2L3BjWWl2S0hDNHlhWTd0SlVRU3R2?=
 =?utf-8?B?UkN3SjhhY3YyVkJmdWhUNXhCZmRSYTY5OHRTY2pNeEdaRHp1ek51QmtKMXBN?=
 =?utf-8?B?VUhSUkRyenZWb0FnRUwzT2x1T1FKak9QQXBpa0VpWUhTLzhRdzBuWjZZZDls?=
 =?utf-8?B?MnY2cEZBZDNHSXhkZko5SDFFNDVEYlMzcStBbzhBcjFvRVFVYzNtS3RvMUZP?=
 =?utf-8?B?NkwwLzUvaHVHN2YxWnh4R0RSOHFEb3lXNVY0UlQ3TThud21zeGV2SmVZcHFi?=
 =?utf-8?B?SCs2RkxvUjhFM25GMkxvSjQ5ZWVQbkdkQklsSTB0R3crLzh0Y2lXS050R0tq?=
 =?utf-8?B?em1rUkFZMXRwSXpxcmphMmRValV3SzZkTjl2RFc3M1ljWGdadGlTYmhtT2sx?=
 =?utf-8?B?MSs5aFk5MnprRC9wS29ES2RMemJEWk1vb1VQQkljMTdHUlBEKzdkWVZoYkRE?=
 =?utf-8?B?UXVuOE9aL1dkVUJLeUZINE5rM3NaRFFCR0JvT3cvZWRudkNRQzBMVUhoM0M4?=
 =?utf-8?B?eFlzWURJb1pSZUpJaTFvbHZnVFlNblZ5Y2pJT293NjB4M0JVSVRNSnlGV28y?=
 =?utf-8?B?UG5xZjAwaVZNZHd0dUdtd0NJWS9pVXlSWXlnclNJMGRZdVk2M3dsOHpkN0lQ?=
 =?utf-8?Q?36YQpWY0vu4W6VXo=3D?=
X-OriginatorOrg: fb.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f242eff3-a7ed-41d3-807a-08da37b0963b
X-MS-Exchange-CrossTenant-AuthSource: SN6PR1501MB2064.namprd15.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 May 2022 02:54:40.2006
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ui224GnYQxuxRoj1LEqFmU2M+LGQRx4cFhT1gjicr6lI6dcMJtZ3l845IOm6maSl
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR15MB1455
X-Proofpoint-GUID: klHr2PTN3KW_r_-iLNODk9yBZwcQ02EA
X-Proofpoint-ORIG-GUID: klHr2PTN3KW_r_-iLNODk9yBZwcQ02EA
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.205,Aquarius:18.0.858,Hydra:6.0.486,FMLib:17.11.64.514
 definitions=2022-05-16_16,2022-05-16_02,2022-02-23_01
X-Spam-Status: No, score=-3.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,
        RCVD_IN_DNSWL_LOW,RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,
        SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no
        version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 5/15/22 1:36 PM, Jiri Olsa wrote:
> Making arch_cpu_idle and rcu_idle_exit noinstr. Both functions run
> in rcu 'not watching' context and if there's tracer attached to
> them, which uses rcu (e.g. kprobe multi interface) it will hit RCU
> warning like:
> 
>    [    3.017540] WARNING: suspicious RCU usage
>    ...
>    [    3.018363]  kprobe_multi_link_handler+0x68/0x1c0
>    [    3.018364]  ? kprobe_multi_link_handler+0x3e/0x1c0
>    [    3.018366]  ? arch_cpu_idle_dead+0x10/0x10
>    [    3.018367]  ? arch_cpu_idle_dead+0x10/0x10
>    [    3.018371]  fprobe_handler.part.0+0xab/0x150
>    [    3.018374]  0xffffffffa00080c8
>    [    3.018393]  ? arch_cpu_idle+0x5/0x10
>    [    3.018398]  arch_cpu_idle+0x5/0x10
>    [    3.018399]  default_idle_call+0x59/0x90
>    [    3.018401]  do_idle+0x1c3/0x1d0
> 
> The call path is following:
> 
> default_idle_call
>    rcu_idle_enter
>    arch_cpu_idle
>    rcu_idle_exit
> 
> The arch_cpu_idle and rcu_idle_exit are the only ones from above
> path that are traceble and cause this problem on my setup.
> 
> Signed-off-by: Jiri Olsa <jolsa@kernel.org>
> ---
>   arch/x86/kernel/process.c | 2 +-
>   kernel/rcu/tree.c         | 2 +-
>   2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
> index b370767f5b19..1345cb0124a6 100644
> --- a/arch/x86/kernel/process.c
> +++ b/arch/x86/kernel/process.c
> @@ -720,7 +720,7 @@ void arch_cpu_idle_dead(void)
>   /*
>    * Called from the generic idle code.
>    */
> -void arch_cpu_idle(void)
> +void noinstr arch_cpu_idle(void)

noinstr includes a lot of attributes:

#define noinstr                                                         \
         noinline notrace __attribute((__section__(".noinstr.text")))    \
         __no_kcsan __no_sanitize_address __no_profile 
__no_sanitize_coverage

should we use notrace here?

>   {
>   	x86_idle();
>   }
> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
> index a4b8189455d5..20d529722f51 100644
> --- a/kernel/rcu/tree.c
> +++ b/kernel/rcu/tree.c
> @@ -896,7 +896,7 @@ static void noinstr rcu_eqs_exit(bool user)
>    * If you add or remove a call to rcu_idle_exit(), be sure to test with
>    * CONFIG_RCU_EQS_DEBUG=y.
>    */
> -void rcu_idle_exit(void)
> +void noinstr rcu_idle_exit(void)
>   {
>   	unsigned long flags;
>