Received: by 10.192.165.156 with SMTP id m28csp23676imm; Thu, 12 Apr 2018 15:44:15 -0700 (PDT) X-Google-Smtp-Source: AIpwx4+mXKqHnwTE63Tsowf+PL66lMC7Mb0uwToeSoOcQ7eYCeMCe8ebQ/nZRn0jskhwyuylhxbX X-Received: by 10.101.77.67 with SMTP id j3mr2152327pgt.210.1523573055187; Thu, 12 Apr 2018 15:44:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1523573055; cv=none; d=google.com; s=arc-20160816; b=VI9qsuNrsSrMkIuLBdDVCbZ6y3wd97IyTb5tL671H1dFI5M2vfeDkdQkW3AHpCxox3 iWmQ5IBDIdBW8aAPWnNmaYYDtOQTvc3hfhXP0Lco+0nTZjNi9dlAZm5ARsMyrXFxkOUK 7GZP9dPk49f8adctDHQWiihds6BMdfb3xcpGE8WsvCwUToDVuClC7Ck5QeMLBNNf5NOE mf4woG/j90BhA4ZvMYQmjYkO+WjSYT9fNhYtTOvv1E3U5Q1sbsz76ouKl+Ea7fH1nalo KvMYHD8n3l9oJE1EpTehWfdiQ6VQ8aky625m4YGcZ+yeuDEoJFHyV06iQmv8G6DHY0wy ZeDw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:user-agent:in-reply-to :content-transfer-encoding:content-disposition:mime-version :references:message-id:subject:cc:to:from:date:dkim-signature :arc-authentication-results; bh=n6YrL5SchzTTp1YUeAKxNJq8NtSO8sfoAA/2/8/xvz4=; b=Lw8LXUxa7ctvNWynj/wHo14kxRgXPy3beRk2tKZxUuFdWWU9Fly34sW+80JED2vzNY pp0DRbL5KJM8xaaZQSb2lvipqMdTE4N8+fbVtoLEwoGkMjg8ZLWcxYJea7QizpfO0PlW sCqwKE3msrDfXbiFLEZ2vkpUhs8jn9bSbDeNLaNusQhgGyBkq9o0x6OwBrkSLy6JJYwN mhs8PSR8L4gsJvriumoNVQBO/4SMOjtBe54STprbfMbYnj1dtGJWt3p41zN30DdC+++J HrKyPVAEm47bCK8TazdkERDoEkDVPCyibgGR/vtg5odu+9/HeFDXGg4BAFVZDVEGZqRL W1lQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ragnatech-se.20150623.gappssmtp.com header.s=20150623 header.b=ul0xWJBA; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id m13si1624692pgs.49.2018.04.12.15.43.59; Thu, 12 Apr 2018 15:44:15 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@ragnatech-se.20150623.gappssmtp.com header.s=20150623 header.b=ul0xWJBA; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753321AbeDLWjL (ORCPT + 99 others); Thu, 12 Apr 2018 18:39:11 -0400 Received: from mail-lf0-f44.google.com ([209.85.215.44]:46470 "EHLO mail-lf0-f44.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753246AbeDLWjI (ORCPT ); Thu, 12 Apr 2018 18:39:08 -0400 Received: by mail-lf0-f44.google.com with SMTP id j68-v6so9921316lfg.13 for ; Thu, 12 Apr 2018 15:39:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ragnatech-se.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=n6YrL5SchzTTp1YUeAKxNJq8NtSO8sfoAA/2/8/xvz4=; b=ul0xWJBA/l+fNa++Gx9aQytkNY/96REC3pw4lbuHeVbSEO2/vi/UagE1Z56dJswd44 q5MvDdE9tkxflEFpZmIvYKSZQX8BDHgxNGfCUHfOFF80j+h99gEl21lc61L7Ute66gYA l3oklg3fGdwatbbHFub8K9LcT1ZHLEc5i934FTszY0L2hHdeY8hJYAMjsuNg+3khnMGK mT/MqBP9z0wNnlzhGHxGw41Ip7Q/oWkATs1EWD2X8BdwCaodTBL4tFe0YE3qZPsoggfH iNzW/GmnYYqVOvC0783HO/z6Lwo/rkw1xbm6YACgnq4Xpb1nbH1rlifi/htHE5adqfOC 4zhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=n6YrL5SchzTTp1YUeAKxNJq8NtSO8sfoAA/2/8/xvz4=; b=AeSWUWq/L1zphhA8ki1OgdjyVVs8pvB1jnjU55YPab+GL/c7Oh23nFnt8Mmb0/V9v5 fLLkbm3WAT9dft9fooosTqHjVI3+dJnmJKQxYbfYvqFUHiWBWdOW+gY4Dm166HQChpWB OX4zQgzH1LsO1F1QmAuXJCQH/ELUAVoPUb4peg3fGUluE39TizPWOb15F1GK6IyWZQxL pm/Ectz5C+2shhlEBKd4jYholvX0dN8Z9TCZW1sR7zCf+RxtGYJmGTluuKd/xttled4W DjF06ou0YKRAmTKB5Qx+Pg4r2r+Tm3gTuI4aY5mhaQM1ib8OtAfDQICbqgGtWt6q07yi cTcQ== X-Gm-Message-State: ALQs6tBw8UTUXHPCIxNhc45hqM1T1fkpLyOl9aFa6FpSfs8K9ngDug8U pTeXkiCorqNmMGk1cssve/1n6w== X-Received: by 2002:a19:1122:: with SMTP id g34-v6mr5913587lfi.6.1523572746054; Thu, 12 Apr 2018 15:39:06 -0700 (PDT) Received: from localhost (89-233-230-99.cust.bredband2.com. [89.233.230.99]) by smtp.gmail.com with ESMTPSA id t8-v6sm934575lfk.9.2018.04.12.15.39.04 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 12 Apr 2018 15:39:05 -0700 (PDT) Date: Fri, 13 Apr 2018 00:39:04 +0200 From: Niklas =?iso-8859-1?Q?S=F6derlund?= To: Vincent Guittot Cc: i@linaro.org, Heiner Kallweit , Peter Zijlstra , "Paul E. McKenney" , Ingo Molnar , linux-kernel , linux-renesas-soc@vger.kernel.org Subject: Re: Potential problem with 31e77c93e432dec7 ("sched/fair: Update blocked load when newly idle") Message-ID: <20180412223904.GJ12256@bigcity.dyn.berto.se> References: <20180412091822.GG12256@bigcity.dyn.berto.se> <20180412111519.GH12256@bigcity.dyn.berto.se> <20180412133031.GA551@linaro.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20180412133031.GA551@linaro.org> User-Agent: Mutt/1.9.2 (2017-12-15) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Vincent, Thanks for helping trying to figure this out. On 2018-04-12 15:30:31 +0200, Vincent Guittot wrote: [snip] > > I'd like to narrow the problem a bit more with the 2 patchies aboves. Can you try > them separatly on top of c18bb396d3d261eb ("Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net")) > and check if one of them fixes the problem ?i I tried your suggested changes based on top of c18bb396d3d261eb. > > (They should apply on linux-next as well) > > First patch always kick ilb instead of doing ilb on local cpu before entering idle > > --- > kernel/sched/fair.c | 3 +-- > 1 file changed, 1 insertion(+), 2 deletions(-) > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > index 0951d1c..b21925b 100644 > --- a/kernel/sched/fair.c > +++ b/kernel/sched/fair.c > @@ -9739,8 +9739,7 @@ static void nohz_newidle_balance(struct rq *this_rq) > * candidate for ilb instead of waking up another idle CPU. > * Kick an normal ilb if we failed to do the update. > */ > - if (!_nohz_idle_balance(this_rq, NOHZ_STATS_KICK, CPU_NEWLY_IDLE)) > - kick_ilb(NOHZ_STATS_KICK); > + kick_ilb(NOHZ_STATS_KICK); > raw_spin_lock(&this_rq->lock); > } This change don't seem to effect the issue. I can still get the single ssh session and the system to lockup by hitting the return key. And opening a second ssh session immediately unblocks both the first ssh session and the serial console. And I can still trigger the console warning by just letting the system be once it locks-up. I do have just as before reset the system a few times to trigger the issue. [ 245.351693] INFO: rcu_sched detected stalls on CPUs/tasks: [ 245.357199] 0-...!: (1 GPs behind) idle=93c/0/0 softirq=2224/2225 fqs=0 [ 245.363988] (detected by 1, t=3025 jiffies, g=337, c=336, q=10) [ 245.370003] Sending NMI from CPU 1 to CPUs 0: [ 245.374368] NMI backtrace for cpu 0 [ 245.374377] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.16.0-10930-ged741fb4567c816f #42 [ 245.374379] Hardware name: Generic R8A7791 (Flattened Device Tree) [ 245.374393] PC is at arch_cpu_idle+0x24/0x40 [ 245.374397] LR is at arch_cpu_idle+0x34/0x40 [ 245.374400] pc : [] lr : [] psr: 60050013 [ 245.374403] sp : c0b01f40 ip : c0b01f50 fp : c0b01f4c [ 245.374405] r10: c0a56a38 r9 : e7fffbc0 r8 : c0b04c00 [ 245.374407] r7 : c0b04c78 r6 : c0b04c2c r5 : ffffe000 r4 : 00000001 [ 245.374410] r3 : c0119100 r2 : e77813a8 r1 : 0002d93c r0 : 00000000 [ 245.374414] Flags: nZCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none [ 245.374417] Control: 10c5387d Table: 6662006a DAC: 00000051 [ 245.374421] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.16.0-10930-ged741fb4567c816f #42 [ 245.374423] Hardware name: Generic R8A7791 (Flattened Device Tree) [ 245.374425] Backtrace: [ 245.374435] [] (dump_backtrace) from [] (show_stack+0x18/0x1c) [ 245.374440] r7:c0b47278 r6:60050193 r5:00000000 r4:c0b73d80 [ 245.374450] [] (show_stack) from [] (dump_stack+0x84/0xa4) [ 245.374456] [] (dump_stack) from [] (show_regs+0x14/0x18) [ 245.374460] r7:c0b47278 r6:c0b01ef0 r5:00000000 r4:c0bc62c8 [ 245.374468] [] (show_regs) from [] (nmi_cpu_backtrace+0xfc/0x118) [ 245.374475] [] (nmi_cpu_backtrace) from [] (handle_IPI+0x22c/0x294) [ 245.374479] r7:c0b47278 r6:c0b01ef0 r5:00000007 r4:c0a775fc [ 245.374488] [] (handle_IPI) from [] (gic_handle_irq+0x8c/0x98) [ 245.374492] r10:c0a56a38 r9:c0b00000 r8:f0803000 r7:c0b47278 r6:c0b01ef0 r5:c0b05244 [ 245.374495] r4:f0802000 r3:00000407 [ 245.374501] [] (gic_handle_irq) from [] (__irq_svc+0x6c/0x90) [ 245.374504] Exception stack(0xc0b01ef0 to 0xc0b01f38) [ 245.374507] 1ee0: 00000000 0002d93c e77813a8 c0119100 [ 245.374512] 1f00: 00000001 ffffe000 c0b04c2c c0b04c78 c0b04c00 e7fffbc0 c0a56a38 c0b01f4c [ 245.374516] 1f20: c0b01f50 c0b01f40 c0108564 c0108554 60050013 ffffffff [ 245.374521] r9:c0b00000 r8:c0b04c00 r7:c0b01f24 r6:ffffffff r5:60050013 r4:c0108554 [ 245.374528] [] (arch_cpu_idle) from [] (default_idle_call+0x30/0x34) [ 245.374535] [] (default_idle_call) from [] (do_idle+0xd8/0x128) [ 245.374540] [] (do_idle) from [] (cpu_startup_entry+0x20/0x24) [ 245.374543] r7:c0b04c08 r6:ffffffff r5:c0b80380 r4:000000c2 [ 245.374549] [] (cpu_startup_entry) from [] (rest_init+0x9c/0xbc) [ 245.374555] [] (rest_init) from [] (start_kernel+0x368/0x3ec) [ 245.374558] r5:c0b80380 r4:c0b803c0 [ 245.374563] [] (start_kernel) from [<00000000>] ( (null)) [ 245.375369] rcu_sched kthread starved for 3028 jiffies! g337 c336 f0x0 RCU_GP_WAIT_FQS(3) ->state=0x402 ->cpu=1 [ 245.641242] RCU grace-period kthread stack dump: [ 245.645859] rcu_sched I 0 9 2 0x00000000 [ 245.651350] Backtrace: [ 245.653804] [] (__schedule) from [] (schedule+0x94/0xb8) [ 245.660857] r10:e77903c0 r9:e77903c0 r8:ffffffff r7:e709bed4 r6:ffffded7 r5:00000000 [ 245.668689] r4:ffffe000 [ 245.671227] [] (schedule) from [] (schedule_timeout+0x380/0x3dc) [ 245.678973] r5:00000000 r4:00000000 [ 245.682557] [] (schedule_timeout) from [] (rcu_gp_kthread+0x668/0xe2c) [ 245.690826] r10:c0b79018 r9:00000151 r8:00000150 r7:00000001 r6:00000000 r5:c0b10ad0 [ 245.698658] r4:c0b10980 [ 245.701197] [] (rcu_gp_kthread) from [] (kthread+0x148/0x160) [ 245.708682] r7:c0b10980 [ 245.711218] [] (kthread) from [] (ret_from_fork+0x14/0x2c) [ 245.718443] Exception stack(0xe709bfb0 to 0xe709bff8) [ 245.723495] bfa0: 00000000 00000000 00000000 00000000 [ 245.731677] bfc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 [ 245.739858] bfe0: 00000000 00000000 00000000 00000000 00000013 00000000 [ 245.746476] r10:00000000 r9:00000000 r8:00000000 r7:00000000 r6:00000000 r5:c013dc90 [ 245.754307] r4:e7026f40 [ 335.351162] INFO: rcu_sched detected stalls on CPUs/tasks: [ 335.356682] 0-...!: (0 ticks this GP) idle=b0a/1/1073741824 softirq=2226/2226 fqs=0 [ 335.364521] (detected by 1, t=6027 jiffies, g=340, c=339, q=77) [ 335.370545] Sending NMI from CPU 1 to CPUs 0: [ 345.374803] NMI backtrace for cpu 1 [ 345.378295] CPU: 1 PID: 1926 Comm: bash Not tainted 4.16.0-10930-ged741fb4567c816f #42 [ 345.386215] Hardware name: Generic R8A7791 (Flattened Device Tree) [ 345.392395] Backtrace: [ 345.394855] [] (dump_backtrace) from [] (show_stack+0x18/0x1c) [ 345.402430] r7:00000000 r6:600f0193 r5:00000000 r4:c0b73d80 [ 345.408097] [] (show_stack) from [] (dump_stack+0x84/0xa4) [ 345.415329] [] (dump_stack) from [] (nmi_cpu_backtrace+0x104/0x118) [ 345.423337] r7:00000000 r6:00000000 r5:00000001 r4:c0bc62c8 [ 345.429001] [] (nmi_cpu_backtrace) from [] (nmi_trigger_cpumask_backtrace+0x8c/0x10c) [ 345.438572] r7:00000000 r6:c010e4d4 r5:00000001 r4:c0b08158 [ 345.444237] [] (nmi_trigger_cpumask_backtrace) from [] (arch_trigger_cpumask_backtrace+0x14/0x1c) [ 345.454850] r7:c0b10980 r6:c0b04cd0 r5:00000001 r4:c0b10980 [ 345.460519] [] (arch_trigger_cpumask_backtrace) from [] (rcu_dump_cpu_stacks+0xb0/0xd0) [ 345.470268] [] (rcu_dump_cpu_stacks) from [] (rcu_check_callbacks+0x584/0x71c) [ 345.479232] r10:0000004d r9:c0b03d00 r8:ffffe000 r7:00000001 r6:c0b10940 r5:e7795080 [ 345.487065] r4:c0b10980 r3:f181863e [ 345.490648] [] (rcu_check_callbacks) from [] (update_process_times+0x38/0x64) [ 345.499524] r10:e7fa9ea0 r9:e67e8000 r8:e7795e00 r7:0000004e r6:00000000 r5:e72d0640 [ 345.507356] r4:ffffe000 [ 345.509903] [] (update_process_times) from [] (tick_nohz_handler+0xc8/0x114) [ 345.518693] r7:0000004e r6:146afc64 r5:e67e9d70 r4:e77917a0 [ 345.524362] [] (tick_nohz_handler) from [] (arch_timer_handler_virt+0x30/0x40) [ 345.533325] r8:c0b05244 r7:00000013 r6:e7017a40 r5:00000000 r4:e7015c00 [ 345.540033] [] (arch_timer_handler_virt) from [] (handle_percpu_devid_irq+0xf0/0x224) [ 345.549607] [] (handle_percpu_devid_irq) from [] (generic_handle_irq+0x20/0x30) [ 345.558657] r10:e7fa9ea0 r9:e67e8000 r8:e7008800 r7:00000013 r6:c0a775fc r5:00000000 [ 345.566489] r4:00000000 r3:c016ebe4 [ 345.570067] [] (generic_handle_irq) from [] (__handle_domain_irq+0xa8/0xbc) [ 345.578777] [] (__handle_domain_irq) from [] (gic_handle_irq+0x60/0x98) [ 345.587132] r9:e67e8000 r8:f0803000 r7:c0b47278 r6:e67e9d70 r5:c0b05244 r4:f0802000 [ 345.594883] [] (gic_handle_irq) from [] (__irq_svc+0x6c/0x90) [ 345.602368] Exception stack(0xe67e9d70 to 0xe67e9db8) [ 345.607421] 9d60: 00000000 e6c433e8 00182b08 00000000 [ 345.615603] 9d80: e6c429b4 00000087 e67e9e34 e6c434c4 00000000 00000087 e7fa9ea0 e67e9e14 [ 345.623784] 9da0: 00000000 e67e9dc0 c01e0f34 c01e0f88 600f0113 ffffffff [ 345.630401] r9:e67e8000 r8:00000000 r7:e67e9da4 r6:ffffffff r5:600f0113 r4:c01e0f88 [ 345.638156] [] (filemap_map_pages) from [] (handle_mm_fault+0x5c8/0x93c) [ 345.646599] r10:e5dc5380 r9:80000007 r8:00000080 r7:b6e02000 r6:c0b04c08 r5:0000008f [ 345.654431] r4:00000000 [ 345.656968] [] (handle_mm_fault) from [] (do_page_fault+0x17c/0x324) [ 345.665063] r10:e5dc5380 r9:80000007 r8:e72d0640 r7:00000054 r6:e5d030c0 r5:b6e023dc [ 345.672895] r4:e67e9fb0 [ 345.675431] [] (do_page_fault) from [] (do_PrefetchAbort+0x4c/0xc8) [ 345.683439] r10:00000010 r9:00179207 r8:c0b0a434 r7:e67e9fb0 r6:b6e023dc r5:c0b04c08 [ 345.691271] r4:00000007 [ 345.693807] [] (do_PrefetchAbort) from [] (ret_from_exception+0x0/0x14) [ 345.702160] Exception stack(0xe67e9fb0 to 0xe67e9ff8) [ 345.707213] 9fa0: 00000000 000e9600 00000001 bec86544 [ 345.715395] 9fc0: 00000001 bec86544 00000000 00000000 bec8654c 00179207 00000010 000e9600 [ 345.723576] 9fe0: 000c8140 bec86538 0002847c b6e023dc 200f0010 ffffffff [ 345.730192] r8:10c5387d r7:10c5387d r6:ffffffff r5:200f0010 r4:b6e023dc [ 345.737364] nfs: server 10.0.1.1 not responding, still trying [ 345.743294] nfs: server 10.0.1.1 not responding, still trying [ 345.744534] nfs: server 10.0.1.1 not responding, still trying [ 345.749262] nfs: server 10.0.1.1 OK [ 345.755391] nfs: server 10.0.1.1 OK [ 345.758641] nfs: server 10.0.1.1 OK > > -- > 2.7.4 > > > The 2nd patch does the opposite and never kick ilb. I have always made the > assumtpion that the problem was coming from _nohz_idle_balance() and I just > wanted to check that's the case > > --- > kernel/sched/fair.c | 3 +-- > 1 file changed, 1 insertion(+), 2 deletions(-) > > diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c > index 0951d1c..1766594 100644 > --- a/kernel/sched/fair.c > +++ b/kernel/sched/fair.c > @@ -9739,8 +9739,7 @@ static void nohz_newidle_balance(struct rq *this_rq) > * candidate for ilb instead of waking up another idle CPU. > * Kick an normal ilb if we failed to do the update. > */ > - if (!_nohz_idle_balance(this_rq, NOHZ_STATS_KICK, CPU_NEWLY_IDLE)) > - kick_ilb(NOHZ_STATS_KICK); > + _nohz_idle_balance(this_rq, NOHZ_STATS_KICK, CPU_NEWLY_IDLE); > raw_spin_lock(&this_rq->lock); > } With this change it seem more difficult to trigger the issue, I usually can trigger it within 2-3 rests of the system but with this applied I had to reset it over 5 times to trigger it. But I can still reproduce the lockup reliably multiple times. I would not trust my feeling that it's harder to reproduce with this change as the method to trigger it is kind of spotty and most likely due to circumstantial circumstances :-) I can still unlock the system by starting a second ssh session and trigger the output on the serial console. [ 193.741481] INFO: rcu_sched detected stalls on CPUs/tasks: [ 193.746991] 1-...!: (0 ticks this GP) idle=83c/0/0 softirq=2445/2445 fqs=0 [ 193.754041] (detected by 0, t=4096 jiffies, g=611, c=610, q=31) [ 193.760054] NMI backtrace for cpu 0 [ 193.763546] CPU: 0 PID: 2193 Comm: bash Not tainted 4.16.0-10929-gc18bb396d3d261eb-dirty #43 [ 193.771984] Hardware name: Generic R8A7791 (Flattened Device Tree) [ 193.778163] Backtrace: [ 193.780625] [] (dump_backtrace) from [] (show_stack+0x18/0x1c) [ 193.788198] r7:00000000 r6:60010193 r5:00000000 r4:c0b73d80 [ 193.793865] [] (show_stack) from [] (dump_stack+0x84/0xa4) [ 193.801093] [] (dump_stack) from [] (nmi_cpu_backtrace+0x104/0x118) [ 193.809098] r7:00000000 r6:00000000 r5:00000000 r4:c0bc62c8 [ 193.814761] [] (nmi_cpu_backtrace) from [] (nmi_trigger_cpumask_backtrace+0x8c/0x10c) [ 193.824329] r7:00000000 r6:c010e4d4 r5:00000000 r4:c0b08158 [ 193.829992] [] (nmi_trigger_cpumask_backtrace) from [] (arch_trigger_cpumask_backtrace+0x14/0x1c) [ 193.840603] r7:c0b10980 r6:c0b04cd0 r5:00000000 r4:c0b10980 [ 193.846268] [] (arch_trigger_cpumask_backtrace) from [] (rcu_dump_cpu_stacks+0xb0/0xd0) [ 193.856015] [] (rcu_dump_cpu_stacks) from [] (rcu_check_callbacks+0x584/0x71c) [ 193.864977] r10:0000001f r9:c0b03d00 r8:ffffe000 r7:00000001 r6:c0b10940 r5:e7786080 [ 193.872810] r4:c0b10980 r3:f181863e [ 193.876392] [] (rcu_check_callbacks) from [] (update_process_times+0x38/0x64) [ 193.885268] r10:00000000 r9:00180b98 r8:e7786e00 r7:0000002d r6:00000001 r5:e5d86180 [ 193.893100] r4:ffffe000 [ 193.895642] [] (update_process_times) from [] (tick_nohz_handler+0xc8/0x114) [ 193.904431] r7:0000002d r6:1bcdda00 r5:e66c7fb0 r4:e77827a0 [ 193.910099] [] (tick_nohz_handler) from [] (arch_timer_handler_virt+0x30/0x40) [ 193.919062] r8:c0b05244 r7:00000013 r6:e7017a40 r5:00000000 r4:e7015c00 [ 193.925769] [] (arch_timer_handler_virt) from [] (handle_percpu_devid_irq+0xf0/0x224) [ 193.935343] [] (handle_percpu_devid_irq) from [] (generic_handle_irq+0x20/0x30) [ 193.944394] r10:00000000 r9:00180b98 r8:e7008800 r7:00000013 r6:c0a775fc r5:00000000 [ 193.952225] r4:00000000 r3:c016ec24 [ 193.955803] [] (generic_handle_irq) from [] (__handle_domain_irq+0xa8/0xbc) [ 193.964513] [] (__handle_domain_irq) from [] (gic_handle_irq+0x60/0x98) [ 193.972869] r9:00180b98 r8:f0803000 r7:c0b47278 r6:e66c7fb0 r5:c0b05244 r4:f0802000 [ 193.980619] [] (gic_handle_irq) from [] (__irq_usr+0x54/0x80) [ 193.988105] Exception stack(0xe66c7fb0 to 0xe66c7ff8) [ 193.993157] 7fa0: 00000000 00000000 b6e8b79c 00000000 [ 194.001339] 7fc0: 00178378 000134e0 b6e8b794 b6e8b000 b6e8b794 00180b98 00000000 b6e8b7c4 [ 194.009520] 7fe0: be955190 be955170 00000000 b6db5c58 20010010 ffffffff [ 194.016137] r9:00180b98 r8:10c5387d r7:10c5387d r6:ffffffff r5:20010010 r4:b6db5c58 [ 194.023888] Sending NMI from CPU 0 to CPUs 1: [ 204.028147] NMI backtrace for cpu 1 [ 204.028152] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.16.0-10929-gc18bb396d3d261eb-dirty #43 [ 204.028155] Hardware name: Generic R8A7791 (Flattened Device Tree) [ 204.028164] PC is at _raw_spin_unlock_irqrestore+0x28/0x2c [ 204.028169] LR is at rcu_report_qs_rnp+0x4c/0x20c [ 204.028172] pc : [] lr : [] psr: 200e0113 [ 204.028175] sp : e70a3db0 ip : e70a3dc0 fp : e70a3dbc [ 204.028177] r10: 00000264 r9 : c0b787eb r8 : 00000001 [ 204.028179] r7 : 00000002 r6 : c0b10980 r5 : c0b10980 r4 : c0b10980 [ 204.028182] r3 : 000022a3 r2 : 00000264 r1 : 600e0113 r0 : c0b10980 [ 204.028187] Flags: nzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment none [ 204.028189] Control: 10c5387d Table: 66b9806a DAC: 00000051 [ 204.028193] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.16.0-10929-gc18bb396d3d261eb-dirty #43 [ 204.028195] Hardware name: Generic R8A7791 (Flattened Device Tree) [ 204.028197] Backtrace: [ 204.028204] [] (dump_backtrace) from [] (show_stack+0x18/0x1c) [ 204.028209] r7:c0b47278 r6:600e0193 r5:00000000 r4:c0b73d80 [ 204.028215] [] (show_stack) from [] (dump_stack+0x84/0xa4) [ 204.028228] [] (dump_stack) from [] (show_regs+0x14/0x18) [ 204.028231] r7:c0b47278 r6:e70a3d60 r5:00000001 r4:c0bc62c8 [ 204.028238] [] (show_regs) from [] (nmi_cpu_backtrace+0xfc/0x118) [ 204.028244] [] (nmi_cpu_backtrace) from [] (handle_IPI+0x22c/0x294) [ 204.028247] r7:c0b47278 r6:e70a3d60 r5:00000007 r4:c0a775fc [ 204.028254] [] (handle_IPI) from [] (gic_handle_irq+0x8c/0x98) [ 204.028259] r10:00000264 r9:e70a2000 r8:f0803000 r7:c0b47278 r6:e70a3d60 r5:c0b05244 [ 204.028261] r4:f0802000 r3:00000007 [ 204.028267] [] (gic_handle_irq) from [] (__irq_svc+0x6c/0x90) [ 204.028270] Exception stack(0xe70a3d60 to 0xe70a3da8) [ 204.028275] 3d60: c0b10980 600e0113 00000264 000022a3 c0b10980 c0b10980 c0b10980 00000002 [ 204.028280] 3d80: 00000001 c0b787eb 00000264 e70a3dbc e70a3dc0 e70a3db0 c01796d4 c07653d8 [ 204.028283] 3da0: 200e0113 ffffffff [ 204.028287] r9:e70a2000 r8:00000001 r7:e70a3d94 r6:ffffffff r5:200e0113 r4:c07653d8 [ 204.028295] [] (_raw_spin_unlock_irqrestore) from [] (rcu_report_qs_rnp+0x4c/0x20c) [ 204.028301] [] (rcu_report_qs_rnp) from [] (rcu_process_callbacks+0x364/0x8cc) [ 204.028305] r10:c0b03080 r9:00000000 r8:00000002 r7:600e0113 r6:c0b10980 r5:c0b10980 [ 204.028306] r4:e7795080 [ 204.028312] [] (rcu_process_callbacks) from [] (__do_softirq+0x23c/0x338) [ 204.028316] r10:c0b03080 r9:00000100 r8:c0b80ec0 r7:00000002 r6:00000009 r5:c0b030a4 [ 204.028318] r4:ffffe000 [ 204.028325] [] (__do_softirq) from [] (irq_exit+0x6c/0xd4) [ 204.028329] r10:00000000 r9:e70a2000 r8:e7008800 r7:00000013 r6:c0a775fc r5:00000000 [ 204.028331] r4:00000000 [ 204.028336] [] (irq_exit) from [] (__handle_domain_irq+0x88/0xbc) [ 204.028338] r5:00000000 r4:00000000 [ 204.028344] [] (__handle_domain_irq) from [] (gic_handle_irq+0x60/0x98) [ 204.028348] r9:e70a2000 r8:f0803000 r7:c0b47278 r6:e70a3f40 r5:c0b05244 r4:f0802000 [ 204.028354] [] (gic_handle_irq) from [] (__irq_svc+0x6c/0x90) [ 204.028356] Exception stack(0xe70a3f40 to 0xe70a3f88) [ 204.028361] 3f40: 00000000 00013844 e77903a8 c0119100 00000002 ffffe000 c0b04c2c c0b04c78 [ 204.028367] 3f60: 4000406a 413fc0f2 00000000 e70a3f9c e70a3fa0 e70a3f90 c0108564 c0108554 [ 204.028370] 3f80: 600e0013 ffffffff [ 204.028374] r9:e70a2000 r8:4000406a r7:e70a3f74 r6:ffffffff r5:600e0013 r4:c0108554 [ 204.028381] [] (arch_cpu_idle) from [] (default_idle_call+0x30/0x34) [ 204.028388] [] (default_idle_call) from [] (do_idle+0xd8/0x128) [ 204.028392] [] (do_idle) from [] (cpu_startup_entry+0x20/0x24) [ 204.028396] r7:c0b80750 r6:10c0387d r5:00000051 r4:00000085 [ 204.028401] [] (cpu_startup_entry) from [] (secondary_start_kernel+0x10c/0x12c) [ 204.028406] [] (secondary_start_kernel) from [<4010250c>] (0x4010250c) [ 204.028408] r5:00000051 r4:6709406a > > -- > 2.7.4 > > > Thanks, > Vincent > > > > > > > > > > Regards, > > > Vincent > > > > > > > > 1. c18bb396d3d261eb ("Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net")) > > > > 2. 31e77c93e432dec7 ("sched/fair: Update blocked load when newly idle") > > > > > > > > -- > > > > Regards, > > > > Niklas S?derlund > > > > -- > > Regards, > > Niklas S?derlund -- Regards, Niklas S?derlund