Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Content-Type: text/plain;
        charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.100.0.2.22\))
Subject: Re: [powerpc][next-20210621] WARNING at kernel/sched/fair.c:3277
 during boot
From:   Sachin Sant <sachinp@linux.vnet.ibm.com>
In-Reply-To: <20210622143154.GA804@vingu-book>
Date:   Tue, 22 Jun 2021 21:29:36 +0530
Cc:     open list <linux-kernel@vger.kernel.org>,
        Linux Next Mailing List <linux-next@vger.kernel.org>,
        linuxppc-dev@lists.ozlabs.org, Odin Ugedal <odin@uged.al>
Content-Transfer-Encoding: quoted-printable
Message-Id: <53968DDE-9E93-4CB4-B5E4-526230B6E154@linux.vnet.ibm.com>
References: <2ED1BDF5-BC0C-47CD-8F33-9A46C738F8CF@linux.vnet.ibm.com>
 <CAKfTPtDrHv4OOfPvwOE2DMNoucXQJ=yvvEpTVKrXghSdKEnZcA@mail.gmail.com>
 <20210622143154.GA804@vingu-book>
To:     Vincent Guittot <vincent.guittot@linaro.org>
Precedence: bulk

>> On Tue, 22 Jun 2021 at 09:39, Sachin Sant =
<sachinp@linux.vnet.ibm.com> wrote:
>>>=20
>>> While booting 5.13.0-rc7-next-20210621 on a PowerVM LPAR following =
warning
>>> is seen
>>>=20
>>> [   30.922154] ------------[ cut here ]------------
>>> [   30.922201] cfs_rq->avg.load_avg || cfs_rq->avg.util_avg || =
cfs_rq->avg.runnable_avg
>>> [   30.922219] WARNING: CPU: 6 PID: 762 at kernel/sched/fair.c:3277 =
update_blocked_averages+0x758/0x780
>>=20
>> Yes. That was exactly the purpose of the patch. There is one last
>> remaining part which could generate this. I'm going to prepare a =
patch
>=20
> Could you try the patch below ? I have been able to reproduce the =
problem locally and this
> fix it on my system:
>=20
I can recreate the issue with this patch.

         Starting Terminate Plymouth Boot Screen...
         Starting Hold until boot process finishes up...
[FAILED] Failed to start Crash recovery kernel arming.
See 'systemctl status kdump.service' for details.
[   10.737913] ------------[ cut here ]------------
[   10.737960] cfs_rq->avg.load_avg || cfs_rq->avg.util_avg || =
cfs_rq->avg.runnable_avg
[   10.737976] WARNING: CPU: 27 PID: 146 at kernel/sched/fair.c:3279 =
update_blocked_averages+0x758/0x780
[   10.738010] Modules linked in: stp llc rfkill sunrpc pseries_rng xts =
vmx_crypto uio_pdrv_genirq uio sch_fq_codel ip_tables xfs libcrc32c =
sr_mod sd_mod cdrom t10_pi sg ibmvscsi ibmveth scsi_transport_srp =
dm_mirror dm_region_hash dm_log dm_mod fuse
[   10.738089] CPU: 27 PID: 146 Comm: ksoftirqd/27 Not tainted =
5.13.0-rc7-next-20210621-dirty #2
[   10.738103] NIP:  c0000000001b2768 LR: c0000000001b2764 CTR: =
c000000000729120
[   10.738116] REGS: c000000015973840 TRAP: 0700   Not tainted  =
(5.13.0-rc7-next-20210621-dirty)
[   10.738130] MSR:  800000000282b033 <SF,VEC,VSX,EE,FP,ME,IR,DR,RI,LE>  =
CR: 48000224  XER: 00000005
[   10.738161] CFAR: c00000000014d120 IRQMASK: 1=20
[   10.738161] GPR00: c0000000001b2764 c000000015973ae0 c0000000029bb900 =
0000000000000048=20
[   10.738161] GPR04: 00000000fffeffff c0000000159737a0 0000000000000027 =
c00000154f9f7e18=20
[   10.738161] GPR08: 0000000000000023 0000000000000001 0000000000000027 =
c00000167f1d7fe8=20
[   10.738161] GPR12: 0000000000000000 c00000154ffd7e80 c00000154fa82580 =
000000000000b78a=20
[   10.738161] GPR16: 000000028007883c 00000000000002ed c000000038d31000 =
0000000000000000=20
[   10.738161] GPR20: 0000000000000000 c0000000029fdfe0 0000000000000000 =
000000000000037b=20
[   10.738161] GPR24: 0000000000000000 c00000154fa82f90 0000000000000001 =
c00000003d4ca400=20
[   10.738161] GPR28: 00000000000002ed c000000038d311c0 c000000038d31100 =
0000000000000000=20
[   10.738281] NIP [c0000000001b2768] =
update_blocked_averages+0x758/0x780
[   10.738290] LR [c0000000001b2764] update_blocked_averages+0x754/0x780
[   10.738299] Call Trace:
[   10.738303] [c000000015973ae0] [c0000000001b2764] =
update_blocked_averages+0x754/0x780 (unreliable)
[   10.738315] [c000000015973c00] [c0000000001be720] =
run_rebalance_domains+0xa0/0xd0
[   10.738326] [c000000015973c30] [c000000000cf9acc] =
__do_softirq+0x15c/0x3d4
[   10.738337] [c000000015973d20] [c000000000158464] =
run_ksoftirqd+0x64/0x90
[   10.738346] [c000000015973d40] [c00000000018fd24] =
smpboot_thread_fn+0x204/0x270
[   10.738357] [c000000015973da0] [c000000000189770] kthread+0x190/0x1a0
[   10.738367] [c000000015973e10] [c00000000000ceec] =
ret_from_kernel_thread+0x5c/0x70
[   10.738381] Instruction dump:
[   10.738388] 3863c808 9be9eefe 4bf9a979 60000000 0fe00000 4bfff980 =
e9210070 e8610088=20
[   10.738410] 39400001 99490003 4bf9a959 60000000 <0fe00000> 4bfffc24 =
3d22fff6 8929eefb=20
[   10.738431] ---[ end trace 9ca80b55840c53f0 ]=E2=80=94

Thanks
-Sachin

> diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
> index 8cc27b847ad8..da91db1c137f 100644
> --- a/kernel/sched/fair.c
> +++ b/kernel/sched/fair.c
> @@ -3037,8 +3037,9 @@ enqueue_load_avg(struct cfs_rq *cfs_rq, struct =
sched_entity *se)
> static inline void
> dequeue_load_avg(struct cfs_rq *cfs_rq, struct sched_entity *se)
> {
> +       u32 divider =3D get_pelt_divider(&se->avg);
>        sub_positive(&cfs_rq->avg.load_avg, se->avg.load_avg);
> -       sub_positive(&cfs_rq->avg.load_sum, se_weight(se) * =
se->avg.load_sum);
> +       cfs_rq->avg.load_sum =3D cfs_rq->avg.load_avg * divider;
> }
> #else
> static inline void
>=20