2020-07-31 00:41:29

by Joel Fernandes

Subject: [PATCH v2 3/3] rcu/tree: Make FQS complaining about offline CPU more aggressive

Make FQS loop consider it an immediate failure if the case of an offline CPU
reporting QS is detected, instead of a full second.

This is because rcu_report_dead() already reports quiescent states and
updates ->qsmaskinitnext under node lock.

Light testing with TREE03 and hotplug shows no warnings.

Convert the warning as well to WARN_ON_ONCE() to reduce log spam.

Signed-off-by: Joel Fernandes (Google) <[email protected]>
---
kernel/rcu/tree.c | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
index a621932cc385..39bdd744ba97 100644
--- a/kernel/rcu/tree.c
+++ b/kernel/rcu/tree.c
@@ -1208,13 +1208,15 @@ static int rcu_implicit_dynticks_qs(struct rcu_data *rdp)
return 1;
}

- /* If waiting too long on an offline CPU, complain. */
- if (!(rdp->grpmask & rcu_rnp_online_cpus(rnp)) &&
- time_after(jiffies, rcu_state.gp_start + HZ)) {
+ /*
+ * Complain if an offline CPU by RCU's books has not reported QS. Node
+ * lock is held ensuring offlining does not race here.
+ */
+ if (!(rdp->grpmask & rcu_rnp_online_cpus(rnp))) {
bool onl;
struct rcu_node *rnp1;

- WARN_ON(1); /* Offline CPUs are supposed to report QS! */
+ WARN_ON_ONCE(1); /* Offline CPUs are supposed to report QS! */
pr_info("%s: grp: %d-%d level: %d ->gp_seq %ld ->completedqs %ld\n",
__func__, rnp->grplo, rnp->grphi, rnp->level,
(long)rnp->gp_seq, (long)rnp->completedqs);
--
2.28.0.163.g6104cc2f0b6-goog

2020-07-31 01:36:43

by Paul E. McKenney

Subject: Re: [PATCH v2 3/3] rcu/tree: Make FQS complaining about offline CPU more aggressive

On Thu, Jul 30, 2020 at 08:40:12PM -0400, Joel Fernandes (Google) wrote:
> Make FQS loop consider it an immediate failure if the case of an offline CPU
> reporting QS is detected, instead of a full second.
>
> This is because rcu_report_dead() already reports quiescent states and
> updates ->qsmaskinitnext under node lock.
>
> Light testing with TREE03 and hotplug shows no warnings.
>
> Convert the warning as well to WARN_ON_ONCE() to reduce log spam.

I will give you a chance to upgrade the above on your V3.
And the comment below.

I do very much like the change to WARN_ON_ONCE(), by the way!

Thanx, Paul

> Signed-off-by: Joel Fernandes (Google) <[email protected]>
> ---
> kernel/rcu/tree.c | 10 ++++++----
> 1 file changed, 6 insertions(+), 4 deletions(-)
>
> diff --git a/kernel/rcu/tree.c b/kernel/rcu/tree.c
> index a621932cc385..39bdd744ba97 100644
> --- a/kernel/rcu/tree.c
> +++ b/kernel/rcu/tree.c
> @@ -1208,13 +1208,15 @@ static int rcu_implicit_dynticks_qs(struct rcu_data *rdp)
> return 1;
> }
>
> - /* If waiting too long on an offline CPU, complain. */
> - if (!(rdp->grpmask & rcu_rnp_online_cpus(rnp)) &&
> - time_after(jiffies, rcu_state.gp_start + HZ)) {
> + /*
> + * Complain if an offline CPU by RCU's books has not reported QS. Node
> + * lock is held ensuring offlining does not race here.
> + */
> + if (!(rdp->grpmask & rcu_rnp_online_cpus(rnp))) {
> bool onl;
> struct rcu_node *rnp1;
>
> - WARN_ON(1); /* Offline CPUs are supposed to report QS! */
> + WARN_ON_ONCE(1); /* Offline CPUs are supposed to report QS! */
> pr_info("%s: grp: %d-%d level: %d ->gp_seq %ld ->completedqs %ld\n",
> __func__, rnp->grplo, rnp->grphi, rnp->level,
> (long)rnp->gp_seq, (long)rnp->completedqs);
> --
> 2.28.0.163.g6104cc2f0b6-goog
>