Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <20191004145741.118292-1-joel@joelfernandes.org> <20191004154102.GA20945@redhat.com>
In-Reply-To: <20191004154102.GA20945@redhat.com>
From:   Joel Fernandes <joel@joelfernandes.org>
Date:   Thu, 16 Jan 2020 16:57:05 -0500
Message-ID: <CAEXW_YS20r3shEwk8GCTY9NcQB+duzTc82ROo5xgWakJwvKRRQ@mail.gmail.com>
Subject: Re: [PATCH] Remove GP_REPLAY state from rcu_sync
To:     Oleg Nesterov <oleg@redhat.com>
Cc:     LKML <linux-kernel@vger.kernel.org>,
        Daniel Bristot de Oliveira <bristot@redhat.com>,
        Peter Zijlstra <peterz@infradead.org>,
        "Paul E. McKenney" <paulmck@kernel.org>, rcu <rcu@vger.kernel.org>,
        Josh Triplett <josh@joshtriplett.org>,
        Lai Jiangshan <jiangshanlai@gmail.com>,
        Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
        "Paul E. McKenney" <paulmck@linux.ibm.com>,
        Steven Rostedt <rostedt@goodmis.org>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Fri, Oct 4, 2019 at 11:41 AM Oleg Nesterov <oleg@redhat.com> wrote:
>
> On 10/04, Joel Fernandes (Google) wrote:
> >
> > But this is not always true if you consider the following events:
>
> I'm afraid I missed your point, but...
>
> > ---------------------->
> > GP num         111111     22222222222222222222222222222222233333333
> > GP state  i    e     p    x                 r              rx     i
> > CPU0 :         rse      rsx
> > CPU1 :                         rse     rsx
> > CPU2 :                                         rse     rsx
> >
> > Here, we had 3 grace periods that elapsed, 1 for the rcu_sync_enter(),
> > and 2 for the rcu_sync_exit(s).
>
> But this is fine?
>
> We only need to ensure that we have a full GP pass between the "last"
> rcu_sync_exit() and GP_XXX -> GP_IDLE transition.
>
> > However, we had 3 rcu_sync_exit()s, not 2. In other words, the
> > rcu_sync_exit() got batched.
> >
> > So my point here is, rcu_sync_exit() does not really always cause a new
> > GP to happen
>
> See above, it should not.
>
> > Then what is the point of the GP_REPLAY state at all if it does not
> > always wait for a new GP?
>
> Again, I don't understand... GP_REPLAY ensures that we will have a full GP
> before rcu_sync_func() sets GP_IDLE, note that it does another "recursive"
> call_rcu() if it sees GP_REPLAY.

I finally got back to this (meanwhile life, job things happened). You
are right, only the last one needs a full GP and it does get one here.
Probably a comment in rcu_sync_exit() explaining this might help the
future reader.

Basically you are saying, if rcu_sync_exit() happens and GP_REPLAY is
already set, we need not worry about starting a new GP because
GP_REPLAY->GP_EXIT->GP_IDLE transition will involve a full GP anyway.
And only if, GP_EXIT is already set, then we must set GP_REPLAY and
wait for a full GP.  This ensures the last rcu_sync_exit() gets a full
GP. I think that was what I was missing. Some reason I thought that
every rcu_sync_exit() needs to start a full GP.

thanks!

- Joel