Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <20190121131816.GC17749@hirez.programming.kicks-ass.net>
 <Pine.LNX.4.44L0.1901211103000.24162-100000@netrider.rowland.org> <20190122090456.GE13777@hirez.programming.kicks-ass.net>
In-Reply-To: <20190122090456.GE13777@hirez.programming.kicks-ass.net>
From:   Kees Cook <keescook@chromium.org>
Date:   Wed, 23 Jan 2019 12:22:44 +1300
Message-ID: <CAGXu5jKHo4CZWAnn4Wvg=MV=xaf3j4mBvxggBHuZETTkMOxkOA@mail.gmail.com>
Subject: Re: [PATCH] kcov: convert kcov.refcount to refcount_t
To:     Peter Zijlstra <peterz@infradead.org>
Cc:     Alan Stern <stern@rowland.harvard.edu>,
        Dmitry Vyukov <dvyukov@google.com>,
        Elena Reshetova <elena.reshetova@intel.com>,
        Andrea Parri <andrea.parri@amarulasolutions.com>,
        "Paul E. McKenney" <paulmck@linux.ibm.com>,
        Will Deacon <will.deacon@arm.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        Andrey Ryabinin <aryabinin@virtuozzo.com>,
        Anders Roxell <anders.roxell@linaro.org>,
        Mark Rutland <mark.rutland@arm.com>,
        LKML <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Tue, Jan 22, 2019 at 10:05 PM Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Mon, Jan 21, 2019 at 11:05:03AM -0500, Alan Stern wrote:
> > On Mon, 21 Jan 2019, Peter Zijlstra wrote:
>
> > > Any additional ordering; like the one you have above; are not strictly
> > > required for the proper functioning of the refcount. Rather, you rely on
> > > additional ordering and will need to provide this explicitly:
> > >
> > >
> > >     if (refcount_dec_and_text(&x->rc)) {
> > >             /*
> > >              * Comment that explains what we order against....
> > >              */
> > >             smp_mb__after_atomic();
> > >             BUG_ON(!x->done*);
> > >             free(x);
> > >     }
> > >
> > >
> > > Also; these patches explicitly mention that refcount_t is weaker,
> > > specifically to make people aware of this difference.
> > >
> > > A full smp_mb() (or two) would also be much more expensive on a number
> > > of platforms and in the vast majority of the cases it is not required.
> >
> > How about adding smp_rmb() into refcount_dec_and_test()?  That would
> > give acq+rel semantics, which seems to be what people will expect.  And
> > it wouldn't be nearly as expensive as a full smp_mb().
>
> Yes, that's a very good suggestion.
>
> I suppose we can add smp_acquire__after_ctrl_dep() on the true branch.
> Then it reall does become rel_acq.
>
> A wee something like so (I couldn't find an arm64 refcount, even though
> I have distinct memories of talk about it).

In the end, arm and arm64 chose to use REFCOUNT_FULL unconditionally,
so there's no arch-specific implementation.

> This isn't compiled, and obviously needs comment/documentation updates
> to go along with it.

Elena, can you do the doc updates?

>
> ---
>  arch/x86/include/asm/refcount.h | 9 ++++++++-
>  lib/refcount.c                  | 7 ++++++-
>  2 files changed, 14 insertions(+), 2 deletions(-)
>
> diff --git a/arch/x86/include/asm/refcount.h b/arch/x86/include/asm/refcount.h
> index dbaed55c1c24..6f7a1eb345b4 100644
> --- a/arch/x86/include/asm/refcount.h
> +++ b/arch/x86/include/asm/refcount.h
> @@ -74,9 +74,16 @@ bool refcount_sub_and_test(unsigned int i, refcount_t *r)
>
>  static __always_inline __must_check bool refcount_dec_and_test(refcount_t *r)
>  {
> -       return GEN_UNARY_SUFFIXED_RMWcc(LOCK_PREFIX "decl",
> +       bool ret = GEN_UNARY_SUFFIXED_RMWcc(LOCK_PREFIX "decl",
>                                         REFCOUNT_CHECK_LT_ZERO,
>                                         r->refs.counter, e, "cx");
> +
> +       if (ret) {
> +               smp_acquire__after_ctrl_dep();
> +               return true;
> +       }
> +
> +       return false;
>  }
>
>  static __always_inline __must_check
> diff --git a/lib/refcount.c b/lib/refcount.c
> index ebcf8cd49e05..8276ad349d48 100644
> --- a/lib/refcount.c
> +++ b/lib/refcount.c
> @@ -190,7 +190,12 @@ bool refcount_sub_and_test_checked(unsigned int i, refcount_t *r)
>
>         } while (!atomic_try_cmpxchg_release(&r->refs, &val, new));
>
> -       return !new;
> +       if (!new) {
> +               smp_acquire__after_ctrl_dep();
> +               return true;
> +       }
> +
> +       return false;
>  }
>  EXPORT_SYMBOL(refcount_sub_and_test_checked);
>

Thanks for this!

-- 
Kees Cook