Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20220131225250.409564-1-ndesaulniers@google.com>
In-Reply-To: <20220131225250.409564-1-ndesaulniers@google.com>
From:   Ard Biesheuvel <ardb@kernel.org>
Date:   Tue, 1 Feb 2022 10:32:11 +0100
Message-ID: <CAMj1kXHz9psgjP7qQpusLOOL5Nm7TO+LauD_-mK=Fxe_g7mmsQ@mail.gmail.com>
Subject: Re: [PATCH] docs/memory-barriers.txt: volatile is not a barrier() substitute
To:     Nick Desaulniers <ndesaulniers@google.com>
Cc:     Jonathan Corbet <corbet@lwn.net>,
        Josh Poimboeuf <jpoimboe@redhat.com>,
        Borislav Petkov <bp@alien8.de>,
        Peter Zijlstra <peterz@infradead.org>, llvm@lists.linux.dev,
        Kees Cook <keescook@chromium.org>,
        Alan Stern <stern@rowland.harvard.edu>,
        Andrea Parri <parri.andrea@gmail.com>,
        Will Deacon <will@kernel.org>,
        Boqun Feng <boqun.feng@gmail.com>,
        Nicholas Piggin <npiggin@gmail.com>,
        David Howells <dhowells@redhat.com>,
        Jade Alglave <j.alglave@ucl.ac.uk>,
        Luc Maranget <luc.maranget@inria.fr>,
        "Paul E. McKenney" <paulmck@kernel.org>,
        Akira Yokosawa <akiyks@gmail.com>,
        Daniel Lustig <dlustig@nvidia.com>,
        Joel Fernandes <joel@joelfernandes.org>,
        "Gustavo A. R. Silva" <gustavoars@kernel.org>,
        Len Baker <len.baker@gmx.com>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        linux-arch <linux-arch@vger.kernel.org>,
        Linux Doc Mailing List <linux-doc@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Mon, 31 Jan 2022 at 23:53, Nick Desaulniers <ndesaulniers@google.com> wrote:
>
> Add text to memory-barriers.txt and deprecated.rst to denote that
> volatile-qualifying an asm statement is not a substitute for either a
> compiler barrier (``barrier();``) or a clobber list.
>
> This way we can point to this in code that strengthens existing
> volatile-qualified asm statements to use a compiler barrier.
>
> Suggested-by: Kees Cook <keescook@chromium.org>
> Signed-off-by: Nick Desaulniers <ndesaulniers@google.com>
> ---
> Example: https://godbolt.org/z/8PW549zz9
>
>  Documentation/memory-barriers.txt    | 24 ++++++++++++++++++++++++
>  Documentation/process/deprecated.rst | 17 +++++++++++++++++
>  2 files changed, 41 insertions(+)
>
> diff --git a/Documentation/memory-barriers.txt b/Documentation/memory-barriers.txt
> index b12df9137e1c..f3908c0812da 100644
> --- a/Documentation/memory-barriers.txt
> +++ b/Documentation/memory-barriers.txt
> @@ -1726,6 +1726,30 @@ of optimizations:
>       respect the order in which the READ_ONCE()s and WRITE_ONCE()s occur,
>       though the CPU of course need not do so.
>
> + (*) Similarly, the compiler is within its rights to reorder instructions

Similar to what? Was this intended to be the second bullet point
rather than the first?

> +     around an asm statement so long as clobbers are not violated. For example,
> +
> +       asm volatile ("");
> +       flag = true;
> +
> +     May be modified by the compiler to:
> +
> +       flag = true;
> +       asm volatile ("");
> +
> +     Marking an asm statement as volatile is not a substitute for barrier(),
> +     and is implicit for asm goto statements and asm statements that do not
> +     have outputs (like the above example). Prefer either:
> +
> +       asm ("":::"memory");
> +       flag = true;
> +
> +     Or:
> +
> +       asm ("");
> +       barrier();
> +       flag = true;
> +

I would expect the memory clobber to only hazard against the
assignment of flag if it results in a store, but looking at your
Godbolt example, this appears to apply even if flag is kept in a
register.

Is that behavior documented/codified anywhere? Or are we relying on
compiler implementation details here?


>   (*) The compiler is within its rights to invent stores to a variable,
>       as in the following example:
>
> diff --git a/Documentation/process/deprecated.rst b/Documentation/process/deprecated.rst
> index 388cb19f5dbb..432816e2f79e 100644
> --- a/Documentation/process/deprecated.rst
> +++ b/Documentation/process/deprecated.rst
> @@ -329,3 +329,20 @@ struct_size() and flex_array_size() helpers::
>          instance->count = count;
>
>          memcpy(instance->items, source, flex_array_size(instance, items, instance->count));
> +
> +Volatile Qualified asm Statements
> +=================================
> +
> +According to `the GCC docs on inline asm
> +https://gcc.gnu.org/onlinedocs/gcc/Extended-Asm.html#Volatile`_:
> +
> +  asm statements that have no output operands and asm goto statements,
> +  are implicitly volatile.
> +
> +For many uses of asm statements, that means adding a volatile qualifier won't
> +hurt (making the implicit explicit), but it will not strengthen the semantics
> +for such cases where it would have been implied. Care should be taken not to
> +confuse ``volatile`` with the kernel's ``barrier()`` macro or an explicit
> +clobber list. See [memory-barriers]_ for more info on ``barrier()``.
> +
> +.. [memory-barriers] Documentation/memory-barriers.txt
> --
> 2.35.0.rc2.247.g8bbb082509-goog
>