Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20221216185012.2342675-1-abdulras@google.com> <CAEUhbmUsFt0H-Dd3XHjuk6sUq0hrV9SFU=1jD2LQ10xywZOZiA@mail.gmail.com>
 <CAO8XFHv_wV_oSuS+e412yr6vjm-44Vf2gJuMO4FvgPeodhWQww@mail.gmail.com>
 <CAEUhbmX6PE_E8PCEp4mo8Fpjtc-Pm-tmpxws0UKCp715qB+qRg@mail.gmail.com>
 <CAO8XFHvrbj86gcBpN7z7n_cGk2wLHND=t0k+QtYgzZGMQOLpsA@mail.gmail.com> <CAEUhbmW-EH4tQr4sXQAJ6iuwO31iWEE=XQXpTMVBp8fX66LDxg@mail.gmail.com>
In-Reply-To: <CAEUhbmW-EH4tQr4sXQAJ6iuwO31iWEE=XQXpTMVBp8fX66LDxg@mail.gmail.com>
From:   Saleem Abdulrasool <abdulras@google.com>
Date:   Tue, 27 Dec 2022 07:24:00 -0800
Message-ID: <CAO8XFHsjciBKho-9Kzw7v8anBZm1+DD5ZnMLmhZNxUUYZTU-FQ@mail.gmail.com>
Subject: Re: [PATCH] riscv: avoid enabling vectorized code generation
To:     Bin Meng <bmeng.cn@gmail.com>
Cc:     Paul Walmsley <paul.walmsley@sifive.com>,
        Palmer Dabbelt <palmer@dabbelt.com>,
        Albert Ou <aou@eecs.berkeley.edu>,
        linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Thu, Dec 22, 2022 at 10:57 PM Bin Meng <bmeng.cn@gmail.com> wrote:
>
> Hi,
>
> On Thu, Dec 22, 2022 at 11:23 PM Saleem Abdulrasool <abdulras@google.com> wrote:
> >
> > On Thu, Dec 22, 2022 at 1:41 AM Bin Meng <bmeng.cn@gmail.com> wrote:
> > >
> > > Hi,
> > >
> > > On Thu, Dec 22, 2022 at 1:39 AM Saleem Abdulrasool <abdulras@google.com> wrote:
> > > >
> > > > On Wed, Dec 21, 2022 at 8:17 AM Bin Meng <bmeng.cn@gmail.com> wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > On Sat, Dec 17, 2022 at 3:12 AM Saleem Abdulrasool <abdulras@google.com> wrote:
> > > > > >
> > > > > > The compiler is free to generate vectorized operations for zero'ing
> > > > > > memory.  The kernel does not use the vector unit on RISCV, similar to
> > > > > > architectures such as x86 where we use `-mno-mmx` et al to prevent the
> > > > > > implicit vectorization.  Perform a similar check for
> > > > > > `-mno-implicit-float` to avoid this on RISC-V targets.
> > > > > >
> > > > > > Signed-off-by: Saleem Abdulrasool <abdulras@google.com>
> > > > > > ---
> > > > > >  arch/riscv/Makefile | 4 ++++
> > > > > >  1 file changed, 4 insertions(+)
> > > > > >
> > > > > > diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile
> > > > > > index 0d13b597cb55..68433476a96e 100644
> > > > > > --- a/arch/riscv/Makefile
> > > > > > +++ b/arch/riscv/Makefile
> > > > > > @@ -89,6 +89,10 @@ KBUILD_AFLAGS_MODULE += $(call as-option,-Wa$(comma)-mno-relax)
> > > > > >  # architectures.  It's faster to have GCC emit only aligned accesses.
> > > > > >  KBUILD_CFLAGS += $(call cc-option,-mstrict-align)
> > > > > >
> > > > > > +# Ensure that we do not vectorize the kernel code when the `v` extension is
> > > > > > +# enabled.  This mirrors the `-mno-mmx` et al on x86.
> > > > > > +KBUILD_CFLAGS += $(call cc-option,-mno-implicit-float)
> > > > >
> > > > > This looks like an LLVM flag, but not GCC.
> > > >
> > > > Correct, this is a clang flag, though I imagine that GCC will need a
> > > > similar flag once it receives support for the V extension.
> > > >
> > > > > Can you elaborate what exact combination (compiler flag and source)
> > > > > would cause an issue?
> > > >
> > > > The particular case that I was using was simply `clang -target
> > > > riscv64-unknown-linux-musl -march=rv64gcv` off of main.
> > > >
> > > > > From your description, I guess it's that when enabling V extension in
> > > > > LLVM, the compiler tries to use vector instructions to zero memory,
> > > > > correct?
> > > >
> > > > Correct.
> > >
> > > Thanks for the confirmation.
> > >
> > > >
> > > > > Can you confirm LLVM does not emit any float instructions (like F/D
> > > > > extensions) because the flag name suggests something like "float"?
> > > >
> > > > The `-mno-implicit-float` should disable any such emission.  I assume
> > > > that you are worried about the case without the flag?  I'm not 100%
> > > > certain without this flag, but the RISCV build with this flag has been
> > > > running smoothly locally for a while.
> > > >
> > > >
> > >
> > > I still have some questions about the `-mno-implicit-float` option's behavior.
> > >
> > > - If this option is not on, does the compiler emit any F/D extension
> > > instruction for zero'ing memory when -march=rv64g? I want to know
> > > whether the `-mno-implicit-float` option only takes effect when "v"
> > > appears on the -march string.
> >
> > AFAIK, and from a quick test, no, it will not.  That also makes sense
> > since the F/D/Q handling is not as likely to be useful for generating
> > a 0-filled array.  No, the use of `-mno-implicit-float` is not guarded
> > by the use of the vector extension, but it does only impact the
> > vectorized code generation (the loop vectorizer, load/store
> > vectorizer, and SLP vectorizer).
>
> Thank you. The quick test you did seems to match what the LLVM commit [1] says:
>
>     "It also disables implicit uses of scalar FP, but I don't know if
> we have any of those for RISC-V."
>
> [1] https://github.com/llvm/llvm-project/commit/549231d38e10de7371adb85f5452d42ad42f4201
>
> >
> > > - If the answer to the above question is no, I wonder why the option
> > > is called `-mno-implicit-float` as float suggests the FPU usage, but
> > > actually it is about vectorization. The Clang documentation says
> > > almost nothing about this option.
> >
> > The flag itself is from GCC, it was added for the ARM architecture, to
> > prefer using the scalar core over the VFP register set as ARM uses the
> > VFP for vectorized operations.  As it so happens, internally in LLVM,
> > the loop vectorizer uses the (internal) `NoImplicitFloat` function
> > attribute to prevent the loop from being vectorized, and the flag that
> > controls this is exposed as `-mimplicit-float` and
> > `-mno-implicit-float`.
> >
>
> It seems GCC does not have such a flag. Thanks for the history
> introduction. It was introduced on Arm to disable vectorized operation
> using VFP, hence it was named as -no-implict-float. But IMHO the
> option is badly named. Maybe -no-implicit-vectorization better fits
> what it really does.

The option is present on ARM GCC, but not RISC-V GCC.  Sure, the
option could be better named - personally, I'd prefer
`-mgeneral-regs-only` to match the x86 convention which leaves it
sufficiently generalised that future extensions would easily fit into
the behavioural control.  Much like the Linux kernel's prime
directive: "we do not break userspace'', the LLVM toolchain has a
similar view point: options which have shipped are considered
permanent.  Even if renamed, it would be an alias and the old option
sticks around in near perpetuity.

> FWIW,
> Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
>
> Regards,
> Bin