Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <20190221221941.29358-1-daniel@iogearbox.net> <tip-ce02ef06fcf7a399a6276adb83f37373d10cbbe1@git.kernel.org>
 <33bf951448e7d916fd4a6ad41cd3d040e9d1f118.camel@infradead.org>
 <CAMe9rOrcOVRWuNWy3xp7NMD=2UbGH+C5zifDhd=-AxrrzEVrmw@mail.gmail.com>
 <79add9a9-543b-a791-ecbe-79edd49f1bb3@iogearbox.net> <CAMe9rOryEsLHossVa=QmaKW1wbW67AOnewkB_RN_WkveBEwnZg@mail.gmail.com>
 <4604e680-7962-f1ee-5b79-711247f4e7d5@iogearbox.net>
In-Reply-To: <4604e680-7962-f1ee-5b79-711247f4e7d5@iogearbox.net>
From:   "H.J. Lu" <hjl.tools@gmail.com>
Date:   Thu, 28 Feb 2019 10:09:43 -0800
Message-ID: <CAMe9rOotJ+s0nR58Si3F_X8V6OcZKB8+q8+wORQ7C1YT1Nx+DQ@mail.gmail.com>
Subject: Re: [tip:x86/build] x86, retpolines: Raise limit for generating
 indirect calls from switch-case
To:     Daniel Borkmann <daniel@iogearbox.net>
Cc:     David Woodhouse <dwmw2@infradead.org>,
        Ingo Molnar <mingo@kernel.org>, bjorn.topel@intel.com,
        David Miller <davem@davemloft.net>, brouer@redhat.com,
        magnus.karlsson@intel.com, Andy Lutomirski <luto@kernel.org>,
        "H. Peter Anvin" <hpa@zytor.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Peter Zijlstra <peterz@infradead.org>,
        Borislav Petkov <bp@alien8.de>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        LKML <linux-kernel@vger.kernel.org>, ast@kernel.org,
        linux-tip-commits@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Thu, Feb 28, 2019 at 9:58 AM Daniel Borkmann <daniel@iogearbox.net> wrot=
e:
>
> On 02/28/2019 05:25 PM, H.J. Lu wrote:
> > On Thu, Feb 28, 2019 at 8:18 AM Daniel Borkmann <daniel@iogearbox.net> =
wrote:
> >> On 02/28/2019 01:53 PM, H.J. Lu wrote:
> >>> On Thu, Feb 28, 2019 at 3:27 AM David Woodhouse <dwmw2@infradead.org>=
 wrote:
> >>>> On Thu, 2019-02-28 at 03:12 -0800, tip-bot for Daniel Borkmann wrote=
:
> >>>>> Commit-ID:  ce02ef06fcf7a399a6276adb83f37373d10cbbe1
> >>>>> Gitweb:     https://git.kernel.org/tip/ce02ef06fcf7a399a6276adb83f3=
7373d10cbbe1
> >>>>> Author:     Daniel Borkmann <daniel@iogearbox.net>
> >>>>> AuthorDate: Thu, 21 Feb 2019 23:19:41 +0100
> >>>>> Committer:  Thomas Gleixner <tglx@linutronix.de>
> >>>>> CommitDate: Thu, 28 Feb 2019 12:10:31 +0100
> >>>>>
> >>>>> x86, retpolines: Raise limit for generating indirect calls from swi=
tch-case
> >>>>>
> >>>>> From networking side, there are numerous attempts to get rid of ind=
irect
> >>>>> calls in fast-path wherever feasible in order to avoid the cost of
> >>>>> retpolines, for example, just to name a few:
> >>>>>
> >>>>>   * 283c16a2dfd3 ("indirect call wrappers: helpers to speed-up indi=
rect calls of builtin")
> >>>>>   * aaa5d90b395a ("net: use indirect call wrappers at GRO network l=
ayer")
> >>>>>   * 028e0a476684 ("net: use indirect call wrappers at GRO transport=
 layer")
> >>>>>   * 356da6d0cde3 ("dma-mapping: bypass indirect calls for dma-direc=
t")
> >>>>>   * 09772d92cd5a ("bpf: avoid retpoline for lookup/update/delete ca=
lls on maps")
> >>>>>   * 10870dd89e95 ("netfilter: nf_tables: add direct calls for all b=
uiltin expressions")
> >>>>>   [...]
> >>>>>
> >>>>> Recent work on XDP from Bj=C3=B6rn and Magnus additionally found th=
at manually
> >>>>> transforming the XDP return code switch statement with more than 5 =
cases
> >>>>> into if-else combination would result in a considerable speedup in =
XDP
> >>>>> layer due to avoidance of indirect calls in CONFIG_RETPOLINE enable=
d
> >>>>> builds.
> >>>>
> >>>> +HJL
> >>>>
> >>>> This is a GCC bug, surely? It should know how expensive each
> >>>> instruction is, and choose which to use accordingly. That should be
> >>>> true even when the indirect branch "instruction" is a retpoline, and
> >>>> thus enormously expensive.
> >>>>
> >>>> I believe this is https://gcc.gnu.org/bugzilla/show_bug.cgi?id=3D869=
52 so
> >>>> please at least reference that bug, and be prepared to turn this hac=
k
> >>>> off when GCC is fixed.
> >>>
> >>> We couldn't find a testcase to show jump table with indirect branch
> >>> is slower than direct branches.
> >>
> >> Ok, I've just checked https://github.com/marxin/microbenchmark/tree/re=
tpoline-table
> >> with the below on top.
> >>
> >>  Makefile | 6 +++---
> >>  switch.c | 2 +-
> >>  test.c   | 6 ++++--
> >>  3 files changed, 8 insertions(+), 6 deletions(-)
> >>
> >> diff --git a/Makefile b/Makefile
> >> index bd83233..ea81520 100644
> >> --- a/Makefile
> >> +++ b/Makefile
> >> @@ -1,16 +1,16 @@
> >>  CC=3Dgcc
> >>  CFLAGS=3D-g -I.
> >> -CFLAGS+=3D-O2 -mindirect-branch=3Dthunk
> >> +CFLAGS+=3D-O2 -mindirect-branch=3Dthunk-inline -mindirect-branch-regi=
ster
> > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> >
> > Does slowdown show up only with -mindirect-branch=3Dthunk-inline?
>
> Not really, numbers are in similar range / outcome. Additionally, I also =
tried
> on a bit bigger machine (Xeon Gold 5120 this time). First is thunk-inline=
, second
> is thunk, and third is w/o raising limit for comparison; first test (from=
 last
> mail) on that machine:

Please re-open:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=3D86952

with new info.

--=20
H.J.