Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <CALCETrW2QHa2TLvnUuVxAAheqcbSZ-5_WRXtDSAGcbG8N+gtdQ@mail.gmail.com>
 <87lf9nk2ku.fsf@oldenburg.str.redhat.com> <CALCETrWxJzf-rm9rqMpdxEtdVe+0OH7XRtWV=UzrgBDiPT=vVQ@mail.gmail.com>
 <CAJvTdKkAzEeAKrEYMU-gBWXoNGyJ09ZGw1gsU0b3uCuo8vrX0A@mail.gmail.com>
 <CALCETrVvapzL79BQNEvOupMHHzriR+n97955tRA+TPE6rgRC4Q@mail.gmail.com>
 <CAJvTdKmDb3TOHbb4w1YNcdYH2Pzr+RsAS_o0q3AFP1Xz55R37g@mail.gmail.com>
 <CALCETrUe4_DvSdq7CRJsUxZm8DSwfHevagxJEbHv88UjcC1Hkg@mail.gmail.com>
 <CAJvTdKmppOcyy=akSo3+dn0KN5GmOKLK5cp3eZ_ymujE-=ExwQ@mail.gmail.com> <CALCETrUxc1Y=4XmNthSSCeQfJ_SKNS762Nt3B-CynE-rhRSBGw@mail.gmail.com>
In-Reply-To: <CALCETrUxc1Y=4XmNthSSCeQfJ_SKNS762Nt3B-CynE-rhRSBGw@mail.gmail.com>
From:   Len Brown <lenb@kernel.org>
Date:   Fri, 16 Apr 2021 17:54:43 -0400
Message-ID: <CAJvTdKmDskNka9tm5TMZkkP8tYa-K01sz8hAMccRSaAYKNPz5Q@mail.gmail.com>
Subject: Re: Candidate Linux ABI for Intel AMX and hypothetical new related features
To:     Andy Lutomirski <luto@kernel.org>
Cc:     Willy Tarreau <w@1wt.eu>, Florian Weimer <fweimer@redhat.com>,
        "Bae, Chang Seok" <chang.seok.bae@intel.com>,
        Dave Hansen <dave.hansen@intel.com>, X86 ML <x86@kernel.org>,
        LKML <linux-kernel@vger.kernel.org>, linux-abi@vger.kernel.org,
        "libc-alpha@sourceware.org" <libc-alpha@sourceware.org>,
        Rich Felker <dalias@libc.org>, Kyle Huey <me@kylehuey.com>,
        Keno Fischer <keno@juliacomputing.com>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Thu, Apr 15, 2021 at 12:24 PM Andy Lutomirski <luto@kernel.org> wrote:
> On Wed, Apr 14, 2021 at 2:48 PM Len Brown <lenb@kernel.org> wrote:

> > > ... the transition penalty into and out of AMX code

The concept of 'transition' exists between AVX and SSE instructions
because it is possible to mix both instruction sets and touch different
parts of the same registers.  The "unused" parts of those registers
need to be tracked to assure that data is not lost when mixing.

This concept is moot with AMX, which has its own dedicated registers.

> What is the actual impact of a trivial function that initializes the
> tile config, does one tiny math op, and then does TILERELEASE?

1. Task takes #NM on first touch of TILE registers
2. Kernel allocates 8KB for that task and dis-arms XFD
3. Kernel context switches XFD with task state

If the task takes a signal *before* TILERELEASE
4. XSAVE transfers AMX state to signal stack, XRESTOR the reverse.

If the task context switches *before* TILERELEASE
5. kernel context switch XSAVES the AMX state to 8KB context switch
buffer, XRESTORE the reverse.

If the task takes a signal *after* TILERELEASE
4. XSAVE does NOT transfer AMX state (or zeros) to signal stack, 8KB
is consumed on signal stack but not touched.  XRESTOR, the reverse.

If the task context switches *after* TILERELEASE
5. kernel contexts switch ignores INIT=1 AMX state.  8KB buffer is quiescent.

As we discussed previously,  there is no impact to frequency from
either INIT=0 or INIT=1 AMX state.
Frequency is impacted by *compute*, and since there isn't any compute
this scenario, there is no frequency impact.

As we discussed previously, for INIT=1 (which the kernel guarantees,
there is also no impact on power.

thanks,
Len Brown, Intel Open Source Technology Center