Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <20210415044258.GA6318@zn.tnic> <20210415052938.GA2325@1wt.eu>
 <20210415054713.GB6318@zn.tnic> <CAJvTdKnjzAMh3N_c7KP3kA=e0LgYHgCANg44oJp3LcSm7dtbSQ@mail.gmail.com>
 <20210419141454.GE9093@zn.tnic> <CAJvTdK=p8mgO3xw9sRxu0c7NTNTG109M442b3UZh8TqLLfkC1Q@mail.gmail.com>
 <20210419191539.GH9093@zn.tnic> <CAJvTdK=VnG94ECcRVoUi8HrCbVEKc8X4_JmRTkqe+vTttf0Wsg@mail.gmail.com>
 <20210419215809.GJ9093@zn.tnic> <CAJvTdKn6JHo02karEs0e5g+6SimS5VUcXKjCkX35WY+xkgAgxw@mail.gmail.com>
 <YIMmwhEr46VPAZa4@zn.tnic> <CAJvTdKnhXnynybS4eNEF_EtF26auyb-mhKLNd1D9_zvCrchZsw@mail.gmail.com>
 <87bl9s8kfb.fsf@oldenburg.str.redhat.com> <5d3d513b-77d6-e2e2-779e-ff3ea33deba3@intel.com>
 <87o8dmmljh.ffs@nanos.tec.linutronix.de>
In-Reply-To: <87o8dmmljh.ffs@nanos.tec.linutronix.de>
From:   Andy Lutomirski <luto@kernel.org>
Date:   Fri, 7 May 2021 11:50:20 -0700
Message-ID: <CALCETrWG7E2LCSyvaay05R4Sa3mU_t5C5W4rYGuA3+dzaBF9tQ@mail.gmail.com>
Subject: Re: Candidate Linux ABI for Intel AMX and hypothetical new related features
To:     Thomas Gleixner <tglx@linutronix.de>
Cc:     Dave Hansen <dave.hansen@intel.com>,
        Florian Weimer <fweimer@redhat.com>,
        Len Brown <lenb@kernel.org>, Borislav Petkov <bp@alien8.de>,
        Willy Tarreau <w@1wt.eu>, Andy Lutomirski <luto@kernel.org>,
        "Bae, Chang Seok" <chang.seok.bae@intel.com>,
        X86 ML <x86@kernel.org>, LKML <linux-kernel@vger.kernel.org>,
        linux-abi@vger.kernel.org,
        "libc-alpha@sourceware.org" <libc-alpha@sourceware.org>,
        Rich Felker <dalias@libc.org>, Kyle Huey <me@kylehuey.com>,
        Keno Fischer <keno@juliacomputing.com>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Fri, May 7, 2021 at 11:44 AM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> On Mon, May 03 2021 at 06:43, Dave Hansen wrote:
> > On 5/2/21 10:18 PM, Florian Weimer wrote:
> >>> 5. If the feature is enabled in XCR0, the user happily uses it.
> >>>
> >>>     For AMX, Linux implements "transparent first use"
> >>>     so that it doesn't have to allocate 8KB context switch
> >>>     buffers for tasks that don't actually use AMX.
> >>>     It does this by arming XFD for all tasks, and taking a #NM
> >>>     to allocate a context switch buffer only for those tasks
> >>>     that actually execute AMX instructions.
> >> What happens if the kernel cannot allocate that additional context
> >> switch buffer?
> >
> > Well, it's vmalloc()'d and currently smaller that the kernel stack,
> > which is also vmalloc()'d.  While it can theoretically fail, if it
> > happens you have bigger problems on your hands.
>
> Such a buffer allocation might also exceed a per process/cgroup
> limitation. Anything else which is accounted happens in syscall context
> which then returns an error on which the application can react.
>
> So what's the consequence when the allocation fails? Kill it right away
> from #NM? Kill it on the first signal? Do nothing and see what happens?
>

It has to be an immediate signal or kill.  A failure to load FPU state
is somewhat tolerable (and has to be for CET), but a failure to *save*
FPU state on a context switch would be a really nasty can of worms.
At the very least we will want arch_prctl(ARCH_ALLOCTE_XSTATE, mask)
to allow HPC workloads to manually allocate the state and get an error
code if it fails.

--Andy