Received: by 2002:a05:6a10:a852:0:0:0:0 with SMTP id d18csp1993124pxy; Sun, 2 May 2021 08:29:38 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxjkBhRvR68WZuuF69V8YviLVIKDEGh6cLP8tA4HSLFovolYCN6q2jdA9S5htVwpr+XnEbm X-Received: by 2002:a05:6a00:ac6:b029:27b:5d2:6e66 with SMTP id c6-20020a056a000ac6b029027b05d26e66mr14525296pfl.14.1619969378592; Sun, 02 May 2021 08:29:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619969378; cv=none; d=google.com; s=arc-20160816; b=nBaR/JQ/t1x7EJSMg6uJlt4sDpraT9dCDNb0bGj2MPTdJhUwncrru8J5tFKPd1PyU0 28VYg+DNxz3oehRFe5h6KCWwA98/x56g5a7Y1isfIUG/3Z/4Y6RGB2jqphj4j6uuzWyC YBHG5sAm/asSqHo651yJ4uzdPjQQDf8pCaHpqyuEzSjAzu+ZQHck7177tj94ZqPcBs4Z HrqsoLgAX8H/Yr4UqE6lyPBWtLfovA50hIZtgwLftoEsxJSLnsSToqzJGStogjKLqjFh 8LBayLrS/eDnopP2dkXwsy3rkDN3HmA3k4GTp7FuvGqA6yblPc4P92B6fA8c4UFV0f5m A71w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:subject:message-id:date:from:in-reply-to :references:mime-version; bh=2Yo5lJFkseirK7+Os1dsvmENxr6gMP+2YIPJxFmkXVk=; b=pqd0+4ZsQzGk0ddOgY06ObSx1EC+mryA1NtMJ4+nl06zLehbdsyEYOwJ5pSzvMYizG gZrVR9S7c6X2X/0FrNU7WMeQme+tssKIx6PAoZ5CTzrW8nVbhtbGTnRt7lr+CfAORXaI Oj57vzQ3wLt3gvhFdydXKjALvVMP6NgIzDzsB/Xlt40iOrWfAoutmLc0tm5uu3j8Gr+D +1rETR57l+4ejlXY5DjmupbdnelMw84IzaEDVH2h2rYNJ8SaHmpxLjougwk6iYydgzQU jluJQqt5pZbsG7xAcgiKUVgL3Xgv86y8Kka3m0GYxjJYVNAF7h1TfZAK7idMCoeO8ZAM IJAw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id v67si10526658pfc.297.2021.05.02.08.29.23; Sun, 02 May 2021 08:29:38 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231598AbhEBP2b (ORCPT + 99 others); Sun, 2 May 2021 11:28:31 -0400 Received: from mail-ed1-f47.google.com ([209.85.208.47]:33367 "EHLO mail-ed1-f47.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230110AbhEBP2a (ORCPT ); Sun, 2 May 2021 11:28:30 -0400 Received: by mail-ed1-f47.google.com with SMTP id b17so487794ede.0; Sun, 02 May 2021 08:27:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=2Yo5lJFkseirK7+Os1dsvmENxr6gMP+2YIPJxFmkXVk=; b=L27R5QPUSfus9OCUuTfdlxzesCCOAJ9/gKhCTFxF7sS1wAggv5IHd5r+DBqOjXB3Jb 7T7rPHJz6XWoId1dl64LZEidIugrfT4uTDijvaN6hkYQjIIHp6fUUbRixtyQ8JD8pz1O mmKUz0hPGg2oEJUjn5xi+s95Hx+ZK9QSqo3ejsSEhw2nK8dgkvfldiWuzL+Dok36k1EN OkeJA9uy6eeOcH+T55EOhJzRNgK+70ysd8nb2+QVgJFSy+Zr9r4OOy731ag4E52cpTeA jTxrhEbV6i+D6GteomWpmaC9kABntV/rx7NMaPxoERSeVEcQYaB4MaCQGHN9BHH0BD2E HeNA== X-Gm-Message-State: AOAM5329xl4Qd+UFBwbz8NLe68oVySQIhPmq7FGmhXdpK2r4BuYoCGDc 7yoBckiCbZhWVJ6zRmSHh56cfTl8+Kts9H9KwQVdho8A X-Received: by 2002:a05:6402:3079:: with SMTP id bs25mr16173762edb.146.1619969258275; Sun, 02 May 2021 08:27:38 -0700 (PDT) MIME-Version: 1.0 References: <20210415044258.GA6318@zn.tnic> <20210415052938.GA2325@1wt.eu> <20210415054713.GB6318@zn.tnic> <20210419141454.GE9093@zn.tnic> <20210419191539.GH9093@zn.tnic> <20210419215809.GJ9093@zn.tnic> In-Reply-To: From: Len Brown Date: Sun, 2 May 2021 11:27:26 -0400 Message-ID: Subject: Re: Candidate Linux ABI for Intel AMX and hypothetical new related features To: Borislav Petkov Cc: Willy Tarreau , Andy Lutomirski , Florian Weimer , "Bae, Chang Seok" , Dave Hansen , X86 ML , LKML , linux-abi@vger.kernel.org, "libc-alpha@sourceware.org" , Rich Felker , Kyle Huey , Keno Fischer Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Apr 23, 2021 at 3:58 PM Borislav Petkov wrote: > > On Fri, Apr 23, 2021 at 03:35:30PM -0400, Len Brown wrote: > > Yes. If a library decides to execute AMX instructions on behalf > > of a task, the kernel will allocate an 8KB context switch buffer > > on behalf of that task. > > Again, the library should ask the kernel first whether it supports AMX. > And the process should decide whether to use AMX - not the library on > its own, on behalf of the process. > > > Granted, if you find a reason to dislike AMX, the mechanisms to disable > > it today are on a system-wide basis, not on a process or task basis. > > Again, I don't dislike the feature. I don't want libraries jumping on > new features without asking the process or the kernel first especially > when those features have performance implications and need kernel > support. Here is how it works: 1. The kernel boots and sees the feature in CPUID. 2. If the kernel supports that feature, it sets XCR0[feature]. For some features, there may be a bunch of kernel support, while simple features may require only state save/restore. 2a. If the kernel doesn't support the feature, XCR0[feature] remains cleared. 3. user-space sees the feature in CPUID 4. user-space sees for the feature via xgetbv[XCR0] 5. If the feature is enabled in XCR0, the user happily uses it. For AMX, Linux implements "transparent first use" so that it doesn't have to allocate 8KB context switch buffers for tasks that don't actually use AMX. It does this by arming XFD for all tasks, and taking a #NM to allocate a context switch buffer only for those tasks that actually execute AMX instructions. 5a. If the feature is not enabled in XCR0, and the tasks uses those instructions anway, the hardware delivers a #UD and the kernel kills the process with a signal. So you already have what you want, WRT user-space being required to ask the kernel if the feature is supported. When the kernel sets XCR0[feature], it tells user-space that the kernel supports the feature; and there is no way that user space can use the feature if the kernel did not set that bit. System calls before (and after) using a feature are not necessary, and would only degrade performance over the existing ABI. Regarding performance implications... The system administrator is empowered to enable or disable a feature in BIOS (clears CPUID bit) or in the kernel (clears XCR0 bit) if they don't like it. Yes, this is a per-system decision, rather than a per-process or a per-thread decision. So the only thing you don't have that you asked for is a way for the main process to control what instructions are used by the libraries that it links with. That one is above my pay grade. Does the application have a *choice* of which libraries they link with? cheers, Len Brown, Intel Open Source Technology Center