IronPort-Data: A9a23:q5mbM67ORl3PhOJH+4mGBAxRtDPHchMFZxGqfqrLsTDasY5as4F+v
 mceCzyHa6zcNDTxfdkjaIXjpkgF6JPdz9c2HlNppHs2Hi5G8cbLO4+Ufxz6V8+wwm8vb2o8t
 plDNYOQRCwQZiWBzvt4GuG59RGQ7YnRGvynTraCYnsrLeNdYH9JoQp5nOIkiZJfj9G8Agec0
 fv/uMSaM1K+s9JOGjt8B5mr9VU+7JwehBtC5gZlPasS4weH/5UoJMl3yZ+ZfiOQrrZ8RoZWd
 86bpJml82XQ+QsaC9/Nut4XpWVTH9Y+lSDX4pZnc/DKbipq/0Te4Y5iXBYoUm9Fii3hojxE4
 I4lWapc6+seFvakdOw1C3G0GszlVEFM0OevzXOX6aR/w6BaGpdFLjoH4EweZOUlFuhL7W5my
 f4CBBcdXDO5gPuS3+mLUeV+u8V8I5y+VG8fkikIITDxK98DGcqGaYOToNhS0XE3m9xEGuvYa
 4wBcz1zYR/cYhpJfFAKFJY5m+TujX76G9FagAvN+exrvC6OlUotgdABM/KMEjCObexTklyVu
 STt+GPhDwtBHNee1SCE4jSngeqncSbTAdpJSuPkqaYw6LGV7m83FxgYXmqDmvyGp3WUWdlcM
 kcp2gN7+MDe82TuFLERRSaQon+NuBEHW9NWF+Q7wA6Iw6vQpQ2eAwAsXj9HM5oitec3QyAn0
 hmCmNaBLThutqCFDHGQ7LGZqRusNiUPa2wPfykJSU0C+daLiJFj0DrMQ8xlHarzicf6cRnz3
 TWisicznfMQgNQN2qH9+krI6xqwqZzZZgo04BjLRGWj7xM/aIPNT4mp71fcxexNIIaQUh+Ku
 31ss8KGxOkKDJyLxGqBTY0lELi36rCFOTvHjFhHG5gn6iTr+ni/cIQW6zZ7TG9pKs8say71Z
 1WVsgRUjLdLMWend71fYoS/Ec0mwKHsU9P/WZj8dddHZJp6XACA+y5qaAib2GWFuFAmlqctO
 JGaWdyhAXYTFeJsyz/eb/0Uz75t2GYlxWrVRpnh5xWhzbeaInWSTN843ECma+k46OaOplzT+
 tMHbc+SkU0DAav5fzXd9pMVIRYSN38nCJvqqstRMOmePg5hH2JnAPjUqV88R7FYc21uvr+g1
 hmAtoVwkjITWVWvxd22V01e
IronPort-HdrOrdr: A9a23:9/zsQ6jCuECkAoYyval6CrJwE3BQXuAji2hC6mlwRA09TyVXrb
 HLoB17726TtN91YhodcL+7VJVoLUmyyXcX2+ks1NWZMjUO0VHAROsO0WKI+Vzd8kPFh4tg/J
 YlX69iCMDhSXhW5PyKhzVQyuxQo+VvJprY59vj8w==
Message-ID: <ed318bd6-25b2-efcf-0cc4-c57699f6654a@citrix.com>
Date:   Fri, 17 Mar 2023 09:55:44 +0000
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.8.0
From:   andrew.cooper3@citrix.com
Subject: Re: [PATCH v5 28/34] x86/fred: fixup fault on ERETU by jumping to
 fred_entrypoint_user
Content-Language: en-GB
To:     Lai Jiangshan <jiangshanlai@gmail.com>, Xin Li <xin3.li@intel.com>
Cc:     linux-kernel@vger.kernel.org, x86@kernel.org, kvm@vger.kernel.org,
        tglx@linutronix.de, mingo@redhat.com, bp@alien8.de,
        dave.hansen@linux.intel.com, hpa@zytor.com, peterz@infradead.org,
        seanjc@google.com, pbonzini@redhat.com, ravi.v.shankar@intel.com
References: <20230307023946.14516-1-xin3.li@intel.com>
 <20230307023946.14516-29-xin3.li@intel.com>
 <CAJhGHyC6LgCwdDTkiy2TaQ8wzBQQfrx8ni7fY8vH-bUT2kR8pg@mail.gmail.com>
In-Reply-To: <CAJhGHyC6LgCwdDTkiy2TaQ8wzBQQfrx8ni7fY8vH-bUT2kR8pg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?aThEQ2VTU29NaUJPbno1d0VUUDZ5M2o4cEdFa2tkejlxOW5RSVRkaEZnVmpq?=
 =?utf-8?B?MTZQQzZQek80bHZ3VmVRVzZ4NjBiUjhKMExBTHNkbmNzTHRVZjYxT0RYaWJs?=
 =?utf-8?B?MWVvMmFYRGxTU3ZYMTNwY0FrMGtJRElxMXNteGNKQmM4QU5IbkR6MFB0bW4y?=
 =?utf-8?B?ZWJPUVlqRC9FMXd6Zk93RldjSWpxNTdSazFiS3liL0ZpOEN5S2pNNXhhamJI?=
 =?utf-8?B?T0Erb0FZR0k4K2QrT0xHYnA3dWN4OUpxSmRXc2pKKzBwOFNxYXpaeGg4UGJH?=
 =?utf-8?B?RTY1emFoRWtteXdWaFY2U3ZyYkZGVFN0eGZGU1VjVlJQaUQ1OThqUzZFdGt4?=
 =?utf-8?B?a29lM2dTRks4OCt0bExkcHgvdFZ1Zy96bk51UE9TS05EOVB1MWlhZ0lvT0tU?=
 =?utf-8?B?cnNHV09rTGlxanVxbUhzL2hnRlg1aUMwZktSZjJuSTFvMnBPSk5wZjdIS0Uv?=
 =?utf-8?B?b3E5SG16TkhhRkhab3NqUlBWUEI2dlp1Q1R3alRhdzlYTWlQcjNwN216L1Zh?=
 =?utf-8?B?ZGNJYXplNHRPNFBlSStVYlJWZE9LMkRLTHR4aUt6b2VTRy9sMHV3MnVaQ0ta?=
 =?utf-8?B?U3dCWEtNZWhjTGlzQVBONU5BOVdNMVVBdDVnN3RodHFmWEFaSTdDOGdpaFdi?=
 =?utf-8?B?WWtRdHdBdXByN1JESEZQaVhWUVZMSnhRVDZDZ00rZkVOWXdRNWVtbGtjYWx5?=
 =?utf-8?B?Qm93SkhsSFlJekFKY2dqdG0zcUFHM2U5MFh1K0tTbmdteWNmdDBtUFluNUVG?=
 =?utf-8?B?aEp5WEJ4LzJnVWdIMDR4NWVYbFhhbnVWUnJzejNsOVJ2aWoyRjNKRlZENmRD?=
 =?utf-8?B?dWlYODd2ZmxVQmFjMndqUWtuekJQM3lBWlhYbzFEc3F6L2ZlMUFlM1ZXMDBs?=
 =?utf-8?B?WTc0N05JekJBMUtSUDlqSDNhNXR4dncybitHYk1hb3FIYVRBRm9BN2w5QTV1?=
 =?utf-8?B?alFLa1dYWUw1RG42NmNsTGlRdzVSZlE2czhtY0wwS1U3TnhlUFQ3NHpoTXIx?=
 =?utf-8?B?bCswYmpDVG1JbUN4SG4zNFZ5ekhicHdQc0lTM09mSjF4eDZjOUkxSzJYWWVz?=
 =?utf-8?B?SW5scVRMdkhDaFBmWjhDM1dYOXMwY3Nia1E2VkVrQktWTEp6TmhCQjRnK3N5?=
 =?utf-8?B?c20rNU9ubHhzT3ZCbjgrUTVsMndmVVVKY09IaUcyR21YaEd4bmNYRzBlczZI?=
 =?utf-8?B?TlplcnRIU2I5Z3lWNW56S2o2d2dmWEZmL3VSNy92MTZBR0lUaDdYQzdxUjA3?=
 =?utf-8?B?ME95QzJLNFIyZWkwSHFQSFFWVjREbkJwcUFjdUdKWENpOFMxc2M4endoRktn?=
 =?utf-8?B?MCsvb2hjbER6bTdtN1FFQllaUG1qMTNtYWdQZE0zV3pjVGJXaVVCWC9RaXdv?=
 =?utf-8?B?V3NJeVpIZ1huM1M2ZnBIMnc3Z2ZwODhQZFJvcmZzWS96RDBUckhkU1pydGpo?=
 =?utf-8?B?SXF3REVjRERUTnIyVm4yTG4zMVo0a1VPSGVSUUVaMmxoU1hwN0I2bXF5VjQv?=
 =?utf-8?B?YU1vZlNIS1hJdEtmeS8yaE44VWFEbUtqamg0Y1B0N2tyOXdTTXNSZFB1WnRW?=
 =?utf-8?B?dUl3NVBmQmNxM1hrTmdUWFNqSkJXWW1QMExDdjBkVnBQRkszeEhOb0dydCtO?=
 =?utf-8?B?VmJHV2RYcUo2aXU5TU1tc09XS2NyWGRYRmVyT21zcjVLTGFGZE1IMWUwVHVT?=
 =?utf-8?B?M1RXMC83NWg4NlZycitUVFRXcEhIcWlsZFJNMnRvQnE3MWZURzJkQ1dvSFJq?=
 =?utf-8?B?bmRkUS9yNmtUTHU5a242VVZFUU9IczlPSVFvZWNZbFkxL0JXOVhadUtiVGRX?=
 =?utf-8?B?MnZuRUdsZnArVllnYWcxUkEwOTB0ZVVSYkVGMlQraDNRRjZMdStQS0trcmI0?=
 =?utf-8?B?RXdjK0lMa09URlpCTElLRlByMzRacmxNNEtRcjBNcHBDUHdqMk9wZU5rWmUx?=
 =?utf-8?B?TlhqS0JML0NKMWdHc21CWUZkbEljaDh2aGt4UzM3SjhwWVlBSGtsd3dsOU9H?=
 =?utf-8?B?eUl1VWRtWWdrelZzNHF2d2FYMFBjNjlEdVZ0bFA1czV4cXYwRDdVamVHWndL?=
 =?utf-8?B?Z2paUHVkNHg2TThaWjQ1ZlRCSnNxa0xvTTRJbXRnRGZhWFVRSE9HWFNwbUda?=
 =?utf-8?B?anJBTWtxRFA2WjBZdW81UDUzeGxVUnc2NGpFZzJIOUUrR0k0RkNNSS9WdTFX?=
 =?utf-8?B?Vnc9PQ==?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: =?utf-8?B?ZEhBMWZZSjlaUWVXWU5RbXBaNWhHWUtIUm1JRE0yOUdBN2tSN2VRNVdUcjJZ?=
 =?utf-8?B?NXdMaXNBY0xRRit2c3NReTJxakhWUzRHRjdlV3ZTckk2ZGJDeHYyaHVoMFpt?=
 =?utf-8?B?RVVKakQvc3hMdjlMSXNiZG9NUWVDRHc3d0hsWmNaczJYT0QrRUNaeXpTREJW?=
 =?utf-8?B?aW1meVJ1VVhXVitPRC9rRURVcGZpNXBDWHBXWjUyYTBnQWNTYVdkbFRYbU05?=
 =?utf-8?B?Smh0Y1dKWDM3eHRsYnFNNGdRUmdwNEhRVnhSVFBzMC9vdkdqTWZOWjRJSWtZ?=
 =?utf-8?B?VUQwaTFZbTI2TjhnMVYyUUt0ZHp3Rkg1bXpnODFobG9heTIzY3dYNlV5TlNC?=
 =?utf-8?B?Y3pDWlIzcDN3elUrMk1WOUNXSW9idWlWMjFRUVJIVGU0U1hCUjdxSjI0U214?=
 =?utf-8?B?ZGMvak9sTk94VzdEMUxlT21VL0Zkb0pma3lWSjIxQ3lqZS9qQnR3cTFMUlZR?=
 =?utf-8?B?V3NrZFVEM1hPMkpiRVVOcjFUampNeXJwQm9zUlBtVjYwY21tdHBnTFpGWHlC?=
 =?utf-8?B?M0pQMmhHTVZBWGtEVkNJUEdHZi9ab0hycDYzNUppVWdENGc4ZDdhU2VSK0Jh?=
 =?utf-8?B?dndxS2ZLQVd0d0RSbFV4ZUNubmxhV0RpZ04yNUo2VWtRc2d3eHp0RGRpcVFJ?=
 =?utf-8?B?djh3YVFuUWl6WDlHQ3pQT2ErRUZ2S2pMeWw2eVkxeVZITmNlbmhSY2JvTmZo?=
 =?utf-8?B?ajlocWJhNHMwWVZUN1gxYXBPQWw5Tnp3RmVrVTVUWXNka2xjYUJ2dVp3RnFL?=
 =?utf-8?B?YmtaeGR6cFhFRC8wRDNKYUNZZDRHd1ZFRkhzYnkzb0VyNDlvbEpWdXRwOE5Q?=
 =?utf-8?B?UTNyUEZyRVFpLzVZMU4xVkN5bzlKMjkyU3gyNDhEc2YzZDU3UCsyVUFVSUh3?=
 =?utf-8?B?VGloSWtKTThPVzZBTC95VWhsQjZqemNqOFg0ZVdSRzJJb2N2RmJkVkpVYStT?=
 =?utf-8?B?U1VmRGZabGc5UnRJbFdvV1BDd2lDNHVlS3dYMnRuZjZJaFM5aDlUY2dRVG5S?=
 =?utf-8?B?S0VsVXdJRUloNW5FZkVidnRBeFJiQzlFMXp5RE8rcTQ4Qkl4Sm9wOUJRcmRo?=
 =?utf-8?B?QmxDUGJubGFYNDNIa2p3YkJQWHo3YjlvTEpsYUFVaHhxMDlZbmVFcngxQUla?=
 =?utf-8?B?b09yV0Z1a1NXSWlycGdMcVU0bmJEaEE4ejVFZ0JVY0pjVGp5SW5DZngzdzBv?=
 =?utf-8?B?ZFFyMnlvbmdpb01QYXhLQ09GNHcvVWp4SlNpaHh3bTFQUEpROEtTNGE2ckxl?=
 =?utf-8?B?ejF1RWxyVUU3NzFZZmtXc1R6aHM2Q3JuRFpDWEl4UExSVlpKdHdxdCthSnVB?=
 =?utf-8?B?NWlKYk5CbEcxSWVEcXZwUS9Ydm14aVhDRDdkVjZXNDFtdDJ2ZTZvNnZRWHZD?=
 =?utf-8?B?ZEdEdTdtTEQ0OUFlYWlyTDVXUXhKVUQ5c0hVcUtCZTVFK3pSenFqTklkeXg3?=
 =?utf-8?Q?Zqn9NIFn?=
X-OriginatorOrg: citrix.com
X-MS-Exchange-CrossTenant-Network-Message-Id: fad7c4b4-e7ce-42d2-78cd-08db26cdcacd
X-MS-Exchange-CrossTenant-AuthSource: BYAPR03MB3623.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Mar 2023 09:55:52.0202
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335836de-42ef-43a2-b145-348c2ee9ca5b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: vEiFgZ6JMNNZ4mm9ZkfP26GDawa2ajvPziLLSeeM8Bj1AaDSRIk0xGs9t6Iz4owq5EEjhU83eE+8wE+KXkJQrLiqDUba6Co2FZFqx+aSL0U=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR03MB7210
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 17/03/2023 9:39 am, Lai Jiangshan wrote:
>> +#ifdef CONFIG_X86_FRED
>> +static bool ex_handler_eretu(const struct exception_table_entry *fixup,
>> +                            struct pt_regs *regs, unsigned long error_code)
>> +{
>> +       struct pt_regs *uregs = (struct pt_regs *)(regs->sp - offsetof(struct pt_regs, ip));
>> +       unsigned short ss = uregs->ss;
>> +       unsigned short cs = uregs->cs;
>> +
>> +       fred_info(uregs)->edata = fred_event_data(regs);
>> +       uregs->ssx = regs->ssx;
>> +       uregs->ss = ss;
>> +       uregs->csx = regs->csx;
>> +       uregs->current_stack_level = 0;
>> +       uregs->cs = cs;
> Hello
>
> If the ERETU instruction had tried to return from NMI to ring3 and just faulted,
> is NMI still blocked?
>
> We know that IRET unconditionally enables NMI, but I can't find any clue in the
> FRED's manual.
>
> In the pseudocode of ERETU in the manual, it seems that NMI is only enabled when
> ERETU succeeds with bit28 in csx set.  If so, this code will fail to reenable
> NMI if bit28 is not explicitly re-set in csx.

IRET clearing NMI blocking is the source of an immense amount of grief,
and ultimately the reason why Linux and others can't use supervisor
shadow stacks at the moment.

Changing this property, so NMIs only get unblocked on successful
execution of an ERET{S,U}, was a key demand of the FRED spec.

i.e. until you have successfully ERET*'d, you're still logically in the
NMI handler and NMIs need to remain blocked even when handling the #GP
from a bad ERET.

~Andrew