Message-ID: <3bb3e080-caee-8bc8-7de9-f44969f16e75@amd.com>
Date:   Sat, 28 Jan 2023 22:24:56 +1100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.6.0
Subject: Re: [Question PATCH kernel] x86/amd/sev/nmi+vc: Fix stack handling
 (why is this happening?)
Content-Language: en-US
To:     Joerg Roedel <joro@8bytes.org>
Cc:     Peter Zijlstra <peterz@infradead.org>, kvm@vger.kernel.org,
        x86@kernel.org, linux-kernel@vger.kernel.org,
        Thomas Gleixner <tglx@linutronix.de>,
        Sean Christopherson <seanjc@google.com>,
        Jiri Kosina <jkosina@suse.cz>, Ingo Molnar <mingo@redhat.com>,
        Dave Hansen <dave.hansen@linux.intel.com>,
        Borislav Petkov <bp@alien8.de>,
        "H. Peter Anvin" <hpa@zytor.com>,
        Tom Lendacky <thomas.lendacky@amd.com>
References: <20230127035616.508966-1-aik@amd.com>
 <Y9OUfofjxDtTmwyV@hirez.programming.kicks-ass.net>
 <Y9OpcoSacyOkPkvl@8bytes.org> <b7880f0b-a592-cf2d-03b9-1ccfd83f8223@amd.com>
 <Y9QI9JwCVvRmtbr+@8bytes.org>
From:   Alexey Kardashevskiy <aik@amd.com>
In-Reply-To: <Y9QI9JwCVvRmtbr+@8bytes.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?REVScnA3Mk1KcHdXVzlwalI0b0lRb0ZZTXcyY0xmWktPejFzVktwS2tDUk94?=
 =?utf-8?B?TWpDNVpZY3BnSkoxRU1yeEFnUEEySWZXM1VTQlAxVnpnSWJINy9RaTh6MUpy?=
 =?utf-8?B?Y1UvYkJBWVo2dkV1cnUzWU43NzFRVGtlRlJ5YURvNWhZb2F4VEFsbWQxdDVF?=
 =?utf-8?B?ejBReGk3L1MzR2NCRlQ3bmNrVEZTTlljbk12bXVmeW5Ra25SNjVBT2FiTm5k?=
 =?utf-8?B?NWg0VFh1NHI1YzhBblpjUGs1cnM4MkdBRXVnTjhNSFhQUFFJT2pEcTdFNzUv?=
 =?utf-8?B?WGsvMSszeU9RcTdWRFR4UTZ2T2RzdHRpU0lMd1dFeERSNWRONTVxL2g4Ukha?=
 =?utf-8?B?Q2pOTVVjK1FrMFlSTDVSbng2bnN6TndWODFhQ3grbkVOUVEwMzlJMkJrR3JU?=
 =?utf-8?B?ZjFwbm9hS0dqdHRCb1ora2lJOWh4c2RnUUpPbUNaVlFWQ1BkcGsxVGw0TW9X?=
 =?utf-8?B?eUIrN2p1Y2h6RGl3Q2c3YXc5ZUk5Mm9keGtWK2hsdzM5eEp4eFpBU2t0Yktq?=
 =?utf-8?B?VUVjdFlYR0M5dCtIOFR0VWpCNFRpUTgxMndwaE5VWUJXcWlLRkh0SXMyQkxO?=
 =?utf-8?B?NTNKZ2V6ZmxYdHdsUWgvbTFCVExnT3RWRVRkdkdUTzJWczJIY3hzQjNvNXQ3?=
 =?utf-8?B?aDBTSkNQNnZJV2xma2NYZzRQOU9yczVvTkpFL2J3Z1hQSmdyUFJ0czU1NlFZ?=
 =?utf-8?B?eEprVkhVZUYvM05TTnNTS3hiamIwMHlCZnMzcTlKS3dPSmwzWjZVaUh0Snl1?=
 =?utf-8?B?SXhqaEw4akhFalVldlk1dXRYbG9qbjlCdEI0RVFXSHFaMk00Rk1BbUo5NjNq?=
 =?utf-8?B?RlZ2Um9UZEZROHNaS1E3M2R5R2NqT2l5YjFtdVlqbk1RWkdkdHh1VHhreGpj?=
 =?utf-8?B?WlJndlZleVJWVjQreHhVTklLTmliQzdwUlFScVBKK1FjTE0xYXFRWWR4WDEz?=
 =?utf-8?B?a0ZkVFk5UjRRdit5TitHcElwYWFzNXNMVXI3SXBWVS8vZG1ac0creUtxNzJK?=
 =?utf-8?B?V0ttSHMyeUZqMUZibUFIZTB0dXEvYnJYOC81RVE0N3BQQUlaV2VxNzE0SDJp?=
 =?utf-8?B?Wm52cmc3VDBxMXRybStGRVY3OWZQNnRvS2VubHRXUGpYd09TY3ZHMDNqdERr?=
 =?utf-8?B?MkpXckMrUElLelQwM3dpajZNOVpURlo2UDhFa3hqQ1BIdmJId2d2UzZCRjlj?=
 =?utf-8?B?U1cwVUFDaTRqZDNLUkJsWnRDY0lNcEc1NlhjejgzcGNDWlhVTThabHQ0aHdL?=
 =?utf-8?B?T2F3K1g3akV3bWlGMzNDSTBtOUtqUko4OTI2Zmx2K0UvUjRDL0ZEaE91bE5M?=
 =?utf-8?B?ai9zYzc5ZnhpTndLTjNNR3RNSkNTbUJPQlFTRlh6Qnl4eWc0SnMvK3dwZGgr?=
 =?utf-8?B?NGxIRWc4cUtndjFob1F5Vm9ZSXpkdFJINXFoSnp5ZW5ySGFLYVV5K2UwRVBI?=
 =?utf-8?B?RmVHSVlCOGlLM1AzU2ZWdjdobkRjV3diSjZ0VVNEczJsNkQwcTNwUzFNZVNX?=
 =?utf-8?B?dkJTeHZTOG1YQXpRNWE1bWVJSlM1ekJab0IxbllIaDhQc1pqWEp4TXNoamVk?=
 =?utf-8?B?dC9yZUhYeUlpck9TQnc1bnU3dXc3Q3orZGFRei9mN3JnMkl0RkttZVBIN28v?=
 =?utf-8?B?RkwvNW9Ldy9TRWl5TEgweHpteFZwWjBQRFRxK1JZN2sxSCsvZW9ZWUlSNEln?=
 =?utf-8?B?V3FVMmdPK0hzaGs2ZWRVNXp0em11dCtsNzVDdzJiRWxNVWx6U1BBenpSMXJn?=
 =?utf-8?B?RXRZc2ZFVWQrM0RSNmcvNnlRckZnelhXb1BDK2phcW50Kzh0UDR4V1BPcU9C?=
 =?utf-8?B?bklXUTNiUmtQWWxVYkFMSU9wN3ZRang4WXh5ZFVTaHlTbXFhQjRrNmsrdm8r?=
 =?utf-8?B?cW9QRERGa2Vzc3o0S0hSYjZiby80cWJ1YWxDb01wZE0rZjlnWTdYd3dsZXJT?=
 =?utf-8?B?cFdlNzlNMmJOUytpZ1NIdzlpSUs5YkVNOVVrWlhPcHVuNm8yMm50dmZzNmZD?=
 =?utf-8?B?UC9GZmhYb0NURW51dVlydEppMW9xc2NtMytFS3laNlFUQnl3Y0wvaStTVXhh?=
 =?utf-8?B?aEtsT1JmY3d1NUxOL09NTmJDZFVhbDBrWWhzRTBBQUNPQ3lUcnBkZGh2cTNO?=
 =?utf-8?Q?vCoSRFCR4WvrNQafhu6dhj4FY?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c89b7aa7-b0f6-4472-aec2-08db01224e8d
X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB2843.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Jan 2023 11:25:06.6637
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: FRuPGNXYI3iQMpRwIw/X44FB9WnyM/qxDKYcYB8pO4eu5AB6IznwPK18A7m5j8wNSaHOnbFqB7MZ46QOUUW2uQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB7953
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 28/1/23 04:25, Joerg Roedel wrote:
> On Fri, Jan 27, 2023 at 10:56:26PM +1100, Alexey Kardashevskiy wrote:
>> https://github.com/aik/linux/commit/d0d6bbb58fcd927ddd1f8e9d42ab121920c7eafc
> 
> Okay, I reproduced the problem here and the root cause turned out to be
> that the compiler moved the DR7 read instruction before the 5-byte NOP
> which becomes the call to sev_es_ist_enter() in SEV-ES guests. This is
> guaranteed to cause #VC exception stack recursion if the NMI was
> triggered on the #VC stack, and that leads to all kinds of undefined
> behavior.

Cool!

(out of curiosity) where do you see these NOPs? "objdump -D vmlinux" 
does not show any, is this after lifepatching?

Meanwhile, this seems to be doing the right thing:


diff --git a/arch/x86/include/asm/debugreg.h 
b/arch/x86/include/asm/debugreg.h
index b049d950612f..687b15297057 100644
--- a/arch/x86/include/asm/debugreg.h
+++ b/arch/x86/include/asm/debugreg.h
@@ -39,7 +39,7 @@ static __always_inline unsigned long 
native_get_debugreg(int regno)
                 asm("mov %%db6, %0" :"=r" (val));
                 break;
         case 7:
-               asm("mov %%db7, %0" :"=r" (val));
+               asm volatile ("mov %%db7, %0" :"=r" (val));


-- 
Alexey