Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: FSGSBASE causing panic on 5.9-rc1
To:     Sean Christopherson <sean.j.christopherson@intel.com>,
        Andy Lutomirski <luto@kernel.org>
Cc:     Dave Hansen <dave.hansen@intel.com>,
        Jim Mattson <jmattson@google.com>,
        Joerg Roedel <joro@8bytes.org>,
        Paolo Bonzini <pbonzini@redhat.com>,
        Vitaly Kuznetsov <vkuznets@redhat.com>,
        Wanpeng Li <wanpengli@tencent.com>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        X86 ML <x86@kernel.org>,
        "Chang S. Bae" <chang.seok.bae@intel.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Sasha Levin <sashal@kernel.org>,
        Borislav Petkov <bp@alien8.de>,
        Peter Zijlstra <peterz@infradead.org>,
        Ingo Molnar <mingo@kernel.org>
References: <f6694ee6-d672-1cf9-deaf-4d600bc4e9eb@amd.com>
 <5756198D-C8BD-4290-BFCA-04424EB230A6@amacapital.net>
 <20200820220507.GA10269@sjchrist-ice>
 <CALCETrXxfZtLSPx+QFMRdnvDE6+zT=Jy8CYX03Va7o8M0VvKng@mail.gmail.com>
 <20200820223431.GB10269@sjchrist-ice>
From:   Tom Lendacky <thomas.lendacky@amd.com>
Message-ID: <30eb9d62-a883-2630-c51f-6f5ee4def07a@amd.com>
Date:   Thu, 20 Aug 2020 19:00:16 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.10.0
In-Reply-To: <20200820223431.GB10269@sjchrist-ice>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-MessageSentRepresentingType: 1
Received: from 255.255.255.255 (255.255.255.255) by SN4PR0201CA0027.namprd02.prod.outlook.com (2603:10b6:803:2e::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3305.25 via Frontend Transport; Fri, 21 Aug 2020 00:00:18 +0000
X-Originating-IP: [67.79.209.213]
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-HT: Tenant
X-MS-Office365-Filtering-Correlation-Id: be79dd8a-a863-4539-0ee9-08d845653157
X-MS-TrafficTypeDiagnostic: DM6PR12MB3081:
X-Microsoft-Antispam-PRVS: <DM6PR12MB308193AFE4D7C8AF41467DDFEC5B0@DM6PR12MB3081.namprd12.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:10000;
X-MS-Exchange-SenderADCheck: 1
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: b/JlpqrFL7mr2+5Sct6C7pJlGkXhj9J7veKLBM/eNIyUuPe02phj4ERrg0NvhS0y38e/gcFFxNbVFfDifZtbheruGGlDPYojgjf4nv4W8YOI8LRRTQtDCDF/TH+/5vFE5ir73aeTeeSKd8N78YblqxBvhXfzqhx85Xmnz0G9wi1kq4EZe6H3kFoJmMTWIy4MIz9upTJ9r5/SdTvRkihZT2EuJprlGS/HUdiiN3kn+4B9HgaL/MNXYz/XhZoNf1P6KpZ33z4HY6NpjEftebwpm/HefuKHguRs+zf0rvlJYz9chGBbsj/NQqHBkG9FpPD7o47+kkspDBU3GNVoyTbF/WvwDB8iu5nRKtHAStQKYOVIxODtgi3yB84yxvDehtjgZBfvTQDBXZpz8qhN8v61avb3uv1UxQ1znnlN06ls5zA=
X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM5PR12MB1355.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(4636009)(346002)(376002)(39860400002)(396003)(136003)(366004)(83380400001)(478600001)(54906003)(5660300002)(53546011)(26005)(66946007)(66556008)(8676002)(36756003)(52116002)(8936002)(2906002)(7416002)(66476007)(186003)(110011004)(31686004)(31696002)(110136005)(6486002)(2616005)(16576012)(86362001)(956004)(316002)(4326008)(43740500002);DIR:OUT;SFP:1101;
X-MS-Exchange-AntiSpam-MessageData: 8i1NFypTcdLt7hizpdeHnICIXh7RFw+mzko/N16UgCz2qrh4XhlKid+xX2J7KbMLWiPYeuUySPx6RAZgw5wC5d3yVeIbJlv0bbyo2ePg8/ggStaGnuKTbNturjDuRsW6hWWGsr9UwswoRzkqscl+JzR0l6R++xKNzV+dA7Ucea6h6G5IkBfziAaF5CSSQ5a9w53u9aoFSIBW+ThSFUdJS0x9x+qTQAePVMsN1z3us2XC2RQzoyAWvPlLQIUGUnsHOpoV4fG70Lm4Q6BRYwxxc8gR3F2HzmMjR3wn275nEas5yq3E4bvVyBw/IJyfFHWzTxl+6OIrujxria+A8EdyB19FNPZ/g+Qu6Gf79DOSnpiBbglhE6pKMZSDrrarrNWEazza34RcXnxap6oe6qmwfjMxZpAbuCJ7gn9gVMQI+VS2+I5MAVqy39hk7Su1rRpTyKV7SwOkfTixMAVh2cgATtB0cEdKsya432e7M0bVIsB5R89GNgS2sCPSfTsahaZpfkujCHnyEU9fXeHfN6U5R14UL/tQ3vVVa4oVrJ3NGp99C/echf6lwW+J65ZEeq2rmlfvuThWuTDw1KcAPmGRS9hils8UiCI/2bN9GQyTWxYvr7FleZ68yr/1A3hAPDyqIxcorIQfBL+5eMDIMoC5iA==
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: be79dd8a-a863-4539-0ee9-08d845653157
X-MS-Exchange-CrossTenant-AuthSource: DM5PR12MB1355.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Aug 2020 00:00:19.7646
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: rkS6oGhK1+PsuPnVCyWAJ86JpdZSgK531Y3Vwzrbv3mSV35V/RlyS2UIjULZc67wudDDMKdKILL8KFYc/Wx90w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB3081
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 8/20/20 5:34 PM, Sean Christopherson wrote:
> On Thu, Aug 20, 2020 at 03:07:10PM -0700, Andy Lutomirski wrote:
>> On Thu, Aug 20, 2020 at 3:05 PM Sean Christopherson
>> <sean.j.christopherson@intel.com> wrote:
>>>
>>> On Thu, Aug 20, 2020 at 01:36:46PM -0700, Andy Lutomirski wrote:
>>>>
>>>>
>>>>> On Aug 20, 2020, at 1:15 PM, Tom Lendacky <thomas.lendacky@amd.com> wrote:
>>>>>
>>>>> ﻿On 8/20/20 3:07 PM, Dave Hansen wrote:
>>>>>> On 8/20/20 12:05 PM, Tom Lendacky wrote:
>>>>>>>> I added a quick hack to save TSC_AUX to a new variable in the SVM
>>>>>>>> struct and then restore it right after VMEXIT (just after where GS is
>>>>>>>> restored in svm_vcpu_enter_exit()) and my guest is no longer crashing.
>>>>>>>
>>>>>>> Sorry, I mean my host is no longer crashing.
>>>>>> Just to make sure I've got this:
>>>>>> 1. Older CPUs didn't have X86_FEATURE_RDPID
>>>>>> 2. FSGSBASE patches started using RDPID in the NMI entry path when
>>>>>>     supported *AND* FSGSBASE was enabled
>>>>>> 3. There was a latent SVM bug which did not restore the RDPID data
>>>>>>     before NMIs were reenabled after VMEXIT
>>>>>> 4. If an NMI comes in the window between VMEXIT and the
>>>>>>     wrmsr(TSC_AUX)... boom
>>>>>
>>>>> Right, which means that the setting of TSC_AUX to the guest value needs to be moved, too.
>>>>>
>>>>
>>>> Depending on how much of a perf hit this is, we could also skip using RDPID
>>>> in the paranoid path on SVM-capable CPUs.
>>>
>>> Doesn't this affect VMX as well?  KVM+VMX doesn't restore TSC_AUX until the
>>> kernel returns to userspace.  I don't see anything that prevents the NMI
>>> RDPID path from affecting Intel CPUs.
>>>
>>> Assuming that's the case, I would strongly prefer this be handled in the
>>> paranoid path.  NMIs are unblocked immediately on VMX VM-Exit, which means
>>> using the MSR load lists in the VMCS, and I hate those with a vengeance.
>>>
>>> Perf overhead on VMX would be 8-10% for VM-Exits that would normally stay
>>> in KVM's run loop, e.g. ~125 cycles for the WMRSR, ~1300-1500 cycles to
>>> handle the most common VM-Exits.  It'd be even higher overhead for the
>>> VMX preemption timer, which is handled without even enabling IRQs and is
>>> a hot path as it's used to emulate the TSC deadline timer for the guest.
>>
>> I'm fine with that -- let's get rid of RDPID unconditionally in the
>> paranoid path.  Want to send a patch that also adds as comment
>> explaining why we're not using RDPID?
> 
> Sure, though I won't object if Tom beats me to the punch :-)

I can do it, but won't be able to get to it until sometime tomorrow.

Thanks,
Tom

>