Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com;
Date:   Wed, 20 Oct 2021 21:05:42 -0500
From:   Michael Roth <michael.roth@amd.com>
To:     Borislav Petkov <bp@alien8.de>
CC:     Brijesh Singh <brijesh.singh@amd.com>, <x86@kernel.org>,
        <linux-kernel@vger.kernel.org>, <kvm@vger.kernel.org>,
        <linux-efi@vger.kernel.org>, <platform-driver-x86@vger.kernel.org>,
        <linux-coco@lists.linux.dev>, <linux-mm@kvack.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Joerg Roedel <jroedel@suse.de>,
        Tom Lendacky <thomas.lendacky@amd.com>,
        "H. Peter Anvin" <hpa@zytor.com>, Ard Biesheuvel <ardb@kernel.org>,
        Paolo Bonzini <pbonzini@redhat.com>,
        Sean Christopherson <seanjc@google.com>,
        "Vitaly Kuznetsov" <vkuznets@redhat.com>,
        Jim Mattson <jmattson@google.com>,
        "Andy Lutomirski" <luto@kernel.org>,
        Dave Hansen <dave.hansen@linux.intel.com>,
        Sergio Lopez <slp@redhat.com>, Peter Gonda <pgonda@google.com>,
        "Peter Zijlstra" <peterz@infradead.org>,
        Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>,
        David Rientjes <rientjes@google.com>,
        Dov Murik <dovmurik@linux.ibm.com>,
        Tobin Feldman-Fitzthum <tobin@ibm.com>,
        Vlastimil Babka <vbabka@suse.cz>,
        "Kirill A . Shutemov" <kirill@shutemov.name>,
        Andi Kleen <ak@linux.intel.com>,
        "Dr . David Alan Gilbert" <dgilbert@redhat.com>,
        <tony.luck@intel.com>, <marcorr@google.com>,
        <sathyanarayanan.kuppuswamy@linux.intel.com>
Subject: Re: [PATCH v6 08/42] x86/sev-es: initialize sev_status/features
 within #VC handler
Message-ID: <20211021020542.v5s7xr4s2j3gsale@amd.com>
References: <20211008180453.462291-1-brijesh.singh@amd.com>
 <20211008180453.462291-9-brijesh.singh@amd.com>
 <YW2EsxcqBucuyoal@zn.tnic>
 <20211018184003.3ob2uxcpd2rpee3s@amd.com>
 <YW3IdfMs61191qnU@zn.tnic>
 <20211020161023.hzbj53ehmzjrt4xd@amd.com>
 <YXBbJwd2M03Ssq6I@zn.tnic>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <YXBbJwd2M03Ssq6I@zn.tnic>
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Oct 2021 02:08:33.0043
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 2e6e95bc-5f79-4d12-9267-08d99437af23
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT016.eop-nam11.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB3400
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Wed, Oct 20, 2021 at 08:08:39PM +0200, Borislav Petkov wrote:
> On Wed, Oct 20, 2021 at 11:10:23AM -0500, Michael Roth wrote:
> > > 1. Code checks SME/SEV support leaf. HV lies and says there's none. So
> > > guest doesn't boot encrypted. Oh well, not a big deal, the cloud vendor
> > > won't be able to give confidentiality to its users => users go away or
> > > do unencrypted like now.
> > > 
> > > Problem is solved by political and economical pressure.
> > > 
> > > 2. Check SEV and SME bit. HV lies here. Oh well, same as the above.
> > 
> > I'd be worried about the possibility that, through some additional exploits
> > or failures in the attestation flow,
> 
> Well, that puts forward an important question: how do you verify
> *reliably* that this is an SNP guest?
> 
> - attestation?
> 
> - CPUID?
> 
> - anything else?
> 
> I don't see this written down anywhere. Because this assumption will
> guide the design in the kernel.

According to the APM at least, (Rev 3.37, 15.34.10, "SEV_STATUS MSR"), the
SEV MSR is the appropriate source for guests to use. This is what is used
in the EFI code as well. So that seems to be the right way to make the
initial determination.

There's a dependency there on the SEV CPUID bit however, since setting the
bit to 0 would generally result in a guest skipping the SEV MSR read and
assuming 0. So for SNP it would be more reliable to make use of the CPUID
table at that point, since it's less-susceptible to manipulation, or do the
#VC-based SEV MSR read (or both).

> 
> > a guest owner was tricked into booting unencrypted on a compromised
> > host and exposing their secrets. Their attestation process might even
> > do some additional CPUID sanity checks, which would at the point
> > be via the SNP CPUID table and look legitimate, unaware that the
> > kernel didn't actually use the SNP CPUID table until after 0x8000001F
> > was parsed (if we were to only initialize it after/as-part-of
> > sme_enable()).
> 
> So what happens with that guest owner later?
> 
> How is she to notice that she booted unencrypted?

Fully-unencrypted should result in a crash due to the reasons below.

But there may exist some carefully crafted outside influences that could
goad the guest into, perhaps, not marking certain pages as private. The
best that can be done to prevent that is to audit/harden all the code in the
boot stack so that it is less susceptible to that kind of outside
manipulation (via mechanisms like SEV-ES, SNP page validation, SNP CPUID
table, SNP restricted injection, etc.)

Then of course that boot stack needs to be part of the attestation process
to provide any meaningful assurances about the resulting guest state.

Outside of the boot stack the guest owner might take some extra precautions.
Perhaps custom some kernel driver to verify encryption/validated status of
guest pages, some checks against the CPUID table to verify it contains sane
values, but not really worth speculating on that aspect as it will be
ultimately dependent on how the cloud vendor decides to handle things after
boot.

> 
> > Fortunately in this scenario I think the guest kernel actually would fail to
> > boot due to the SNP hardware unconditionally treating code/page tables as
> > encrypted pages. I tested some of these scenarios just to check, but not
> > all, and I still don't feel confident enough about it to say that there's
> > not some way to exploit this by someone who is more clever/persistant than
> > me.
> 
> All this design needs to be preceded with: "We protect against cases A,
> B and C and not against D, E, etc."
> 
> So that it is clear to all parties involved what we're working with and
> what we're protecting against and what we're *not* protecting against.

That would indeed be useful. Perhaps as a nice big comment in sme_enable()
and/or the proposed sev_init() so that those invariants can be maintained,
or updated in sync with future changes. I'll look into that for the next
spin and check with Brijesh on the details.

> 
> End of mail 2, more later.
> 
> -- 
> Regards/Gruss,
>     Boris.
> 
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpeople.kernel.org%2Ftglx%2Fnotes-about-netiquette&amp;data=04%7C01%7Cmichael.roth%40amd.com%7C70ce657823a441516fc808d993f4a402%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637703501243595370%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=nIqCXolZUNWTV6eBfLscXRfQDWJZk5fwBMghKVbIeaw%3D&amp;reserved=0