Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [RFC] KVM: mm: fd-based approach for supporting KVM guest private
 memory
To:     jejb@linux.ibm.com, Andy Lutomirski <luto@kernel.org>,
        David Hildenbrand <david@redhat.com>,
        Sean Christopherson <seanjc@google.com>
Cc:     Paolo Bonzini <pbonzini@redhat.com>,
        Vitaly Kuznetsov <vkuznets@redhat.com>,
        Wanpeng Li <wanpengli@tencent.com>,
        Jim Mattson <jmattson@google.com>,
        Joerg Roedel <joro@8bytes.org>, kvm list <kvm@vger.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        Borislav Petkov <bp@alien8.de>,
        Andrew Morton <akpm@linux-foundation.org>,
        Joerg Roedel <jroedel@suse.de>,
        Andi Kleen <ak@linux.intel.com>,
        David Rientjes <rientjes@google.com>,
        Vlastimil Babka <vbabka@suse.cz>,
        Thomas Gleixner <tglx@linutronix.de>,
        "Peter Zijlstra (Intel)" <peterz@infradead.org>,
        Ingo Molnar <mingo@redhat.com>,
        Varad Gautam <varad.gautam@suse.com>,
        Dario Faggioli <dfaggioli@suse.com>,
        the arch/x86 maintainers <x86@kernel.org>,
        linux-mm@kvack.org, linux-coco@lists.linux.dev,
        "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>,
        "Kirill A . Shutemov" <kirill@shutemov.name>,
        Sathyanarayanan Kuppuswamy 
        <sathyanarayanan.kuppuswamy@linux.intel.com>,
        Dave Hansen <dave.hansen@intel.com>,
        Yu Zhang <yu.c.zhang@linux.intel.com>
References: <20210824005248.200037-1-seanjc@google.com>
 <307d385a-a263-276f-28eb-4bc8dd287e32@redhat.com>
 <YSlkzLblHfiiPyVM@google.com>
 <61ea53ce-2ba7-70cc-950d-ca128bcb29c5@redhat.com>
 <YS6lIg6kjNPI1EgF@google.com>
 <f413cc20-66fc-cf1e-47ab-b8f099c89583@redhat.com>
 <9ec3636a-6434-4c98-9d8d-addc82858c41@www.fastmail.com>
 <bd22ef54224d15ee89130728c408f70da0516eaa.camel@linux.ibm.com>
From:   Tom Lendacky <thomas.lendacky@amd.com>
Message-ID: <f9fb2962-4b67-00e6-01e3-48a57d466932@amd.com>
Date:   Tue, 7 Sep 2021 10:00:12 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.13.0
In-Reply-To: <bd22ef54224d15ee89130728c408f70da0516eaa.camel@linux.ibm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ejRUSTZ1ZGtZTWMxUXkrb0hyTFVWMjI0SENweGZTbFM1UHpsa0FHSDhlQU0y?=
 =?utf-8?B?M1ZpMnNHaGhISERhQnFFQytBRk5VYTh6eVZQSHYyb0VyNUJSak12ZFYzTnN6?=
 =?utf-8?B?STZYckEzOUtuYWJWUytyNVdEejBma0I0ZVFxVkVKWUNISUVyM3Nud0oxbkNw?=
 =?utf-8?B?QVlScmhkZDZIY2xkTUNrV28wM21kNkJ0RURkc1d1RGp2ZStSQksrTFhoK1F1?=
 =?utf-8?B?V3VCZnZVOHRIUmlIMERPUnljOVczK2dUVnVHTVN5TStIUVVQSzFncXFUZDJO?=
 =?utf-8?B?YkdIOWNtT3JNUi9HdlNBcU9QZ3RaUXY3aWg1NDhuZHlOUG9iY0xaWEtjTVNK?=
 =?utf-8?B?V1FEUWJsNWM5VVJzTzh0Tmp6VExabHJBWlFpQVhtSFdWbW9CVEdMWEY5cUNt?=
 =?utf-8?B?OHdOaXNyOFM5dGpqcHdIczJTU0JSNHNHUWc1WFo0d1R6R05XQ0dzTUZTTHhL?=
 =?utf-8?B?YWFkVU5lZ0N6RDJRakJ3UUZjbWF5aW5OV2xnZ3FnTEZlbjlieXZEWDdTUlNR?=
 =?utf-8?B?R21KYjlYVmVNWnZES0tEWSs0cmtLakNYN2h4cElZb21tMmRZRlVONEhtVmVz?=
 =?utf-8?B?SGpRSzdGUGRjek55b2E2N1JmVGVweVFnQnZqNTh1K0YyYjQrc3NTc1RUekJz?=
 =?utf-8?B?U3gray9yanNjK2tpV0tFZUFob0hzdTBzaHBaU3FlQ2dEQzBDMTJsY01QWjNq?=
 =?utf-8?B?UnU2YXNuc1UwR2p1OTBmMStBNlVKR2tvcW0xWWZHS0F3VFN1ZVA4K0NlOERC?=
 =?utf-8?B?SzNIakU3dnJVNHpHS1NlZFV4UlNoNXJYY0lYV0g4R05XZWlnWnhEWlpSbExi?=
 =?utf-8?B?SVo0azFyeGVhaDg3K0x1QXFUNHZXYVoyOUZRMXRSMGl5eUNMRjRsZGZjanJC?=
 =?utf-8?B?Q2FqMFRRcDdMdnJsbzFHbnlFOW1ncUxZZzQrVnFDR2JjblRka09vRExrSG4w?=
 =?utf-8?B?dXluS3drUGl2THhTMWQ4SGY4ZzA5WFB2TVAxSkhTSHdrOGFnRkJENnJCZXVu?=
 =?utf-8?B?MVR4c3Y4NjMwc3VBUFZGdjR3RFBBUjQwMzlwUmdmSkMrNHR2WnNmVHMrclR0?=
 =?utf-8?B?RWpQbjVOdGhnSk5abmRMU252Q0pUOEo4YlBSTDRWUVhpeFdFeXNsc0VTUGdI?=
 =?utf-8?B?cE5VRVpYV1ZVMldJaXBSQ0lLSTBRRk91OWZ3NjNqQ1VFSjlZMkxqVFdCSmMz?=
 =?utf-8?B?RFNVUnMwNHhBTTZ0VERqd3RKSy83WlJMWjkwYzRNWndpOGd3NmQzRnRVZjJ4?=
 =?utf-8?B?cUlmeFFZc1BFUk9XNUQvQWlqWHUyTHBTbTFnM1RudWVUNzJjNlNGNDFNWS9T?=
 =?utf-8?B?RFNObGRwdUMrK25VYVhuM3RyRjk1QmNxRXpWaHpRZ2IwUmZkMkw2ZmY4OWdr?=
 =?utf-8?B?cVVUNXB6ZGNUK0dPQk85cWp6NU52NWExcWRJVUpvcGl3MHRLNkdiNlpHSFhN?=
 =?utf-8?B?WmpTUllJRDBBRUhZa1NxMHIrSzgxaXhWdUVMVmJkSDlkd3c1T3AwVTJEbHFY?=
 =?utf-8?B?SWJPVThlcEc2UUNrN0FianNMSTMxNzl0T20xbVpkQ0VnZkxMb1hhWDJQR3Fj?=
 =?utf-8?B?MU8rRmtWd0szZk5wUUhZMzdOMkNXWHhrTUpWQzM0Zm92OFgxMXd4cXVSSVBj?=
 =?utf-8?B?L2FKaW93OHM0VjZwUUlMTitjOWZLMkpLbkxaL3pYc0w1TnNPemJqaUErQm9F?=
 =?utf-8?B?VG1QZnhlaVU0SDJ3cUE4MU9XUVlub2VLWnJJRnVtQ29IZit5aDQ4R0F2eG1l?=
 =?utf-8?Q?OVQM1ZvkLfLiOJNH37uajoRZh3g4uOU/p2oy6Zf?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: d3af2eed-abd0-418d-9086-08d9721032f5
X-MS-Exchange-CrossTenant-AuthSource: DM4PR12MB5229.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 15:00:15.3735
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: zmo8ucCQufib9yEWRDqnY1BGJZ90RQ44RMw3omDkBHVUSdawfo+pmS9f7fDnbm2BupYeBmK1luLxA3roPgcX0Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5054
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 9/1/21 11:18 AM, James Bottomley wrote:
> On Wed, 2021-09-01 at 08:54 -0700, Andy Lutomirski wrote:
> [...]
>> If you want to swap a page on TDX, you can't.  Sorry, go directly to
>> jail, do not collect $200.
> 
> Actually, even on SEV-ES you can't either.  You can read the encrypted
> page and write it out if you want, but unless you swap it back to the
> exact same physical memory location, the encryption key won't work.
> Since we don't guarantee this for swap, I think swap won't actually
> work for any confidential computing environment.

There are SEV/SEV-ES and SEV-SNP APIs to swap a page out and in. There is 
also an API to copy (for SEV/SEV-ES) or move (for SEV-SNP) a page in 
memory to help with balancing if needed.

These aren't currently implemented in the kernel, but can be. It requires 
the changes going in for live migration to recognize that a page is 
encrypted/private and invoke the APIs to transform the page before doing 
the operation.

Thanks,
Tom

> 
>> So I think there are literally zero code paths that currently call
>> try_to_unmap() that will actually work like that on TDX.  If we run
>> out of memory on a TDX host, we can kill the guest completely and
>> reclaim all of its memory (which probably also involves killing QEMU
>> or whatever other user program is in charge), but that's really our
>> only option.
> 
> I think our only option for swap is guest co-operation.  We're going to
> have to inflate a balloon or something in the guest and have the guest
> driver do some type of bounce of the page, where it becomes an
> unencrypted page in the guest (so the host can read it without the
> physical address keying of the encryption getting in the way) but
> actually encrypted with a swap transfer key known only to the guest.  I
> assume we can use the page acceptance infrastructure currently being
> discussed elsewhere to do swap back in as well ... the host provides
> the guest with the encrypted swap page and the guest has to decrypt it
> and place it in encrypted guest memory.
> 
> That way the swapped memory is securely encrypted, but can be swapped
> back in.
> 
> James
> 
>