Received-SPF: pass (google.com: domain of linux-crypto+bounces-4279-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Message-ID: <1ce87335-2ea7-41c4-8442-36210656cdca@intel.com>
Date: Tue, 21 May 2024 11:41:31 +1200
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v15 13/20] KVM: SEV: Implement gmem hook for initializing
 private pages
To: Sean Christopherson <seanjc@google.com>
CC: "isaku.yamahata@gmail.com" <isaku.yamahata@gmail.com>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>, Rick P Edgecombe
	<rick.p.edgecombe@intel.com>, "michael.roth@amd.com" <michael.roth@amd.com>,
	"pankaj.gupta@amd.com" <pankaj.gupta@amd.com>, "tglx@linutronix.de"
	<tglx@linutronix.de>, "tobin@ibm.com" <tobin@ibm.com>,
	"liam.merwick@oracle.com" <liam.merwick@oracle.com>, "alpergun@google.com"
	<alpergun@google.com>, Tony Luck <tony.luck@intel.com>, "jmattson@google.com"
	<jmattson@google.com>, "luto@kernel.org" <luto@kernel.org>,
	"ak@linux.intel.com" <ak@linux.intel.com>, "pbonzini@redhat.com"
	<pbonzini@redhat.com>, "pgonda@google.com" <pgonda@google.com>,
	"srinivas.pandruvada@linux.intel.com" <srinivas.pandruvada@linux.intel.com>,
	"slp@redhat.com" <slp@redhat.com>, "rientjes@google.com"
	<rientjes@google.com>, "peterz@infradead.org" <peterz@infradead.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"dovmurik@linux.ibm.com" <dovmurik@linux.ibm.com>, "thomas.lendacky@amd.com"
	<thomas.lendacky@amd.com>, "x86@kernel.org" <x86@kernel.org>, "bp@alien8.de"
	<bp@alien8.de>, "vkuznets@redhat.com" <vkuznets@redhat.com>, "vbabka@suse.cz"
	<vbabka@suse.cz>, "ashish.kalra@amd.com" <ashish.kalra@amd.com>,
	"linux-coco@lists.linux.dev" <linux-coco@lists.linux.dev>,
	"nikunj.dadhania@amd.com" <nikunj.dadhania@amd.com>, Jorg Rodel
	<jroedel@suse.de>, "mingo@redhat.com" <mingo@redhat.com>,
	"sathyanarayanan.kuppuswamy@linux.intel.com"
	<sathyanarayanan.kuppuswamy@linux.intel.com>, "hpa@zytor.com"
	<hpa@zytor.com>, "kirill@shutemov.name" <kirill@shutemov.name>,
	"jarkko@kernel.org" <jarkko@kernel.org>, "ardb@kernel.org" <ardb@kernel.org>,
	"linux-crypto@vger.kernel.org" <linux-crypto@vger.kernel.org>,
	"linux-mm@kvack.org" <linux-mm@kvack.org>, "dave.hansen@linux.intel.com"
	<dave.hansen@linux.intel.com>
References: <20240501085210.2213060-1-michael.roth@amd.com>
 <20240501085210.2213060-14-michael.roth@amd.com>
 <41d8ba3a48d33de82baa67ef5ee88e5f8995aea8.camel@intel.com>
 <ZkuJ27DKOCkqogHn@google.com>
 <e771a7ba-0445-483e-9c42-66bd5b331dce@intel.com>
 <ZkvZkPvHqqPnVa9k@google.com>
Content-Language: en-US
From: "Huang, Kai" <kai.huang@intel.com>
In-Reply-To: <ZkvZkPvHqqPnVa9k@google.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?WjMxT1lGWVVUSlFIZEIrejhvcWJRc1U3QXNOL0xTUTVhdmxqaVJKYkZTdDlz?=
 =?utf-8?B?U2hJS1d5M0twZkFqakRuSEU5dDNmdmZWR2kwYXBtRUpaem90Sm9vRjJIMHRo?=
 =?utf-8?B?blBTQTVTck5OVTZzZ05lclFMVXB6U0FaaGhMM0l3ODZ1blBnZFJ3a3pKY0w4?=
 =?utf-8?B?QlNlSDh4QVRIbWpSQVlSMHY2MW1abGRsVng5ZHcwaE5FVFRFN1lLNFFPV1I3?=
 =?utf-8?B?dFVDUUJyanJZYmFFNUN5MGRxMkdEWkFXV1JmTUxIUTI2eU44cDc5U0F6cEdH?=
 =?utf-8?B?cVlSMlc1S0twZDE3SW1BWXM1U3RNdGc5Rk1mRk4vaHRURkR2NU9hSmVKTnht?=
 =?utf-8?B?NCtDaUZseWYzSWUxVzFzVi9JbVY3WmNXVDQwOWpMeC9iT2ViNFQ5S2VVSnFo?=
 =?utf-8?B?VlQyMERmM3RBT3VWWmZJUjZJUzh1V0VoYTYvS3NOYm5aVjRqbnVYTEZuekZM?=
 =?utf-8?B?cWJ1ZURRZURlbW5Mc1FVQVRBOXhMODRieDFUM3RDczRVbFBpUmdyb3pGOHli?=
 =?utf-8?B?eWMwSFFRK2R1S0NFTjZHamUrYlIxWGFMWU1paXczTFBVQlZjVmEzV2t5cXg0?=
 =?utf-8?B?KzdkZjdiRUVuZ2U2eER2TUZjMkhyckJxTFMramNiL08rQ0hWTmxyZEJmQ3M2?=
 =?utf-8?B?SXBlNzQ0MTFicmdWUXlYeDBNYjRZOWsxVit4TnZPcS9pQTl4MTRrMUxTNEY4?=
 =?utf-8?B?emVPb1JxTzJLNElWZyt0ZU8zTmtRVkF6YjlDcGFpN1BIbkc1a3JNOXg4bXpj?=
 =?utf-8?B?ZmtxVy9CUU5Md0VpbHMyMXk4bklTb1V0Wm5CL3ZWNi9lK1ZCTjd2eWFUSndR?=
 =?utf-8?B?QW1IeWJXT3BNWWRFaTZUcWJDNEwvRHpYcUdnSHdZVnN1QzBleUVBb3lyWFZw?=
 =?utf-8?B?djhOV1diNGlMNUhoUkFqcWF5WjY4VFd3R21zcCtHK3ptNzN3bzFnZGxVOXo1?=
 =?utf-8?B?RlVndkZwQXdvQTB6SHp5bzZMVHQ5ZDk1MFpGbTFuUktSS3Y0SlVpWnU0ODZO?=
 =?utf-8?B?TmdaM0l6YTBScFpBdGtKdmNwV2YwTERoU200TGZUK3dMNWt3ODR1TzlEVE00?=
 =?utf-8?B?OStTTHFUaENZUlY0bHBabDlkTG4zdXphM21YRFB1R1pRd3p0WEsxeEp1clZI?=
 =?utf-8?B?VEt5cjhheTRXT0FDMUsrazBoaTRIcU9RUkRteGVyalJ4WjRzMk5na200dExp?=
 =?utf-8?B?RDBrdHhjR1VMZkNiWUdpZDM2RmUvYVBhdHhEZjJVUDV0Y2JhL3JrRXp1SFZr?=
 =?utf-8?B?QmVtYUhVOTI1TDV0Vk9uM0dFRUV6Wm9mNWxNYXZEMFpSU0R5aFJwT1pod1dB?=
 =?utf-8?B?MVlHVkJIbFE0cUJVOHNaMWdMZW1OTklpQmhlK1BRZS9xVlpuRlM0SWNSc0xT?=
 =?utf-8?B?RjNiQTRHRkdkZzJrUThsb01GS0VXL0VydHdIcUgxSjY4VmI2YzBXT0gxVjV1?=
 =?utf-8?B?WkY4SXhGSGNkMnpoaTR5a1ZFVTN4ZFBUdXFLYXh0dU94cVZIUWJaV2lVN0pq?=
 =?utf-8?B?bFc0THVVNEpnZk1GM0tCUjhYM0o2WXNubDFEK0hQcGR5bDdjN1dpTUdCcEFt?=
 =?utf-8?B?MWVUN1pWMVllUW1jQ2tXaUdydFEyYWZQamszdWE5VEZYOG0zdVB6eFk2d285?=
 =?utf-8?B?N0h5RFJmUmhjMUg0RWpwY1VzNjhHZjdYVHEvR3NVUmJnMDF0NHk2RTdOWXZl?=
 =?utf-8?B?U01TUi9FWmlyZW0wWXZHY2NURk9naXlZWW9kS3FYNlBxeFZXMlF1azlGM3hU?=
 =?utf-8?B?YW85cmI2QzJmdVlKNEJqVXZVQ3ViR3g2aStmM25EVWt4MXh6ejBUS2h6S3hi?=
 =?utf-8?B?ekxuaUxlMThXcUc3M1NYSXk2QzQ0RlBCUE5mOE5PZjMxdmc3MjNqbktGV2J0?=
 =?utf-8?B?YnhtZDVJWjJ6em1keEN1S2ZlVkRnKy9hZmFsSXFreHB3WjhPODNsTTlnTWNQ?=
 =?utf-8?B?bmZGWFpSYUoyVG1PNE9xd3ZWeVJsMVptWE5OU0djVWp6Umx4N1hramhjRUFO?=
 =?utf-8?B?Q2xWVHluOEU5RllpdnpCcGNiQTd6VXFWaHhpL000UlpicWltTHpZc2NJeUNp?=
 =?utf-8?B?ZHlTZTlId0kzc0xxeVlBci9QSWlvYjkyb0FyaDExSldQWXM5MU4rWExEZVJv?=
 =?utf-8?Q?JsnPRCsjCEmD5aI4M68AoZvf0?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 388f4dc0-6c6b-43c6-31c3-08dc79266aa7
X-MS-Exchange-CrossTenant-AuthSource: BL1PR11MB5978.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 23:41:48.6717
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: OkKtdKjS7jT0Wu+cf6ZoSyRCR8d2b4ygh1oxqE0pZrZYIVERNAlrlhrUAEvOy/oIwY79ubshrZBVEp07h/ltZw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB6066
X-OriginatorOrg: intel.com


On 21/05/2024 11:15 am, Sean Christopherson wrote:
> On Tue, May 21, 2024, Kai Huang wrote:
>> On 21/05/2024 5:35 am, Sean Christopherson wrote:
>>> On Mon, May 20, 2024, Kai Huang wrote:
>>>> I am wondering whether this can be done in the KVM page fault handler?
>>>
>>> No, because the state of a pfn in the RMP is tied to the guest_memfd inode,
>>> not to the file descriptor, i.e. not to an individual VM.
>>
>> It's strange that as state of a PFN of SNP doesn't bind to individual VM, at
>> least for the private pages.  The command rpm_make_private() indeed reflects
>> the mapping between PFN <-> <GFN, SSID>.
> 
> s/SSID/ASID
> 
> KVM allows a single ASID to be bound to multiple "struct kvm" instances, e.g.
> for intra-host migration.  If/when trusted I/O is a thing, presumably KVM will
> also need to share the ASID with other entities, e.g. IOMMUFD.

But is this the case for SNP?  I thought due to the nature of private 
pages, they cannot be shared between VMs?  So to me this RMP entry 
mapping for PFN <-> GFN for private page should just be per-VM.

> 
>> 	rc = rmp_make_private(pfn_aligned, gfn_to_gpa(gfn_aligned),
>> 			level, sev->asid, false);
>>
>>> And the NPT page tables are treated as ephemeral for SNP.
>>
>> Do you mean private mappings for SNP guest can be zapped from the VM (the
>> private pages are still there unchanged) and re-mapped later w/o needing to
>> have guest's explicit acceptance?
> 
> Correct.
> 
>> If so, I think "we can zap" doesn't mean "we need to zap"?
> 
> Correct.
> 
>> Because the privates are now pinned anyway.
> 
> Pinning is an orthogonal issue.  And it's not so much that the pfns are pinned
> as it is that guest_memfd simply doesn't support page migration or swap at this
> time.

Yes.

> 
> Regardless of whether or not guest_memfd supports page migration, KVM needs to
> track the state of the physical page in guest_memfd, e.g. if it's been assigned
> to the ASID versus if it's still in a shared state.

I am not certain this can impact whether we want to do RMP commands via 
guest_memfd() hooks or TDP MMU hooks?

> 
>> If we truly want to zap private mappings for SNP, IIUC it can be done by
>> distinguishing whether a VM needs to use a separate private table, which is
>> TDX-only.
> 
> I wouldn't say we "want" to zap private mappings for SNP, rather that it's a lot
> less work to keep KVM's existing behavior (literally do nothing) than it is to
> rework the MMU and whatnot to not zap SPTEs.  

My thinking too.

> And there's no big motivation to
> avoid zapping because SNP VMs are unlikely to delete memslots.

I think we should also consider MMU notifier?

> 
> If it turns out that it's easy to preserve SNP mappings after TDX lands, then we
> can certainly go that route, but AFAIK there's no reason to force the issue.

No I am certainly not saying we should do SNP after TDX.  Sorry I didn't 
closely monitor the status of this SNP patchset.

My intention is just wanting to make the TDP MMU common code change more 
useful (since we need that for TDX anyway), i.e., not effectively just 
for TDX if possible:

Currently the TDP MMU hooks are called depending whether the page table 
type is private (or mirrored whatever), but I think conceptually, we 
should decide whether to call TDP MMU hooks based on whether faulting 
GPA is private, _AND_ when the hook is available.

https://lore.kernel.org/lkml/5e8119c0-31f5-4aa9-a496-4ae10bd745a3@intel.com/

If invoking SNP RMP commands is feasible in TDP MMU hooks, then I think 
there's value of letting SNP code to use them too.  And we can simply 
split one patch out to only add the TDP MMU hooks for SNP to land first.