Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Message-ID: <efd76cc3-98af-f70a-74e6-f15f84fd927a@amd.com>
Date:   Thu, 27 Jan 2022 21:59:44 +0530
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.3.2
Cc:     Paolo Bonzini <pbonzini@redhat.com>,
        Sean Christopherson <seanjc@google.com>,
        Vitaly Kuznetsov <vkuznets@redhat.com>,
        Wanpeng Li <wanpengli@tencent.com>,
        Jim Mattson <jmattson@google.com>,
        Joerg Roedel <joro@8bytes.org>,
        Brijesh Singh <brijesh.singh@amd.com>,
        Tom Lendacky <thomas.lendacky@amd.com>,
        kvm list <kvm@vger.kernel.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Bharata B Rao <bharata@amd.com>
Subject: Re: [RFC PATCH 3/6] KVM: SVM: Implement demand page pinning
Content-Language: en-US
To:     Peter Gonda <pgonda@google.com>
References: <20220118110621.62462-1-nikunj@amd.com>
 <20220118110621.62462-4-nikunj@amd.com>
 <CAMkAt6rxeGZ3SpF9UoSW0U5XWmTNe-iSMc5jgCmOLP587J03Aw@mail.gmail.com>
 <04698792-95b8-f5b6-5b2c-375806626de6@amd.com>
 <CAMkAt6rCU3K-dk7edxa9iKrOZ0uh5442K0U5Fbux=js0q6qQ+g@mail.gmail.com>
From:   "Nikunj A. Dadhania" <nikunj@amd.com>
In-Reply-To: <CAMkAt6rCU3K-dk7edxa9iKrOZ0uh5442K0U5Fbux=js0q6qQ+g@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?V0N2UVFNT2ZzKzhkclc0MHFuWUlWcEhzTWVZZ0g2aGN4YUJVNEVQVWw2R1pG?=
 =?utf-8?B?Sy9KTnZ5QVRxYlVyd0VkSDAvLzFvS1hvQ3hVT2xVYUdEYVhwNTNSNjY4QWZv?=
 =?utf-8?B?ZHRHVTdhSXRMSXRiMTgyR1pBT2kzZ0hsVHJ5LzkrRWUxMDJWc1dGTEhDZmdp?=
 =?utf-8?B?cytxN0t5L21HNTRybnUwS2JTR3dMRU9UV1ZRT0wyd24xN3RjVWlqRGpOSjRH?=
 =?utf-8?B?Wm0yWWtOQ0I1K1pCM09UZHlXVVBwM2ErcUhFZlR5TVJTeDRZV2JKay9hZXVI?=
 =?utf-8?B?aFltVk9MQVJtbWVxK3ZJbzhtaUozNGtoRkx4Z1k3Mks0aW1KZEpQWkd2Y0tX?=
 =?utf-8?B?SkJRdXpBQWxXbkJsQUROUFduUzBqdnEzcnUreUlXMmg4UGlFeDFUWStqVk1L?=
 =?utf-8?B?RGk4VU9XNlRWZmNjMnREdzVvQlhXakVBK3V3Nmx5WERzNE5HQmFpbjhMc3NU?=
 =?utf-8?B?NG1QUjg4ZHpVM1QvdUx4NDhyc2NRbnFTdEwyaWQva2FvcWxiSDczNU9ZU0dF?=
 =?utf-8?B?K2pzOU9lVko4aEFZaitxZkhCeWdVS2RJOHVDSUw5Z0dGM015VHEzekR2UTBJ?=
 =?utf-8?B?eWFnZlZDdkFJRzJaSUd1UE1pK1l5WmFWaDZNc0IrcFRGK1dyT0tIR21LekJV?=
 =?utf-8?B?Z3ZNZXNjWVRGVmJuYVRPK2ZkT2VjNlZjcENoRTk5OVpQaUdEODg3SUJzcWov?=
 =?utf-8?B?Ui9rcTArZ3dxOGxyVG10NEJ4NkNRbUZNVm13eU44YStqbUZPSmpZdXRUVVdu?=
 =?utf-8?B?VkI1ZUhmTXVnQVNhZE85emtuOXpBVktiM1lUMW1XbCtYeEdycTJqMGZCL0ty?=
 =?utf-8?B?ZHdPQVRZcmUrQ2U0ZThyam14TVNwNnN3cDFLdWVWaVRHREEzcmlZL1FsdVJG?=
 =?utf-8?B?WlFUUjNsQ01yNXA1YlgyMnpDRDZQWUhaV0lFS2ZkbHhlSytlT0dPcGhkMjVz?=
 =?utf-8?B?Q0t0SFFmY2tCU1J1Y2lVdEJQdEtNOG5PQnF0aTdpR0o1bDlBQ1VBdEEwMWs1?=
 =?utf-8?B?aEJwODlVSTMwTlI0RUc2UEVzaC9QZmgxM3JXVmxhQzBJOUxHcVhUR25zbFFh?=
 =?utf-8?B?YTZYUDFMNkhZcEx6ZUtMckpzelJUbmZWS3lVbkdsS0wvR2l3aVFzSXlTUnMz?=
 =?utf-8?B?WWhBTDhwVEx3Qkt0bm9ORFk1V2gxbVBrbkl0MGxJLzJTOHgyN1V4V0FXek1t?=
 =?utf-8?B?Nk14dGZXdWxQK2Q5RUkrQjVBWk1sbkFNVjNIZTViSFpZVk5NVFhMa2NDbEVS?=
 =?utf-8?B?QXh2azc5WXpYRWV4ZVQyd3NCcm5zOTVHc3RoUGpVaEJ1ellDSmpmbXo1SDhs?=
 =?utf-8?B?WUpxem1rWVZjWC90eVNNeFIrNXRreGFySmdpMlBDRC9WaHZ4ME9Ldy9TaUFx?=
 =?utf-8?B?TVNYeHlDUzJpZkVSS01mNmlWdGZ0WTZlRWdCQlF1ckxwMTRmMTI3NGFhS3RB?=
 =?utf-8?B?NURGaURzN25lQjVoRDVyY1lJb2wyQVVUTXpid2ZNdGNKRmUwTDN0TzJITDBk?=
 =?utf-8?B?MEVKU2s4TFo0SkNwR3kzWEJ6aFA1aUJ1UldGQlBPVWVCNHFPSzZBcWZ5N3Y3?=
 =?utf-8?B?S3VBRzdJcE5sZWEyUm15ZGI3aHcxRFFKTCtURi9SU3FzbW56REs3em5YUm01?=
 =?utf-8?B?WGs3V2VTNUpXaXJqZGh6aG9pVmxLeG1aOVhjTWphcmJKQlJ1bTRySE1iZFlZ?=
 =?utf-8?B?TEdsWWp1em1mUU84M3NJNmxYUkp4YytjVloyN3ZMQmM4K2UySU1KS3podWxm?=
 =?utf-8?B?UzNvYk9FdG1Vb2p3WVc3WUZlZjJCTVVJcXFWVWIrZ25FQ2ttZlFnUWd1UE1Z?=
 =?utf-8?B?OUVPWG9zak5BRVBhZ3prRjR6dGlwZkhrNGtoK2paRjhrVy85b2ZKSWZXTlk3?=
 =?utf-8?B?czJ3UGpmdHlHUG1aUnJVRm9tR0RlQXdrMElyVzU0M2p6R1VPS1hPdCtNdE9t?=
 =?utf-8?B?WUg3NE1ya2lOdk1SSzhBVGhZVmhMa2p4YlJrc3hxUFlLQ0taRUg3YitsVmcy?=
 =?utf-8?B?cHZPcEU0WGRvTnNpcnVBRlFIOFV0ZWVsRXRBUUVnbTZGTjlRS1FQRjYzbk1y?=
 =?utf-8?B?TE91NmtVdnNwSHljamhtVE5PZFpoVVoxd0F4bWhKVTl1UDFSWFdCa3RPZVps?=
 =?utf-8?B?YWlnOEkxeDNydjJKWGVSaHBPL3FVODhwT0lvY1lXOUV3Q01xSm5rWkVUbVpL?=
 =?utf-8?Q?U/PxHJ1qaUXxDETVkJslVDg=3D?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 52c56b82-fba7-409a-af33-08d9e1b242b1
X-MS-Exchange-CrossTenant-AuthSource: DM5PR12MB2470.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Jan 2022 16:29:59.3379
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: UN4gBlaL/uRrEtlsUNM5aSOP0Udlbl++bPDgQZbFrROg0nRVJLZiZl1HxBfBRmpT1BUzPhWP45cMgCIsAUFWmQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB3651
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 1/25/2022 11:29 PM, Peter Gonda wrote:
> On Tue, Jan 25, 2022 at 10:49 AM Nikunj A. Dadhania <nikunj@amd.com> wrote:
>>
>> Hi Peter
>>
>> On 1/25/2022 10:17 PM, Peter Gonda wrote:
>>>> @@ -1637,8 +1627,6 @@ static void sev_migrate_from(struct kvm_sev_info *dst,
>>>>         src->handle = 0;
>>>>         src->pages_locked = 0;
>>>>         src->enc_context_owner = NULL;
>>>> -
>>>> -       list_cut_before(&dst->regions_list, &src->regions_list, &src->regions_list);
>>> I think we need to move the pinned SPTE entries into the target, and
>>> repin the pages in the target here. Otherwise the pages will be
>>> unpinned when the source is cleaned up. Have you thought about how
>>> this could be done?

Right, copying just the list doesn't look to be sufficient. 

In destination kvm context, will have to go over the source region list of 
pinned pages and pin them. Roughly something like the below:

struct list_head *head = &src->pinned_regions_list;
struct pinned_region *new, old;

if (!list_empty(head)) {
	list_for_each_safe(pos, q, head) {
		old = list_entry(pos, struct pinned_region, list);
		/* alloc new region and initialize with old */
		new = kzalloc(sizeof(*region), GFP_KERNEL_ACCOUNT);
		new->uaddr = old->uaddr;
		new->len = old->len;
		new->npages = old->npages;
		/* pin memory */
		new->pages = sev_pin_memory(kvm, new->uaddr, new->npages);
		list_add_tail(&new->list, &dst->pinned_regions_list);
		...
	}
}

>>>
>> I am testing migration with pinned_list, I see that all the guest pages are
>> transferred/pinned on the other side during migration. I think that there is
>> assumption that all private pages needs to be moved.
>>
>> QEMU: target/i386/sev.c:bool sev_is_gfn_in_unshared_region(unsigned long gfn)
>>
>> Will dig more on this.
> 
> The code you linked appears to be for a remote migration. 

Yes, that is correct.

> This
> function is for an "intra-host" migration meaning we are just moving
> the VMs memory and state to a new userspace VMM on the same not an
> entirely new host.

Regards
Nikunj