Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Message-ID: <0e523405-f52c-b152-1dd3-aa65a9caee3c@amd.com>
Date:   Wed, 19 Jan 2022 12:03:34 +0530
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.3.2
Cc:     Sean Christopherson <seanjc@google.com>,
        Vitaly Kuznetsov <vkuznets@redhat.com>,
        Wanpeng Li <wanpengli@tencent.com>,
        Jim Mattson <jmattson@google.com>,
        Joerg Roedel <joro@8bytes.org>,
        Brijesh Singh <brijesh.singh@amd.com>,
        Tom Lendacky <thomas.lendacky@amd.com>,
        Peter Gonda <pgonda@google.com>, kvm@vger.kernel.org,
        linux-kernel@vger.kernel.org, Paolo Bonzini <pbonzini@redhat.com>,
        Bharata B Rao <bharata@amd.com>
Subject: Re: [RFC PATCH 6/6] KVM: SVM: Pin SEV pages in MMU during
 sev_launch_update_data()
Content-Language: en-US
To:     "Maciej S. Szmigiero" <mail@maciej.szmigiero.name>
References: <20220118110621.62462-1-nikunj@amd.com>
 <20220118110621.62462-7-nikunj@amd.com>
 <010ef70c-31a2-2831-a2a7-950db14baf23@maciej.szmigiero.name>
From:   "Nikunj A. Dadhania" <nikunj@amd.com>
In-Reply-To: <010ef70c-31a2-2831-a2a7-950db14baf23@maciej.szmigiero.name>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ekJPb0RSSzE4enBOM21Mc0J3aWkxd21vRHFrT2hUeUVlb2NjQzB3K2VTMnU1?=
 =?utf-8?B?T2h2USsrRWhjamlObHdKOTJLeS83dFNlTk1BTGtkK0J4OW5sdjhlMS85OTl1?=
 =?utf-8?B?NEhVQ2RydFZqYkhKNzViUUlOb1NuSVlnbE5sc1NtS29UTmw0REt2TTFvb0wr?=
 =?utf-8?B?dGJZUTJsWTN4UjJnYmMrVVlNWUt5Y0lTSTFDZWdueXV0ZU15NW5mYzB4WjBx?=
 =?utf-8?B?K0hkL3NKMHFsN2dNSldkZWVDdUJhS3Z0MHhia2lncEY3WE54ZEhsTzFOQzFI?=
 =?utf-8?B?YzBSR09nY0U5VzBDQWxyVmhUSExMaElzdzczL1FzS205U0ZvQWw5M0Jsb3py?=
 =?utf-8?B?MUVPVEJiRVplSnNKck1KU2xnTlh5MVI0QUxTa1dpUnJQSld1b2QwRWJPVlZp?=
 =?utf-8?B?Z3U3bXplSDNqbjFlNHdQbXBPdkFoeXZzNEpKM0FVcmwvT2NVYThkZUNBdHIw?=
 =?utf-8?B?WGNEalVlTmtEQnY4UC9ZeW8zSXNpd1hvVDBaTEpHSlJyUVNTYlVXdmpJOWFG?=
 =?utf-8?B?YU9qM0ZSL3NiNjlhaVM1TDdXK014cXFHQmsyMzRNS3Qzek5aNXdHd2tvSXBa?=
 =?utf-8?B?NFZVbm1rQVRoVXUwcytUaERMRlc0MGZIajRNTXplR08ydmowQ1lsdnJUNWNj?=
 =?utf-8?B?cDliVlJMcXVNckVJc1V6RUhXcnp5UzdTQ1BaKzRNRjM2MVk2ZEU4N0szQ3Jt?=
 =?utf-8?B?MXdsMlVuaDJjbjVJT0RpbjVvNmI4blMzZDZEZUJWb2p4ZjdPTFpNM3R0Uit2?=
 =?utf-8?B?ZzYzdS9GV05hMktPVkFsSGE4UzBIWnFkWmcybXF2UXZLR0RRaW16SThXSGVN?=
 =?utf-8?B?TUhPWk5wcnY1VlFHSXd2NUVDQUNrUExhUzhaUDA4cFd1U3RneElWOC9USVJp?=
 =?utf-8?B?blJSVzh2QzdWMVA4RGR2RGdrT0lQeTFhd2ErdDBHbzIzOWRFV0d1MG10NEcx?=
 =?utf-8?B?cXVnY3pzcDlncXZkV0ZYQnl4VVQ0TkF6L3Z2S3NrakpEWXZCTzVBR1FSU0FU?=
 =?utf-8?B?Y2dCRTBtVXMxYlc4b0xoU2JIUExoOVJDN3NKNCtIc2JtS24wT2FRSlJCMk56?=
 =?utf-8?B?dW93UnBZNWlyY2pWbnhZejEvNE5ZQ2hVM1JjTUJLeUxPak5GbEFNRFlUM3Jh?=
 =?utf-8?B?bDBIZkhyYkRPeU94SWdaNlJuM0IyeWFoa2VQYTNiSk1BSmZmQzFRU0dJQnpz?=
 =?utf-8?B?cllGSDkvZmdacXo2ZEkwSmcvQ1FQVUNMa0JLU0RLKy9iZ3hJMUdwS2RkU015?=
 =?utf-8?B?M1lDSkloZzJlRExZMmVPQ1ZPUCsrL2xOWlE4WVZRa3Rya2hTL3VFK09kQ1ds?=
 =?utf-8?B?MjRZaFZRUm1lVFg0VVNzSW1nQThxUXNZdFIzYlVFemIveEREY1VKR3lUVXkw?=
 =?utf-8?B?UmRlRVJraEhVb0pTSVQyZFY1MHhjREpuN002aTdhamthc2FMbllSbnpUZHFC?=
 =?utf-8?B?NHZYQkplKzlVbzlxNDJsZU4veXloR1BkeVRjRkZrd0NaWCtuV1NsNkJkQStP?=
 =?utf-8?B?MDg2VVVDaXU5bWVabUpITXc4dXRpdXpYVjlQNzVCMTZjNnVKZm15ZHZnUDQx?=
 =?utf-8?B?TDZmQzI3Q0FpdjNzdWh0UVVydkttU2d2enJXakNTMGtzYU9OMUtHV2pOMEJD?=
 =?utf-8?B?QXhvSzJvVEdPUWdVNjdjK2Y1cCtha0hrRkJLbGhQYTVQRmZscEJwNHU4ejFz?=
 =?utf-8?B?SWtnbWZMUWo4M3hsam8zeE54Zm5BMHpzNEloZnJJVTJPN2RZVGJuNzR0NHlp?=
 =?utf-8?B?N2lPSmR0amoraFdMSGxyU0VmSlFHS203enRYYWROYmQvSEFaNHAzSUFsdDlD?=
 =?utf-8?B?aGlyZ3djY2J0UUt4SDdGODA1V1NOQ3hQVVZDd2xXUFZkVHg0MnZEdTRyZjQ1?=
 =?utf-8?B?UzBJOElzVWFDY20yTWI4bmV4QlNjeWZyNVlBSjBSUjdEbVdVUUM5RHNQbnFo?=
 =?utf-8?B?MmlrM3l4N1pHWXNoa0pPK3NjdDZDeDJxZ2Z5NElRTWprZ3pwNW1ma1NuVVVy?=
 =?utf-8?B?Q1NOM3g4cmZvRUR4Q3YrOHZuZFRlVEQrNnFFNVdkZUU2ajFlbjhmeTBGRHZI?=
 =?utf-8?B?OXpxMGo4R2lQaStwbnFYUFhQK2ZoNUJDcGhJdjBvZGV3TXNzK0NPemZqMis2?=
 =?utf-8?B?UUJINVBNNkp4bHBRVVZlK1h0eTIyYTFMNzRMRllTekR0K3N4d2F5VEdJVEhM?=
 =?utf-8?Q?Kc+bVj45VeC0a4y73m7ayQE=3D?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: beeeeaa5-3d11-4885-b00d-08d9db15ab94
X-MS-Exchange-CrossTenant-AuthSource: DM5PR12MB2470.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jan 2022 06:33:57.3923
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: DxVWKZN5W7rpnQDWaWLzcUuRkq8YKqXuRmreOBaXnK6KMjw2i5OaOvgHTFnvAIgmgvoQ7b/Kq2vFbGb+tzxtbQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8PR12MB5413
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Maciej,

On 1/18/2022 8:30 PM, Maciej S. Szmigiero wrote:
> Hi Nikunj,
> 
> On 18.01.2022 12:06, Nikunj A Dadhania wrote:
>> From: Sean Christopherson <sean.j.christopherson@intel.com>
>>
>> Pin the memory for the data being passed to launch_update_data()
>> because it gets encrypted before the guest is first run and must
>> not be moved which would corrupt it.
>>
>> Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
>> [ * Changed hva_to_gva() to take an extra argument and return gpa_t.
>>    * Updated sev_pin_memory_in_mmu() error handling.
>>    * As pinning/unpining pages is handled within MMU, removed
>>      {get,put}_user(). ]
>> Signed-off-by: Nikunj A Dadhania <nikunj@amd.com>
>> ---
>>   arch/x86/kvm/svm/sev.c | 122 ++++++++++++++++++++++++++++++++++++++++-
>>   1 file changed, 119 insertions(+), 3 deletions(-)
>>
>> diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
>> index 14aeccfc500b..1ae714e83a3c 100644
>> --- a/arch/x86/kvm/svm/sev.c
>> +++ b/arch/x86/kvm/svm/sev.c
>> @@ -22,6 +22,7 @@
>>   #include <asm/trapnr.h>
>>   #include <asm/fpu/xcr.h>
>>   +#include "mmu.h"
>>   #include "x86.h"
>>   #include "svm.h"
>>   #include "svm_ops.h"
>> @@ -490,6 +491,110 @@ static unsigned long get_num_contig_pages(unsigned long idx,
>>       return pages;
>>   }
>>   +#define SEV_PFERR_RO (PFERR_USER_MASK)
>> +#define SEV_PFERR_RW (PFERR_WRITE_MASK | PFERR_USER_MASK)
>> +
>> +static struct kvm_memory_slot *hva_to_memslot(struct kvm *kvm,
>> +                          unsigned long hva)
>> +{
>> +    struct kvm_memslots *slots = kvm_memslots(kvm);
>> +    struct kvm_memory_slot *memslot;
>> +    int bkt;
>> +
>> +    kvm_for_each_memslot(memslot, bkt, slots) {
>> +        if (hva >= memslot->userspace_addr &&
>> +            hva < memslot->userspace_addr +
>> +            (memslot->npages << PAGE_SHIFT))
>> +            return memslot;
>> +    }
>> +
>> +    return NULL;
>> +}
> 
> We have kvm_for_each_memslot_in_hva_range() now, please don't do a linear
> search through memslots.
> You might need to move the aforementioned macro from kvm_main.c to some
> header file, though.

Sure, let me try optimizing with this newly added macro.

> 
>> +static gpa_t hva_to_gpa(struct kvm *kvm, unsigned long hva, bool *ro)
>> +{
>> +    struct kvm_memory_slot *memslot;
>> +    gpa_t gpa_offset;
>> +
>> +    memslot = hva_to_memslot(kvm, hva);
>> +    if (!memslot)
>> +        return UNMAPPED_GVA;
>> +
>> +    *ro = !!(memslot->flags & KVM_MEM_READONLY);
>> +    gpa_offset = hva - memslot->userspace_addr;
>> +    return ((memslot->base_gfn << PAGE_SHIFT) + gpa_offset);
>> +}
>> +
>> +static struct page **sev_pin_memory_in_mmu(struct kvm *kvm, unsigned long addr,
>> +                       unsigned long size,
>> +                       unsigned long *npages)
>> +{
>> +    struct kvm_sev_info *sev = &to_kvm_svm(kvm)->sev_info;
>> +    struct kvm_vcpu *vcpu;
>> +    struct page **pages;
>> +    unsigned long i;
>> +    u32 error_code;
>> +    kvm_pfn_t pfn;
>> +    int idx, ret = 0;
>> +    gpa_t gpa;
>> +    bool ro;
>> +
>> +    pages = sev_alloc_pages(sev, addr, size, npages);
>> +    if (IS_ERR(pages))
>> +        return pages;
>> +
>> +    vcpu = kvm_get_vcpu(kvm, 0);
>> +    if (mutex_lock_killable(&vcpu->mutex)) {
>> +        kvfree(pages);
>> +        return ERR_PTR(-EINTR);
>> +    }
>> +
>> +    vcpu_load(vcpu);
>> +    idx = srcu_read_lock(&kvm->srcu);
>> +
>> +    kvm_mmu_load(vcpu);
>> +
>> +    for (i = 0; i < *npages; i++, addr += PAGE_SIZE) {
>> +        if (signal_pending(current)) {
>> +            ret = -ERESTARTSYS;
>> +            break;
>> +        }
>> +
>> +        if (need_resched())
>> +            cond_resched();
>> +
>> +        gpa = hva_to_gpa(kvm, addr, &ro);
>> +        if (gpa == UNMAPPED_GVA) {
>> +            ret = -EFAULT;
>> +            break;
>> +        }
> 
> This function is going to have worst case O(n²) complexity if called with
> the whole VM memory (or O(n * log(n)) when hva_to_memslot() is modified
> to use kvm_for_each_memslot_in_hva_range()).

I understand your concern and will address it. BTW, this is called for a small 
fragment of VM memory( <10MB), that needs to be pinned before the guest execution 
starts.

> That's really bad for something that can be done in O(n) time - look how
> kvm_for_each_memslot_in_gfn_range() does it over gfns.
> 

I saw one use of kvm_for_each_memslot_in_gfn_range() in __kvm_zap_rmaps(), and 
that too calls slot_handle_level_range() which has a for_each_slot_rmap_range().
How would that be O(n) ?

kvm_for_each_memslot_in_gfn_range() {
	...
	slot_handle_level_range()
	...
}

slot_handle_level_range() {
	...
	for_each_slot_rmap_range() {
		...
	}
	...
}

Regards,
Nikunj