Subject: Re: [PATCH] x86: kvm: Avoid guest page table walk when gpa_available
 is set
To: =?UTF-8?B?UmFkaW0gS3LEjW3DocWZ?= <rkrcmar@redhat.com>
References: <1493049146-19261-1-git-send-email-brijesh.singh@amd.com>
 <20170424205236.GE5713@potion>
CC: <brijesh.singh@amd.com>, <pbonzini@redhat.com>, <joro@8bytes.org>,
        <kvm@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
        <tglx@linutronix.de>, <mingo@redhat.com>, <hpa@zytor.com>,
        <x86@kernel.org>, <Thomas.Lendacky@amd.com>
From: Brijesh Singh <brijesh.singh@amd.com>
Message-ID: <77f51978-5937-0c94-13b6-885345921b03@amd.com>
Date: Mon, 24 Apr 2017 17:14:21 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101
 Thunderbird/45.8.0
MIME-Version: 1.0
In-Reply-To: <20170424205236.GE5713@potion>
Content-Type: text/plain; charset="windows-1252"; format=flowed
Content-Transfer-Encoding: 7bit
SpamDiagnosticOutput: 1:99
SpamDiagnosticMetadata: NSPM
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Apr 2017 22:14:24.9053 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR1201MB0139
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
Content-Length: 3944
Lines: 126

Hi Radim,


>>  	/* GPA available (AMD only) */
>>  	bool gpa_available;
>> +	gpa_t gpa_val;
>
> Can't we pass this information through function parameters?
>
> (I'd rather avoid intractable variables.)
>

I also wanted to avoid adding yet another variable but we can't depend on
cr2 parameters passed into x86_emulate_instruction().

The x86_emulate_instruction() function is called from two places:

1) handling the page-fault.
pf_interception [svm.c]
  kvm_mmu_page_fault [mmu.c]
   x86_emulate_instruction [x86.c]

2) completing the IO/MMIO's from previous instruction decode
kvm_arch_vcpu_ioctl_run
  complete_emulated_io
    emulate_instruction
     x86_emulate_instruction(vcpu, 0, emulation_type, NULL, 0)

In #1, we are guaranteed that cr2 variable will contain a valid GPA but
in #2, CR2 is set to zero.

>>  };
>>
>>  struct kvm_lpage_info {
>> diff --git a/arch/x86/kvm/svm.c b/arch/x86/kvm/svm.c
>> index 5fba706..8827e4b 100644
>> --- a/arch/x86/kvm/svm.c
>> +++ b/arch/x86/kvm/svm.c
>> @@ -4159,6 +4159,10 @@ static int handle_exit(struct kvm_vcpu *vcpu)
>>
>>  	vcpu->arch.gpa_available = (exit_code == SVM_EXIT_NPF);
>>
>> +	/* On #NPF, exit_info_2 contain a valid GPA */
>> +	if (vcpu->arch.gpa_available)
>> +		vcpu->arch.gpa_val = svm->vmcb->control.exit_info_2;
>
> How is vcpu->arch.gpa_val used between here and the NPF handler?
>

handle_exit [svm.c]
  pf_interception  [svm.c]
/* it invokes the fault handler with CR2 = svm->vmcb->control.exit_info_2 */
   kvm_mmu_page_fault [mmu.c]
    x86_emulate_instruction  [x86.c]
     emulator_read_write_onepage  [x86.c]
      /*
       *this is where we walk the guest page table to translate
       * a GVA to GPA. If gpa_available is set then we use the
       * gpa_val instead of walking the pgtable.
       */

>> +
>>  	if (!is_cr_intercept(svm, INTERCEPT_CR0_WRITE))
>>  		vcpu->arch.cr0 = svm->vmcb->save.cr0;
>>  	if (npt_enabled)
>> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
>> @@ -4653,18 +4653,16 @@ static int emulator_read_write_onepage(unsigned long addr, void *val,
>>  	 * occurred.
>>  	 */
>>  	if (vcpu->arch.gpa_available &&
>> -	    emulator_can_use_gpa(ctxt) &&
>> -	    vcpu_is_mmio_gpa(vcpu, addr, exception->address, write) &&
>> -	    (addr & ~PAGE_MASK) == (exception->address & ~PAGE_MASK)) {
>> +		emulator_can_use_gpa(ctxt) &&
>> +		(addr & ~PAGE_MASK) == (exception->address & ~PAGE_MASK)) {
>>  		gpa = exception->address;
>> -		goto mmio;
>> +		ret = vcpu_is_mmio_gpa(vcpu, addr, gpa, write);
>> +	} else {
>> +		ret = vcpu_mmio_gva_to_gpa(vcpu, addr, &gpa, exception, write);
>> +		if (ret < 0)
>> +			return X86EMUL_PROPAGATE_FAULT;
>>  	}
>>
>> -	ret = vcpu_mmio_gva_to_gpa(vcpu, addr, &gpa, exception, write);
>> -
>> -	if (ret < 0)
>> -		return X86EMUL_PROPAGATE_FAULT;
>> -
>>  	/* For APIC access vmexit */
>>  	if (ret)
>>  		goto mmio;
>> @@ -5675,8 +5673,14 @@ int x86_emulate_instruction(struct kvm_vcpu *vcpu,
>>  	}
>>
>>  restart:
>> -	/* Save the faulting GPA (cr2) in the address field */
>> -	ctxt->exception.address = cr2;
>> +	/*
>> +	 * Save the faulting GPA (cr2) in the address field
>> +	 * NOTE: If gpa_available is set then gpa_val will contain a valid GPA
>> +	 */
>> +	if (vcpu->arch.gpa_available)
>> +		ctxt->exception.address = vcpu->arch.gpa_val;
>> +	else
>> +		ctxt->exception.address = cr2;
>
> And related, shouldn't vcpu->arch.gpa_val be in cr2?
>

See my previous comment. In some cases CR2 may be set to zero
(e.g when completing the instruction from previous io/mmio page-fault).

If we are decide to add the gpa_val then we can remove above if
statement from x86_emulate_instruction() and update emulator_read_write_onepage
to use the vcpu->arch.gpa_val instead of exception->address.

if (vcpu->arch.gpa_available &&
      emulator_can_use_gpa(ctxt) &&
      (addr & ~PAGE_MASK) == (exception->address & ~PAGE_MASK)) {
       gpa = vcpu=>arch.gpa_val;
       ...
       ...
    }

-Brijesh