Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 2 Aug 2023 19:56:01 +0800
From:   Wu Zongyo <wuzongyo@mail.ustc.edu.cn>
To:     Tom Lendacky <thomas.lendacky@amd.com>,
        Sean Christopherson <seanjc@google.com>
Cc:     linux-kernel@vger.kernel.org, kvm@vger.kernel.org, x86@kernel.org,
        linux-coco@lists.linux.dev
Subject: Re: [Question] int3 instruction generates a #UD in SEV VM
Message-ID: <ZMpEUVsv5hSmrcH8@iZuf6hx7901barev1c282cZ>
Reply-To: Wu Zongyo <wuzongyo@mail.ustc.edu.cn>
References: <8eb933fd-2cf3-d7a9-32fe-2a1d82eac42a@mail.ustc.edu.cn>
 <ZMfFaF2M6Vrh/QdW@google.com>
 <4ebb3e20-a043-8ad3-ef6c-f64c2443412c@amd.com>
 <544b7f95-4b34-654d-a57b-3791a6f4fd5f@mail.ustc.edu.cn>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <544b7f95-4b34-654d-a57b-3791a6f4fd5f@mail.ustc.edu.cn>
Precedence: bulk

On Mon, Jul 31, 2023 at 11:45:29PM +0800, wuzongyong wrote:
> 
> On 2023/7/31 23:03, Tom Lendacky wrote:
> > On 7/31/23 09:30, Sean Christopherson wrote:
> >> On Sat, Jul 29, 2023, wuzongyong wrote:
> >>> Hi,
> >>> I am writing a firmware in Rust to support SEV based on project td-shim[1].
> >>> But when I create a SEV VM (just SEV, no SEV-ES and no SEV-SNP) with the firmware,
> >>> the linux kernel crashed because the int3 instruction in int3_selftest() cause a
> >>> #UD.
> >>
> >> ...
> >>
> >>> BTW, if a create a normal VM without SEV by qemu & OVMF, the int3 instruction always generates a
> >>> #BP.
> >>> So I am confused now about the behaviour of int3 instruction, could anyone help to explain the behaviour?
> >>> Any suggestion is appreciated!
> >>
> >> Have you tried my suggestions from the other thread[*]?
> Firstly, I'm sorry for sending muliple mails with the same content. I thought the mails I sent previously 
> didn't be sent successfully.
> And let's talk the problem here.
> >>
> >>    : > > I'm curious how this happend. I cannot find any condition that would
> >>    : > > cause the int3 instruction generate a #UD according to the AMD's spec.
> >>    :
> >>    : One possibility is that the value from memory that gets executed diverges from the
> >>    : value that is read out be the #UD handler, e.g. due to patching (doesn't seem to
> >>    : be the case in this test), stale cache/tlb entries, etc.
> >>    :
> >>    : > > BTW, it worked nomarlly with qemu and ovmf.
> >>    : >
> >>    : > Does this happen every time you boot the guest with your firmware? What
> >>    : > processor are you running on?
> >>    :
> Yes, every time.
> The processor I used is EPYC 7T83.
> >>    : And have you ruled out KVM as the culprit?  I.e. verified that KVM is NOT injecting
> >>    : a #UD.  That obviously shouldn't happen, but it should be easy to check via KVM
> >>    : tracepoints.
> >
> > I have a feeling that KVM is injecting the #UD, but it will take instrumenting KVM to see which path the #UD is being injected from.
> >
> > Wu Zongyo, can you add some instrumentation to figure that out if the trace points towards KVM injecting the #UD?
> Ok, I will try to do that.
You're right. The #UD is injected by KVM.

The path I found is:
    svm_vcpu_run
        svm_complete_interrupts
	    kvm_requeue_exception // vector = 3
	        kvm_make_request

    vcpu_enter_guest
        kvm_check_and_inject_events
	    svm_inject_exception
	        svm_update_soft_interrupt_rip
		    __svm_skip_emulated_instruction
		        x86_emulate_instruction
			    svm_can_emulate_instruction
			        kvm_queue_exception(vcpu, UD_VECTOR)

Does this mean a #PF intercept occur when the guest try to deliver a
#BP through the IDT? But why?

Thanks

> >
> > Thanks,
> > Tom
> >
> >>
> >> [*] https://lore.kernel.org/all/ZMFd5kkehlkIfnBA@google.com