Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <4ebb3e20-a043-8ad3-ef6c-f64c2443412c@amd.com>
Date:   Mon, 31 Jul 2023 10:03:34 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.13.0
Subject: Re: [Question] int3 instruction generates a #UD in SEV VM
To:     Sean Christopherson <seanjc@google.com>,
        wuzongyong <wuzongyo@mail.ustc.edu.cn>
Cc:     linux-kernel@vger.kernel.org, kvm@vger.kernel.org, x86@kernel.org,
        linux-coco@lists.linux.dev
References: <8eb933fd-2cf3-d7a9-32fe-2a1d82eac42a@mail.ustc.edu.cn>
 <ZMfFaF2M6Vrh/QdW@google.com>
Content-Language: en-US
From:   Tom Lendacky <thomas.lendacky@amd.com>
In-Reply-To: <ZMfFaF2M6Vrh/QdW@google.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?d3gyTWJ2SnFUaS9GT3BYUFVDQ3ZXMkk5bkl0ZVhnS21SWHc3QmtLTC85UW1w?=
 =?utf-8?B?b3FEWSt6R2Z2QTRZeWc2S3d1L0hIa0I3RERRWmE4VDJKYzhpRk1UT0RWdzg1?=
 =?utf-8?B?UzZSWUtUQWMyRjFVVnpCSlBkbDl0U3Y0NytjWm9GN1Y0Sm5hUUFkRE8zRFY4?=
 =?utf-8?B?VkJ6R1dMQTdjYTdFUGVEWlZ2NjRpY0RMdWRBOVF4bzViUWx1b0dLTWVWNDBB?=
 =?utf-8?B?dDYxaDl0Q29ETXp6b1JWZTZDVVdiOE1XcjdmeHp4YVZJaFdUMkFPNVZ6T3dZ?=
 =?utf-8?B?eTNEU2JKWEUxdFhZQWJvYzhNaXlGc3YyWis5VmtTVVVpeHk1SSt0b0x3b1Nx?=
 =?utf-8?B?N3ovWi9BdXJQUXpLM3Q4Q0RxemJYMnRlOXpaZ3RDVCtKR0toek9lbkNGYy9S?=
 =?utf-8?B?MDZmTDB3bFY5SVRsWFd4NzIzOERDR25HWVhEWWZxb29Oc0wxWHQzTEQydFpw?=
 =?utf-8?B?R2FsbHU2M3I0djhUeWZrd0J4dEoweTgwN3haTlhSNjZnQ1l2aUdIRHJxNWdQ?=
 =?utf-8?B?N0gxTEhoRStwMGtCaTB4LzNXVTZYc3NvN3FlWm5EK1hZQm9WdjdUaFNYWUJ3?=
 =?utf-8?B?U2NVd1AzZzNpY21XSGk0dGczRkJGbm9tK1hLSlVYL3FDZmhXQ0c4VlBxQjlu?=
 =?utf-8?B?UHZFL2kzMzlyTGcrQU1nM0RIQXlseC9ObUY4Y3hULys5SWhBVVhJdEpsVmhN?=
 =?utf-8?B?TjdrOEh1aGZHNWV0U1JLeU9SSWR4d0Q5OGRIeEVHc1R6Um9uelVabVdiZHoz?=
 =?utf-8?B?cFRxTGpWbkJFT3MrN0c1MDJFSjNCUStWVGVBci9hTGVmdjJRZE9sSjBwc3M0?=
 =?utf-8?B?ZFdvN3VRVTRleGdsWVpzSzNHZHpjdnAxeXB6b01aTTJLNkxOOWo5UWROM3li?=
 =?utf-8?B?a1QvMmF0bGkwaFkvUjJ0UlU5N0dtVHBUMnNtc2x4MjhzODMzVlVvT2dzZGVj?=
 =?utf-8?B?VXhTSE1wRVFHQURROXBWODkwWHc1QnZId0d6RFB4R0dBTWdGbllwQmh3ZVlp?=
 =?utf-8?B?Z2txY1k0Vkd5QTlremhUWEpHa1RXVXdlMk9DR2dHbVFWUU1KbG1ITHFBUi9W?=
 =?utf-8?B?Q3FlSGhNTUlXRlh6NmZVWWozaGlORmlOMmUzR3d2YmlZUHQrdjRYRU9NK0NE?=
 =?utf-8?B?em91aDNuYjNiY1ZuRTRyNExDWk5waDgxdkIya3haN3VVRk93MkxUWFZ4WUgr?=
 =?utf-8?B?eWh1WjQxNFphRmRnQlFxQ083V0pGa2xYdXNOUUhPRXJiOStCOVRMWHI0Mkkr?=
 =?utf-8?B?Q2E2bmxyUStnbDNhSEtXTTVCMjdaS01VV1pJSkYrR2s5OEFyNFdqZFV3WCs4?=
 =?utf-8?B?N1ZDbW5udlZldlY5N3VnTk9hc21PaTNOTCtJZHJoYnJUaWRrQXByeDNjeGpp?=
 =?utf-8?B?R2ZVWndvOTgrUlpTWkdIT09SalpXYXdnQmtEaDhxNHM2dC80SGZJaEwrcXkv?=
 =?utf-8?B?cXFiZnJneWJHMG5ndkYrTmtBaEN2Q3RwdGZGNmVRUXpoUlkyTHBmVzJJbThv?=
 =?utf-8?B?VFh0bXlTRlROZ1R3Ty9rcFh5QWZ4VFpDOVpuclpvSWh1TnZUbkZwZlo3Y3h0?=
 =?utf-8?B?ZjluUjRCWGhNdm9DRW5oTUlTcXJvdGhtQy9nZnhDU3ZxQ0JkRHQyaHJKY1pQ?=
 =?utf-8?B?ZGltbTBIU2NUSm4ybTVVT3I0UyttNmJlaXRRcy9BUnVaRFdvMi9wNWNIS3Za?=
 =?utf-8?B?bjZuTjlLTTllK1NXa1orY2xjaHZhR0N5Zk5pNTI0YW5wWFNYVFFidjIrQ1F3?=
 =?utf-8?B?ZzlveDQ1VmUwUmsrYlBzcnQ2VGhBNEFZdXJ0bXlWMlBuWWNqVTE0L3pyblZH?=
 =?utf-8?B?SG8ySkxPdXZaSm9MQzE0UHAvRkJTMVVFZk9OSys4NmxEb1AxWm5uMHpQdDRx?=
 =?utf-8?B?M0ZRTmpJSGx1dlg0cEMrcTFOcFN2MGx1eW9LbGZHY3EzM2paRzUyZkxiMmxI?=
 =?utf-8?B?eWZ4WDZjcmhFRldOVVZIWnFiZGpCUnVxY01DNUYrMkltQUVTajdQaGl1NGt0?=
 =?utf-8?B?dFJhQms4REtJSVdsT1NiZHdicjI0SFNxQVJqY25ZRDJYdDdXVEltUFBJU0hm?=
 =?utf-8?B?UXVqTzg0a09GM1FjMVp3SVV6Nm9mZnpyRXhKcE9qdW52SUlKa0hpczFyQ3V2?=
 =?utf-8?Q?VPuaALK5ae52iVCra14/qihpd?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4634bdb9-d5db-4a65-73de-08db91d75151
X-MS-Exchange-CrossTenant-AuthSource: DM4PR12MB5229.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Jul 2023 15:03:37.3202
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: lxbuVkOFkeNZUX5wXXmbNw1VtMl4a+DBVCTbA+Kuz0bVOkXg9xkTU0Ts414KiF4sb9xF9xZr69GtUj4/3uI0cQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB8804
X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,NICE_REPLY_A,
        RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 7/31/23 09:30, Sean Christopherson wrote:
> On Sat, Jul 29, 2023, wuzongyong wrote:
>> Hi,
>> I am writing a firmware in Rust to support SEV based on project td-shim[1].
>> But when I create a SEV VM (just SEV, no SEV-ES and no SEV-SNP) with the firmware,
>> the linux kernel crashed because the int3 instruction in int3_selftest() cause a
>> #UD.
> 
> ...
> 
>> BTW, if a create a normal VM without SEV by qemu & OVMF, the int3 instruction always generates a
>> #BP.
>> So I am confused now about the behaviour of int3 instruction, could anyone help to explain the behaviour?
>> Any suggestion is appreciated!
> 
> Have you tried my suggestions from the other thread[*]?
> 
>    : > > I'm curious how this happend. I cannot find any condition that would
>    : > > cause the int3 instruction generate a #UD according to the AMD's spec.
>    :
>    : One possibility is that the value from memory that gets executed diverges from the
>    : value that is read out be the #UD handler, e.g. due to patching (doesn't seem to
>    : be the case in this test), stale cache/tlb entries, etc.
>    :
>    : > > BTW, it worked nomarlly with qemu and ovmf.
>    : >
>    : > Does this happen every time you boot the guest with your firmware? What
>    : > processor are you running on?
>    :
>    : And have you ruled out KVM as the culprit?  I.e. verified that KVM is NOT injecting
>    : a #UD.  That obviously shouldn't happen, but it should be easy to check via KVM
>    : tracepoints.

I have a feeling that KVM is injecting the #UD, but it will take 
instrumenting KVM to see which path the #UD is being injected from.

Wu Zongyo, can you add some instrumentation to figure that out if the 
trace points towards KVM injecting the #UD?

Thanks,
Tom

> 
> [*] https://lore.kernel.org/all/ZMFd5kkehlkIfnBA@google.com