Received-SPF: pass (google.com: domain of linux-kernel+bounces-147542-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Message-ID: <66cc2113-3417-42d0-bf47-d707816cbb53@oracle.com>
Date: Tue, 16 Apr 2024 16:57:42 -0400
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] KVM/x86: Do not clear SIPI while in SMM
To: Paolo Bonzini <pbonzini@redhat.com>, kvm@vger.kernel.org
Cc: seanjc@google.com, linux-kernel@vger.kernel.org
References: <20240416204729.2541743-1-boris.ostrovsky@oracle.com>
 <c7091688-8af5-4e70-b2d7-6d0a7134dbbe@redhat.com>
Content-Language: en-US
From: boris.ostrovsky@oracle.com
In-Reply-To: <c7091688-8af5-4e70-b2d7-6d0a7134dbbe@redhat.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?utf-8?B?cW5scjFuMzBFRWk4NWJpWUVHTzFZb1QzR0M4UzducTU1NGsrRm1iOFdnekRq?=
 =?utf-8?B?NW93QzlraWtKOWIvdUZIN0x3cUpRUUlPNEdUVjAxL1FXYnY0OXVIbHlUZm1F?=
 =?utf-8?B?M3p0VU1CbHZQSk1GazlpdWVFZENReEdNdjlMVkpuZWsrTldFbXFqTVpNa2tF?=
 =?utf-8?B?NmkvTWxQNE1UaFhleTVvUVhEL2gvTlVBQnNyKy85NFJNVVYybU9iNlBWTm5v?=
 =?utf-8?B?dkx2MU9FN3llNGM0Rkt0SStjWksrcVVBWkkrbUdrN2pzTE00bHZQVUljNHFV?=
 =?utf-8?B?VWJ2dlZpaE5EalB0bXNQU05IU0Q4Ymc5Qm9vVFhQVXI3THdvQngwVGdSUmQ3?=
 =?utf-8?B?YUxaaHN3d1VsekhCQU9RU1lsWWRPZ1lrbWFkVzNzZ3pDajBFL3d2RG45TU5i?=
 =?utf-8?B?V1ZncnVIbjhyeTdaQ3pFUVEwWDg0U2UzQ0pOUjZMOWZ4L0g5bkxpNmRYQVZW?=
 =?utf-8?B?T3hoRVU0d1VsN2ExazBvK3RseEM3cEI1UWhGM2lIdlY0U3pDQVBHbDkxWkxK?=
 =?utf-8?B?ckl2UUpnSGc1bldOOUV2dXA4TWdZWXlFTWlXYWpoZ1hYWll2RGdGeW4rU1cx?=
 =?utf-8?B?NG96QlNjQmJObERjQzNjT3pGOXVBRU9SOVM1L3ZDNTJQQnlwYmxEQkxxM0Nn?=
 =?utf-8?B?WXR4c25wSlpISDdQcWFieUlSZENWV0c2WHZiQlVabmtneWlrUzZ5TDJGekRT?=
 =?utf-8?B?L3dmZ3pHdlc3UWJINGtsemRyRzJPSVNHV0hPQzFxdWVwMDhhVlM1a09rNGUx?=
 =?utf-8?B?dHM2T3NvY3dtREFEREhjWTFhZFpPQzRsSzJXb1J5Tk9NMno4MS9lejZsU1BZ?=
 =?utf-8?B?SEc2SS9sd2RVb05lMVVmWE1sbk1MaDZ3ZkpMZ1crdG9xSzlwTE5jNUl3Tk0v?=
 =?utf-8?B?UzVMajlyYmhrUzYrUWlpbFFzeWkxWXpGQnZKV3ZaV2ZybzdWRFZVWjUwcVhr?=
 =?utf-8?B?WXV3VlBTNS9TRW13S2taRE9uandKNlJKV2V4anVyMlN0K0IrMmVGaENqUjJ6?=
 =?utf-8?B?TXAwa250RkRBaklkVXlXMHpFeS8rd2RRZkVBM1k4L1Jxa3QxblJlS1orMk9u?=
 =?utf-8?B?NGpQV1lqYUVEUE1Wck94TnRnMTFKb3cySzVOcFBkcGFyWnQzUXNCNUZ6V0VZ?=
 =?utf-8?B?dnBOUzVKTkhKZWFPckV6c295UWxFS01FaitsQ0tXck9FYUJLMmxIOG9KS0dK?=
 =?utf-8?B?RW8wbVhrajdybmhMWnVuTXBuQ3E4RStiZFIxcjdYeVk5MFM2blJYcytyNDFr?=
 =?utf-8?B?RHZsMTVMWVpscGdLZC9sdlpTcURXZ1NEcVdWb2x4Q3lBU3ZsLzlwTDZGY0J4?=
 =?utf-8?B?UXRTMVAxMXRORXFLa0JUTDBvQUlSeUNlQmRIcmlHTmZrSkhvb2ZTdkh5cm56?=
 =?utf-8?B?d3p0VDFlQUU0eVVGS1hWanZJVXo0WndzT3QwdnkwckZRc0x3cEE0YkQzekhO?=
 =?utf-8?B?cVVNLzdVWUZRZzRZbWVPYWJqY3QydjV6VFB2TFNVYVN6VkFlamNmT012b1FW?=
 =?utf-8?B?WDdwd2JONHZIN0d5aTNUUHlyRC9CNU56ZjUrTHkxOUphTnkzOUovNDVlaUVJ?=
 =?utf-8?B?TnhHOGkxOTIxMUhFaTlIVUY2NFZkVE5JVEdPcktZRS9hVFhOMzhvMjcxcDJ6?=
 =?utf-8?B?SHZaTWlqei9MazdWMXU5R3EvcjVIVmkycUdGcDFFK0UyTVlaZFJKK1JlQ0N3?=
 =?utf-8?B?WU96ZTdlYlAveGdZdXlXVFh2dEcxbU1OZmhINHRRT0RlSVpmRUE1bXdmT2hl?=
 =?utf-8?B?UlkyblZXMlBib0h1M0hjUlZKY0xxZmdVUlQzbTZpNTY2cWtOa3grN0hKZmVq?=
 =?utf-8?B?TnV2Tk5HaTJEVTVhd2RScGQ3d25Cd1I3a0JQSm1EYm9tSHdiNGlkUUR6U3lH?=
 =?utf-8?B?UnFIeFlLb3ZMOVlmcTdxczZNYmJkYUEyOHBHRmhOcDFJeUVTcm50SFZLVjU5?=
 =?utf-8?B?WE5XSkxWemdPcThrQys5MjljUzUzUVNXU2M0cmhXb0c3RDB3bGlvWkFoZzNk?=
 =?utf-8?B?WDVYQmtZSklsVS9IRkV3cEZjVWl1TGVBR3dNWTZsTW9aazAyN0FIb1F0K1ZK?=
 =?utf-8?B?WDdlTTZRanFUWlRYcHE3WHZycEprb3MyOHJrUzdMUG1ManZnTm03cU5ZMm1r?=
 =?utf-8?B?L2xvVStnN2c2UE1kWm9QMmNKZVlPU05pZVpZaFliVVBEWXY4Z29Ma0tWc1Nl?=
 =?utf-8?B?QWc9PQ==?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	YlEXbIoHrAMM5zNiahX8l6z3LQvyWqDUOki4vYyF8wa+Oc7pBX5E5YQDjBnQwhO+XmiOYlMNYmrazj/mTEfPypBthFv7ad6tDKFqn9aXRPzQovqnjEYCEOSV85dfYjD6ZiPQBPDGysb4ClFLN9bUbq/AmoOIVwVtG+WgyQMjaiwBegbiqn1ToGaOY1AozQve8b5Ynv5Uie7f/Gs/I3moSs8Jw6cibMKupdgXvQdu08i12eOhzOIDexmkAChVY4UBcm8wlFBn4A97J0UoiMA4FObjoFpfxmHhbXkpcQBVnI51hDS8/cfypiJDQdP5D1MPVQssow2roHJeERrLHPDmJ7IV6Kcq2vnJILo4+B649nUrpZgqTkwQMuo9PI+gC1/Sjw17YqI1a9gZp3OQsDln6MX3CC+I+roFzuWIaEFXsW4WPcRN0pbXvAsIEJs5DU/XY8N5Qs8N/sIXgUkuTfU/lNpyw2KvWQgqVyzlvHGSX9IaGkhG372r0eNGFXJzgZcIG+567qHbhvrASlxROh7cZXYgpUsV5GWXJUARVocife9Fix3waYll0K8SXYclITta/t2b603VK5YLS0lut2EBIcryHX2wPD+sVQHweuMHpME=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 10629171-8cdb-4a6e-0b95-08dc5e57dde2
X-MS-Exchange-CrossTenant-AuthSource: BLAPR10MB5009.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Apr 2024 20:57:45.9097
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: +CrpEs85SnHmYiSphOJTatVSFxGkuYHdd6NoSovbvDVTZiVNWy+r4mkM6B5qJybXfT2XOh0cM+MfDHB72jEWr77XLjEunKVmVo7csxSERIo=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA2PR10MB4571
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2024-04-16_18,2024-04-16_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 spamscore=0 mlxlogscore=999
 mlxscore=0 adultscore=0 phishscore=0 malwarescore=0 suspectscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2404010000
 definitions=main-2404160134
X-Proofpoint-ORIG-GUID: mEi8LRUlAww14aVtlPxaoRAQJ2_nD1Gs
X-Proofpoint-GUID: mEi8LRUlAww14aVtlPxaoRAQJ2_nD1Gs


On 4/16/24 4:53 PM, Paolo Bonzini wrote:
> On 4/16/24 22:47, Boris Ostrovsky wrote:
>> When a processor is running in SMM and receives INIT message the 
>> interrupt
>> is left pending until SMM is exited. On the other hand, SIPI, which
>> typically follows INIT, is discarded. This presents a problem since 
>> sender
>> has no way of knowing that its SIPI has been dropped, which results in
>> processor failing to come up.
>>
>> Keeping the SIPI pending avoids this scenario.
>
> This is incorrect - it's yet another ugly legacy facet of x86, but we 
> have to live with it.  SIPI is discarded because the code is supposed 
> to retry it if needed ("INIT-SIPI-SIPI").


I couldn't find in the SDM/APM a definitive statement about whether SIPI 
is supposed to be dropped.


>
> The sender should set a flag as early as possible in the SIPI code so 
> that it's clear that it was not received; and an extra SIPI is not a 
> problem, it will be ignored anyway and will not cause trouble if 
> there's a race.
>
> What is the reproducer for this?
>

Hotplugging/unplugging cpus in a loop, especially if you oversubscribe 
the guest, will get you there in 10-15 minutes.

Typically (although I think not always) this is happening when OVMF if 
trying to rendezvous and a processor is missing and is sent an extra SMI.


-boris