Received-SPF: pass (google.com: domain of linux-kernel+bounces-176131-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
From: Dongli Zhang <dongli.zhang@oracle.com>
To: x86@kernel.org
Cc: tglx@linutronix.de, mingo@redhat.com, Borislavbp@alien8.de,
        dave.hansen@linux.intel.com, hpa@zytor.com, joe.jin@oracle.com,
        linux-kernel@vger.kernel.org, virtualization@lists.linux.dev
Subject: [PATCH 1/1] x86/vector: Fix vector leak during CPU offline
Date: Fri, 10 May 2024 12:06:23 -0700
Message-Id: <20240510190623.117031-1-dongli.zhang@oracle.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?us-ascii?Q?DAjITnozhBWV80DBaf9ttLEtSkSYBQXblL0bQ+htYa4OMSoiL3rub1+REwVB?=
 =?us-ascii?Q?Wzrgi11v+DOzKWF0tGn44Pl4iFqQJv0QquPyQeDV95ifOx/8XVOxAmd//EcC?=
 =?us-ascii?Q?/8nB3m4dr8wlavCeeikSTeX7UEFZsLq1od/FtvM9CaeoMOTK0jPWZWPL/Y5+?=
 =?us-ascii?Q?w1ooMJItkZ2JstVqV1iCInIrB60fwrZ/5zKqefkb8TysBLXo6/Gkf4UGpTKc?=
 =?us-ascii?Q?wE7jOrxVoCHHvsYI/l5Zje2r92QBsvCrYagcaqW4pt1T9HI3ns0f9nnRbSm4?=
 =?us-ascii?Q?Wck3UjsLs2/B+6EpB1I9kxxgqDdjhrUjh5CTOnrDutlNOvXC6onSZhLlBKO+?=
 =?us-ascii?Q?e0kuFoMYucdwtbcaf+4QJqeQW1NTKQYvALvAIoy5I3TKnNd0iSWir9zIa5Yt?=
 =?us-ascii?Q?K61Lii8nMmcAr87HbiEPbJrhj97+e3jMkJQBm9ndeO0WacdzmL2yCnpMBIRq?=
 =?us-ascii?Q?QedDe107XzTwaEmeflT2B3XLniJHB8GE+OyWpe699eTQPagrRaiJowF8gsGa?=
 =?us-ascii?Q?mlI45Mbs4OCeQmZuUUfnn3lxfT0TEIw3NDCrDv75J6vGun0DoIF/1z3ne5tW?=
 =?us-ascii?Q?762SdJ72hQ1o8NrwMcs2KFVrNyROdU1TEl2wvXa3bEmTtZSjMvzvOkwhRnBE?=
 =?us-ascii?Q?Cm+YtHEHmD2Nt6bdFwQhbxQ/IUP3F3VINrVRSkT4knByBhkgoOy5urLbwuyn?=
 =?us-ascii?Q?GE5uga7Et96ODc20p36mX4Edtp0nuL6WSdDBF0CC/vzP5Sm9huQFxxqBg1KU?=
 =?us-ascii?Q?wlHmbTu8rOP1gmqQWdyeoCcpJIEt2IMTDZ6HqWxPinunGb0FzFVpz/KPG3ib?=
 =?us-ascii?Q?ArQCxu7l9TrKtrvvzwRojmNatHetuC3FXzy2aI5SROsMHI3cBeqK2YE4P1Or?=
 =?us-ascii?Q?RilrplM7eNiuuKdXzFyEGQXOE6Z7O/v8OncGQiQclFSz0lWezNwkFsWRrnW+?=
 =?us-ascii?Q?J+PE+6NtmEXUw+4B1o8Wgsnoq3eyPpMDg51p77KOal6iMjOeMFFUgmaS+TYU?=
 =?us-ascii?Q?FBIOtSPMuZmHBhmpBVfJBR4WCB/YgutzjeRBWuIvm9jePeIPmSAidqet6HDU?=
 =?us-ascii?Q?QrZqANcxk++ftAfQLil0d8MfjLfO8dvyZ/qWhpNin6lpuJdH+k9Uof5q9Vqv?=
 =?us-ascii?Q?8Oe+/HzPLwO2E5DKiqypuc1vAZHZFU1IsIDymNzQfbJX5OKbRtOzQaffRzly?=
 =?us-ascii?Q?j19ITL/OPe1SmCXCSMIZipabtE/jauLuPVTSItQMZtpoCjVcY4JTJ4BvI3cY?=
 =?us-ascii?Q?T4cP5KumJ78RuAuLpq8wVBA4oUVDcd9uYRVU/tOkaQDkjHiv3JwiWx9KZrHe?=
 =?us-ascii?Q?dNHdu8O3NfqdAxqVBGQmDcoZb9CZskzqU34HnCMfcS4M32ID+ALuHQygAq1J?=
 =?us-ascii?Q?78hirr1GdUmqIZGwmfdSDSKkAFh05Cw8oY7KkPQ/bc2cdfF92AR5YROXIJou?=
 =?us-ascii?Q?jiLEFh681acePEaqRbMjdTZ9TiofXjrRuaJmqxUvvcJJxUG5PpwkNdtV7415?=
 =?us-ascii?Q?2Znu9u+UJORlNHKKy1gWLatfupFIgnyg2AqAKSxsFZAnNTech37rkg+HyPtc?=
 =?us-ascii?Q?Y7JS19T5+fnYx9c5WnZt4jJj39jlPu/gPF+NyvRs?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	CO958NW/uBZPi6bgtioq4Xiq4YNyiCK1/m471jRhLdvnTD60rx1yddr1bB+L8LxnR4eNLZ1qXxgem+n6PWluvChI94Z++fOiIlW0g0uvuxFpLRcw6WsLjlM7K+v0oZmq10dwlL/wzPJ6/WgA9sPWui2Khrh3D2zID337zQZHxLaCsNNiT5rcG5sLAaDUo7aVnEVV7fCHw53zcQJaxJfbfkwAUIKjMDRDB4d/c+2EjefAePJFGursvHfCGfMyuOHVOZmUjoYOBSvmDrc0++1CmGMoeGXw9H7z5GdooykbzMCXv32xMJcKkQJ5VIzUm/CO1VUE5/7dJmPHH5PJKM5MgmWGcTMlignHHzs2mlLkBLU+7pRQrmljmlgBa3KflN/vJBQDSePLzutlEdjrctn6LdIYnfjj5Q+B4B8fWdDdtK+JtShXXkjxgcS3u5Bt+q6VjcmvFcwKYZfliYQS0UQbc+3os6KAiXOg1GFnX6QPZlG/rErVW4aWcqL3rD4sUJjY6ST9tYxywqoC/r/69e4Lbvglm/oxyHAiRHh61Mii4zv0OBT53wDSlEX5ddvilcACXDkbkk4oeuxU7qbSepncaHyDqDUaM2zJnjI9qm3bygE=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: efb45953-ea36-45a4-469e-08dc71250085
X-MS-Exchange-CrossTenant-AuthSource: SA0PR10MB6425.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 May 2024 19:11:31.8058
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: TNxG3fvWjY+sEFFSxA8r3+wMPRiHiCurR+5A8ygkAvpWEI1q7Y28E0Qt2Dei3goWZUkHIrYm3Fl7C46bd1mgMg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR10MB7113
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.650,FMLib:17.11.176.26
 definitions=2024-05-10_14,2024-05-10_02,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 phishscore=0
 suspectscore=0 spamscore=0 bulkscore=0 mlxlogscore=999 adultscore=0
 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2405010000 definitions=main-2405100137
X-Proofpoint-ORIG-GUID: IOHuvzSYcqHGZuzSdoB72_iEW2nMAomr
X-Proofpoint-GUID: IOHuvzSYcqHGZuzSdoB72_iEW2nMAomr

The absence of IRQD_MOVE_PCNTXT prevents immediate effectiveness of
interrupt affinity reconfiguration via procfs. Instead, the change is
deferred until the next instance of the interrupt being triggered on the
original CPU.

When the interrupt next triggers on the original CPU, the new affinity is
enforced within __irq_move_irq(). A vector is allocated from the new CPU,
but if the old vector on the original CPU remains online, it is not
immediately reclaimed. Instead, apicd->move_in_progress is flagged, and the
reclaiming process is delayed until the next trigger of the interrupt on
the new CPU.

Upon the subsequent triggering of the interrupt on the new CPU,
irq_complete_move() adds a task to the old CPU's vector_cleanup list if it
remains online. Subsequently, the timer on the old CPU iterates over its
vector_cleanup list, reclaiming vectors.

However, if the old CPU is offline before the interrupt triggers again on
the new CPU, irq_complete_move() simply resets both apicd->move_in_progress
and apicd->prev_vector to 0. Consequently, the vector remains unreclaimed
in vector_matrix, resulting in a CPU vector leak.

To address this issue, the fix borrows from the comments and implementation
of apic_update_vector(): "If the target CPU is offline then the regular
release mechanism via the cleanup vector is not possible and the vector can
be immediately freed in the underlying matrix allocator.".

Cc: Joe Jin <joe.jin@oracle.com>
Signed-off-by: Dongli Zhang <dongli.zhang@oracle.com>
---
 arch/x86/kernel/apic/vector.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/arch/x86/kernel/apic/vector.c b/arch/x86/kernel/apic/vector.c
index 185738c72766..aad189a3bac9 100644
--- a/arch/x86/kernel/apic/vector.c
+++ b/arch/x86/kernel/apic/vector.c
@@ -1036,6 +1036,15 @@ static void __vector_schedule_cleanup(struct apic_chip_data *apicd)
 			add_timer_on(&cl->timer, cpu);
 		}
 	} else {
+		/*
+		 * This call borrows from the comments and implementation
+		 * of apic_update_vector(): "If the target CPU is offline
+		 * then the regular release mechanism via the cleanup
+		 * vector is not possible and the vector can be immediately
+		 * freed in the underlying matrix allocator.".
+		 */
+		irq_matrix_free(vector_matrix, apicd->prev_cpu,
+				apicd->prev_vector, apicd->is_managed);
 		apicd->prev_vector = 0;
 	}
 	raw_spin_unlock(&vector_lock);
-- 
2.39.3