Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <1918c63f-0065-b468-3691-de6e314f4374@suse.com>
Date:   Thu, 3 Nov 2022 14:38:07 +0100
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Thunderbird/102.4.1
Subject: Re: [PATCH v4] x86/xen: Add support for
 HVMOP_set_evtchn_upcall_vector
Content-Language: en-US
To:     Jane Malalane <jane.malalane@citrix.com>
Cc:     Juergen Gross <jgross@suse.com>,
        Boris Ostrovsky <boris.ostrovsky@oracle.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
        "H. Peter Anvin" <hpa@zytor.com>,
        Stefano Stabellini <sstabellini@kernel.org>,
        Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
        Maximilian Heyne <mheyne@amazon.de>,
        xen-devel@lists.xenproject.org, LKML <linux-kernel@vger.kernel.org>
References: <20220729070416.23306-1-jane.malalane@citrix.com>
From:   Jan Beulich <jbeulich@suse.com>
In-Reply-To: <20220729070416.23306-1-jane.malalane@citrix.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?NWpDMFptaHF1d1pxOWRYeVQ1Sk5uNk9rSGRDZU5iM1oxMDdya0tXMkpBa204?=
 =?utf-8?B?Skg0QS9NUlZ3eUZqVUhVRWRTRUoxa0hZMzRneGpkOFhpVEhrVWpRUlNZWHhZ?=
 =?utf-8?B?eFg1SDQ3T2p6ZTNGZ0RnVkdzUlBReXh2VFQrWk5uTGtaVGhPQUVSN0tYOWdv?=
 =?utf-8?B?NGJGUWsyVzRTY1NhdVlUNCttb3BKREdyQk5XblJyZXYxVUdidFhrUG12cUsr?=
 =?utf-8?B?MS9wNXd2V1JQSVVDRkxhT0c3ZTdHYy9MYzlFU3VPZkVGaXhsdEpmRWI0SHN3?=
 =?utf-8?B?ZTFSUXkrdE1YSm1VUVNsNHRDZk9vbERFdlNzRmFqNWNFci93UmxncmNVYUhZ?=
 =?utf-8?B?VVVQdWZvdm1aMUlNSDlBVzNxaTZPalNZVFpwcXpEUGNzclBxdHZGR2VjeC9E?=
 =?utf-8?B?a1hpaGlmMlFOcTdQam8zay80Q1RnS2g4VkVIVk9TZVBGTjBEM2d0M2J2RkUw?=
 =?utf-8?B?UDNXNVcwZUplZWRTbi9VNEJNem00TjBzbFF2QmpkV3dnM1NadndlQlZwa1Vw?=
 =?utf-8?B?NU1VWkFnK3JBdXh6bmhpU3Z3UXBwbi9JVk1EelI1UEdZeWRldVRWcUlCRzU5?=
 =?utf-8?B?Y2JIL2hqYnFBRXlzTVBUSHUwNzczZjJONHdVOXNWZ3J2UzdZTUNCL1Jpc3lZ?=
 =?utf-8?B?bTJKRjViMThmNlA0NU5VSU8wMHVqOUUzSlcva3dBdk9TWjg1cGtPc1ZXQmVU?=
 =?utf-8?B?My9MNGVBKzIrSTg5dkxtZDY0clR2M0FBSDBnNWYwaVlCY0k1dWF1SXFybFY4?=
 =?utf-8?B?WGxva3B1Rmk2UVR6OURPekpvWS9jNGk0dEdYUTRRcEgzNnJHd1BlNVoycUhz?=
 =?utf-8?B?S2ZsSnNwY1FNRm1FdEkzZS9PaXpsVHhMYWhFbDk2YVVXUGVVRHU5eEZoWVJs?=
 =?utf-8?B?MlExc3Z1SmNYV3RWNi9SeHpDN0ZDU2lvZDRNQVhMclVnMllHRnZsNkhaRHFS?=
 =?utf-8?B?dGNyOG1IblFicXI1b25DVHFldFROd0VXYzdQc3J4dER6dDNRUWt4ai9rM21x?=
 =?utf-8?B?N3pjOTVRcC9hU3BOWnlSRkYxdkg5YmRZYUZtZEgwVnpmMGhQMGMwa3N5aDVn?=
 =?utf-8?B?eVVBR0RnUm8xV3ZFU0pmZnQ1blArNnNGYTY0L3V0TTVBczlxMGFtME53Qi9C?=
 =?utf-8?B?aDdUYTliU1p2VksxSWFjZGZlY0dKSC82Z2VCU3pxK3dBNWtlR0hkdHUzcE1j?=
 =?utf-8?B?azMvSHhLVFUxSldod0MwdzFwQmhDekQrQWRPZ1RHblJvWmVucXBXZ1c1eGE4?=
 =?utf-8?B?RXgraytpS1RORktxdTVQa255eUhUd3hhYkgvNStocUl6UytmSHYyS1lNVjFI?=
 =?utf-8?B?SFRRM01hczJjRWtrM0RzdVpGQWI0S29abVVTRXN4UXhrdytlWHVYSGdUY1p6?=
 =?utf-8?B?dGE4bXByMEZraCtKcDE1bHhuOVl3TVU5cUtRSXJ2Z2JTTFR3V1FpenFuMU1S?=
 =?utf-8?B?NWdaSk9pWWxPempOYjFaTHF1TEt1SVFVWno0U0pDT0VmaEp0dlVyZWJnMGtt?=
 =?utf-8?B?US9hVlVTb2RLV1VKOVNLemk4V2VFZ3Q2Z2MrbWlSbVNTMWk4Znp3Rk9Kbmhj?=
 =?utf-8?B?Y0ZyaGJiRFFBd21uUkRJR2RKbjRqYnVvYUMwZWx6K2ptYklVcmVGQncycXU0?=
 =?utf-8?B?UXlqRW1Ea3hQLytKcHRxVjVwU0xOYUQrTytFR28wNEkvclk3TUtnRXRZODlM?=
 =?utf-8?B?MlVobkQ3d3c4MTk5WDR0OUI3eVB6Ukg3NndZRGRBenY4SFQva2RxMFNRRjFM?=
 =?utf-8?B?WHNrSjNMSjhaZk9OVVd1S3FKcTFLUlBmWHNFUFpta2liZm1UZndiWHlCTElJ?=
 =?utf-8?B?LzczYS90MGxBcWtxU3NBQWdvQnF3aU1UVG9Oc1hkcW5TZlhHN3JLYnI1SDE3?=
 =?utf-8?B?NXFwalFPVWhpbXpxQ0F1d2E0NmdnTWpSeGhGYU1kdG1Xc3dDbjExb1hrdDhT?=
 =?utf-8?B?MjZnajFIQXpxckRsb3RTeVN3RitONGREY0p5a1pVZnNGMkFCdjlSRUF3N2xS?=
 =?utf-8?B?cE5zN2h2T25abFE1VGVkUmRkSlhxRkhQTnJGUFpxQTVSUXArY20vN1k2eW1P?=
 =?utf-8?B?ajljL1lHckNkRXM4UnR5TnMzL0d4NThUVnA2Tk45a282YnNzZFFPYnB5ak5p?=
 =?utf-8?Q?TxemHabsi/L4YNT6uNIqwJYtO?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8f438a99-6dc3-4ffc-0b4a-08dabda0a575
X-MS-Exchange-CrossTenant-AuthSource: VE1PR04MB6560.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2022 13:38:09.5924
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: GS5b3e/8f3kjUo6tQodGbGEJk4Czbt+2SmMhDGtCUfWa5jyosncZBNLSScEE8eMOAAchCvnsct3gu4iSMPv+kg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DBAPR04MB7479
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 29.07.2022 09:04, Jane Malalane wrote:
> @@ -125,6 +130,9 @@ DEFINE_IDTENTRY_SYSVEC(sysvec_xen_hvm_callback)
>  {
>  	struct pt_regs *old_regs = set_irq_regs(regs);
>  
> +	if (xen_percpu_upcall)
> +		ack_APIC_irq();
> +
>  	inc_irq_stat(irq_hv_callback_count);
>  
>  	xen_hvm_evtchn_do_upcall();
> @@ -168,6 +176,15 @@ static int xen_cpu_up_prepare_hvm(unsigned int cpu)
>  	if (!xen_have_vector_callback)
>  		return 0;
>  
> +	if (xen_percpu_upcall) {
> +		rc = xen_set_upcall_vector(cpu);

From all I can tell at least for APs this happens before setup_local_apic().
With there being APIC interaction in this operation mode, as seen e.g. in
the earlier hunk above, I think this is logically wrong. And it leads to
apic_pending_intr_clear() issuing its warning: The vector registration, as
an intentional side effect, marks the vector as pending. Unless IRQs were
enabled at any point between the registration and the check, there's
simply no way for the corresponding IRR bit to be dealt with (by
propagating to ISR when the interrupt is delivered, and then being cleared
from ISR by EOI).

As a note to x86 maintainers: This comment there

	/*
	 * If the ISR map is not empty. ACK the APIC and run another round
	 * to verify whether a pending IRR has been unblocked and turned
	 * into a ISR.
	 */

is pretty clearly wrong - with IRQs disabled there's no way for a pending
IRR bit to be "unblocked and turned into a ISR" one. And even if IRQs
were enabled TPR would still prevent the handling of any bits potentially
set in the 16....31 range.

Jan