Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: [PATCH 2/6] xen/x86: restore (fix) xen_set_pte_init() behavior
From:   Jan Beulich <jbeulich@suse.com>
To:     Juergen Gross <jgross@suse.com>,
        Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc:     Stefano Stabellini <sstabellini@kernel.org>,
        lkml <linux-kernel@vger.kernel.org>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
References: <022b1a5e-4121-6bae-f07c-4ad5eac12481@suse.com>
Message-ID: <57ce1289-0297-e96e-79e1-cedafb5d9bf6@suse.com>
Date:   Thu, 30 Sep 2021 14:35:22 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <022b1a5e-4121-6bae-f07c-4ad5eac12481@suse.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?eFRnVmthazE2TnZIdTNjRWltRXNxVFRDbDdINzFXWlMvUFV6QzFIRFBEN2sr?=
 =?utf-8?B?dFFMOThPbDZqdkptU1FGWkkvUTc4RERYTThkVDlVSUpGWFBsR1RnNXdOc3lm?=
 =?utf-8?B?Z0l6NTJna2FkSEhZQUVYdEhuZll6UU5sUEVQQ2ZwWHF0YVpScDBmUVNyRnBQ?=
 =?utf-8?B?aytWWnFNa01GU0RFR0M1S3lrd0dqQVc1R1RhK3l1MDRIY0kzTHZVdXc2VEN5?=
 =?utf-8?B?NjhqOEw0YUdlajluUFdZRjlvYTRFRUxoL1hicGw3WnNqbnlJcjJrZHdET2pt?=
 =?utf-8?B?ZUJUVS9xNHJHTVJHbEcrWDQvdG9EOUJoZjBVNTR3dGgxSnRKYXliMDJ5Zkc2?=
 =?utf-8?B?U3g0ZFl6NkFsUldwT2FocHN2MDN6MmFiVFl1djBTSm9Gck5SM2VobzR1TUk1?=
 =?utf-8?B?WGp5UEFnc2gvSzNnSGtSYlJRWGlDNFE1UXlFU1UxaGVBdFpvM0pCMVlBL1FD?=
 =?utf-8?B?TVZ2RU1Sdmo4ZS9aSzRSc0FJc3RCYU5hcUVnMGxGaFVtS0xBSTlZL1NaWm1Y?=
 =?utf-8?B?NW91WUZmckhMbVJHYy8vb1FaemkxQTQzWDVHTHh3R1JOVkhQZ0xRYjljdjg3?=
 =?utf-8?B?ZnRIZDVaRDQvazRFT2Y2RHpzUm40c29DanVYdUVNNFdRTkxUWnFEeDY0Zmx1?=
 =?utf-8?B?cGZObjVDZFlYNEc5em1vMzZCN0Z0SUg1aG8ycDFGMVpFbVVOWnFrREthUmww?=
 =?utf-8?B?N1QwUlprc3BHcVlEbFFmZzNqY2lTWlhYSGtOZjFMNHhzWVhVZU5IT3VlUUdW?=
 =?utf-8?B?UlBGd29RdFA4Nnk0UDREallOZ1N4bkFLRTRHN0Ezb1dQZTRxRTJRQUdpL05Y?=
 =?utf-8?B?RmpFYnoyaDlYTkFDOUsrU2JMT3JXMndoaUxjT1hiOUdsTU1IS2pJNU5uQnZX?=
 =?utf-8?B?UDJhbkJwK0QyN3ZZeEx0cWs4S2l5NTNWQzRKUWNlK05iaG16c0swdkxLQXJt?=
 =?utf-8?B?V0lWNTI4b2tpMm5xdUFzWlhCOXY4YnhtelkveUd5bU5HOVYybnp3OGJvaDJ0?=
 =?utf-8?B?WFNRWVFhVDg5Zmo0Q1d0eElHMGtTSTkrM25vZlRUT3lpNkJNZGZJTE41WnJC?=
 =?utf-8?B?bjE5MlVVTXVlMm8va1BNSHlRYk5OOFdkTE9ZcFRvNGIxdDVmNm9ua25keHB1?=
 =?utf-8?B?bCtIbXpKaFAvTEl5cHppSEFSdTVEeEpQdzRLN3JjT2paREw2QTU3bFJMcW5l?=
 =?utf-8?B?SWxNUG1aVThySDVlQXVobkIrZFNNcTJ4M2xFWVhFVm52NXlpZ3MxYnZkMm1J?=
 =?utf-8?B?M0xJTUN6V1NWdmFPQmVGVmhYZHNTQUJDTWlZYjJpRi9kRy8xRUVKZ3R4MndR?=
 =?utf-8?B?cCtJSkZjekRxUU04U2RpZUEvb2VMZkpWQ2Q2VlRtaTdBblI1Y3VUTlJjajhu?=
 =?utf-8?B?WnF2NFkvTko4dkVqY0JRdzdVbFRleGx1MkFPWU9YbnRZYU03b3FYWjZ0Slg3?=
 =?utf-8?B?WDFBZHN2Ync5OWNGQjJPcCtjTm5HR0FyRWE1aVdlRkJ2aE5WZ2lkbWxWYUNu?=
 =?utf-8?B?OXlESCtpb1hqblhXZlptaEk5QitYU3phY2g3cGd6a0lwZVYxMEhwMUlGM1pY?=
 =?utf-8?B?aUtzWFJuMmJiMVNMeCtHemh0dHNjcVN1Q2poeW9SQWIwWUpJY1BrSU5TaFFz?=
 =?utf-8?B?RSt0S3ZUd3lVa2JvMHo1NHlTV1lRdE1DTEgxYWY3MGREZnRxS24zNXFEUzBy?=
 =?utf-8?B?MTA4Sk1raGszWXljSVN4RU83MzlqZWlST1A0dkJFQUczU2QrNnhxcVNISEg2?=
 =?utf-8?Q?dlV7EnSH3Vo+EakO3a/jG/FhhROjRzqBJAxvm2V?=
X-OriginatorOrg: suse.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 74700795-3594-4d3b-fd63-08d9840ec687
X-MS-Exchange-CrossTenant-AuthSource: VI1PR04MB5600.eurprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Sep 2021 12:35:24.5804
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: f7a17af6-1c5c-4a36-aa8b-f5be247aa4ba
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Z+QpJ6hdD1dvtUjfh20b2SeBFtCK/5mrepLbPm7M8obAJPEYT8rkLVValbSZp1jWDTxr/UUqZCjnonlfRIpSDA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VE1PR04MB6383
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Commit f7c90c2aa400 ("x86/xen: don't write ptes directly in 32-bit PV
guests") needlessly (and heavily) penalized 64-bit guests here: The
majority of the early page table updates is to writable pages (which get
converted to r/o only after all the writes are done), in particular
those involved in building the direct map (which consists of all 4k
mappings in PV). On my test system this accounts for almost 16 million
hypercalls when each could simply have been a plain memory write.

Switch back to using native_set_pte(), except for updates of early
ioremap tables (where a suitable accessor exists to recognize them).
With 32-bit PV support gone, this doesn't need to be further
conditionalized (albeit backports thereof may need adjustment).

To avoid a fair number (almost 256k on my test system) of trap-and-
emulate cases appearing as a result, switch the hook in
xen_pagetable_init().

Finally commit d6b186c1e2d8 ("x86/xen: avoid m2p lookup when setting
early page table entries") inserted a function ahead of
xen_set_pte_init(), separating it from its comment (which may have been
part of the reason why the performance regression wasn't anticipated /
recognized while codeing / reviewing the change mentioned further up).
Move the function up and adjust that comment to describe the new
behavior.

Signed-off-by: Jan Beulich <jbeulich@suse.com>

--- a/arch/x86/xen/mmu_pv.c
+++ b/arch/x86/xen/mmu_pv.c
@@ -1194,6 +1194,13 @@ static void __init xen_pagetable_p2m_set
 
 static void __init xen_pagetable_init(void)
 {
+	/*
+	 * The majority of further PTE writes is to pagetables already
+	 * announced as such to Xen. Hence it is more efficient to use
+	 * hypercalls for these updates.
+	 */
+	pv_ops.mmu.set_pte = __xen_set_pte;
+
 	paging_init();
 	xen_post_allocator_init();
 
@@ -1422,10 +1429,18 @@ static void xen_pgd_free(struct mm_struc
  *
  * Many of these PTE updates are done on unpinned and writable pages
  * and doing a hypercall for these is unnecessary and expensive.  At
- * this point it is not possible to tell if a page is pinned or not,
- * so always write the PTE directly and rely on Xen trapping and
+ * this point it is rarely possible to tell if a page is pinned, so
+ * mostly write the PTE directly and rely on Xen trapping and
  * emulating any updates as necessary.
  */
+static void __init xen_set_pte_init(pte_t *ptep, pte_t pte)
+{
+	if (unlikely(is_early_ioremap_ptep(ptep)))
+		__xen_set_pte(ptep, pte);
+	else
+		native_set_pte(ptep, pte);
+}
+
 __visible pte_t xen_make_pte_init(pteval_t pte)
 {
 	unsigned long pfn;
@@ -1447,11 +1462,6 @@ __visible pte_t xen_make_pte_init(pteval
 }
 PV_CALLEE_SAVE_REGS_THUNK(xen_make_pte_init);
 
-static void __init xen_set_pte_init(pte_t *ptep, pte_t pte)
-{
-	__xen_set_pte(ptep, pte);
-}
-
 /* Early in boot, while setting up the initial pagetable, assume
    everything is pinned. */
 static void __init xen_alloc_pte_init(struct mm_struct *mm, unsigned long pfn)