Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 21 Jul 2023 20:49:44 +0200
From:   Peter Zijlstra <peterz@infradead.org>
To:     "Michael Kelley (LINUX)" <mikelley@microsoft.com>
Cc:     KY Srinivasan <kys@microsoft.com>,
        Haiyang Zhang <haiyangz@microsoft.com>,
        "wei.liu@kernel.org" <wei.liu@kernel.org>,
        Dexuan Cui <decui@microsoft.com>,
        "tglx@linutronix.de" <tglx@linutronix.de>,
        "mingo@redhat.com" <mingo@redhat.com>,
        "bp@alien8.de" <bp@alien8.de>,
        "dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
        "hpa@zytor.com" <hpa@zytor.com>, "x86@kernel.org" <x86@kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
        "stable@vger.kernel.org" <stable@vger.kernel.org>
Subject: Re: [PATCH 1/1] x86/hyperv: Disable IBT when hypercall page lacks
 ENDBR instruction
Message-ID: <20230721184944.GP4253@hirez.programming.kicks-ass.net>
References: <1689885237-32662-1-git-send-email-mikelley@microsoft.com>
 <20230720211553.GA3615208@hirez.programming.kicks-ass.net>
 <SN6PR2101MB16933FAC4E09E15D824EB2FDD73FA@SN6PR2101MB1693.namprd21.prod.outlook.com>
 <20230721075848.GA3630545@hirez.programming.kicks-ass.net>
 <BYAPR21MB168878CCD076762A1EA58635D73FA@BYAPR21MB1688.namprd21.prod.outlook.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <BYAPR21MB168878CCD076762A1EA58635D73FA@BYAPR21MB1688.namprd21.prod.outlook.com>
Precedence: bulk

On Fri, Jul 21, 2023 at 02:00:35PM +0000, Michael Kelley (LINUX) wrote:

> > Well, we have a lot of infrastructure for this already. Specifically
> > this is very like the paravirt patching.
> > 
> > Also, direct calls are both faster and have less speculation issues, so
> > it might still be worth looking at.
> > 
> > The way to do something like this would be:
> > 
> > 
> > 	asm volatile ("   ANNOTATE_RETPOLINE_SAFE	\n\t"
> > 		      "1: call *hv_hypercall_page	\n\t"
> > 		      ".pushsection .hv_call_sites	\n\t"
> > 		      ".long 1b - .			\n\t"
> > 		      ".popsection			\n\t");
> > 
> > 
> > And then (see alternative.c for many other examples):
> > 
> > 
> > patch_hypercalls()
> > {
> > 	s32 *s;
> > 
> > 	for (s = __hv_call_sites_begin; s < __hv_call_sites_end; s++) {
> > 		void *addr = (void *)s + *s;
> > 		struct insn insn;
> > 
> > 		ret = insn_decode_kernel(&insn, addr);
> > 		if (WARN_ON_ONCE(ret < 0))
> > 			continue;
> > 
> > 		/*
> > 		 * indirect call: ff 15 disp32
> > 		 * direct call:   2e e8 disp32
> > 		 */
> > 		if (insn.length == 6 &&
> > 		    insn.opcode.bytes[0] == 0xFF &&
> > 		    X86_MODRM_REG(insn.modrm.bytes[0]) == 2) {
> > 
> > 			/* verify it was calling hy_hypercall_page */
> > 			if (WARN_ON_ONCE(addr + 6 + insn.displacement.value != &hv_hypercall_page))
> > 				continue;
> > 
> > 			/*
> > 			 * write a CS padded direct call -- assumes the
> > 			 * hypercall page is in the 2G immediate range
> > 			 * of the kernel text
> 
> Probably not true -- the hypercall page has a vmalloc address.

See module_alloc(), that uses vmalloc but constrains the address to stay
within the 2G immediate address limit.

> > 			 */
> > 			addr[0] = 0x2e; /* CS prefix */
> > 			addr[1] = CALL_INSN_OPCODE;
> > 			(s32 *)&Addr[2] = *hv_hypercall_page - (addr + 6);
			*(s32 *)...
> > 		}
> > 	}
> > }
> > 
> > 
> > See, easy :-)
> 
> OK, worth looking into.  This is a corner of the Linux kernel code that
> I've never looked at before.  I appreciate the pointers.

No problem, I've been doing too much of this the past few years :-)

> Hypercall sites also exist in loadable modules, so would need to hook
> into module_finalize() as well.  Processing a new section type looks
> straightforward.

Yep,

> But altogether, this feels like more change than should go as a bug
> fix to be backported to stable kernels.  It's something to look at for a
> future kernel release.

Agreed!