Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 21 Jul 2023 09:58:48 +0200
From:   Peter Zijlstra <peterz@infradead.org>
To:     "Michael Kelley (LINUX)" <mikelley@microsoft.com>
Cc:     KY Srinivasan <kys@microsoft.com>,
        Haiyang Zhang <haiyangz@microsoft.com>,
        "wei.liu@kernel.org" <wei.liu@kernel.org>,
        Dexuan Cui <decui@microsoft.com>,
        "tglx@linutronix.de" <tglx@linutronix.de>,
        "mingo@redhat.com" <mingo@redhat.com>,
        "bp@alien8.de" <bp@alien8.de>,
        "dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
        "hpa@zytor.com" <hpa@zytor.com>, "x86@kernel.org" <x86@kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
        "stable@vger.kernel.org" <stable@vger.kernel.org>
Subject: Re: [PATCH 1/1] x86/hyperv: Disable IBT when hypercall page lacks
 ENDBR instruction
Message-ID: <20230721075848.GA3630545@hirez.programming.kicks-ass.net>
References: <1689885237-32662-1-git-send-email-mikelley@microsoft.com>
 <20230720211553.GA3615208@hirez.programming.kicks-ass.net>
 <SN6PR2101MB16933FAC4E09E15D824EB2FDD73FA@SN6PR2101MB1693.namprd21.prod.outlook.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <SN6PR2101MB16933FAC4E09E15D824EB2FDD73FA@SN6PR2101MB1693.namprd21.prod.outlook.com>
Precedence: bulk

On Fri, Jul 21, 2023 at 12:41:35AM +0000, Michael Kelley (LINUX) wrote:

> > Other than that, this seems fairly straight forward. One thing I
> > wondered about; wouldn't it be possible to re-write the indirect
> > hypercall thingies to a direct call? I mean, once we have the hypercall
> > page mapped, the address is known right?
> 
> Yes, the address is known.  It does not change across things like
> hibernation.  But the indirect call instruction is part of an inline assembly
> sequence, so the call instructions that need re-writing are scattered
> throughout the code. There's also the SEV-SNP case from the
> latest version of Tianyu Lan's patch set [1] where vmmcall may be used
> instead, based on your recent enhancement for nested ALTERNATIVE.
> Re-writing seems like that's more complexity than warranted for a
> mostly interim situation until the Hyper-V patch is available and
> users install it.

Well, we have a lot of infrastructure for this already. Specifically
this is very like the paravirt patching.

Also, direct calls are both faster and have less speculation issues, so
it might still be worth looking at.

The way to do something like this would be:


	asm volatile ("   ANNOTATE_RETPOLINE_SAFE	\n\t"
		      "1: call *hv_hypercall_page	\n\t"
		      ".pushsection .hv_call_sites	\n\t"
		      ".long 1b - .			\n\t"
		      ".popsection			\n\t");


And then (see alternative.c for many other examples):


patch_hypercalls()
{
	s32 *s;

	for (s = __hv_call_sites_begin; s < __hv_call_sites_end; s++) {
		void *addr = (void *)s + *s;
		struct insn insn;

		ret = insn_decode_kernel(&insn, addr);
		if (WARN_ON_ONCE(ret < 0))
			continue;

		/*
		 * indirect call: ff 15 disp32
		 * direct call:   2e e8 disp32
		 */
		if (insn.length == 6 &&
		    insn.opcode.bytes[0] == 0xFF &&
		    X86_MODRM_REG(insn.modrm.bytes[0]) == 2) {

			/* verify it was calling hy_hypercall_page */
			if (WARN_ON_ONCE(addr + 6 + insn.displacement.value != &hv_hypercall_page))
				continue;

			/*
			 * write a CS padded direct call -- assumes the
			 * hypercall page is in the 2G immediate range
			 * of the kernel text
			 */
			addr[0] = 0x2e; /* CS prefix */
			addr[1] = CALL_INSN_OPCODE;
			(s32 *)&Addr[2] = *hv_hypercall_page - (addr + 6);
		}
	}
}


See, easy :-)