Date: Mon, 10 Apr 2017 16:19:59 +0000
From: "Naveen N. Rao" <naveen.n.rao@linux.vnet.ibm.com>
Subject: Re: [PATCH] ppc64/kprobe: Fix oops when kprobed on 'stdu' instruction
To: mpe@ellerman.id.au, Ravi Bangoria <ravi.bangoria@linux.vnet.ibm.com>
Cc: aneesh.kumar@linux.vnet.ibm.com, chris@distroguy.com,
        linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org,
        npiggin@gmail.com, paulus@samba.org, viro@zeniv.linux.org.uk
References: <1491837657-4918-1-git-send-email-ravi.bangoria@linux.vnet.ibm.com>
In-Reply-To: <1491837657-4918-1-git-send-email-ravi.bangoria@linux.vnet.ibm.com>
User-Agent: astroid/0.8 (https://github.com/astroidmail/astroid)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8BIT
Message-Id: <1491840547.fs94dx4913.astroid@naverao1-tp.none>
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 3514
Lines: 96

Excerpts from Ravi Bangoria's message of April 10, 2017 20:50:
> If we set a kprobe on a 'stdu' instruction on powerpc64, we see a kernel 
> OOPS:
> 
>   [ 1275.165932] Bad kernel stack pointer cd93c840 at c000000000009868
>   [ 1275.166378] Oops: Bad kernel stack pointer, sig: 6 [#1]
>   ...
>   GPR00: c000001fcd93cb30 00000000cd93c840 c0000000015c5e00 00000000cd93c840
>   ...
>   [ 1275.178305] NIP [c000000000009868] resume_kernel+0x2c/0x58
>   [ 1275.178594] LR [c000000000006208] program_check_common+0x108/0x180
> 
> Basically, on 64 bit system, when user probes on 'stdu' instruction,
> kernel does not emulate actual store in emulate_step itself because it
> may corrupt exception frame. So kernel does actual store operation in
> exception return code i.e. resume_kernel().
> 
> resume_kernel() loads the saved stack pointer from memory using lwz,
> effectively loading a corrupt (32bit) address, causing the kernel crash.
> 
> Fix this by loading the 64bit value instead.

Thanks for fixing this!

> 
> Fixes: 8e9f69371536 ("powerpc/kprobe: Don't emulate store when kprobe stwu r1")

I think this should really be:
Fixes: be96f63375a1 ("powerpc: Split out instruction analysis part of 
emulate_step()")

...since the original commit just handled stwu on powerpc64 as well. In 
some ways, the 64-bit part of that commit wasn't that useful, but it 
never addressed stdu directly.

> Signed-off-by: Ravi Bangoria <ravi.bangoria@linux.vnet.ibm.com>
> ---
> History:
>   Commit 8e9f69371536 ("powerpc/kprobe: Don't emulate store when kprobe
>   stwu r1") fixed exception frame corruption for 32 bit system which uses
>   'stwu' instruction for stack frame allocation. This commit also added
>   code for 64 bit system but did not enabled it for 'stdu' instruction.
>   So 'stdu' instruction on 64 bit machine was emulating actual store in
>   emulate_step() itself until...
> 
>   Commit be96f63375a1 ("powerpc: Split out instruction analysis part of
>   emulate_step()"), enabled it for 'stdu' instruction on 64 bit machine.
> 
>   Since then it's broken. So this should also go into stable.

Hmm... so I think kprobe on 'stdu' has always been broken on powerpc64.  
We haven't noticed since most stdu operations were probably landing in 
the red zone so the exception frame never got corrupted. In that sense, 
this fix is needed for BE ever since load/store emulation was added.

For LE, this is only getting exposed now due to your recent patch to 
enable load/store emulation on LE.

> 
>  arch/powerpc/kernel/entry_64.S | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/powerpc/kernel/entry_64.S b/arch/powerpc/kernel/entry_64.S
> index 6432d4b..530f6e9 100644
> --- a/arch/powerpc/kernel/entry_64.S
> +++ b/arch/powerpc/kernel/entry_64.S
> @@ -689,7 +689,7 @@ resume_kernel:
> 
>  	addi	r8,r1,INT_FRAME_SIZE	/* Get the kprobed function entry */
> 
> -	lwz	r3,GPR1(r1)
> +	ld	r3,GPR1(r1)
>  	subi	r3,r3,INT_FRAME_SIZE	/* dst: Allocate a trampoline exception frame */
>  	mr	r4,r1			/* src:  current exception frame */
>  	mr	r1,r3			/* Reroute the trampoline frame to r1 */
> @@ -704,7 +704,7 @@ resume_kernel:
>  	bdnz	2b
> 
>  	/* Do real store operation to complete stwu */

Can you also update the above comment to refer to 'stdu'?
Apart from that, for this patch:
Reviewed-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com>

- Naveen


> -	lwz	r5,GPR1(r1)
> +	ld	r5,GPR1(r1)
>  	std	r8,0(r5)
> 
>  	/* Clear _TIF_EMULATE_STACK_STORE flag */
> -- 
> 1.9.3
> 
>