Received-SPF: pass (google.com: domain of linux-kernel+bounces-67110-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Date: Thu, 15 Feb 2024 09:57:39 -0500
From: Steven Rostedt <rostedt@goodmis.org>
To: "Masami Hiramatsu (Google)" <mhiramat@kernel.org>
Cc: Alexei Starovoitov <alexei.starovoitov@gmail.com>, Florent Revest
 <revest@chromium.org>, linux-trace-kernel@vger.kernel.org, LKML
 <linux-kernel@vger.kernel.org>, Martin KaFai Lau <martin.lau@linux.dev>,
 bpf <bpf@vger.kernel.org>, Sven Schnelle <svens@linux.ibm.com>, Alexei
 Starovoitov <ast@kernel.org>, Jiri Olsa <jolsa@kernel.org>, Arnaldo
 Carvalho de Melo <acme@kernel.org>, Daniel Borkmann <daniel@iogearbox.net>,
 Alan Maguire <alan.maguire@oracle.com>, Mark Rutland
 <mark.rutland@arm.com>, Peter Zijlstra <peterz@infradead.org>, Thomas
 Gleixner <tglx@linutronix.de>, Guo Ren <guoren@kernel.org>
Subject: Re: [PATCH v7 20/36] function_graph: Improve push operation for
 several interrupts
Message-ID: <20240215095739.41a2fac7@gandalf.local.home>
In-Reply-To: <170723227198.502590.10431025573751489041.stgit@devnote2>
References: <170723204881.502590.11906735097521170661.stgit@devnote2>
	<170723227198.502590.10431025573751489041.stgit@devnote2>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

On Wed,  7 Feb 2024 00:11:12 +0900
"Masami Hiramatsu (Google)" <mhiramat@kernel.org> wrote:

> From: Masami Hiramatsu (Google) <mhiramat@kernel.org>
> 
> Improve push and data reserve operation on the shadow stack for
> several sequencial interrupts.
> 
> To push a ret_stack or data entry on the shadow stack, we need to
> prepare an index (offset) entry before updating the stack pointer
> (curr_ret_stack) so that unwinder from interrupts can find the
> next return address from the shadow stack. Currently we do write index,
> update the curr_ret_stack, and rewrite it again. But that is not enough
> for the case if two interrupts happens and the first one breaks it.
> For example,
> 
>  1. write reserved index entry at ret_stack[new_index - 1] and ret addr.
>  2. interrupt comes.
>     2.1. push new index and ret addr on ret_stack.
>     2.2. pop it. (corrupt entries on new_index - 1)
>  3. return from interrupt.
>  4. update curr_ret_stack = new_index
>  5. interrupt comes again.
>     5.1. unwind <------ may not work.

I'm curious if you saw this happen?

That is, did you trigger this or only noticed it by inspection?

This code is already quite complex, I would like to simplify it more before
we try to fix rare race conditions that only affect the unwinder.

Let's hold off on this change.

-- Steve


> 
> To avoid this issue, this introduces a new rsrv_ret_stack stack
> reservation pointer and a new push code (slow path) to commit
> previous reserved code forcibly.
> 
>  0. update rsrv_ret_stack = new_index.
>  1. write reserved index entry at ret_stack[new_index - 1] and ret addr.
>  2. interrupt comes.
>     2.0. if rsrv_ret_stack != curr_ret_stack, add reserved index
>         entry on ret_stack[rsrv_ret_stack - 1] to point the previous
> 	ret_stack pointed by ret_stack[curr_ret_stack - 1]. and
> 	update curr_ret_stack = rsrv_ret_stack.
>     2.1. push new index and ret addr on ret_stack.
>     2.2. pop it. (corrupt entries on new_index - 1)
>  3. return from interrupt.
>  4. update curr_ret_stack = new_index
>  5. interrupt comes again.
>     5.1. unwind works, because curr_ret_stack points the previously
>         saved ret_stack.
>     5.2. this can do push/pop operations too.
> 6. return from interrupt.
> 7. rewrite reserved index entry at ret_stack[new_index] again.
> 
> This maybe a bit heavier but safer.
> 
> Signed-off-by: Masami Hiramatsu (Google) <mhiramat@kernel.org>