Received: by 2002:a05:6358:16cc:b0:ea:6187:17c9 with SMTP id r12csp12126374rwl; Tue, 3 Jan 2023 09:20:57 -0800 (PST) X-Google-Smtp-Source: AMrXdXvK/BNk+VXowlfxqznmhU9HUPRebQ5pwuv1TBYpJ3xnt0uEviY5joUKgWxItmd4kKakblD8 X-Received: by 2002:a62:1915:0:b0:581:65ab:8cba with SMTP id 21-20020a621915000000b0058165ab8cbamr27117983pfz.31.1672766457134; Tue, 03 Jan 2023 09:20:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1672766457; cv=none; d=google.com; s=arc-20160816; b=AH1YGIm1hgMd+NzL+dyVkQbBvPMoxRRZHstCLv4EyRm82rXIb0tVYFZqo317JdpoZX dgFzY/GbGEiA03cV5vOfuF5cuzMI/Yx2z9skEOaPGpwPDQvCINb9HJ8324joQZiQIr2C 5LFH5IiIu5RwL6EeVYu5rrWHVkPba0dQKIVoAqsMur4WsdN8lR3uIy6CMHrjjrGqllaP B4mU2K/1xTRndDj3Cq2VI6oyzDtZZaaY+BuHBhOz7DA0h8H/9bNhrsRuKhCsBEltjXRr R2KvW7o1KEyXxL8tWdkqSp3+Zp4c0ijPqXbAlEEvvlJoq4YyuC8Z6muLaOr0WaFb+8Cc x//Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:to:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:from :dkim-signature:dkim-filter:secumail-id:dkim-signature; bh=6OkPdE13irN1BDSOxxXFQDK4XI14Zdk6/rpCDpMH2Wc=; b=fqTzNcmqM5CQMJNK9ipnlVUhTI5SmJQ0Ipj0S7TpgaYGuT4+1zvpbTSmO64wctciXR wvzPEaK61O70JO6Q520YSbicqB815W76fSwsae7OfmjiRNoDrnU+/F7UWQiSHha4Dc8t E1nInKx1tr4tmLsESL9/CGKJV5PRu31BLnvHYV9zdOQxTsBPVl81tnkgP/od8Z7A3zfa x1uxSZqCeSEd+c2mghMVKgjw9vF9Gigjwz7SChZpzOKVzldTR0LSlwR17f5+mxQ+y5GB CaJIStHhSQ01Nfo2meAQlpinxEQwjZqCHkgafFEmH7xSZvtJD4pBz4x0GEo50veMyZoL VZpw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@kalray.eu header.s=sec-sig-email header.b=mUBpEe4I; dkim=neutral (body hash did not verify) header.i=@kalray.eu header.s=32AE1B44-9502-11E5-BA35-3734643DEF29 header.b=dCm29yk4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=NONE dis=NONE) header.from=kalray.eu Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h18-20020a63c012000000b00478950f15c7si32734065pgg.253.2023.01.03.09.20.49; Tue, 03 Jan 2023 09:20:57 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=fail header.i=@kalray.eu header.s=sec-sig-email header.b=mUBpEe4I; dkim=neutral (body hash did not verify) header.i=@kalray.eu header.s=32AE1B44-9502-11E5-BA35-3734643DEF29 header.b=dCm29yk4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=QUARANTINE sp=NONE dis=NONE) header.from=kalray.eu Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237615AbjACQuX (ORCPT + 61 others); Tue, 3 Jan 2023 11:50:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33302 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238123AbjACQuK (ORCPT ); Tue, 3 Jan 2023 11:50:10 -0500 Received: from fx301.security-mail.net (smtpout30.security-mail.net [85.31.212.37]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A0B1CE00E for ; Tue, 3 Jan 2023 08:50:04 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by fx301.security-mail.net (Postfix) with ESMTP id ECBE6269354A for ; Tue, 3 Jan 2023 17:44:33 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kalray.eu; s=sec-sig-email; t=1672764274; bh=IrxMFxPjU/kO/u0ljyWLn4YOh/RjjppFKmUJaw2Hi0I=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=mUBpEe4IFoXcYBQ9CE3AWVtBx7rdEgClEIZkfRr307N76UlmJFYLCTe8fDTrzAkHX UGF0AO9gE+hW+GKkDyVKVzlpwYyn672ouMQxQB25YwfBwyWOV6WKiGSw+cGaLUN1Zt lcvs2yO2qVaNoleLAk5Z6l+sxzjvKo1U5PPIFd90= Received: from fx301 (localhost [127.0.0.1]) by fx301.security-mail.net (Postfix) with ESMTP id D3E0A2693546; Tue, 3 Jan 2023 17:44:33 +0100 (CET) Received: from zimbra2.kalray.eu (unknown [217.181.231.53]) by fx301.security-mail.net (Postfix) with ESMTPS id E7CB12693554; Tue, 3 Jan 2023 17:44:31 +0100 (CET) Received: from zimbra2.kalray.eu (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTPS id AD88527E03FA; Tue, 3 Jan 2023 17:44:31 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by zimbra2.kalray.eu (Postfix) with ESMTP id 8EAA527E03FD; Tue, 3 Jan 2023 17:44:31 +0100 (CET) Received: from zimbra2.kalray.eu ([127.0.0.1]) by localhost (zimbra2.kalray.eu [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id xaw2SfyK9t4B; Tue, 3 Jan 2023 17:44:31 +0100 (CET) Received: from junon.lin.mbt.kalray.eu (unknown [192.168.37.161]) by zimbra2.kalray.eu (Postfix) with ESMTPSA id 654FC27E03FB; Tue, 3 Jan 2023 17:44:31 +0100 (CET) X-Virus-Scanned: E-securemail Secumail-id: DKIM-Filter: OpenDKIM Filter v2.10.3 zimbra2.kalray.eu 8EAA527E03FD DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kalray.eu; s=32AE1B44-9502-11E5-BA35-3734643DEF29; t=1672764271; bh=iUITjDsI6ikn2R9v3MFjzcJ1Ct9kuay/gAQ3TaiWEEo=; h=From:To:Date:Message-Id:MIME-Version; b=dCm29yk4tremVX59tBfx3Oqj7snj1HkqhP2G78AcP7KBxRv/LFFSKXppHN36/j79A 8hlx3X10iO7JbgYktL2MtZjKb4oMBGl714mQ2rXXxnFxZTwNnwdbqmVfddXgFcYTxG R0QinLRr6aIbONKJ4t24Tb9YhK3MkJuKtT2TzvKY= From: Yann Sionneau Cc: Yann Sionneau , linux-kernel@vger.kernel.org, Clement Leger , Guillaume Thouvenin , Julian Vetter , Luc Michel , Marius Gligor Subject: [RFC PATCH 08/25] kvx: Add exception/interrupt handling Date: Tue, 3 Jan 2023 17:43:42 +0100 Message-ID: <20230103164359.24347-9-ysionneau@kalray.eu> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20230103164359.24347-1-ysionneau@kalray.eu> References: <20230103164359.24347-1-ysionneau@kalray.eu> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=utf-8 X-ALTERMIMEV2_out: done X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Add the exception and interrupt handling machanism for basic kvx support. CC: linux-kernel@vger.kernel.org Co-developed-by: Clement Leger Signed-off-by: Clement Leger Co-developed-by: Guillaume Thouvenin Signed-off-by: Guillaume Thouvenin Co-developed-by: Julian Vetter Signed-off-by: Julian Vetter Co-developed-by: Luc Michel Signed-off-by: Luc Michel Co-developed-by: Marius Gligor Signed-off-by: Marius Gligor Co-developed-by: Yann Sionneau Signed-off-by: Yann Sionneau --- arch/kvx/include/asm/break_hook.h | 69 +++++++++ arch/kvx/include/asm/bug.h | 67 ++++++++ arch/kvx/include/asm/dame.h | 31 ++++ arch/kvx/include/asm/hardirq.h | 14 ++ arch/kvx/include/asm/hw_irq.h | 14 ++ arch/kvx/include/asm/ipi.h | 16 ++ arch/kvx/include/asm/irqflags.h | 58 +++++++ arch/kvx/include/asm/stacktrace.h | 44 ++++++ arch/kvx/include/asm/traps.h | 76 ++++++++++ arch/kvx/kernel/dame_handler.c | 113 ++++++++++++++ arch/kvx/kernel/irq.c | 78 ++++++++++ arch/kvx/kernel/traps.c | 243 ++++++++++++++++++++++++++++++ arch/kvx/platform/ipi.c | 110 ++++++++++++++ 13 files changed, 933 insertions(+) create mode 100644 arch/kvx/include/asm/break_hook.h create mode 100644 arch/kvx/include/asm/bug.h create mode 100644 arch/kvx/include/asm/dame.h create mode 100644 arch/kvx/include/asm/hardirq.h create mode 100644 arch/kvx/include/asm/hw_irq.h create mode 100644 arch/kvx/include/asm/ipi.h create mode 100644 arch/kvx/include/asm/irqflags.h create mode 100644 arch/kvx/include/asm/stacktrace.h create mode 100644 arch/kvx/include/asm/traps.h create mode 100644 arch/kvx/kernel/dame_handler.c create mode 100644 arch/kvx/kernel/irq.c create mode 100644 arch/kvx/kernel/traps.c create mode 100644 arch/kvx/platform/ipi.c diff --git a/arch/kvx/include/asm/break_hook.h b/arch/kvx/include/asm/break_hook.h new file mode 100644 index 000000000000..333b2c440c81 --- /dev/null +++ b/arch/kvx/include/asm/break_hook.h @@ -0,0 +1,69 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef __ASM_KVX_BREAK_HOOK_H_ +#define __ASM_KVX_BREAK_HOOK_H_ + +#include + +#include +#include + +/* + * The following macros define the different causes of break: + * We use the `set $vsfr0 = $rXX` instruction which will raise a trap into the + * debugger. The trapping instruction is read and decoded to extract the source + * register number. The source register number is used to differentiate the + * trap cause. + */ +#define BREAK_CAUSE_BUG KVX_REG_R1 +#define BREAK_CAUSE_KGDB_DYN KVX_REG_R2 +#define BREAK_CAUSE_KGDB_COMP KVX_REG_R3 +#define BREAK_CAUSE_BKPT KVX_REG_R63 + +/** + * enum break_ret - Break return value + * @BREAK_HOOK_HANDLED: Hook handled successfully + * @BREAK_HOOK_ERROR: Hook was not handled + */ +enum break_ret { + BREAK_HOOK_HANDLED = 0, + BREAK_HOOK_ERROR = 1, +}; + +/* + * The following macro assembles a `set` instruction targeting $vsfr0 + * using the source register whose number is __id. + */ +#define KVX_BREAK_INSN(__id) \ + KVX_INSN_SET_SYLLABLE_0(KVX_INSN_PARALLEL_EOB, KVX_SFR_VSFR0, __id) + +#define KVX_BREAK_INSN_SIZE (KVX_INSN_SET_SIZE * KVX_INSN_SYLLABLE_WIDTH) + +struct pt_regs; + +/** + * struct break_hook - Break hook description + * @node: List node + * @handler: handler called when break matches this hook + * @imm: Immediate value expected for break insn + * @mode: Hook mode (user/kernel) + */ +struct break_hook { + struct list_head node; + int (*handler)(struct break_hook *brk_hook, struct pt_regs *regs); + u8 id; + u8 mode; +}; + +void kvx_skip_break_insn(struct pt_regs *regs); + +void break_hook_register(struct break_hook *brk_hook); +void break_hook_unregister(struct break_hook *brk_hook); + +int break_hook_handler(u64 es, struct pt_regs *regs); + +#endif /* __ASM_KVX_BREAK_HOOK_H_ */ diff --git a/arch/kvx/include/asm/bug.h b/arch/kvx/include/asm/bug.h new file mode 100644 index 000000000000..62f556b00d5a --- /dev/null +++ b/arch/kvx/include/asm/bug.h @@ -0,0 +1,67 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_BUG_H +#define _ASM_KVX_BUG_H + +#include +#include +#include + +#include + +#ifdef CONFIG_GENERIC_BUG + +#define BUG_INSN KVX_BREAK_INSN(BREAK_CAUSE_BUG) + +#define __BUG_ENTRY_ADDR ".dword 1b" + +#ifdef CONFIG_DEBUG_BUGVERBOSE +#define __BUG_ENTRY_LAST_MEMBER flags +#define __BUG_ENTRY \ + __BUG_ENTRY_ADDR "\n\t" \ + ".dword %0\n\t" \ + ".short %1\n\t" +#else +#define __BUG_ENTRY_LAST_MEMBER file +#define __BUG_ENTRY \ + __BUG_ENTRY_ADDR "\n\t" +#endif + +#define BUG() \ +do { \ + __asm__ __volatile__ ( \ + "1:\n\t" \ + ".word " __stringify(BUG_INSN) "\n" \ + ".pushsection __bug_table,\"a\"\n\t" \ + "2:\n\t" \ + __BUG_ENTRY \ + ".fill 1, %2, 0\n\t" \ + ".popsection" \ + : \ + : "i" (__FILE__), "i" (__LINE__), \ + "i" (sizeof(struct bug_entry) - \ + offsetof(struct bug_entry, __BUG_ENTRY_LAST_MEMBER))); \ + unreachable(); \ +} while (0) + +#else /* CONFIG_GENERIC_BUG */ +#define BUG() \ +do { \ + __asm__ __volatile__ (".word " __stringify(BUG_INSN) "\n"); \ + unreachable(); \ +} while (0) +#endif /* CONFIG_GENERIC_BUG */ + +#define HAVE_ARCH_BUG + +struct pt_regs; + +void die(struct pt_regs *regs, unsigned long ea, const char *str); + +#include + +#endif /* _ASM_KVX_BUG_H */ diff --git a/arch/kvx/include/asm/dame.h b/arch/kvx/include/asm/dame.h new file mode 100644 index 000000000000..8befd767bbee --- /dev/null +++ b/arch/kvx/include/asm/dame.h @@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_DAME_H +#define _ASM_KVX_DAME_H + +#include +#include + +static inline void dame_irq_check(struct pt_regs *regs) +{ +#ifdef CONFIG_SECURE_DAME_HANDLING + unsigned long ilr; + /* If we are returning to the kernel, no need to check for DAME */ + if (!user_mode(regs)) + return; + + /* Else, make sure we do a barrier to trig any pending DAME IRQ */ + __builtin_kvx_barrier(); + + /* Check if we triggered a DAME */ + ilr = kvx_sfr_get(ILR); + if (ilr & KVX_SFR_ILR_IT16_MASK) + panic("DAME error encountered while in kernel !!!!\n"); +#endif +} + +#endif /* _ASM_KVX_DAME_H */ diff --git a/arch/kvx/include/asm/hardirq.h b/arch/kvx/include/asm/hardirq.h new file mode 100644 index 000000000000..f82630f7e4e2 --- /dev/null +++ b/arch/kvx/include/asm/hardirq.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_HARDIRQ_H +#define _ASM_KVX_HARDIRQ_H + +#define __ARCH_IRQ_EXIT_IRQS_DISABLED 1 + +#include + +#endif /* _ASM_KVX_HARDIRQ_H */ diff --git a/arch/kvx/include/asm/hw_irq.h b/arch/kvx/include/asm/hw_irq.h new file mode 100644 index 000000000000..f073dba3b1c5 --- /dev/null +++ b/arch/kvx/include/asm/hw_irq.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * derived from arch/mips/include/asm/ide.h + * + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_HW_IRQ_H +#define _ASM_KVX_HW_IRQ_H + +void kvx_init_core_irq(void); + +#endif /* _ASM_KVX_HW_IRQ_H */ diff --git a/arch/kvx/include/asm/ipi.h b/arch/kvx/include/asm/ipi.h new file mode 100644 index 000000000000..137407a075e6 --- /dev/null +++ b/arch/kvx/include/asm/ipi.h @@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_IPI_H +#define _ASM_KVX_IPI_H + +#include + +int kvx_ipi_ctrl_probe(irqreturn_t (*ipi_irq_handler)(int, void *)); + +void kvx_ipi_send(const struct cpumask *mask); + +#endif /* _ASM_KVX_IPI_H */ diff --git a/arch/kvx/include/asm/irqflags.h b/arch/kvx/include/asm/irqflags.h new file mode 100644 index 000000000000..681c890b3fcd --- /dev/null +++ b/arch/kvx/include/asm/irqflags.h @@ -0,0 +1,58 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_IRQFLAGS_H +#define _ASM_KVX_IRQFLAGS_H + +#include + +#include + +static inline notrace unsigned long arch_local_save_flags(void) +{ + return kvx_sfr_get(PS) & (1 << KVX_SFR_PS_IE_SHIFT); +} + +static inline notrace unsigned long arch_local_irq_save(void) +{ + unsigned long flags = arch_local_save_flags(); + + kvx_sfr_set_field(PS, IE, 0); + + return flags; +} + +static inline notrace void arch_local_irq_restore(unsigned long flags) +{ + /* If flags are set, interrupt are enabled), set the IE bit */ + if (flags) + kvx_sfr_set_field(PS, IE, 1); + else + kvx_sfr_set_field(PS, IE, 0); +} + +static inline notrace void arch_local_irq_enable(void) +{ + kvx_sfr_set_field(PS, IE, 1); +} + +static inline notrace void arch_local_irq_disable(void) +{ + kvx_sfr_set_field(PS, IE, 0); +} + +static inline notrace bool arch_irqs_disabled_flags(unsigned long flags) +{ + return (flags & (1 << KVX_SFR_PS_IE_SHIFT)) == 0; +} + +static inline notrace bool arch_irqs_disabled(void) +{ + return arch_irqs_disabled_flags(kvx_sfr_get(PS)); +} + + +#endif /* _ASM_KVX_IRQFLAGS_H */ diff --git a/arch/kvx/include/asm/stacktrace.h b/arch/kvx/include/asm/stacktrace.h new file mode 100644 index 000000000000..0feed6bd4424 --- /dev/null +++ b/arch/kvx/include/asm/stacktrace.h @@ -0,0 +1,44 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#ifndef _ASM_KVX_STACKTRACE_H +#define _ASM_KVX_STACKTRACE_H + +#include + +/** + * Structure of a frame on the stack + */ +struct stackframe { + unsigned long fp; /* Next frame pointer */ + unsigned long ra; /* Return address */ +}; + +static inline bool on_task_stack(struct task_struct *tsk, unsigned long sp) +{ + unsigned long low = (unsigned long) task_stack_page(tsk); + unsigned long high = low + THREAD_SIZE; + + if (sp < low || sp >= high) + return false; + + return true; +} + +void show_stacktrace(struct task_struct *task, struct pt_regs *regs); + + +void walk_stackframe(struct task_struct *task, struct stackframe *frame, + bool (*fn)(unsigned long, void *), void *arg); + +static inline void start_stackframe(struct stackframe *frame, + unsigned long fp, + unsigned long pc) +{ + frame->fp = fp; + frame->ra = pc; +} +#endif /* _ASM_KVX_STACKTRACE_H */ diff --git a/arch/kvx/include/asm/traps.h b/arch/kvx/include/asm/traps.h new file mode 100644 index 000000000000..77a663968135 --- /dev/null +++ b/arch/kvx/include/asm/traps.h @@ -0,0 +1,76 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Guillaume Thouvenin + * Marius Gligor + */ + +#ifndef _ASM_KVX_TRAPS_H +#define _ASM_KVX_TRAPS_H + +#include + +#define KVX_TRAP_RESET 0x0 +#define KVX_TRAP_OPCODE 0x1 +#define KVX_TRAP_PRIVILEGE 0x2 +#define KVX_TRAP_DMISALIGN 0x3 +#define KVX_TRAP_PSYSERROR 0x4 +#define KVX_TRAP_DSYSERROR 0x5 +#define KVX_TRAP_PDECCERROR 0x6 +#define KVX_TRAP_DDECCERROR 0x7 +#define KVX_TRAP_PPARERROR 0x8 +#define KVX_TRAP_DPARERROR 0x9 +#define KVX_TRAP_PSECERROR 0xA +#define KVX_TRAP_DSECERROR 0xB +#define KVX_TRAP_NOMAPPING 0xC +#define KVX_TRAP_PROTECTION 0xD +#define KVX_TRAP_WRITETOCLEAN 0xE +#define KVX_TRAP_ATOMICTOCLEAN 0xF +#define KVX_TRAP_TPAR 0x10 +#define KVX_TRAP_DOUBLE_ECC 0x11 +#define KVX_TRAP_VSFR 0x12 +#define KVX_TRAP_PL_OVERFLOW 0x13 + +#define KVX_TRAP_COUNT 0x14 + +#define KVX_TRAP_SFRI_NOT_BCU 0 +#define KVX_TRAP_SFRI_GET 1 +#define KVX_TRAP_SFRI_IGET 2 +#define KVX_TRAP_SFRI_SET 4 +#define KVX_TRAP_SFRI_WFXL 5 +#define KVX_TRAP_SFRI_WFXM 6 +#define KVX_TRAP_SFRI_RSWAP 7 + +/* Access type on memory trap */ +#define KVX_TRAP_RWX_FETCH 1 +#define KVX_TRAP_RWX_WRITE 2 +#define KVX_TRAP_RWX_READ 4 +#define KVX_TRAP_RWX_ATOMIC 6 + +#ifndef __ASSEMBLY__ + +typedef void (*trap_handler_func) (uint64_t es, uint64_t ea, + struct pt_regs *regs); + +#define trap_cause(__es) kvx_sfr_field_val(__es, ES, HTC) + +#define trap_sfri(__es) \ + kvx_sfr_field_val((__es), ES, SFRI) + +#define trap_gprp(__es) \ + kvx_sfr_field_val((__es), ES, GPRP) + +#define trap_sfrp(__es) \ + kvx_sfr_field_val((__es), ES, SFRP) + +#ifdef CONFIG_MMU +extern void do_page_fault(uint64_t es, uint64_t ea, struct pt_regs *regs); +extern void do_writetoclean(uint64_t es, uint64_t ea, struct pt_regs *regs); +#endif + +void user_do_sig(struct pt_regs *regs, int signo, int code, unsigned long addr); + +#endif /* __ASSEMBLY__ */ + +#endif diff --git a/arch/kvx/kernel/dame_handler.c b/arch/kvx/kernel/dame_handler.c new file mode 100644 index 000000000000..ce190bee8211 --- /dev/null +++ b/arch/kvx/kernel/dame_handler.c @@ -0,0 +1,113 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +static unsigned int kvx_dame_irq; + +static const char *error_str[KVX_SFR_ES_ITI_WIDTH] = { + "PSE", + "PILSY", + "PILDE", + "PILPA", + "DSE", + "DILSY", + "DILDE", + "DILPA", + "DDEE", + "DSYE" +}; + +static irqreturn_t dame_irq_handler(int irq, void *dev_id) +{ + int bit; + struct pt_regs *regs = get_irq_regs(); + unsigned long error_status = kvx_sfr_field_val(regs->es, ES, ITI); + + if (error_status) { + pr_err("Memory Error:\n"); + for_each_set_bit(bit, &error_status, KVX_SFR_ES_ITI_WIDTH) + pr_err("- %s\n", error_str[bit]); + } + + /* + * If the DAME happened in user mode, we can handle it properly + * by killing the user process. + * Otherwise, if we are in kernel, we are fried... + */ + if (user_mode(regs)) + force_sig_fault(SIGBUS, BUS_ADRERR, (void __user *) NULL); + else + die(regs, 0, "DAME error encountered while in kernel !!!!\n"); + + return IRQ_HANDLED; +} + +static int kvx_dame_starting_cpu(unsigned int cpu) +{ + enable_percpu_irq(kvx_dame_irq, IRQ_TYPE_NONE); + + return 0; +} + +static int kvx_dame_dying_cpu(unsigned int cpu) +{ + disable_percpu_irq(kvx_dame_irq); + + return 0; +} + +static int __init dame_handler_init(void) +{ + struct device_node *dame_node; + int ret; + + dame_node = of_find_compatible_node(NULL, NULL, + "kalray,kvx-dame-handler"); + if (!dame_node) { + pr_err("Failed to find dame handler device tree node\n"); + return -ENODEV; + } + + kvx_dame_irq = irq_of_parse_and_map(dame_node, 0); + of_node_put(dame_node); + + if (!kvx_dame_irq) { + pr_err("Failed to parse dame irq\n"); + return -ENODEV; + } + + ret = request_percpu_irq(kvx_dame_irq, dame_irq_handler, "dame", + &kvx_dame_irq); + if (ret) { + pr_err("Failed to request dame irq\n"); + return -ENODEV; + } + + ret = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, + "kvx/dame_handler:online", + kvx_dame_starting_cpu, + kvx_dame_dying_cpu); + if (ret <= 0) { + pr_err("Failed to setup cpuhp\n"); + return ret; + } + + pr_info("DAME handler registered\n"); + + return 0; +} + +core_initcall(dame_handler_init); diff --git a/arch/kvx/kernel/irq.c b/arch/kvx/kernel/irq.c new file mode 100644 index 000000000000..5d7c8cfba1dd --- /dev/null +++ b/arch/kvx/kernel/irq.c @@ -0,0 +1,78 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + */ + +#include +#include +#include +#include +#include +#include + +#include + +#define IT_MASK(__it) (KVX_SFR_ILL_ ## __it ## _MASK) +#define IT_LEVEL(__it, __level) \ + (__level##ULL << KVX_SFR_ILL_ ## __it ## _SHIFT) + +void do_IRQ(unsigned long hwirq_mask, struct pt_regs *regs) +{ + struct pt_regs *old_regs = set_irq_regs(regs); + int irq; + unsigned int hwirq; + + trace_hardirqs_off(); + + irq_enter(); + + while (hwirq_mask) { + hwirq = __ffs(hwirq_mask); + irq = irq_find_mapping(NULL, hwirq); + generic_handle_irq(irq); + hwirq_mask &= ~BIT_ULL(hwirq); + } + + irq_exit(); + set_irq_regs(old_regs); + + dame_irq_check(regs); +} + +/* + * Early Hardware specific Interrupt setup + * -Called very early (start_kernel -> setup_arch -> setup_processor) + * -Needed for each CPU + */ +void kvx_init_core_irq(void) +{ + /* + * On KVX, Kernel only care about the following IT: + * - IT0: Timer 0 + * - IT2: Watchdog + * - IT4: APIC IT 1 + * - IT24: IPI + */ + uint64_t mask = IT_MASK(IT0) | IT_MASK(IT2) | IT_MASK(IT4) | + IT_MASK(IT24); + + /* + * Specific priorities for ITs: + * - Watchdog has the highest priority: 3 + * - Timer has priority 2 + * - APIC entries have lowest priority: 1 + */ + uint64_t value = IT_LEVEL(IT0, 0x2) | IT_LEVEL(IT2, 0x3) | + IT_LEVEL(IT4, 0x1) | IT_LEVEL(IT24, 0x1); + + kvx_sfr_set_mask(ILL, mask, value); + + /* Set core level to 0 */ + kvx_sfr_set_field(PS, IL, 0); +} + +void __init init_IRQ(void) +{ + irqchip_init(); +} diff --git a/arch/kvx/kernel/traps.c b/arch/kvx/kernel/traps.c new file mode 100644 index 000000000000..3a1706b666c2 --- /dev/null +++ b/arch/kvx/kernel/traps.c @@ -0,0 +1,243 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Guillaume Thouvenin + * Marius Gligor + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include +#include + +int show_unhandled_signals = 1; + +static DEFINE_SPINLOCK(die_lock); + +static trap_handler_func trap_handler_table[KVX_TRAP_COUNT] = { NULL }; + +/* Trap names associated to the trap numbers */ +static const char * const trap_name[] = { + "RESET", + "OPCODE", + "PRIVILEGE", + "DMISALIGN", + "PSYSERROR", + "DSYSERROR", + "PDECCERROR", + "DDECCERROR", + "PPARERROR", + "DPARERROR", + "PSECERROR", + "DSECERROR", + /* MMU related traps */ + "NOMAPPING", + "PROTECTION", + "WRITETOCLEAN", + "ATOMICTOCLEAN", + "TPAR", + "DOUBLE_ECC", + "VSFR", + "PL_OVERFLOW" +}; + +void die(struct pt_regs *regs, unsigned long ea, const char *str) +{ + static int die_counter; + int ret; + + oops_enter(); + + spin_lock_irq(&die_lock); + console_verbose(); + bust_spinlocks(1); + + pr_emerg("%s [#%d]\n", str, ++die_counter); + print_modules(); + show_regs(regs); + + if (!user_mode(regs)) + show_stacktrace(NULL, regs); + + ret = notify_die(DIE_OOPS, str, regs, ea, 0, SIGSEGV); + + bust_spinlocks(0); + add_taint(TAINT_DIE, LOCKDEP_NOW_UNRELIABLE); + spin_unlock_irq(&die_lock); + oops_exit(); + + if (in_interrupt()) + panic("Fatal exception in interrupt"); + if (panic_on_oops) + panic("Fatal exception"); + if (ret != NOTIFY_STOP) + make_task_dead(SIGSEGV); +} + +void user_do_sig(struct pt_regs *regs, int signo, int code, unsigned long addr) +{ + struct task_struct *tsk = current; + + if (show_unhandled_signals && unhandled_signal(tsk, signo) + && printk_ratelimit()) { + pr_info("%s[%d]: unhandled signal %d code 0x%x at 0x%lx", + tsk->comm, task_pid_nr(tsk), signo, code, addr); + print_vma_addr(KERN_CONT " in ", instruction_pointer(regs)); + pr_cont("\n"); + show_regs(regs); + } + if (signo == SIGKILL) { + force_sig(signo); + return; + } + force_sig_fault(signo, code, (void __user *) addr); +} + +static void panic_or_kill(uint64_t es, uint64_t ea, struct pt_regs *regs, + int signo, int sigcode) +{ + if (user_mode(regs)) { + user_do_sig(regs, signo, sigcode, ea); + return; + } + + pr_alert(CUT_HERE "ERROR: TRAP %s received at 0x%.16llx\n", + trap_name[trap_cause(es)], regs->spc); + die(regs, ea, "Oops"); + make_task_dead(SIGKILL); +} + +int is_valid_bugaddr(unsigned long pc) +{ + /* + * Since the bug was reported, this means that the break hook handling + * already check the faulting instruction so there is no need for + * additionnal check here. This is a BUG for sure. + */ + return 1; +} + +static int bug_break_handler(struct break_hook *brk_hook, struct pt_regs *regs) +{ + enum bug_trap_type type; + + type = report_bug(regs->spc, regs); + switch (type) { + case BUG_TRAP_TYPE_NONE: + return BREAK_HOOK_ERROR; + case BUG_TRAP_TYPE_WARN: + break; + case BUG_TRAP_TYPE_BUG: + die(regs, regs->spc, "Kernel BUG"); + break; + } + + /* Skip over break insn if we survived ! */ + kvx_skip_break_insn(regs); + + return BREAK_HOOK_HANDLED; +} + +static struct break_hook bug_break_hook = { + .handler = bug_break_handler, + .id = BREAK_CAUSE_BUG, + .mode = MODE_KERNEL, +}; + +#define GEN_TRAP_HANDLER(__name, __sig, __code) \ +static void __name ## _trap_handler(uint64_t es, uint64_t ea, \ + struct pt_regs *regs) \ +{ \ + panic_or_kill(es, ea, regs, __sig, __code); \ +} + +GEN_TRAP_HANDLER(default, SIGKILL, SI_KERNEL); +GEN_TRAP_HANDLER(privilege, SIGILL, ILL_PRVREG); +GEN_TRAP_HANDLER(dmisalign, SIGBUS, BUS_ADRALN); +GEN_TRAP_HANDLER(syserror, SIGBUS, BUS_ADRERR); +GEN_TRAP_HANDLER(opcode, SIGILL, ILL_ILLOPC); + +static void register_trap_handler(unsigned int trap_nb, trap_handler_func fn) +{ + + if (trap_nb >= KVX_TRAP_COUNT || fn == NULL) + panic("Failed to register handler #%d\n", trap_nb); + + trap_handler_table[trap_nb] = fn; +} + +static void do_vsfr_fault(uint64_t es, uint64_t ea, struct pt_regs *regs) +{ + if (break_hook_handler(es, regs) == BREAK_HOOK_HANDLED) + return; + + panic_or_kill(es, ea, regs, SIGILL, ILL_PRVREG); +} + +void __init trap_init(void) +{ + int i; + + break_hook_register(&bug_break_hook); + + for (i = 0; i < KVX_TRAP_COUNT; i++) + register_trap_handler(i, default_trap_handler); +#ifdef CONFIG_MMU + register_trap_handler(KVX_TRAP_NOMAPPING, do_page_fault); + register_trap_handler(KVX_TRAP_PROTECTION, do_page_fault); + register_trap_handler(KVX_TRAP_WRITETOCLEAN, do_writetoclean); +#endif + + register_trap_handler(KVX_TRAP_PSYSERROR, syserror_trap_handler); + register_trap_handler(KVX_TRAP_DSYSERROR, syserror_trap_handler); + register_trap_handler(KVX_TRAP_PRIVILEGE, privilege_trap_handler); + register_trap_handler(KVX_TRAP_OPCODE, opcode_trap_handler); + register_trap_handler(KVX_TRAP_DMISALIGN, dmisalign_trap_handler); + register_trap_handler(KVX_TRAP_VSFR, do_vsfr_fault); +} + +/** + * trap_handler - trap handler called by _trap_handler routine in trap_handler.S + * This handler will redirect to other trap handlers if present + * If not then it will do a generic action + * @es: Exception Syndrome register value + * @ea: Exception Address register + * @regs: pointer to registers saved when trapping + */ +void trap_handler(uint64_t es, uint64_t ea, struct pt_regs *regs) +{ + enum ctx_state prev_state = exception_enter(); + int htc = trap_cause(es); + trap_handler_func trap_func = trap_handler_table[htc]; + + trace_hardirqs_off(); + + /* Normal traps number should and must be between 0 and 15 included */ + if (unlikely(htc >= KVX_TRAP_COUNT)) { + pr_err("Invalid trap %d !\n", htc); + goto done; + } + + /* If irqs were enabled in the preempted context, reenable them */ + if (regs->sps & KVX_SFR_PS_IE_MASK) + local_irq_enable(); + + trap_func(es, ea, regs); + +done: + dame_irq_check(regs); + exception_exit(prev_state); +} diff --git a/arch/kvx/platform/ipi.c b/arch/kvx/platform/ipi.c new file mode 100644 index 000000000000..73d1de27a628 --- /dev/null +++ b/arch/kvx/platform/ipi.c @@ -0,0 +1,110 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017-2023 Kalray Inc. + * Author(s): Clement Leger + * Luc Michel + */ + +#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define IPI_INTERRUPT_OFFSET 0x0 +#define IPI_MASK_OFFSET 0x20 + +/* + * IPI controller can signal RM and PE0 -> 15 + * In order to restrict that to the PE, write the corresponding mask + */ +#define KVX_IPI_CPU_MASK (~0xFFFF) + +struct kvx_ipi_ctrl { + void __iomem *regs; + unsigned int ipi_irq; +}; + +static struct kvx_ipi_ctrl kvx_ipi_controller; + +/** + * @kvx_pwr_ctrl_cpu_poweron Wakeup a cpu + * + * cpu: cpu to wakeup + */ +void kvx_ipi_send(const struct cpumask *mask) +{ + const unsigned long *maskb = cpumask_bits(mask); + + WARN_ON(*maskb & KVX_IPI_CPU_MASK); + writel(*maskb, kvx_ipi_controller.regs + IPI_INTERRUPT_OFFSET); +} + +static int kvx_ipi_starting_cpu(unsigned int cpu) +{ + enable_percpu_irq(kvx_ipi_controller.ipi_irq, IRQ_TYPE_NONE); + + return 0; +} + +static int kvx_ipi_dying_cpu(unsigned int cpu) +{ + disable_percpu_irq(kvx_ipi_controller.ipi_irq); + + return 0; +} + +int __init kvx_ipi_ctrl_probe(irqreturn_t (*ipi_irq_handler)(int, void *)) +{ + struct device_node *np; + int ret; + unsigned int ipi_irq; + void __iomem *ipi_base; + + np = of_find_compatible_node(NULL, NULL, "kalray,kvx-ipi-ctrl"); + BUG_ON(!np); + + ipi_base = of_iomap(np, 0); + BUG_ON(!ipi_base); + + kvx_ipi_controller.regs = ipi_base; + + /* Init mask for interrupts to PE0 -> PE15 */ + writel(KVX_IPI_CPU_MASK, kvx_ipi_controller.regs + IPI_MASK_OFFSET); + + ipi_irq = irq_of_parse_and_map(np, 0); + of_node_put(np); + if (!ipi_irq) { + pr_err("Failed to parse irq: %d\n", ipi_irq); + return -EINVAL; + } + + ret = request_percpu_irq(ipi_irq, ipi_irq_handler, + "kvx_ipi", &kvx_ipi_controller); + if (ret) { + pr_err("can't register interrupt %d (%d)\n", + ipi_irq, ret); + return ret; + } + kvx_ipi_controller.ipi_irq = ipi_irq; + + ret = cpuhp_setup_state(CPUHP_AP_IRQ_KVX_STARTING, + "kvx/ipi:online", + kvx_ipi_starting_cpu, + kvx_ipi_dying_cpu); + if (ret < 0) { + pr_err("Failed to setup hotplug state"); + return ret; + } + + pr_info("controller probed\n"); + + return 0; +} -- 2.37.2