Received: by 2002:a05:6902:102b:0:0:0:0 with SMTP id x11csp878514ybt; Wed, 8 Jul 2020 14:10:49 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwEQzcrALIAjoA8nxEm8wPGh3GsT2TWqnbWBScUdgvogvoEvpQC767Q4zJWldNZOWVHK8Pf X-Received: by 2002:a17:907:94cf:: with SMTP id dn15mr57584244ejc.457.1594242649656; Wed, 08 Jul 2020 14:10:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594242649; cv=none; d=google.com; s=arc-20160816; b=U4scOv61ikF30/EKU7t9mF2T/2XvnbLOCuHB2qH1mWIDL+NVPeWp95A8YXOdtbzYl7 pu/bP+XNW9mvvBoQ3+3vKYZ5bhLwnA3AXajUCi+Ckh7n0rq/Kl6z/LhONqad3bS9qDvn JrFWe3jSV3s+GsiKLfc4jkB4QNX81y7XlheghRDERH89wKfYz9s0xBwqSnLsXC4MKSis lF2KUriiyOv+tNRBww0xSECuLBmZB1GT0wYkOx73dLTIe+RxK2VzeRMRv4USo5XoPafJ MzCDdIC3VpyPpNv1BZVJgvIUk48sWI3awfQ6JO5fWyGetCJ/qJrzi4xau4VOvdkFYpGH O7Yg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=BRdWrZGotFU/V/N69RBCWuEXsKoewH6WQsweVR7ALZI=; b=PqCu9vsNtPPn9NSM+MWxAvzLK8Jo3QmhRZ+U2BHtpcibZiTFd4kvBd/1LeUhX/NnoC dvUKYjGYNjwx8XIoJ7+J/9tBUnEoq+bwSfVfuahAIaHzvizNNP6WxWkjfhRwpmGXIrHh A6gJwwVCDVgU3RjGaKru8SGwGH9a9s5x2EPIQ4S1q57erEbP6UE4VzWzlES7Qgx9baOv rUGRtCBEV14KQLxCBsQIREdLPMgJbtuCi/Yi66RI1gS420z3c5iNKwVShL5KLHb+j91h eMvzm2PJ5eqvftoXADxcOubntEi311Eq5bgSRtQXJTomsavTotenW4Ww80F60iry0kS+ ksHA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@gmail.com header.s=20161025 header.b=YbRsINFQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id dk23si689865edb.219.2020.07.08.14.10.26; Wed, 08 Jul 2020 14:10:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=fail header.i=@gmail.com header.s=20161025 header.b=YbRsINFQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726184AbgGHVKQ (ORCPT + 99 others); Wed, 8 Jul 2020 17:10:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55890 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725787AbgGHVKP (ORCPT ); Wed, 8 Jul 2020 17:10:15 -0400 Received: from mail-wm1-x344.google.com (mail-wm1-x344.google.com [IPv6:2a00:1450:4864:20::344]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 30DCBC061A0B; Wed, 8 Jul 2020 14:10:15 -0700 (PDT) Received: by mail-wm1-x344.google.com with SMTP id f18so4599503wml.3; Wed, 08 Jul 2020 14:10:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BRdWrZGotFU/V/N69RBCWuEXsKoewH6WQsweVR7ALZI=; b=YbRsINFQuj4rsIvUK1OvMoLvTYf9ohCqUDZGyBsOUBI3xxHIV+sh/Njeh0iTFSfrm7 nf2DXUHUvXzafp9O+7dfPtZWLreUJkV7xdBl9yr1xSMjbrDpILgJ7cmk8GUwE45rdFvk 7+CIS4sSr7qsGv2SYKoAsRWs1ERetwZ6R2d70aPbkxz3tv7n+UvK2nuSZk4QB+OP5rLb JzTk7d1DU4nNJuvteFyS03a3Y7A1fkarmms0pCRX+2RLyA5oTERnh4J3Y95M3CdfIXWp aLbbh1t0UXAEekvcWViq+7HAn1MfESw1eamf8rR3WjF+ZYAdSXjKU7kiyPaOOwVn1/UT EJ9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:from:to:cc:subject:date:message-id :in-reply-to:references:mime-version:content-transfer-encoding; bh=BRdWrZGotFU/V/N69RBCWuEXsKoewH6WQsweVR7ALZI=; b=geybeQ7rhn/aTSoNGX4x7PFEN4Qq/zTVx0YUQTW80veZ+Vlq8vmCDhXigGiyKaWVHZ uS4Vrb47CARp9Xe8g8vaL7KDOnM72v8OB1cRkHATbZIygZ1CZOKymkjlEptZLCGZSMc9 6U2fIc/0h75uT5a2j2B525EOzGp51SsLsoElBsLE84bMZUIqVsv7G85Jo1l4W8Nz4hYe JuEXb9bpYDAWtoaLitMk+h7xde8YR3fivPlAHgG8KbRdbtdOrS2qdGxFkoHITA0D8fob TaIBOvGCe2vODJSC8HxkJgpxSwV7pgm9hTNxciSRYioj45RAH3lbP2CJr+rBx8Z7Kj/9 XXVA== X-Gm-Message-State: AOAM530sLBb17UGtGvc5TzwJSOFBLlwujv8yz40TMrFXj1R9gF8GSDPX nB8Yq3GX3IoEB/Y1gphBXcg= X-Received: by 2002:a1c:4804:: with SMTP id v4mr11080060wma.139.1594242613783; Wed, 08 Jul 2020 14:10:13 -0700 (PDT) Received: from localhost.localdomain ([80.71.140.73]) by smtp.gmail.com with ESMTPSA id u23sm1971515wru.94.2020.07.08.14.10.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Jul 2020 14:10:13 -0700 (PDT) From: Emil Renner Berthing To: linux-riscv@lists.infradead.org Cc: Emil Renner Berthing , Palmer Dabbelt , =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , Paul Walmsley , Jonathan Corbet , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 2/2] riscv: Add jump-label implementation Date: Wed, 8 Jul 2020 23:09:48 +0200 Message-Id: <20200708210948.34197-2-kernel@esmil.dk> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200708210948.34197-1-kernel@esmil.dk> References: <20200708210948.34197-1-kernel@esmil.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Add jump-label implementation based on the ARM64 version and add CONFIG_JUMP_LABEL=y to the defconfigs. Signed-off-by: Emil Renner Berthing Reviewed-by: Björn Töpel --- Tested on the HiFive Unleashed board. Changes since v1: - WARN and give up gracefully if the jump offset cannot be represented in a JAL instruction. - Add missing braces. - Add CONFIG_JUMP_LABEL=y to defconfigs. All suggested by Björn Töpel. Changes since RFC: - Use RISCV_PTR and RISCV_LGPTR macros to match struct jump_table also in 32bit kernels. - Remove unneeded branch ? 1 : 0, thanks Björn - Fix \n\n instead of \n\t mistake .../core/jump-labels/arch-support.txt | 2 +- arch/riscv/Kconfig | 2 + arch/riscv/configs/defconfig | 1 + arch/riscv/configs/nommu_k210_defconfig | 1 + arch/riscv/configs/nommu_virt_defconfig | 1 + arch/riscv/configs/rv32_defconfig | 1 + arch/riscv/include/asm/jump_label.h | 59 +++++++++++++++++++ arch/riscv/kernel/Makefile | 2 + arch/riscv/kernel/jump_label.c | 49 +++++++++++++++ 9 files changed, 117 insertions(+), 1 deletion(-) create mode 100644 arch/riscv/include/asm/jump_label.h create mode 100644 arch/riscv/kernel/jump_label.c diff --git a/Documentation/features/core/jump-labels/arch-support.txt b/Documentation/features/core/jump-labels/arch-support.txt index 632a1c7aefa2..760243d18ed7 100644 --- a/Documentation/features/core/jump-labels/arch-support.txt +++ b/Documentation/features/core/jump-labels/arch-support.txt @@ -23,7 +23,7 @@ | openrisc: | TODO | | parisc: | ok | | powerpc: | ok | - | riscv: | TODO | + | riscv: | ok | | s390: | ok | | sh: | TODO | | sparc: | ok | diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index fd639937e251..d2f5c53fdc19 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -46,6 +46,8 @@ config RISCV select GENERIC_TIME_VSYSCALL if MMU && 64BIT select HANDLE_DOMAIN_IRQ select HAVE_ARCH_AUDITSYSCALL + select HAVE_ARCH_JUMP_LABEL + select HAVE_ARCH_JUMP_LABEL_RELATIVE select HAVE_ARCH_KASAN if MMU && 64BIT select HAVE_ARCH_KGDB select HAVE_ARCH_KGDB_QXFER_PKT diff --git a/arch/riscv/configs/defconfig b/arch/riscv/configs/defconfig index 4da4886246a4..d58c93efb603 100644 --- a/arch/riscv/configs/defconfig +++ b/arch/riscv/configs/defconfig @@ -17,6 +17,7 @@ CONFIG_BPF_SYSCALL=y CONFIG_SOC_SIFIVE=y CONFIG_SOC_VIRT=y CONFIG_SMP=y +CONFIG_JUMP_LABEL=y CONFIG_MODULES=y CONFIG_MODULE_UNLOAD=y CONFIG_NET=y diff --git a/arch/riscv/configs/nommu_k210_defconfig b/arch/riscv/configs/nommu_k210_defconfig index b48138e329ea..cd1df62b13c7 100644 --- a/arch/riscv/configs/nommu_k210_defconfig +++ b/arch/riscv/configs/nommu_k210_defconfig @@ -33,6 +33,7 @@ CONFIG_SMP=y CONFIG_NR_CPUS=2 CONFIG_CMDLINE="earlycon console=ttySIF0" CONFIG_CMDLINE_FORCE=y +CONFIG_JUMP_LABEL=y # CONFIG_BLOCK is not set CONFIG_BINFMT_FLAT=y # CONFIG_COREDUMP is not set diff --git a/arch/riscv/configs/nommu_virt_defconfig b/arch/riscv/configs/nommu_virt_defconfig index cf74e179bf90..f27596e9663e 100644 --- a/arch/riscv/configs/nommu_virt_defconfig +++ b/arch/riscv/configs/nommu_virt_defconfig @@ -30,6 +30,7 @@ CONFIG_MAXPHYSMEM_2GB=y CONFIG_SMP=y CONFIG_CMDLINE="root=/dev/vda rw earlycon=uart8250,mmio,0x10000000,115200n8 console=ttyS0" CONFIG_CMDLINE_FORCE=y +CONFIG_JUMP_LABEL=y # CONFIG_BLK_DEV_BSG is not set CONFIG_PARTITION_ADVANCED=y # CONFIG_MSDOS_PARTITION is not set diff --git a/arch/riscv/configs/rv32_defconfig b/arch/riscv/configs/rv32_defconfig index 05bbf5240569..3a55f0e00d6c 100644 --- a/arch/riscv/configs/rv32_defconfig +++ b/arch/riscv/configs/rv32_defconfig @@ -17,6 +17,7 @@ CONFIG_BPF_SYSCALL=y CONFIG_SOC_VIRT=y CONFIG_ARCH_RV32I=y CONFIG_SMP=y +CONFIG_JUMP_LABEL=y CONFIG_MODULES=y CONFIG_MODULE_UNLOAD=y CONFIG_NET=y diff --git a/arch/riscv/include/asm/jump_label.h b/arch/riscv/include/asm/jump_label.h new file mode 100644 index 000000000000..d5fb342bfccf --- /dev/null +++ b/arch/riscv/include/asm/jump_label.h @@ -0,0 +1,59 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2020 Emil Renner Berthing + * + * Based on arch/arm64/include/asm/jump_label.h + */ +#ifndef __ASM_JUMP_LABEL_H +#define __ASM_JUMP_LABEL_H + +#ifndef __ASSEMBLY__ + +#include + +#define JUMP_LABEL_NOP_SIZE 4 + +static __always_inline bool arch_static_branch(struct static_key *key, + bool branch) +{ + asm_volatile_goto( + " .option push \n\t" + " .option norelax \n\t" + " .option norvc \n\t" + "1: nop \n\t" + " .option pop \n\t" + " .pushsection __jump_table, \"aw\" \n\t" + " .align " RISCV_LGPTR " \n\t" + " .long 1b - ., %l[label] - . \n\t" + " " RISCV_PTR " %0 - . \n\t" + " .popsection \n\t" + : : "i"(&((char *)key)[branch]) : : label); + + return false; +label: + return true; +} + +static __always_inline bool arch_static_branch_jump(struct static_key *key, + bool branch) +{ + asm_volatile_goto( + " .option push \n\t" + " .option norelax \n\t" + " .option norvc \n\t" + "1: jal zero, %l[label] \n\t" + " .option pop \n\t" + " .pushsection __jump_table, \"aw\" \n\t" + " .align " RISCV_LGPTR " \n\t" + " .long 1b - ., %l[label] - . \n\t" + " " RISCV_PTR " %0 - . \n\t" + " .popsection \n\t" + : : "i"(&((char *)key)[branch]) : : label); + + return false; +label: + return true; +} + +#endif /* __ASSEMBLY__ */ +#endif /* __ASM_JUMP_LABEL_H */ diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index b355cf485671..a5287ab9f7f2 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -53,4 +53,6 @@ endif obj-$(CONFIG_HOTPLUG_CPU) += cpu-hotplug.o obj-$(CONFIG_KGDB) += kgdb.o +obj-$(CONFIG_JUMP_LABEL) += jump_label.o + clean: diff --git a/arch/riscv/kernel/jump_label.c b/arch/riscv/kernel/jump_label.c new file mode 100644 index 000000000000..1bab1abc1aa5 --- /dev/null +++ b/arch/riscv/kernel/jump_label.c @@ -0,0 +1,49 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2020 Emil Renner Berthing + * + * Based on arch/arm64/kernel/jump_label.c + */ +#include +#include +#include +#include + +#define RISCV_INSN_NOP 0x00000013U +#define RISCV_INSN_JAL 0x0000006fU + +void arch_jump_label_transform(struct jump_entry *entry, + enum jump_label_type type) +{ + void *addr = (void *)jump_entry_code(entry); + u32 insn; + + if (type == JUMP_LABEL_JMP) { + long offset = jump_entry_target(entry) - jump_entry_code(entry); + + if (WARN_ON(offset & 1 || offset < -524288 || offset >= 524288)) + return; + + insn = RISCV_INSN_JAL | + (((u32)offset & GENMASK(19, 12)) << (12 - 12)) | + (((u32)offset & GENMASK(11, 11)) << (20 - 11)) | + (((u32)offset & GENMASK(10, 1)) << (21 - 1)) | + (((u32)offset & GENMASK(20, 20)) << (31 - 20)); + } else { + insn = RISCV_INSN_NOP; + } + + patch_text_nosync(addr, &insn, sizeof(insn)); +} + +void arch_jump_label_transform_static(struct jump_entry *entry, + enum jump_label_type type) +{ + /* + * We use the same instructions in the arch_static_branch and + * arch_static_branch_jump inline functions, so there's no + * need to patch them up here. + * The core will call arch_jump_label_transform when those + * instructions need to be replaced. + */ +} -- 2.27.0