kevin/linux
1
Fork 0
Code
master
linux/arch/riscv/kernel/traps.c

433 lines
10 KiB
C
Raw Permalink Normal View History

treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 286 Based on 1 normalized pattern(s): this program is free software you can redistribute it and or modify it under the terms of the gnu general public license as published by the free software foundation version 2 this program is distributed in the hope that it will be useful but without any warranty without even the implied warranty of merchantability or fitness for a particular purpose see the gnu general public license for more details extracted by the scancode license scanner the SPDX license identifier GPL-2.0-only has been chosen to replace the boilerplate/reference in 97 file(s). Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Reviewed-by: Allison Randal <allison@lohutok.net> Reviewed-by: Alexios Zavras <alexios.zavras@intel.com> Cc: linux-spdx@vger.kernel.org Link: https://lkml.kernel.org/r/20190529141901.025053186@linutronix.de Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2019-05-29 07:18:00 -07:00
// SPDX-License-Identifier: GPL-2.0-only
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
/*
* Copyright (C) 2012 Regents of the University of California
*/
riscv: add missing header file includes sparse identifies several missing prototypes caused by missing preprocessor include directives: arch/riscv/kernel/cpufeature.c:16:6: warning: symbol 'has_fpu' was not declared. Should it be static? arch/riscv/kernel/process.c:26:6: warning: symbol 'arch_cpu_idle' was not declared. Should it be static? arch/riscv/kernel/reset.c:15:6: warning: symbol 'pm_power_off' was not declared. Should it be static? arch/riscv/kernel/syscall_table.c:15:6: warning: symbol 'sys_call_table' was not declared. Should it be static? arch/riscv/kernel/traps.c:149:13: warning: symbol 'trap_init' was not declared. Should it be static? arch/riscv/kernel/vdso.c:54:5: warning: symbol 'arch_setup_additional_pages' was not declared. Should it be static? arch/riscv/kernel/smp.c:64:6: warning: symbol 'arch_match_cpu_phys_id' was not declared. Should it be static? arch/riscv/kernel/module-sections.c:89:5: warning: symbol 'module_frob_arch_sections' was not declared. Should it be static? arch/riscv/mm/context.c:42:6: warning: symbol 'switch_mm' was not declared. Should it be static? Fix by including the appropriate header files in the appropriate source files. This patch should have no functional impact. Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com> Reviewed-by: Christoph Hellwig <hch@lst.de>
2019-10-17 15:21:28 -07:00
#include <linux/cpu.h>
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
#include <linux/kernel.h>
#include <linux/init.h>
riscv: Support RANDOMIZE_KSTACK_OFFSET Inspired from arm64's implement -- commit 70918779aec9 ("arm64: entry: Enable random_kstack_offset support") Add support of kernel stack offset randomization while handling syscall, the offset is defaultly limited by KSTACK_OFFSET_MAX() (i.e. 10 bits). In order to avoid trigger stack canaries (due to __builtin_alloca) and slowing down the entry path, use __no_stack_protector attribute to disable stack protector for do_trap_ecall_u() at the function level. Acked-by: Palmer Dabbelt <palmer@rivosinc.com> Reviewed-by: Kees Cook <keescook@chromium.org> Signed-off-by: Song Shuai <songshuaishuai@tinylab.org> Link: https://lore.kernel.org/r/20231109133751.212079-1-songshuaishuai@tinylab.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-11-09 06:37:51 -07:00
#include <linux/randomize_kstack.h>
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
#include <linux/sched.h>
#include <linux/sched/debug.h>
#include <linux/sched/signal.h>
#include <linux/signal.h>
#include <linux/kdebug.h>
#include <linux/uaccess.h>
riscv: Add kprobes supported This patch enables "kprobe & kretprobe" to work with ftrace interface. It utilized software breakpoint as single-step mechanism. Some instructions which can't be single-step executed must be simulated in kernel execution slot, such as: branch, jal, auipc, la ... Some instructions should be rejected for probing and we use a blacklist to filter, such as: ecall, ebreak, ... We use ebreak & c.ebreak to replace origin instruction and the kprobe handler prepares an executable memory slot for out-of-line execution with a copy of the original instruction being probed. In execution slot we add ebreak behind original instruction to simulate a single-setp mechanism. The patch is based on packi's work [1] and csky's work [2]. - The kprobes_trampoline.S is all from packi's patch - The single-step mechanism is new designed for riscv without hw single-step trap - The simulation codes are from csky - Frankly, all codes refer to other archs' implementation [1] https://lore.kernel.org/linux-riscv/20181113195804.22825-1-me@packi.ch/ [2] https://lore.kernel.org/linux-csky/20200403044150.20562-9-guoren@kernel.org/ Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Co-developed-by: Patrick Stählin <me@packi.ch> Signed-off-by: Patrick Stählin <me@packi.ch> Acked-by: Masami Hiramatsu <mhiramat@kernel.org> Tested-by: Zong Li <zong.li@sifive.com> Reviewed-by: Pekka Enberg <penberg@kernel.org> Cc: Patrick Stählin <me@packi.ch> Cc: Palmer Dabbelt <palmerdabbelt@google.com> Cc: Björn Töpel <bjorn.topel@gmail.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-12-17 09:01:42 -07:00
#include <linux/kprobes.h>
riscv: Only consider swbp/ss handlers for correct privileged mode RISC-V software breakpoint trap handlers are used for {k,u}probes. When trapping from kernelmode, only the kernelmode handlers should be considered. Vice versa, only usermode handlers for usermode traps. This is not the case on RISC-V, which can trigger a bug if a userspace process uses uprobes, and a WARN() is triggered from kernelmode (which is implemented via {c.,}ebreak). The kernel will trap on the kernelmode {c.,}ebreak, look for uprobes handlers, realize incorrectly that uprobes need to be handled, and exit the trap handler early. The trap returns to re-executing the {c.,}ebreak, and enter an infinite trap-loop. The issue was found running the BPF selftest [1]. Fix this issue by only considering the swbp/ss handlers for kernel/usermode respectively. Also, move CONFIG ifdeffery from traps.c to the asm/{k,u}probes.h headers. Note that linux/uprobes.h only include asm/uprobes.h if CONFIG_UPROBES is defined, which is why asm/uprobes.h needs to be unconditionally included in traps.c Link: https://lore.kernel.org/linux-riscv/87v8d19aun.fsf@all.your.base.are.belong.to.us/ # [1] Fixes: 74784081aac8 ("riscv: Add uprobes supported") Reviewed-by: Guo Ren <guoren@kernel.org> Reviewed-by: Nam Cao <namcaov@gmail.com> Tested-by: Puranjay Mohan <puranjay12@gmail.com> Signed-off-by: Björn Töpel <bjorn@rivosinc.com> Link: https://lore.kernel.org/r/20230912065619.62020-1-bjorn@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-09-11 23:56:19 -07:00
#include <linux/uprobes.h>
#include <asm/uprobes.h>
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
#include <linux/mm.h>
#include <linux/module.h>
#include <linux/irq.h>
RISC-V: Add fast call path of crash_kexec() Currently, almost all archs (x86, arm64, mips...) support fast call of crash_kexec() when "regs && kexec_should_crash()" is true. But RISC-V not, it can only enter crash system via panic(). However panic() doesn't pass the regs of the real accident scene to crash_kexec(), it caused we can't get accurate backtrace via gdb, $ riscv64-linux-gnu-gdb vmlinux vmcore Reading symbols from vmlinux... [New LWP 95] #0 console_unlock () at kernel/printk/printk.c:2557 2557 if (do_cond_resched) (gdb) bt #0 console_unlock () at kernel/printk/printk.c:2557 #1 0x0000000000000000 in ?? () With the patch we can get the accurate backtrace, $ riscv64-linux-gnu-gdb vmlinux vmcore Reading symbols from vmlinux... [New LWP 95] #0 0xffffffe00063a4e0 in test_thread (data=<optimized out>) at drivers/test_crash.c:81 81 *(int *)p = 0xdead; (gdb) (gdb) bt #0 0xffffffe00064d5c0 in test_thread (data=<optimized out>) at drivers/test_crash.c:81 #1 0x0000000000000000 in ?? () Test code to produce NULL address dereference in test_crash.c, void *p = NULL; *(int *)p = 0xdead; Reviewed-by: Guo Ren <guoren@kernel.org> Tested-by: Xianting Tian <xianting.tian@linux.alibaba.com> Signed-off-by: Xianting Tian <xianting.tian@linux.alibaba.com> Link: https://lore.kernel.org/r/20220606082308.2883458-1-xianting.tian@linux.alibaba.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-06-06 01:23:08 -07:00
#include <linux/kexec.h>
riscv: entry: Convert to generic entry This patch converts riscv to use the generic entry infrastructure from kernel/entry/*. The generic entry makes maintainers' work easier and codes more elegant. Here are the changes: - More clear entry.S with handle_exception and ret_from_exception - Get rid of complex custom signal implementation - Move syscall procedure from assembly to C, which is much more readable. - Connect ret_from_fork & ret_from_kernel_thread to generic entry. - Wrap with irqentry_enter/exit and syscall_enter/exit_from_user_mode - Use the standard preemption code instead of custom Suggested-by: Huacai Chen <chenhuacai@kernel.org> Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Tested-by: Yipeng Zou <zouyipeng@huawei.com> Tested-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Cc: Ben Hutchings <ben@decadent.org.uk> Link: https://lore.kernel.org/r/20230222033021.983168-5-guoren@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-21 20:30:18 -07:00
#include <linux/entry-common.h>
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
riscv: traps: Fix no prototype warnings Fix all W=1 compilation warnings:'no previous prototype for' in arch/riscv/kernel/traps.c: arch/riscv/kernel/traps.c:96:15: warning: no previous prototype for ‘do_trap_unknown’ [-Wmissing-prototypes] 96 | DO_ERROR_INFO(do_trap_unknown, | ^~~~~~~~~~~~~~~ arch/riscv/kernel/traps.c:91:27: note: in definition of macro ‘DO_ERROR_INFO’ 91 | asmlinkage __visible void name(struct pt_regs *regs) \ | ^~~~ arch/riscv/kernel/traps.c:98:15: warning: no previous prototype for ‘do_trap_insn_misaligned’ [-Wmissing-prototypes] 98 | DO_ERROR_INFO(do_trap_insn_misaligned, | ^~~~~~~~~~~~~~~~~~~~~~~ arch/riscv/kernel/traps.c:91:27: note: in definition of macro ‘DO_ERROR_INFO’ 91 | asmlinkage __visible void name(struct pt_regs *regs) \ | ^~~~ arch/riscv/kernel/traps.c:100:15: warning: no previous prototype for ‘do_trap_insn_fault’ [-Wmissing-prototypes] ... Reported-by: Hulk Robot <hulkci@huawei.com> Signed-off-by: Nanyong Sun <sunnanyong@huawei.com> [Palmer: fix checkpatch warnings] Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-03-05 04:33:24 -07:00
#include <asm/asm-prototypes.h>
riscv: Add dump stack in show_regs Like commit 1149aad10b1e ("arm64: Add dump_backtrace() in show_regs"), dump the stack in riscv show_regs as common code expects. Reviewed-by: Atish Patra <atish.patra@wdc.com> Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-01-11 05:40:12 -07:00
#include <asm/bug.h>
riscv: Add CFI error handling With CONFIG_CFI_CLANG, the compiler injects a type preamble immediately before each function and a check to validate the target function type before indirect calls: ; type preamble .word <id> function: ... ; indirect call check lw t1, -4(a0) lui t2, <hi20> addiw t2, t2, <lo12> beq t1, t2, .Ltmp0 ebreak .Ltmp0: jarl a0 Implement error handling code for the ebreak traps emitted for the checks. This produces the following oops on a CFI failure (generated using lkdtm): [ 21.177245] CFI failure at lkdtm_indirect_call+0x22/0x32 [lkdtm] (target: lkdtm_increment_int+0x0/0x18 [lkdtm]; expected type: 0x3ad55aca) [ 21.178483] Kernel BUG [#1] [ 21.178671] Modules linked in: lkdtm [ 21.179037] CPU: 1 PID: 104 Comm: sh Not tainted 6.3.0-rc6-00037-g37d5ec6297ab #1 [ 21.179511] Hardware name: riscv-virtio,qemu (DT) [ 21.179818] epc : lkdtm_indirect_call+0x22/0x32 [lkdtm] [ 21.180106] ra : lkdtm_CFI_FORWARD_PROTO+0x48/0x7c [lkdtm] [ 21.180426] epc : ffffffff01387092 ra : ffffffff01386f14 sp : ff20000000453cf0 [ 21.180792] gp : ffffffff81308c38 tp : ff6000000243f080 t0 : ff20000000453b78 [ 21.181157] t1 : 000000003ad55aca t2 : 000000007e0c52a5 s0 : ff20000000453d00 [ 21.181506] s1 : 0000000000000001 a0 : ffffffff0138d170 a1 : ffffffff013870bc [ 21.181819] a2 : b5fea48dd89aa700 a3 : 0000000000000001 a4 : 0000000000000fff [ 21.182169] a5 : 0000000000000004 a6 : 00000000000000b7 a7 : 0000000000000000 [ 21.182591] s2 : ff20000000453e78 s3 : ffffffffffffffea s4 : 0000000000000012 [ 21.183001] s5 : ff600000023c7000 s6 : 0000000000000006 s7 : ffffffff013882a0 [ 21.183653] s8 : 0000000000000008 s9 : 0000000000000002 s10: ffffffff0138d878 [ 21.184245] s11: ffffffff0138d878 t3 : 0000000000000003 t4 : 0000000000000000 [ 21.184591] t5 : ffffffff8133df08 t6 : ffffffff8133df07 [ 21.184858] status: 0000000000000120 badaddr: 0000000000000000 cause: 0000000000000003 [ 21.185415] [<ffffffff01387092>] lkdtm_indirect_call+0x22/0x32 [lkdtm] [ 21.185772] [<ffffffff01386f14>] lkdtm_CFI_FORWARD_PROTO+0x48/0x7c [lkdtm] [ 21.186093] [<ffffffff01383552>] lkdtm_do_action+0x22/0x34 [lkdtm] [ 21.186445] [<ffffffff0138350c>] direct_entry+0x128/0x13a [lkdtm] [ 21.186817] [<ffffffff8033ed8c>] full_proxy_write+0x58/0xb2 [ 21.187352] [<ffffffff801d4fe8>] vfs_write+0x14c/0x33a [ 21.187644] [<ffffffff801d5328>] ksys_write+0x64/0xd4 [ 21.187832] [<ffffffff801d53a6>] sys_write+0xe/0x1a [ 21.188171] [<ffffffff80003996>] ret_from_syscall+0x0/0x2 [ 21.188595] Code: 0513 0f65 a303 ffc5 53b7 7e0c 839b 2a53 0363 0073 (9002) 9582 [ 21.189178] ---[ end trace 0000000000000000 ]--- [ 21.189590] Kernel panic - not syncing: Fatal exception Reviewed-by: Kees Cook <keescook@chromium.org> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> # ISA bits Tested-by: Nathan Chancellor <nathan@kernel.org> Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Link: https://lore.kernel.org/r/20230710183544.999540-12-samitolvanen@google.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-07-10 11:35:49 -07:00
#include <asm/cfi.h>
riscv: traps: add missing prototype Sparse complains: arch/riscv/kernel/traps.c:213:6: warning: symbol 'shadow_stack' was not declared. Should it be static? The variable is used in entry.S, so declare shadow_stack there alongside SHADOW_OVERFLOW_STACK_SIZE. Fixes: 31da94c25aea ("riscv: add VMAP_STACK overflow detection") Signed-off-by: Conor Dooley <conor.dooley@microchip.com> Cc: stable@vger.kernel.org Link: https://lore.kernel.org/r/20220814141237.493457-5-mail@conchuod.ie Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-08-14 07:12:38 -07:00
#include <asm/csr.h>
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
#include <asm/processor.h>
#include <asm/ptrace.h>
riscv: entry: Convert to generic entry This patch converts riscv to use the generic entry infrastructure from kernel/entry/*. The generic entry makes maintainers' work easier and codes more elegant. Here are the changes: - More clear entry.S with handle_exception and ret_from_exception - Get rid of complex custom signal implementation - Move syscall procedure from assembly to C, which is much more readable. - Connect ret_from_fork & ret_from_kernel_thread to generic entry. - Wrap with irqentry_enter/exit and syscall_enter/exit_from_user_mode - Use the standard preemption code instead of custom Suggested-by: Huacai Chen <chenhuacai@kernel.org> Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Tested-by: Yipeng Zou <zouyipeng@huawei.com> Tested-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Cc: Ben Hutchings <ben@decadent.org.uk> Link: https://lore.kernel.org/r/20230222033021.983168-5-guoren@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-21 20:30:18 -07:00
#include <asm/syscall.h>
riscv: traps: add missing prototype Sparse complains: arch/riscv/kernel/traps.c:213:6: warning: symbol 'shadow_stack' was not declared. Should it be static? The variable is used in entry.S, so declare shadow_stack there alongside SHADOW_OVERFLOW_STACK_SIZE. Fixes: 31da94c25aea ("riscv: add VMAP_STACK overflow detection") Signed-off-by: Conor Dooley <conor.dooley@microchip.com> Cc: stable@vger.kernel.org Link: https://lore.kernel.org/r/20220814141237.493457-5-mail@conchuod.ie Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-08-14 07:12:38 -07:00
#include <asm/thread_info.h>
riscv: Allocate user's vector context in the first-use trap Vector unit is disabled by default for all user processes. Thus, a process will take a trap (illegal instruction) into kernel at the first time when it uses Vector. Only after then, the kernel allocates V context and starts take care of the context for that user process. Suggested-by: Richard Henderson <richard.henderson@linaro.org> Link: https://lore.kernel.org/r/3923eeee-e4dc-0911-40bf-84c34aee962d@linaro.org Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Link: https://lore.kernel.org/r/20230605110724.21391-12-andy.chiu@sifive.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-06-05 04:07:08 -07:00
#include <asm/vector.h>
riscv: stack: Support HAVE_IRQ_EXIT_ON_IRQ_STACK Add independent irq stacks for percpu to prevent kernel stack overflows. It is also compatible with VMAP_STACK by arch_alloc_vmap_stack. Tested-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Cc: Clément Léger <cleger@rivosinc.com> Link: https://lore.kernel.org/r/20230614013018.2168426-2-guoren@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-06-13 18:30:16 -07:00
#include <asm/irq_stack.h>
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
int show_unhandled_signals = 1;
static DEFINE_SPINLOCK(die_lock);
riscv: add userland instruction dump to RISC-V splats Add userland instruction dump and rename dump_kernel_instr() to dump_instr(). An example: [ 0.822439] Freeing unused kernel image (initmem) memory: 6916K [ 0.823817] Run /init as init process [ 0.839411] init[1]: unhandled signal 4 code 0x1 at 0x000000000005be18 in bb[10000+5fb000] [ 0.840751] CPU: 0 PID: 1 Comm: init Not tainted 5.14.0-rc4-00049-gbd644290aa72-dirty #187 [ 0.841373] Hardware name: , BIOS [ 0.841743] epc : 000000000005be18 ra : 0000000000079e74 sp : 0000003fffcafda0 [ 0.842271] gp : ffffffff816e9dc8 tp : 0000000000000000 t0 : 0000000000000000 [ 0.842947] t1 : 0000003fffc9fdf0 t2 : 0000000000000000 s0 : 0000000000000000 [ 0.843434] s1 : 0000000000000000 a0 : 0000003fffca0190 a1 : 0000003fffcafe18 [ 0.843891] a2 : 0000000000000000 a3 : 0000000000000000 a4 : 0000000000000000 [ 0.844357] a5 : 0000000000000000 a6 : 0000000000000000 a7 : 0000000000000000 [ 0.844803] s2 : 0000000000000000 s3 : 0000000000000000 s4 : 0000000000000000 [ 0.845253] s5 : 0000000000000000 s6 : 0000000000000000 s7 : 0000000000000000 [ 0.845722] s8 : 0000000000000000 s9 : 0000000000000000 s10: 0000000000000000 [ 0.846180] s11: 0000000000d144e0 t3 : 0000000000000000 t4 : 0000000000000000 [ 0.846616] t5 : 0000000000000000 t6 : 0000000000000000 [ 0.847204] status: 0000000200000020 badaddr: 00000000f0028053 cause: 0000000000000002 [ 0.848219] Code: f06f ff5f 3823 fa11 0113 fb01 2e23 0201 0293 0000 (8053) f002 [ 0.851016] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000004 Signed-off-by: Yunhui Cui <cuiyunhui@bytedance.com> Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Link: https://lore.kernel.org/r/20230912021349.28302-1-cuiyunhui@bytedance.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-09-11 19:13:49 -07:00
static int copy_code(struct pt_regs *regs, u16 *val, const u16 *insns)
{
const void __user *uaddr = (__force const void __user *)insns;
if (!user_mode(regs))
return get_kernel_nofault(*val, insns);
/* The user space code from other tasks cannot be accessed. */
if (regs != task_pt_regs(current))
return -EPERM;
return copy_from_user_nofault(val, uaddr, sizeof(*val));
}
static void dump_instr(const char *loglvl, struct pt_regs *regs)
riscv: Add instruction dump to RISC-V splats Add instruction dump (Code:) output to RISC-V splats. Dump 16b parcels. An example: Unable to handle kernel NULL pointer dereference at virtual address 0000000000000000 Oops [#1] Modules linked in: CPU: 0 PID: 1 Comm: swapper/0 Not tainted 6.2.0-rc3-00302-g840ff44c571d-dirty #27 Hardware name: riscv-virtio,qemu (DT) epc : kernel_init+0xc8/0x10e ra : kernel_init+0x70/0x10e epc : ffffffff80bd9a40 ra : ffffffff80bd99e8 sp : ff2000000060bec0 gp : ffffffff81730b28 tp : ff6000007ff00000 t0 : 7974697275636573 t1 : 0000000000000000 t2 : 3030303270393d6e s0 : ff2000000060bee0 s1 : ffffffff81732028 a0 : 0000000000000000 a1 : ff60000080dd1780 a2 : 0000000000000002 a3 : ffffffff8176a470 a4 : 0000000000000000 a5 : 000000000000000a a6 : 0000000000000081 a7 : ff60000080dd1780 s2 : 0000000000000000 s3 : 0000000000000000 s4 : 0000000000000000 s5 : 0000000000000000 s6 : 0000000000000000 s7 : 0000000000000000 s8 : 0000000000000000 s9 : 0000000000000000 s10: 0000000000000000 s11: 0000000000000000 t3 : ffffffff81186018 t4 : 0000000000000022 t5 : 000000000000003d t6 : 0000000000000000 status: 0000000200000120 badaddr: 0000000000000000 cause: 000000000000000f [<ffffffff80003528>] ret_from_exception+0x0/0x16 Code: 862a d179 608c a517 0069 0513 2be5 d0ef db2e 47a9 (c11c) a517 ---[ end trace 0000000000000000 ]--- Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b SMP: stopping secondary CPUs ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b ]--- Signed-off-by: Björn Töpel <bjorn@rivosinc.com> Link: https://lore.kernel.org/r/20230119074738.708301-2-bjorn@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-01-19 00:47:37 -07:00
{
char str[sizeof("0000 ") * 12 + 2 + 1], *p = str;
const u16 *insns = (u16 *)instruction_pointer(regs);
long bad;
u16 val;
int i;
for (i = -10; i < 2; i++) {
riscv: add userland instruction dump to RISC-V splats Add userland instruction dump and rename dump_kernel_instr() to dump_instr(). An example: [ 0.822439] Freeing unused kernel image (initmem) memory: 6916K [ 0.823817] Run /init as init process [ 0.839411] init[1]: unhandled signal 4 code 0x1 at 0x000000000005be18 in bb[10000+5fb000] [ 0.840751] CPU: 0 PID: 1 Comm: init Not tainted 5.14.0-rc4-00049-gbd644290aa72-dirty #187 [ 0.841373] Hardware name: , BIOS [ 0.841743] epc : 000000000005be18 ra : 0000000000079e74 sp : 0000003fffcafda0 [ 0.842271] gp : ffffffff816e9dc8 tp : 0000000000000000 t0 : 0000000000000000 [ 0.842947] t1 : 0000003fffc9fdf0 t2 : 0000000000000000 s0 : 0000000000000000 [ 0.843434] s1 : 0000000000000000 a0 : 0000003fffca0190 a1 : 0000003fffcafe18 [ 0.843891] a2 : 0000000000000000 a3 : 0000000000000000 a4 : 0000000000000000 [ 0.844357] a5 : 0000000000000000 a6 : 0000000000000000 a7 : 0000000000000000 [ 0.844803] s2 : 0000000000000000 s3 : 0000000000000000 s4 : 0000000000000000 [ 0.845253] s5 : 0000000000000000 s6 : 0000000000000000 s7 : 0000000000000000 [ 0.845722] s8 : 0000000000000000 s9 : 0000000000000000 s10: 0000000000000000 [ 0.846180] s11: 0000000000d144e0 t3 : 0000000000000000 t4 : 0000000000000000 [ 0.846616] t5 : 0000000000000000 t6 : 0000000000000000 [ 0.847204] status: 0000000200000020 badaddr: 00000000f0028053 cause: 0000000000000002 [ 0.848219] Code: f06f ff5f 3823 fa11 0113 fb01 2e23 0201 0293 0000 (8053) f002 [ 0.851016] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000004 Signed-off-by: Yunhui Cui <cuiyunhui@bytedance.com> Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Link: https://lore.kernel.org/r/20230912021349.28302-1-cuiyunhui@bytedance.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-09-11 19:13:49 -07:00
bad = copy_code(regs, &val, &insns[i]);
riscv: Add instruction dump to RISC-V splats Add instruction dump (Code:) output to RISC-V splats. Dump 16b parcels. An example: Unable to handle kernel NULL pointer dereference at virtual address 0000000000000000 Oops [#1] Modules linked in: CPU: 0 PID: 1 Comm: swapper/0 Not tainted 6.2.0-rc3-00302-g840ff44c571d-dirty #27 Hardware name: riscv-virtio,qemu (DT) epc : kernel_init+0xc8/0x10e ra : kernel_init+0x70/0x10e epc : ffffffff80bd9a40 ra : ffffffff80bd99e8 sp : ff2000000060bec0 gp : ffffffff81730b28 tp : ff6000007ff00000 t0 : 7974697275636573 t1 : 0000000000000000 t2 : 3030303270393d6e s0 : ff2000000060bee0 s1 : ffffffff81732028 a0 : 0000000000000000 a1 : ff60000080dd1780 a2 : 0000000000000002 a3 : ffffffff8176a470 a4 : 0000000000000000 a5 : 000000000000000a a6 : 0000000000000081 a7 : ff60000080dd1780 s2 : 0000000000000000 s3 : 0000000000000000 s4 : 0000000000000000 s5 : 0000000000000000 s6 : 0000000000000000 s7 : 0000000000000000 s8 : 0000000000000000 s9 : 0000000000000000 s10: 0000000000000000 s11: 0000000000000000 t3 : ffffffff81186018 t4 : 0000000000000022 t5 : 000000000000003d t6 : 0000000000000000 status: 0000000200000120 badaddr: 0000000000000000 cause: 000000000000000f [<ffffffff80003528>] ret_from_exception+0x0/0x16 Code: 862a d179 608c a517 0069 0513 2be5 d0ef db2e 47a9 (c11c) a517 ---[ end trace 0000000000000000 ]--- Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b SMP: stopping secondary CPUs ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b ]--- Signed-off-by: Björn Töpel <bjorn@rivosinc.com> Link: https://lore.kernel.org/r/20230119074738.708301-2-bjorn@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-01-19 00:47:37 -07:00
if (!bad) {
p += sprintf(p, i == 0 ? "(%04hx) " : "%04hx ", val);
} else {
printk("%sCode: Unable to access instruction at 0x%px.\n",
loglvl, &insns[i]);
return;
}
}
printk("%sCode: %s\n", loglvl, str);
}
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
void die(struct pt_regs *regs, const char *str)
{
static int die_counter;
int ret;
RISC-V: Avoid dereferening NULL regs in die() I don't think we can actually die() without a regs pointer, but the compiler was warning about a NULL check after a dereference. It seems prudent to just avoid the possibly-NULL dereference, given that when die()ing the system is already toast so who knows how we got there. Reported-by: kernel test robot <lkp@intel.com> Reported-by: Dan Carpenter <dan.carpenter@oracle.com> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Link: https://lore.kernel.org/r/20220920200037.6727-1-palmer@rivosinc.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-09-20 13:00:37 -07:00
long cause;
riscv: Avoid enabling interrupts in die() While working on something else, I noticed that the kernel would start accepting interrupts again after crashing in an interrupt handler. Since the kernel is already in inconsistent state, enabling interrupts is dangerous and opens up risk of kernel state deteriorating further. Interrupts do get enabled via what looks like an unintended side effect of spin_unlock_irq, so switch to the more cautious spin_lock_irqsave/spin_unlock_irqrestore instead. Fixes: 76d2a0493a17 ("RISC-V: Init and Halt Code") Signed-off-by: Mattias Nissler <mnissler@rivosinc.com> Reviewed-by: Björn Töpel <bjorn@kernel.org> Link: https://lore.kernel.org/r/20230215144828.3370316-1-mnissler@rivosinc.com Cc: stable@vger.kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-15 07:48:28 -07:00
unsigned long flags;
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
oops_enter();
riscv: Avoid enabling interrupts in die() While working on something else, I noticed that the kernel would start accepting interrupts again after crashing in an interrupt handler. Since the kernel is already in inconsistent state, enabling interrupts is dangerous and opens up risk of kernel state deteriorating further. Interrupts do get enabled via what looks like an unintended side effect of spin_unlock_irq, so switch to the more cautious spin_lock_irqsave/spin_unlock_irqrestore instead. Fixes: 76d2a0493a17 ("RISC-V: Init and Halt Code") Signed-off-by: Mattias Nissler <mnissler@rivosinc.com> Reviewed-by: Björn Töpel <bjorn@kernel.org> Link: https://lore.kernel.org/r/20230215144828.3370316-1-mnissler@rivosinc.com Cc: stable@vger.kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-15 07:48:28 -07:00
spin_lock_irqsave(&die_lock, flags);
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
console_verbose();
bust_spinlocks(1);
pr_emerg("%s [#%d]\n", str, ++die_counter);
print_modules();
riscv: Add instruction dump to RISC-V splats Add instruction dump (Code:) output to RISC-V splats. Dump 16b parcels. An example: Unable to handle kernel NULL pointer dereference at virtual address 0000000000000000 Oops [#1] Modules linked in: CPU: 0 PID: 1 Comm: swapper/0 Not tainted 6.2.0-rc3-00302-g840ff44c571d-dirty #27 Hardware name: riscv-virtio,qemu (DT) epc : kernel_init+0xc8/0x10e ra : kernel_init+0x70/0x10e epc : ffffffff80bd9a40 ra : ffffffff80bd99e8 sp : ff2000000060bec0 gp : ffffffff81730b28 tp : ff6000007ff00000 t0 : 7974697275636573 t1 : 0000000000000000 t2 : 3030303270393d6e s0 : ff2000000060bee0 s1 : ffffffff81732028 a0 : 0000000000000000 a1 : ff60000080dd1780 a2 : 0000000000000002 a3 : ffffffff8176a470 a4 : 0000000000000000 a5 : 000000000000000a a6 : 0000000000000081 a7 : ff60000080dd1780 s2 : 0000000000000000 s3 : 0000000000000000 s4 : 0000000000000000 s5 : 0000000000000000 s6 : 0000000000000000 s7 : 0000000000000000 s8 : 0000000000000000 s9 : 0000000000000000 s10: 0000000000000000 s11: 0000000000000000 t3 : ffffffff81186018 t4 : 0000000000000022 t5 : 000000000000003d t6 : 0000000000000000 status: 0000000200000120 badaddr: 0000000000000000 cause: 000000000000000f [<ffffffff80003528>] ret_from_exception+0x0/0x16 Code: 862a d179 608c a517 0069 0513 2be5 d0ef db2e 47a9 (c11c) a517 ---[ end trace 0000000000000000 ]--- Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b SMP: stopping secondary CPUs ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b ]--- Signed-off-by: Björn Töpel <bjorn@rivosinc.com> Link: https://lore.kernel.org/r/20230119074738.708301-2-bjorn@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-01-19 00:47:37 -07:00
if (regs) {
RISC-V: Avoid dereferening NULL regs in die() I don't think we can actually die() without a regs pointer, but the compiler was warning about a NULL check after a dereference. It seems prudent to just avoid the possibly-NULL dereference, given that when die()ing the system is already toast so who knows how we got there. Reported-by: kernel test robot <lkp@intel.com> Reported-by: Dan Carpenter <dan.carpenter@oracle.com> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Link: https://lore.kernel.org/r/20220920200037.6727-1-palmer@rivosinc.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-09-20 13:00:37 -07:00
show_regs(regs);
riscv: add userland instruction dump to RISC-V splats Add userland instruction dump and rename dump_kernel_instr() to dump_instr(). An example: [ 0.822439] Freeing unused kernel image (initmem) memory: 6916K [ 0.823817] Run /init as init process [ 0.839411] init[1]: unhandled signal 4 code 0x1 at 0x000000000005be18 in bb[10000+5fb000] [ 0.840751] CPU: 0 PID: 1 Comm: init Not tainted 5.14.0-rc4-00049-gbd644290aa72-dirty #187 [ 0.841373] Hardware name: , BIOS [ 0.841743] epc : 000000000005be18 ra : 0000000000079e74 sp : 0000003fffcafda0 [ 0.842271] gp : ffffffff816e9dc8 tp : 0000000000000000 t0 : 0000000000000000 [ 0.842947] t1 : 0000003fffc9fdf0 t2 : 0000000000000000 s0 : 0000000000000000 [ 0.843434] s1 : 0000000000000000 a0 : 0000003fffca0190 a1 : 0000003fffcafe18 [ 0.843891] a2 : 0000000000000000 a3 : 0000000000000000 a4 : 0000000000000000 [ 0.844357] a5 : 0000000000000000 a6 : 0000000000000000 a7 : 0000000000000000 [ 0.844803] s2 : 0000000000000000 s3 : 0000000000000000 s4 : 0000000000000000 [ 0.845253] s5 : 0000000000000000 s6 : 0000000000000000 s7 : 0000000000000000 [ 0.845722] s8 : 0000000000000000 s9 : 0000000000000000 s10: 0000000000000000 [ 0.846180] s11: 0000000000d144e0 t3 : 0000000000000000 t4 : 0000000000000000 [ 0.846616] t5 : 0000000000000000 t6 : 0000000000000000 [ 0.847204] status: 0000000200000020 badaddr: 00000000f0028053 cause: 0000000000000002 [ 0.848219] Code: f06f ff5f 3823 fa11 0113 fb01 2e23 0201 0293 0000 (8053) f002 [ 0.851016] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000004 Signed-off-by: Yunhui Cui <cuiyunhui@bytedance.com> Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Link: https://lore.kernel.org/r/20230912021349.28302-1-cuiyunhui@bytedance.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-09-11 19:13:49 -07:00
dump_instr(KERN_EMERG, regs);
riscv: Add instruction dump to RISC-V splats Add instruction dump (Code:) output to RISC-V splats. Dump 16b parcels. An example: Unable to handle kernel NULL pointer dereference at virtual address 0000000000000000 Oops [#1] Modules linked in: CPU: 0 PID: 1 Comm: swapper/0 Not tainted 6.2.0-rc3-00302-g840ff44c571d-dirty #27 Hardware name: riscv-virtio,qemu (DT) epc : kernel_init+0xc8/0x10e ra : kernel_init+0x70/0x10e epc : ffffffff80bd9a40 ra : ffffffff80bd99e8 sp : ff2000000060bec0 gp : ffffffff81730b28 tp : ff6000007ff00000 t0 : 7974697275636573 t1 : 0000000000000000 t2 : 3030303270393d6e s0 : ff2000000060bee0 s1 : ffffffff81732028 a0 : 0000000000000000 a1 : ff60000080dd1780 a2 : 0000000000000002 a3 : ffffffff8176a470 a4 : 0000000000000000 a5 : 000000000000000a a6 : 0000000000000081 a7 : ff60000080dd1780 s2 : 0000000000000000 s3 : 0000000000000000 s4 : 0000000000000000 s5 : 0000000000000000 s6 : 0000000000000000 s7 : 0000000000000000 s8 : 0000000000000000 s9 : 0000000000000000 s10: 0000000000000000 s11: 0000000000000000 t3 : ffffffff81186018 t4 : 0000000000000022 t5 : 000000000000003d t6 : 0000000000000000 status: 0000000200000120 badaddr: 0000000000000000 cause: 000000000000000f [<ffffffff80003528>] ret_from_exception+0x0/0x16 Code: 862a d179 608c a517 0069 0513 2be5 d0ef db2e 47a9 (c11c) a517 ---[ end trace 0000000000000000 ]--- Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b SMP: stopping secondary CPUs ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b ]--- Signed-off-by: Björn Töpel <bjorn@rivosinc.com> Link: https://lore.kernel.org/r/20230119074738.708301-2-bjorn@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-01-19 00:47:37 -07:00
}
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
RISC-V: Avoid dereferening NULL regs in die() I don't think we can actually die() without a regs pointer, but the compiler was warning about a NULL check after a dereference. It seems prudent to just avoid the possibly-NULL dereference, given that when die()ing the system is already toast so who knows how we got there. Reported-by: kernel test robot <lkp@intel.com> Reported-by: Dan Carpenter <dan.carpenter@oracle.com> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Link: https://lore.kernel.org/r/20220920200037.6727-1-palmer@rivosinc.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-09-20 13:00:37 -07:00
cause = regs ? regs->cause : -1;
ret = notify_die(DIE_OOPS, str, regs, 0, cause, SIGSEGV);
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
RISC-V: Avoid dereferening NULL regs in die() I don't think we can actually die() without a regs pointer, but the compiler was warning about a NULL check after a dereference. It seems prudent to just avoid the possibly-NULL dereference, given that when die()ing the system is already toast so who knows how we got there. Reported-by: kernel test robot <lkp@intel.com> Reported-by: Dan Carpenter <dan.carpenter@oracle.com> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Link: https://lore.kernel.org/r/20220920200037.6727-1-palmer@rivosinc.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-09-20 13:00:37 -07:00
if (kexec_should_crash(current))
RISC-V: Add fast call path of crash_kexec() Currently, almost all archs (x86, arm64, mips...) support fast call of crash_kexec() when "regs && kexec_should_crash()" is true. But RISC-V not, it can only enter crash system via panic(). However panic() doesn't pass the regs of the real accident scene to crash_kexec(), it caused we can't get accurate backtrace via gdb, $ riscv64-linux-gnu-gdb vmlinux vmcore Reading symbols from vmlinux... [New LWP 95] #0 console_unlock () at kernel/printk/printk.c:2557 2557 if (do_cond_resched) (gdb) bt #0 console_unlock () at kernel/printk/printk.c:2557 #1 0x0000000000000000 in ?? () With the patch we can get the accurate backtrace, $ riscv64-linux-gnu-gdb vmlinux vmcore Reading symbols from vmlinux... [New LWP 95] #0 0xffffffe00063a4e0 in test_thread (data=<optimized out>) at drivers/test_crash.c:81 81 *(int *)p = 0xdead; (gdb) (gdb) bt #0 0xffffffe00064d5c0 in test_thread (data=<optimized out>) at drivers/test_crash.c:81 #1 0x0000000000000000 in ?? () Test code to produce NULL address dereference in test_crash.c, void *p = NULL; *(int *)p = 0xdead; Reviewed-by: Guo Ren <guoren@kernel.org> Tested-by: Xianting Tian <xianting.tian@linux.alibaba.com> Signed-off-by: Xianting Tian <xianting.tian@linux.alibaba.com> Link: https://lore.kernel.org/r/20220606082308.2883458-1-xianting.tian@linux.alibaba.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-06-06 01:23:08 -07:00
crash_kexec(regs);
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
bust_spinlocks(0);
add_taint(TAINT_DIE, LOCKDEP_NOW_UNRELIABLE);
riscv: Avoid enabling interrupts in die() While working on something else, I noticed that the kernel would start accepting interrupts again after crashing in an interrupt handler. Since the kernel is already in inconsistent state, enabling interrupts is dangerous and opens up risk of kernel state deteriorating further. Interrupts do get enabled via what looks like an unintended side effect of spin_unlock_irq, so switch to the more cautious spin_lock_irqsave/spin_unlock_irqrestore instead. Fixes: 76d2a0493a17 ("RISC-V: Init and Halt Code") Signed-off-by: Mattias Nissler <mnissler@rivosinc.com> Reviewed-by: Björn Töpel <bjorn@kernel.org> Link: https://lore.kernel.org/r/20230215144828.3370316-1-mnissler@rivosinc.com Cc: stable@vger.kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-15 07:48:28 -07:00
spin_unlock_irqrestore(&die_lock, flags);
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
oops_exit();
if (in_interrupt())
panic("Fatal exception in interrupt");
if (panic_on_oops)
panic("Fatal exception");
if (ret != NOTIFY_STOP)
exit: Add and use make_task_dead. There are two big uses of do_exit. The first is it's design use to be the guts of the exit(2) system call. The second use is to terminate a task after something catastrophic has happened like a NULL pointer in kernel code. Add a function make_task_dead that is initialy exactly the same as do_exit to cover the cases where do_exit is called to handle catastrophic failure. In time this can probably be reduced to just a light wrapper around do_task_dead. For now keep it exactly the same so that there will be no behavioral differences introducing this new concept. Replace all of the uses of do_exit that use it for catastraphic task cleanup with make_task_dead to make it clear what the code is doing. As part of this rename rewind_stack_do_exit rewind_stack_and_make_dead. Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
2021-06-28 12:52:01 -07:00
make_task_dead(SIGSEGV);
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
}
signal/riscv: Remove tsk parameter from do_trap The do_trap function is always called with tsk == current. Make that obvious by removing the tsk parameter. This also makes it clear that do_trap calls force_sig_fault on the current task. Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
2019-02-05 18:10:48 -07:00
void do_trap(struct pt_regs *regs, int signo, int code, unsigned long addr)
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
{
signal/riscv: Remove tsk parameter from do_trap The do_trap function is always called with tsk == current. Make that obvious by removing the tsk parameter. This also makes it clear that do_trap calls force_sig_fault on the current task. Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
2019-02-05 18:10:48 -07:00
struct task_struct *tsk = current;
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
if (show_unhandled_signals && unhandled_signal(tsk, signo)
&& printk_ratelimit()) {
pr_info("%s[%d]: unhandled signal %d code 0x%x at 0x" REG_FMT,
tsk->comm, task_pid_nr(tsk), signo, code, addr);
riscv: remove duplicate macros from ptrace.h No need to have two names for the same thing. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2019-04-15 02:14:40 -07:00
print_vma_addr(KERN_CONT " in ", instruction_pointer(regs));
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
pr_cont("\n");
riscv: Add dump stack in show_regs Like commit 1149aad10b1e ("arm64: Add dump_backtrace() in show_regs"), dump the stack in riscv show_regs as common code expects. Reviewed-by: Atish Patra <atish.patra@wdc.com> Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-01-11 05:40:12 -07:00
__show_regs(regs);
riscv: use KERN_INFO in do_trap Print the instruction dump with info instead of emergency level. The unhandled signal message is only for informational purpose. Fixes: b8a03a634129 ("riscv: add userland instruction dump to RISC-V splats") Signed-off-by: Andreas Schwab <schwab@suse.de> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Reviewed-by: Atish Patra <atishp@rivosinc.com> Reviewed-by: Yunhui Cui <cuiyunhui@bytedance.com> Link: https://lore.kernel.org/r/mvmy1aegrhm.fsf@suse.de Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2024-03-19 08:40:05 -07:00
dump_instr(KERN_INFO, regs);
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
}
signal: Remove the task parameter from force_sig_fault As synchronous exceptions really only make sense against the current task (otherwise how are you synchronous) remove the task parameter from from force_sig_fault to make it explicit that is what is going on. The two known exceptions that deliver a synchronous exception to a stopped ptraced task have already been changed to force_sig_fault_to_task. The callers have been changed with the following emacs regular expression (with obvious variations on the architectures that take more arguments) to avoid typos: force_sig_fault[(]\([^,]+\)[,]\([^,]+\)[,]\([^,]+\)[,]\W+current[)] -> force_sig_fault(\1,\2,\3) Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
2019-05-23 09:04:24 -07:00
force_sig_fault(signo, code, (void __user *)addr);
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
}
static void do_trap_error(struct pt_regs *regs, int signo, int code,
unsigned long addr, const char *str)
{
riscv: Add uprobes supported This patch adds support for uprobes on riscv architecture. Just like kprobe, it support single-step and simulate instructions. Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Reviewed-by: Pekka Enberg <penberg@kernel.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-12-17 09:01:44 -07:00
current->thread.bad_cause = regs->cause;
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
if (user_mode(regs)) {
signal/riscv: Remove tsk parameter from do_trap The do_trap function is always called with tsk == current. Make that obvious by removing the tsk parameter. This also makes it clear that do_trap calls force_sig_fault on the current task. Signed-off-by: "Eric W. Biederman" <ebiederm@xmission.com>
2019-02-05 18:10:48 -07:00
do_trap(regs, signo, code, addr);
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
} else {
if (!fixup_exception(regs))
die(regs, str);
}
}
riscv: integrate alternatives better into the main architecture Right now the alternatives need to be explicitly enabled and erratas are limited to SiFive ones. We want to use alternatives not only for patching soc erratas, but in the future also for handling different behaviour depending on the existence of future extensions. So move the core alternatives over to the kernel subdirectory and move the CONFIG_RISCV_ALTERNATIVE to be a hidden symbol which we expect relevant erratas and extensions to just select if needed. Signed-off-by: Heiko Stuebner <heiko@sntech.de> Reviewed-by: Philipp Tomsich <philipp.tomsich@vrull.eu> Link: https://lore.kernel.org/r/20220511192921.2223629-2-heiko@sntech.de Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2022-05-11 12:29:10 -07:00
#if defined(CONFIG_XIP_KERNEL) && defined(CONFIG_RISCV_ALTERNATIVE)
riscv: entry: Add noinstr to prevent instrumentation inserted Without noinstr the compiler is free to insert instrumentation (think all the k*SAN, KCov, GCov, ftrace etc..) which can call code we're not yet ready to run this early in the entry path, for instance it could rely on RCU which isn't on yet, or expect lockdep state. (by peterz) Link: https://lore.kernel.org/linux-riscv/YxcQ6NoPf3AH0EXe@hirez.programming.kicks-ass.net/ Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Suggested-by: Peter Zijlstra <peterz@infradead.org> Tested-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Link: https://lore.kernel.org/r/20230222033021.983168-4-guoren@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-21 20:30:17 -07:00
#define __trap_section __noinstr_section(".xip.traps")
riscv: xip: support runtime trap patching RISCV_ERRATA_ALTERNATIVE patches text at runtime which is currently not possible when the kernel is executed from the flash in XIP mode. Since runtime patching concerns only traps at the moment, let's just have all the traps reside in RAM anyway if RISCV_ERRATA_ALTERNATIVE is set. Thus, these functions will be patch-able even when the .text section is in flash. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-05-31 01:53:42 -07:00
#else
riscv: entry: Add noinstr to prevent instrumentation inserted Without noinstr the compiler is free to insert instrumentation (think all the k*SAN, KCov, GCov, ftrace etc..) which can call code we're not yet ready to run this early in the entry path, for instance it could rely on RCU which isn't on yet, or expect lockdep state. (by peterz) Link: https://lore.kernel.org/linux-riscv/YxcQ6NoPf3AH0EXe@hirez.programming.kicks-ass.net/ Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Suggested-by: Peter Zijlstra <peterz@infradead.org> Tested-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Link: https://lore.kernel.org/r/20230222033021.983168-4-guoren@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-21 20:30:17 -07:00
#define __trap_section noinstr
riscv: xip: support runtime trap patching RISCV_ERRATA_ALTERNATIVE patches text at runtime which is currently not possible when the kernel is executed from the flash in XIP mode. Since runtime patching concerns only traps at the moment, let's just have all the traps reside in RAM anyway if RISCV_ERRATA_ALTERNATIVE is set. Thus, these functions will be patch-able even when the .text section is in flash. Signed-off-by: Vitaly Wool <vitaly.wool@konsulko.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-05-31 01:53:42 -07:00
#endif
riscv: entry: Convert to generic entry This patch converts riscv to use the generic entry infrastructure from kernel/entry/*. The generic entry makes maintainers' work easier and codes more elegant. Here are the changes: - More clear entry.S with handle_exception and ret_from_exception - Get rid of complex custom signal implementation - Move syscall procedure from assembly to C, which is much more readable. - Connect ret_from_fork & ret_from_kernel_thread to generic entry. - Wrap with irqentry_enter/exit and syscall_enter/exit_from_user_mode - Use the standard preemption code instead of custom Suggested-by: Huacai Chen <chenhuacai@kernel.org> Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Tested-by: Yipeng Zou <zouyipeng@huawei.com> Tested-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Cc: Ben Hutchings <ben@decadent.org.uk> Link: https://lore.kernel.org/r/20230222033021.983168-5-guoren@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-21 20:30:18 -07:00
#define DO_ERROR_INFO(name, signo, code, str) \
asmlinkage __visible __trap_section void name(struct pt_regs *regs) \
{ \
if (user_mode(regs)) { \
irqentry_enter_from_user_mode(regs); \
do_trap_error(regs, signo, code, regs->epc, "Oops - " str); \
irqentry_exit_to_user_mode(regs); \
} else { \
irqentry_state_t state = irqentry_nmi_enter(regs); \
do_trap_error(regs, signo, code, regs->epc, "Oops - " str); \
irqentry_nmi_exit(regs, state); \
} \
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
}
DO_ERROR_INFO(do_trap_unknown,
SIGILL, ILL_ILLTRP, "unknown exception");
DO_ERROR_INFO(do_trap_insn_misaligned,
SIGBUS, BUS_ADRALN, "instruction address misaligned");
DO_ERROR_INFO(do_trap_insn_fault,
SIGSEGV, SEGV_ACCERR, "instruction access fault");
riscv: Allocate user's vector context in the first-use trap Vector unit is disabled by default for all user processes. Thus, a process will take a trap (illegal instruction) into kernel at the first time when it uses Vector. Only after then, the kernel allocates V context and starts take care of the context for that user process. Suggested-by: Richard Henderson <richard.henderson@linaro.org> Link: https://lore.kernel.org/r/3923eeee-e4dc-0911-40bf-84c34aee962d@linaro.org Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Link: https://lore.kernel.org/r/20230605110724.21391-12-andy.chiu@sifive.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-06-05 04:07:08 -07:00
asmlinkage __visible __trap_section void do_trap_insn_illegal(struct pt_regs *regs)
{
riscv: vector: only enable interrupts in the first-use trap The function irqentry_exit_to_user_mode() must be called with interrupt disabled. The caller of do_trap_insn_illegal() also assumes running without interrupts. So, we should turn off interrupts after riscv_v_first_use_handler() returns. Fixes: cd054837243b ("riscv: Allocate user's vector context in the first-use trap") Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Link: https://lore.kernel.org/r/20230625155416.18629-1-andy.chiu@sifive.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-06-25 08:54:15 -07:00
bool handled;
riscv: Allocate user's vector context in the first-use trap Vector unit is disabled by default for all user processes. Thus, a process will take a trap (illegal instruction) into kernel at the first time when it uses Vector. Only after then, the kernel allocates V context and starts take care of the context for that user process. Suggested-by: Richard Henderson <richard.henderson@linaro.org> Link: https://lore.kernel.org/r/3923eeee-e4dc-0911-40bf-84c34aee962d@linaro.org Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Link: https://lore.kernel.org/r/20230605110724.21391-12-andy.chiu@sifive.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-06-05 04:07:08 -07:00
if (user_mode(regs)) {
irqentry_enter_from_user_mode(regs);
local_irq_enable();
riscv: vector: only enable interrupts in the first-use trap The function irqentry_exit_to_user_mode() must be called with interrupt disabled. The caller of do_trap_insn_illegal() also assumes running without interrupts. So, we should turn off interrupts after riscv_v_first_use_handler() returns. Fixes: cd054837243b ("riscv: Allocate user's vector context in the first-use trap") Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Link: https://lore.kernel.org/r/20230625155416.18629-1-andy.chiu@sifive.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-06-25 08:54:15 -07:00
handled = riscv_v_first_use_handler(regs);
local_irq_disable();
if (!handled)
riscv: Allocate user's vector context in the first-use trap Vector unit is disabled by default for all user processes. Thus, a process will take a trap (illegal instruction) into kernel at the first time when it uses Vector. Only after then, the kernel allocates V context and starts take care of the context for that user process. Suggested-by: Richard Henderson <richard.henderson@linaro.org> Link: https://lore.kernel.org/r/3923eeee-e4dc-0911-40bf-84c34aee962d@linaro.org Signed-off-by: Andy Chiu <andy.chiu@sifive.com> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Link: https://lore.kernel.org/r/20230605110724.21391-12-andy.chiu@sifive.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-06-05 04:07:08 -07:00
do_trap_error(regs, SIGILL, ILL_ILLOPC, regs->epc,
"Oops - illegal instruction");
irqentry_exit_to_user_mode(regs);
} else {
irqentry_state_t state = irqentry_nmi_enter(regs);
do_trap_error(regs, SIGILL, ILL_ILLOPC, regs->epc,
"Oops - illegal instruction");
irqentry_nmi_exit(regs, state);
}
}
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
DO_ERROR_INFO(do_trap_load_fault,
SIGSEGV, SEGV_ACCERR, "load access fault");
riscv: Unaligned load/store handling for M_MODE Add handlers for unaligned load and store traps that may be generated by applications. Code heavily inspired from the OpenSBI project. Handling of the unaligned access traps is suitable for applications compiled with or without compressed instructions and is independent of the kernel CONFIG_RISCV_ISA_C option value. Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Signed-off-by: Anup Patel <anup.patel@wdc.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-15 17:47:36 -07:00
riscv: entry: Convert to generic entry This patch converts riscv to use the generic entry infrastructure from kernel/entry/*. The generic entry makes maintainers' work easier and codes more elegant. Here are the changes: - More clear entry.S with handle_exception and ret_from_exception - Get rid of complex custom signal implementation - Move syscall procedure from assembly to C, which is much more readable. - Connect ret_from_fork & ret_from_kernel_thread to generic entry. - Wrap with irqentry_enter/exit and syscall_enter/exit_from_user_mode - Use the standard preemption code instead of custom Suggested-by: Huacai Chen <chenhuacai@kernel.org> Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Tested-by: Yipeng Zou <zouyipeng@huawei.com> Tested-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Cc: Ben Hutchings <ben@decadent.org.uk> Link: https://lore.kernel.org/r/20230222033021.983168-5-guoren@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-21 20:30:18 -07:00
asmlinkage __visible __trap_section void do_trap_load_misaligned(struct pt_regs *regs)
riscv: Unaligned load/store handling for M_MODE Add handlers for unaligned load and store traps that may be generated by applications. Code heavily inspired from the OpenSBI project. Handling of the unaligned access traps is suitable for applications compiled with or without compressed instructions and is independent of the kernel CONFIG_RISCV_ISA_C option value. Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Signed-off-by: Anup Patel <anup.patel@wdc.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-15 17:47:36 -07:00
{
riscv: entry: Convert to generic entry This patch converts riscv to use the generic entry infrastructure from kernel/entry/*. The generic entry makes maintainers' work easier and codes more elegant. Here are the changes: - More clear entry.S with handle_exception and ret_from_exception - Get rid of complex custom signal implementation - Move syscall procedure from assembly to C, which is much more readable. - Connect ret_from_fork & ret_from_kernel_thread to generic entry. - Wrap with irqentry_enter/exit and syscall_enter/exit_from_user_mode - Use the standard preemption code instead of custom Suggested-by: Huacai Chen <chenhuacai@kernel.org> Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Tested-by: Yipeng Zou <zouyipeng@huawei.com> Tested-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Cc: Ben Hutchings <ben@decadent.org.uk> Link: https://lore.kernel.org/r/20230222033021.983168-5-guoren@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-21 20:30:18 -07:00
if (user_mode(regs)) {
irqentry_enter_from_user_mode(regs);
if (handle_misaligned_load(regs))
do_trap_error(regs, SIGBUS, BUS_ADRALN, regs->epc,
"Oops - load address misaligned");
irqentry_exit_to_user_mode(regs);
} else {
irqentry_state_t state = irqentry_nmi_enter(regs);
if (handle_misaligned_load(regs))
do_trap_error(regs, SIGBUS, BUS_ADRALN, regs->epc,
"Oops - load address misaligned");
irqentry_nmi_exit(regs, state);
}
riscv: Unaligned load/store handling for M_MODE Add handlers for unaligned load and store traps that may be generated by applications. Code heavily inspired from the OpenSBI project. Handling of the unaligned access traps is suitable for applications compiled with or without compressed instructions and is independent of the kernel CONFIG_RISCV_ISA_C option value. Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Signed-off-by: Anup Patel <anup.patel@wdc.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-15 17:47:36 -07:00
}
riscv: entry: Convert to generic entry This patch converts riscv to use the generic entry infrastructure from kernel/entry/*. The generic entry makes maintainers' work easier and codes more elegant. Here are the changes: - More clear entry.S with handle_exception and ret_from_exception - Get rid of complex custom signal implementation - Move syscall procedure from assembly to C, which is much more readable. - Connect ret_from_fork & ret_from_kernel_thread to generic entry. - Wrap with irqentry_enter/exit and syscall_enter/exit_from_user_mode - Use the standard preemption code instead of custom Suggested-by: Huacai Chen <chenhuacai@kernel.org> Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Tested-by: Yipeng Zou <zouyipeng@huawei.com> Tested-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Cc: Ben Hutchings <ben@decadent.org.uk> Link: https://lore.kernel.org/r/20230222033021.983168-5-guoren@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-21 20:30:18 -07:00
asmlinkage __visible __trap_section void do_trap_store_misaligned(struct pt_regs *regs)
riscv: Unaligned load/store handling for M_MODE Add handlers for unaligned load and store traps that may be generated by applications. Code heavily inspired from the OpenSBI project. Handling of the unaligned access traps is suitable for applications compiled with or without compressed instructions and is independent of the kernel CONFIG_RISCV_ISA_C option value. Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Signed-off-by: Anup Patel <anup.patel@wdc.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-15 17:47:36 -07:00
{
riscv: entry: Convert to generic entry This patch converts riscv to use the generic entry infrastructure from kernel/entry/*. The generic entry makes maintainers' work easier and codes more elegant. Here are the changes: - More clear entry.S with handle_exception and ret_from_exception - Get rid of complex custom signal implementation - Move syscall procedure from assembly to C, which is much more readable. - Connect ret_from_fork & ret_from_kernel_thread to generic entry. - Wrap with irqentry_enter/exit and syscall_enter/exit_from_user_mode - Use the standard preemption code instead of custom Suggested-by: Huacai Chen <chenhuacai@kernel.org> Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Tested-by: Yipeng Zou <zouyipeng@huawei.com> Tested-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Cc: Ben Hutchings <ben@decadent.org.uk> Link: https://lore.kernel.org/r/20230222033021.983168-5-guoren@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-21 20:30:18 -07:00
if (user_mode(regs)) {
irqentry_enter_from_user_mode(regs);
if (handle_misaligned_store(regs))
do_trap_error(regs, SIGBUS, BUS_ADRALN, regs->epc,
"Oops - store (or AMO) address misaligned");
irqentry_exit_to_user_mode(regs);
} else {
irqentry_state_t state = irqentry_nmi_enter(regs);
if (handle_misaligned_store(regs))
do_trap_error(regs, SIGBUS, BUS_ADRALN, regs->epc,
"Oops - store (or AMO) address misaligned");
irqentry_nmi_exit(regs, state);
}
riscv: Unaligned load/store handling for M_MODE Add handlers for unaligned load and store traps that may be generated by applications. Code heavily inspired from the OpenSBI project. Handling of the unaligned access traps is suitable for applications compiled with or without compressed instructions and is independent of the kernel CONFIG_RISCV_ISA_C option value. Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com> Signed-off-by: Anup Patel <anup.patel@wdc.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-15 17:47:36 -07:00
}
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
DO_ERROR_INFO(do_trap_store_fault,
SIGSEGV, SEGV_ACCERR, "store (or AMO) access fault");
DO_ERROR_INFO(do_trap_ecall_s,
SIGILL, ILL_ILLTRP, "environment call from S-mode");
DO_ERROR_INFO(do_trap_ecall_m,
SIGILL, ILL_ILLTRP, "environment call from M-mode");
riscv: Add the support for c.ebreak check in is_valid_bugaddr() The macro __BUG_INSN currently is defined as the "ebreak" opcode. The is_valid_bugaddr() function compares the instruction pointed to by $sepc with macro __BUG_INSN to check whether the current trap exception is caused by an "ebreak" instruction. However, this check flow is possibly erroneous because if C extension is supported, the expected trap instruction "ebreak" is possibly translated to "c.ebreak" by the assembler. Therefore, it requires a mechanism to distinguish the length of the instruction in $spec and compare it to the correct trap instruction. Signed-off-by: Vincent Chen <vincentc@andestech.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2019-03-04 20:23:34 -07:00
static inline unsigned long get_break_insn_length(unsigned long pc)
{
bug_insn_t insn;
maccess: rename probe_kernel_address to get_kernel_nofault Better describe what this helper does, and match the naming of copy_from_kernel_nofault. Also switch the argument order around, so that it acts and looks like get_user(). Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2020-06-17 00:37:55 -07:00
if (get_kernel_nofault(insn, (bug_insn_t *)pc))
riscv: Add the support for c.ebreak check in is_valid_bugaddr() The macro __BUG_INSN currently is defined as the "ebreak" opcode. The is_valid_bugaddr() function compares the instruction pointed to by $sepc with macro __BUG_INSN to check whether the current trap exception is caused by an "ebreak" instruction. However, this check flow is possibly erroneous because if C extension is supported, the expected trap instruction "ebreak" is possibly translated to "c.ebreak" by the assembler. Therefore, it requires a mechanism to distinguish the length of the instruction in $spec and compare it to the correct trap instruction. Signed-off-by: Vincent Chen <vincentc@andestech.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2019-03-04 20:23:34 -07:00
return 0;
riscv: add macro to get instruction length Extract the calculation of instruction length for common use. Signed-off-by: Zong Li <zong.li@sifive.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-03-09 09:55:42 -07:00
return GET_INSN_LENGTH(insn);
riscv: Add the support for c.ebreak check in is_valid_bugaddr() The macro __BUG_INSN currently is defined as the "ebreak" opcode. The is_valid_bugaddr() function compares the instruction pointed to by $sepc with macro __BUG_INSN to check whether the current trap exception is caused by an "ebreak" instruction. However, this check flow is possibly erroneous because if C extension is supported, the expected trap instruction "ebreak" is possibly translated to "c.ebreak" by the assembler. Therefore, it requires a mechanism to distinguish the length of the instruction in $spec and compare it to the correct trap instruction. Signed-off-by: Vincent Chen <vincentc@andestech.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2019-03-04 20:23:34 -07:00
}
riscv: Only consider swbp/ss handlers for correct privileged mode RISC-V software breakpoint trap handlers are used for {k,u}probes. When trapping from kernelmode, only the kernelmode handlers should be considered. Vice versa, only usermode handlers for usermode traps. This is not the case on RISC-V, which can trigger a bug if a userspace process uses uprobes, and a WARN() is triggered from kernelmode (which is implemented via {c.,}ebreak). The kernel will trap on the kernelmode {c.,}ebreak, look for uprobes handlers, realize incorrectly that uprobes need to be handled, and exit the trap handler early. The trap returns to re-executing the {c.,}ebreak, and enter an infinite trap-loop. The issue was found running the BPF selftest [1]. Fix this issue by only considering the swbp/ss handlers for kernel/usermode respectively. Also, move CONFIG ifdeffery from traps.c to the asm/{k,u}probes.h headers. Note that linux/uprobes.h only include asm/uprobes.h if CONFIG_UPROBES is defined, which is why asm/uprobes.h needs to be unconditionally included in traps.c Link: https://lore.kernel.org/linux-riscv/87v8d19aun.fsf@all.your.base.are.belong.to.us/ # [1] Fixes: 74784081aac8 ("riscv: Add uprobes supported") Reviewed-by: Guo Ren <guoren@kernel.org> Reviewed-by: Nam Cao <namcaov@gmail.com> Tested-by: Puranjay Mohan <puranjay12@gmail.com> Signed-off-by: Björn Töpel <bjorn@rivosinc.com> Link: https://lore.kernel.org/r/20230912065619.62020-1-bjorn@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-09-11 23:56:19 -07:00
static bool probe_single_step_handler(struct pt_regs *regs)
{
bool user = user_mode(regs);
return user ? uprobe_single_step_handler(regs) : kprobe_single_step_handler(regs);
}
static bool probe_breakpoint_handler(struct pt_regs *regs)
{
bool user = user_mode(regs);
return user ? uprobe_breakpoint_handler(regs) : kprobe_breakpoint_handler(regs);
}
riscv: entry: Convert to generic entry This patch converts riscv to use the generic entry infrastructure from kernel/entry/*. The generic entry makes maintainers' work easier and codes more elegant. Here are the changes: - More clear entry.S with handle_exception and ret_from_exception - Get rid of complex custom signal implementation - Move syscall procedure from assembly to C, which is much more readable. - Connect ret_from_fork & ret_from_kernel_thread to generic entry. - Wrap with irqentry_enter/exit and syscall_enter/exit_from_user_mode - Use the standard preemption code instead of custom Suggested-by: Huacai Chen <chenhuacai@kernel.org> Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Tested-by: Yipeng Zou <zouyipeng@huawei.com> Tested-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Cc: Ben Hutchings <ben@decadent.org.uk> Link: https://lore.kernel.org/r/20230222033021.983168-5-guoren@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-21 20:30:18 -07:00
void handle_break(struct pt_regs *regs)
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
{
riscv: Only consider swbp/ss handlers for correct privileged mode RISC-V software breakpoint trap handlers are used for {k,u}probes. When trapping from kernelmode, only the kernelmode handlers should be considered. Vice versa, only usermode handlers for usermode traps. This is not the case on RISC-V, which can trigger a bug if a userspace process uses uprobes, and a WARN() is triggered from kernelmode (which is implemented via {c.,}ebreak). The kernel will trap on the kernelmode {c.,}ebreak, look for uprobes handlers, realize incorrectly that uprobes need to be handled, and exit the trap handler early. The trap returns to re-executing the {c.,}ebreak, and enter an infinite trap-loop. The issue was found running the BPF selftest [1]. Fix this issue by only considering the swbp/ss handlers for kernel/usermode respectively. Also, move CONFIG ifdeffery from traps.c to the asm/{k,u}probes.h headers. Note that linux/uprobes.h only include asm/uprobes.h if CONFIG_UPROBES is defined, which is why asm/uprobes.h needs to be unconditionally included in traps.c Link: https://lore.kernel.org/linux-riscv/87v8d19aun.fsf@all.your.base.are.belong.to.us/ # [1] Fixes: 74784081aac8 ("riscv: Add uprobes supported") Reviewed-by: Guo Ren <guoren@kernel.org> Reviewed-by: Nam Cao <namcaov@gmail.com> Tested-by: Puranjay Mohan <puranjay12@gmail.com> Signed-off-by: Björn Töpel <bjorn@rivosinc.com> Link: https://lore.kernel.org/r/20230912065619.62020-1-bjorn@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-09-11 23:56:19 -07:00
if (probe_single_step_handler(regs))
riscv: Add kprobes supported This patch enables "kprobe & kretprobe" to work with ftrace interface. It utilized software breakpoint as single-step mechanism. Some instructions which can't be single-step executed must be simulated in kernel execution slot, such as: branch, jal, auipc, la ... Some instructions should be rejected for probing and we use a blacklist to filter, such as: ecall, ebreak, ... We use ebreak & c.ebreak to replace origin instruction and the kprobe handler prepares an executable memory slot for out-of-line execution with a copy of the original instruction being probed. In execution slot we add ebreak behind original instruction to simulate a single-setp mechanism. The patch is based on packi's work [1] and csky's work [2]. - The kprobes_trampoline.S is all from packi's patch - The single-step mechanism is new designed for riscv without hw single-step trap - The simulation codes are from csky - Frankly, all codes refer to other archs' implementation [1] https://lore.kernel.org/linux-riscv/20181113195804.22825-1-me@packi.ch/ [2] https://lore.kernel.org/linux-csky/20200403044150.20562-9-guoren@kernel.org/ Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Co-developed-by: Patrick Stählin <me@packi.ch> Signed-off-by: Patrick Stählin <me@packi.ch> Acked-by: Masami Hiramatsu <mhiramat@kernel.org> Tested-by: Zong Li <zong.li@sifive.com> Reviewed-by: Pekka Enberg <penberg@kernel.org> Cc: Patrick Stählin <me@packi.ch> Cc: Palmer Dabbelt <palmerdabbelt@google.com> Cc: Björn Töpel <bjorn.topel@gmail.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-12-17 09:01:42 -07:00
return;
riscv: Only consider swbp/ss handlers for correct privileged mode RISC-V software breakpoint trap handlers are used for {k,u}probes. When trapping from kernelmode, only the kernelmode handlers should be considered. Vice versa, only usermode handlers for usermode traps. This is not the case on RISC-V, which can trigger a bug if a userspace process uses uprobes, and a WARN() is triggered from kernelmode (which is implemented via {c.,}ebreak). The kernel will trap on the kernelmode {c.,}ebreak, look for uprobes handlers, realize incorrectly that uprobes need to be handled, and exit the trap handler early. The trap returns to re-executing the {c.,}ebreak, and enter an infinite trap-loop. The issue was found running the BPF selftest [1]. Fix this issue by only considering the swbp/ss handlers for kernel/usermode respectively. Also, move CONFIG ifdeffery from traps.c to the asm/{k,u}probes.h headers. Note that linux/uprobes.h only include asm/uprobes.h if CONFIG_UPROBES is defined, which is why asm/uprobes.h needs to be unconditionally included in traps.c Link: https://lore.kernel.org/linux-riscv/87v8d19aun.fsf@all.your.base.are.belong.to.us/ # [1] Fixes: 74784081aac8 ("riscv: Add uprobes supported") Reviewed-by: Guo Ren <guoren@kernel.org> Reviewed-by: Nam Cao <namcaov@gmail.com> Tested-by: Puranjay Mohan <puranjay12@gmail.com> Signed-off-by: Björn Töpel <bjorn@rivosinc.com> Link: https://lore.kernel.org/r/20230912065619.62020-1-bjorn@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-09-11 23:56:19 -07:00
if (probe_breakpoint_handler(regs))
riscv: Add uprobes supported This patch adds support for uprobes on riscv architecture. Just like kprobe, it support single-step and simulate instructions. Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Reviewed-by: Pekka Enberg <penberg@kernel.org> Cc: Oleg Nesterov <oleg@redhat.com> Cc: Masami Hiramatsu <mhiramat@kernel.org> Cc: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-12-17 09:01:44 -07:00
return;
current->thread.bad_cause = regs->cause;
riscv: Add kprobes supported This patch enables "kprobe & kretprobe" to work with ftrace interface. It utilized software breakpoint as single-step mechanism. Some instructions which can't be single-step executed must be simulated in kernel execution slot, such as: branch, jal, auipc, la ... Some instructions should be rejected for probing and we use a blacklist to filter, such as: ecall, ebreak, ... We use ebreak & c.ebreak to replace origin instruction and the kprobe handler prepares an executable memory slot for out-of-line execution with a copy of the original instruction being probed. In execution slot we add ebreak behind original instruction to simulate a single-setp mechanism. The patch is based on packi's work [1] and csky's work [2]. - The kprobes_trampoline.S is all from packi's patch - The single-step mechanism is new designed for riscv without hw single-step trap - The simulation codes are from csky - Frankly, all codes refer to other archs' implementation [1] https://lore.kernel.org/linux-riscv/20181113195804.22825-1-me@packi.ch/ [2] https://lore.kernel.org/linux-csky/20200403044150.20562-9-guoren@kernel.org/ Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Co-developed-by: Patrick Stählin <me@packi.ch> Signed-off-by: Patrick Stählin <me@packi.ch> Acked-by: Masami Hiramatsu <mhiramat@kernel.org> Tested-by: Zong Li <zong.li@sifive.com> Reviewed-by: Pekka Enberg <penberg@kernel.org> Cc: Patrick Stählin <me@packi.ch> Cc: Palmer Dabbelt <palmerdabbelt@google.com> Cc: Björn Töpel <bjorn.topel@gmail.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-12-17 09:01:42 -07:00
riscv: cleanup do_trap_break If we always compile the get_break_insn_length inline function we can remove the ifdefs and let dead code elimination take care of the warn branch that is now unreadable because the report_bug stub always returns BUG_TRAP_TYPE_BUG. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-17 10:37:30 -07:00
if (user_mode(regs))
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 05:10:32 -07:00
force_sig_fault(SIGTRAP, TRAP_BRKPT, (void __user *)regs->epc);
riscv: Add KGDB support The skeleton of RISC-V KGDB port. Signed-off-by: Vincent Chen <vincent.chen@sifive.com> Reviewed-by: Palmer Dabbelt <palmerdabbelt@google.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2020-04-15 19:38:05 -07:00
#ifdef CONFIG_KGDB
else if (notify_die(DIE_TRAP, "EBREAK", regs, 0, regs->cause, SIGTRAP)
== NOTIFY_STOP)
return;
#endif
riscv: Add CFI error handling With CONFIG_CFI_CLANG, the compiler injects a type preamble immediately before each function and a check to validate the target function type before indirect calls: ; type preamble .word <id> function: ... ; indirect call check lw t1, -4(a0) lui t2, <hi20> addiw t2, t2, <lo12> beq t1, t2, .Ltmp0 ebreak .Ltmp0: jarl a0 Implement error handling code for the ebreak traps emitted for the checks. This produces the following oops on a CFI failure (generated using lkdtm): [ 21.177245] CFI failure at lkdtm_indirect_call+0x22/0x32 [lkdtm] (target: lkdtm_increment_int+0x0/0x18 [lkdtm]; expected type: 0x3ad55aca) [ 21.178483] Kernel BUG [#1] [ 21.178671] Modules linked in: lkdtm [ 21.179037] CPU: 1 PID: 104 Comm: sh Not tainted 6.3.0-rc6-00037-g37d5ec6297ab #1 [ 21.179511] Hardware name: riscv-virtio,qemu (DT) [ 21.179818] epc : lkdtm_indirect_call+0x22/0x32 [lkdtm] [ 21.180106] ra : lkdtm_CFI_FORWARD_PROTO+0x48/0x7c [lkdtm] [ 21.180426] epc : ffffffff01387092 ra : ffffffff01386f14 sp : ff20000000453cf0 [ 21.180792] gp : ffffffff81308c38 tp : ff6000000243f080 t0 : ff20000000453b78 [ 21.181157] t1 : 000000003ad55aca t2 : 000000007e0c52a5 s0 : ff20000000453d00 [ 21.181506] s1 : 0000000000000001 a0 : ffffffff0138d170 a1 : ffffffff013870bc [ 21.181819] a2 : b5fea48dd89aa700 a3 : 0000000000000001 a4 : 0000000000000fff [ 21.182169] a5 : 0000000000000004 a6 : 00000000000000b7 a7 : 0000000000000000 [ 21.182591] s2 : ff20000000453e78 s3 : ffffffffffffffea s4 : 0000000000000012 [ 21.183001] s5 : ff600000023c7000 s6 : 0000000000000006 s7 : ffffffff013882a0 [ 21.183653] s8 : 0000000000000008 s9 : 0000000000000002 s10: ffffffff0138d878 [ 21.184245] s11: ffffffff0138d878 t3 : 0000000000000003 t4 : 0000000000000000 [ 21.184591] t5 : ffffffff8133df08 t6 : ffffffff8133df07 [ 21.184858] status: 0000000000000120 badaddr: 0000000000000000 cause: 0000000000000003 [ 21.185415] [<ffffffff01387092>] lkdtm_indirect_call+0x22/0x32 [lkdtm] [ 21.185772] [<ffffffff01386f14>] lkdtm_CFI_FORWARD_PROTO+0x48/0x7c [lkdtm] [ 21.186093] [<ffffffff01383552>] lkdtm_do_action+0x22/0x34 [lkdtm] [ 21.186445] [<ffffffff0138350c>] direct_entry+0x128/0x13a [lkdtm] [ 21.186817] [<ffffffff8033ed8c>] full_proxy_write+0x58/0xb2 [ 21.187352] [<ffffffff801d4fe8>] vfs_write+0x14c/0x33a [ 21.187644] [<ffffffff801d5328>] ksys_write+0x64/0xd4 [ 21.187832] [<ffffffff801d53a6>] sys_write+0xe/0x1a [ 21.188171] [<ffffffff80003996>] ret_from_syscall+0x0/0x2 [ 21.188595] Code: 0513 0f65 a303 ffc5 53b7 7e0c 839b 2a53 0363 0073 (9002) 9582 [ 21.189178] ---[ end trace 0000000000000000 ]--- [ 21.189590] Kernel panic - not syncing: Fatal exception Reviewed-by: Kees Cook <keescook@chromium.org> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> # ISA bits Tested-by: Nathan Chancellor <nathan@kernel.org> Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Link: https://lore.kernel.org/r/20230710183544.999540-12-samitolvanen@google.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-07-10 11:35:49 -07:00
else if (report_bug(regs->epc, regs) == BUG_TRAP_TYPE_WARN ||
handle_cfi_failure(regs) == BUG_TRAP_TYPE_WARN)
riscv: abstract out CSR names for supervisor vs machine mode Many of the privileged CSRs exist in a supervisor and machine version that are used very similarly. Provide versions of the CSR names and fields that map to either the S-mode or M-mode variant depending on a new CONFIG_RISCV_M_MODE kconfig symbol. Contains contributions from Damien Le Moal <Damien.LeMoal@wdc.com> and Paul Walmsley <paul.walmsley@sifive.com>. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Gleixner <tglx@linutronix.de> # for drivers/clocksource, drivers/irqchip [paul.walmsley@sifive.com: updated to apply] Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-28 05:10:32 -07:00
regs->epc += get_break_insn_length(regs->epc);
riscv: cleanup do_trap_break If we always compile the get_break_insn_length inline function we can remove the ifdefs and let dead code elimination take care of the warn branch that is now unreadable because the report_bug stub always returns BUG_TRAP_TYPE_BUG. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Anup Patel <anup@brainfault.org> Signed-off-by: Paul Walmsley <paul.walmsley@sifive.com>
2019-10-17 10:37:30 -07:00
else
die(regs, "Kernel BUG");
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
}
riscv: entry: Convert to generic entry This patch converts riscv to use the generic entry infrastructure from kernel/entry/*. The generic entry makes maintainers' work easier and codes more elegant. Here are the changes: - More clear entry.S with handle_exception and ret_from_exception - Get rid of complex custom signal implementation - Move syscall procedure from assembly to C, which is much more readable. - Connect ret_from_fork & ret_from_kernel_thread to generic entry. - Wrap with irqentry_enter/exit and syscall_enter/exit_from_user_mode - Use the standard preemption code instead of custom Suggested-by: Huacai Chen <chenhuacai@kernel.org> Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Tested-by: Yipeng Zou <zouyipeng@huawei.com> Tested-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Cc: Ben Hutchings <ben@decadent.org.uk> Link: https://lore.kernel.org/r/20230222033021.983168-5-guoren@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-21 20:30:18 -07:00
asmlinkage __visible __trap_section void do_trap_break(struct pt_regs *regs)
{
if (user_mode(regs)) {
irqentry_enter_from_user_mode(regs);
handle_break(regs);
irqentry_exit_to_user_mode(regs);
} else {
irqentry_state_t state = irqentry_nmi_enter(regs);
handle_break(regs);
irqentry_nmi_exit(regs, state);
}
}
riscv: Support RANDOMIZE_KSTACK_OFFSET Inspired from arm64's implement -- commit 70918779aec9 ("arm64: entry: Enable random_kstack_offset support") Add support of kernel stack offset randomization while handling syscall, the offset is defaultly limited by KSTACK_OFFSET_MAX() (i.e. 10 bits). In order to avoid trigger stack canaries (due to __builtin_alloca) and slowing down the entry path, use __no_stack_protector attribute to disable stack protector for do_trap_ecall_u() at the function level. Acked-by: Palmer Dabbelt <palmer@rivosinc.com> Reviewed-by: Kees Cook <keescook@chromium.org> Signed-off-by: Song Shuai <songshuaishuai@tinylab.org> Link: https://lore.kernel.org/r/20231109133751.212079-1-songshuaishuai@tinylab.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-11-09 06:37:51 -07:00
asmlinkage __visible __trap_section __no_stack_protector
void do_trap_ecall_u(struct pt_regs *regs)
riscv: entry: Convert to generic entry This patch converts riscv to use the generic entry infrastructure from kernel/entry/*. The generic entry makes maintainers' work easier and codes more elegant. Here are the changes: - More clear entry.S with handle_exception and ret_from_exception - Get rid of complex custom signal implementation - Move syscall procedure from assembly to C, which is much more readable. - Connect ret_from_fork & ret_from_kernel_thread to generic entry. - Wrap with irqentry_enter/exit and syscall_enter/exit_from_user_mode - Use the standard preemption code instead of custom Suggested-by: Huacai Chen <chenhuacai@kernel.org> Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Tested-by: Yipeng Zou <zouyipeng@huawei.com> Tested-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Cc: Ben Hutchings <ben@decadent.org.uk> Link: https://lore.kernel.org/r/20230222033021.983168-5-guoren@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-21 20:30:18 -07:00
{
if (user_mode(regs)) {
riscv: entry: set a0 = -ENOSYS only when syscall != -1 When we test seccomp with 6.4 kernel, we found errno has wrong value. If we deny NETLINK_AUDIT with EAFNOSUPPORT, after f0bddf50586d, we will get ENOSYS instead. We got same result with commit 9c2598d43510 ("riscv: entry: Save a0 prior syscall_enter_from_user_mode()"). After analysing code, we think that regs->a0 = -ENOSYS should only be executed when syscall != -1. In __seccomp_filter, when seccomp rejected this syscall with specified errno, they will set a0 to return number as syscall ABI, and then return -1. This return number is finally pass as return number of syscall_enter_from_user_mode, and then is compared with NR_syscalls after converted to ulong (so it will be ULONG_MAX). The condition syscall < NR_syscalls will always be false, so regs->a0 = -ENOSYS is always executed. It covered a0 set by seccomp, so we always get ENOSYS when match seccomp RET_ERRNO rule. Fixes: f0bddf50586d ("riscv: entry: Convert to generic entry") Reported-by: Felix Yan <felixonmars@archlinux.org> Co-developed-by: Ruizhe Pan <c141028@gmail.com> Signed-off-by: Ruizhe Pan <c141028@gmail.com> Co-developed-by: Shiqi Zhang <shiqi@isrc.iscas.ac.cn> Signed-off-by: Shiqi Zhang <shiqi@isrc.iscas.ac.cn> Signed-off-by: Celeste Liu <CoelacanthusHex@gmail.com> Tested-by: Felix Yan <felixonmars@archlinux.org> Tested-by: Emil Renner Berthing <emil.renner.berthing@canonical.com> Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Reviewed-by: Guo Ren <guoren@kernel.org> Link: https://lore.kernel.org/r/20230801141607.435192-1-CoelacanthusHex@gmail.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-08-01 07:15:16 -07:00
long syscall = regs->a7;
riscv: entry: Convert to generic entry This patch converts riscv to use the generic entry infrastructure from kernel/entry/*. The generic entry makes maintainers' work easier and codes more elegant. Here are the changes: - More clear entry.S with handle_exception and ret_from_exception - Get rid of complex custom signal implementation - Move syscall procedure from assembly to C, which is much more readable. - Connect ret_from_fork & ret_from_kernel_thread to generic entry. - Wrap with irqentry_enter/exit and syscall_enter/exit_from_user_mode - Use the standard preemption code instead of custom Suggested-by: Huacai Chen <chenhuacai@kernel.org> Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Tested-by: Yipeng Zou <zouyipeng@huawei.com> Tested-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Cc: Ben Hutchings <ben@decadent.org.uk> Link: https://lore.kernel.org/r/20230222033021.983168-5-guoren@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-21 20:30:18 -07:00
regs->epc += 4;
regs->orig_a0 = regs->a0;
riscv: entry: always initialize regs->a0 to -ENOSYS Otherwise when the tracer changes syscall number to -1, the kernel fails to initialize a0 with -ENOSYS and subsequently fails to return the error code of the failed syscall to userspace. For example, it will break strace syscall tampering. Fixes: 52449c17bdd1 ("riscv: entry: set a0 = -ENOSYS only when syscall != -1") Reported-by: "Dmitry V. Levin" <ldv@strace.io> Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Cc: stable@vger.kernel.org Signed-off-by: Celeste Liu <CoelacanthusHex@gmail.com> Link: https://lore.kernel.org/r/20240627142338.5114-2-CoelacanthusHex@gmail.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2024-06-27 07:23:39 -07:00
regs->a0 = -ENOSYS;
riscv: entry: Convert to generic entry This patch converts riscv to use the generic entry infrastructure from kernel/entry/*. The generic entry makes maintainers' work easier and codes more elegant. Here are the changes: - More clear entry.S with handle_exception and ret_from_exception - Get rid of complex custom signal implementation - Move syscall procedure from assembly to C, which is much more readable. - Connect ret_from_fork & ret_from_kernel_thread to generic entry. - Wrap with irqentry_enter/exit and syscall_enter/exit_from_user_mode - Use the standard preemption code instead of custom Suggested-by: Huacai Chen <chenhuacai@kernel.org> Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Tested-by: Yipeng Zou <zouyipeng@huawei.com> Tested-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Cc: Ben Hutchings <ben@decadent.org.uk> Link: https://lore.kernel.org/r/20230222033021.983168-5-guoren@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-21 20:30:18 -07:00
riscv: Discard vector state on syscalls The RISC-V vector specification states: Executing a system call causes all caller-saved vector registers (v0-v31, vl, vtype) and vstart to become unspecified. The vector registers are set to all 1s, vill is set (invalid), and the vector status is set to Dirty. That way we can prevent userspace from accidentally relying on the stated save. Rémi pointed out [1] that writing to the registers might be superfluous, and setting vill is sufficient. Link: https://lore.kernel.org/linux-riscv/12784326.9UPPK3MAeB@basile.remlab.net/ # [1] Suggested-by: Darius Rad <darius@bluespec.com> Suggested-by: Palmer Dabbelt <palmer@rivosinc.com> Suggested-by: Rémi Denis-Courmont <remi@remlab.net> Signed-off-by: Björn Töpel <bjorn@rivosinc.com> Link: https://lore.kernel.org/r/20230629142228.1125715-1-bjorn@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-06-29 07:22:28 -07:00
riscv_v_vstate_discard(regs);
riscv: entry: Save a0 prior syscall_enter_from_user_mode() The RISC-V calling convention passes the first argument, and the return value in the a0 register. For this reason, the a0 register needs some extra care; When handling syscalls, the a0 register is saved into regs->orig_a0, so a0 can be properly restored for, e.g. interrupted syscalls. This functionality was broken with the introduction of the generic entry patches. Here, a0 was saved into orig_a0 after calling syscall_enter_from_user_mode(), which can change regs->a0 for some paths, incorrectly restoring a0. This is resolved, by saving a0 prior doing the syscall_enter_from_user_mode() call. Fixes: f0bddf50586d ("riscv: entry: Convert to generic entry") Reviewed-by: Heiko Stuebner <heiko.stuebner@vrull.eu> Tested-by: Heiko Stuebner <heiko.stuebner@vrull.eu> Signed-off-by: Björn Töpel <bjorn@rivosinc.com> Reported-by: Conor Dooley <conor.dooley@microchip.com> Reviewed-by: Conor Dooley <conor.dooley@microchip.com> Tested-by: Conor Dooley <conor.dooley@microchip.com> Tested-by: Geert Uytterhoeven <geert+renesas@glider.be> Tested-by: Andy Chiu <andy.chiu@sifive.com> Link: https://lore.kernel.org/r/20230403065207.1070974-1-bjorn@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-04-02 23:52:07 -07:00
syscall = syscall_enter_from_user_mode(regs, syscall);
riscv: Support RANDOMIZE_KSTACK_OFFSET Inspired from arm64's implement -- commit 70918779aec9 ("arm64: entry: Enable random_kstack_offset support") Add support of kernel stack offset randomization while handling syscall, the offset is defaultly limited by KSTACK_OFFSET_MAX() (i.e. 10 bits). In order to avoid trigger stack canaries (due to __builtin_alloca) and slowing down the entry path, use __no_stack_protector attribute to disable stack protector for do_trap_ecall_u() at the function level. Acked-by: Palmer Dabbelt <palmer@rivosinc.com> Reviewed-by: Kees Cook <keescook@chromium.org> Signed-off-by: Song Shuai <songshuaishuai@tinylab.org> Link: https://lore.kernel.org/r/20231109133751.212079-1-songshuaishuai@tinylab.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-11-09 06:37:51 -07:00
add_random_kstack_offset();
riscv: entry: set a0 = -ENOSYS only when syscall != -1 When we test seccomp with 6.4 kernel, we found errno has wrong value. If we deny NETLINK_AUDIT with EAFNOSUPPORT, after f0bddf50586d, we will get ENOSYS instead. We got same result with commit 9c2598d43510 ("riscv: entry: Save a0 prior syscall_enter_from_user_mode()"). After analysing code, we think that regs->a0 = -ENOSYS should only be executed when syscall != -1. In __seccomp_filter, when seccomp rejected this syscall with specified errno, they will set a0 to return number as syscall ABI, and then return -1. This return number is finally pass as return number of syscall_enter_from_user_mode, and then is compared with NR_syscalls after converted to ulong (so it will be ULONG_MAX). The condition syscall < NR_syscalls will always be false, so regs->a0 = -ENOSYS is always executed. It covered a0 set by seccomp, so we always get ENOSYS when match seccomp RET_ERRNO rule. Fixes: f0bddf50586d ("riscv: entry: Convert to generic entry") Reported-by: Felix Yan <felixonmars@archlinux.org> Co-developed-by: Ruizhe Pan <c141028@gmail.com> Signed-off-by: Ruizhe Pan <c141028@gmail.com> Co-developed-by: Shiqi Zhang <shiqi@isrc.iscas.ac.cn> Signed-off-by: Shiqi Zhang <shiqi@isrc.iscas.ac.cn> Signed-off-by: Celeste Liu <CoelacanthusHex@gmail.com> Tested-by: Felix Yan <felixonmars@archlinux.org> Tested-by: Emil Renner Berthing <emil.renner.berthing@canonical.com> Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Reviewed-by: Guo Ren <guoren@kernel.org> Link: https://lore.kernel.org/r/20230801141607.435192-1-CoelacanthusHex@gmail.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-08-01 07:15:16 -07:00
if (syscall >= 0 && syscall < NR_syscalls)
riscv: entry: Convert to generic entry This patch converts riscv to use the generic entry infrastructure from kernel/entry/*. The generic entry makes maintainers' work easier and codes more elegant. Here are the changes: - More clear entry.S with handle_exception and ret_from_exception - Get rid of complex custom signal implementation - Move syscall procedure from assembly to C, which is much more readable. - Connect ret_from_fork & ret_from_kernel_thread to generic entry. - Wrap with irqentry_enter/exit and syscall_enter/exit_from_user_mode - Use the standard preemption code instead of custom Suggested-by: Huacai Chen <chenhuacai@kernel.org> Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Tested-by: Yipeng Zou <zouyipeng@huawei.com> Tested-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Cc: Ben Hutchings <ben@decadent.org.uk> Link: https://lore.kernel.org/r/20230222033021.983168-5-guoren@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-21 20:30:18 -07:00
syscall_handler(regs, syscall);
riscv: entry: always initialize regs->a0 to -ENOSYS Otherwise when the tracer changes syscall number to -1, the kernel fails to initialize a0 with -ENOSYS and subsequently fails to return the error code of the failed syscall to userspace. For example, it will break strace syscall tampering. Fixes: 52449c17bdd1 ("riscv: entry: set a0 = -ENOSYS only when syscall != -1") Reported-by: "Dmitry V. Levin" <ldv@strace.io> Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Cc: stable@vger.kernel.org Signed-off-by: Celeste Liu <CoelacanthusHex@gmail.com> Link: https://lore.kernel.org/r/20240627142338.5114-2-CoelacanthusHex@gmail.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2024-06-27 07:23:39 -07:00
riscv: Support RANDOMIZE_KSTACK_OFFSET Inspired from arm64's implement -- commit 70918779aec9 ("arm64: entry: Enable random_kstack_offset support") Add support of kernel stack offset randomization while handling syscall, the offset is defaultly limited by KSTACK_OFFSET_MAX() (i.e. 10 bits). In order to avoid trigger stack canaries (due to __builtin_alloca) and slowing down the entry path, use __no_stack_protector attribute to disable stack protector for do_trap_ecall_u() at the function level. Acked-by: Palmer Dabbelt <palmer@rivosinc.com> Reviewed-by: Kees Cook <keescook@chromium.org> Signed-off-by: Song Shuai <songshuaishuai@tinylab.org> Link: https://lore.kernel.org/r/20231109133751.212079-1-songshuaishuai@tinylab.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-11-09 06:37:51 -07:00
/*
* Ultimately, this value will get limited by KSTACK_OFFSET_MAX(),
* so the maximum stack offset is 1k bytes (10 bits).
*
* The actual entropy will be further reduced by the compiler when
* applying stack alignment constraints: 16-byte (i.e. 4-bit) aligned
* for RV32I or RV64I.
*
* The resulting 6 bits of entropy is seen in SP[9:4].
*/
choose_random_kstack_offset(get_random_u16());
riscv: entry: Convert to generic entry This patch converts riscv to use the generic entry infrastructure from kernel/entry/*. The generic entry makes maintainers' work easier and codes more elegant. Here are the changes: - More clear entry.S with handle_exception and ret_from_exception - Get rid of complex custom signal implementation - Move syscall procedure from assembly to C, which is much more readable. - Connect ret_from_fork & ret_from_kernel_thread to generic entry. - Wrap with irqentry_enter/exit and syscall_enter/exit_from_user_mode - Use the standard preemption code instead of custom Suggested-by: Huacai Chen <chenhuacai@kernel.org> Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Tested-by: Yipeng Zou <zouyipeng@huawei.com> Tested-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Cc: Ben Hutchings <ben@decadent.org.uk> Link: https://lore.kernel.org/r/20230222033021.983168-5-guoren@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-21 20:30:18 -07:00
syscall_exit_to_user_mode(regs);
} else {
irqentry_state_t state = irqentry_nmi_enter(regs);
do_trap_error(regs, SIGILL, ILL_ILLTRP, regs->epc,
"Oops - environment call from U-mode");
irqentry_nmi_exit(regs, state);
}
}
#ifdef CONFIG_MMU
asmlinkage __visible noinstr void do_page_fault(struct pt_regs *regs)
{
irqentry_state_t state = irqentry_enter(regs);
handle_page_fault(regs);
local_irq_disable();
irqentry_exit(regs, state);
}
#endif
riscv: stack: Support HAVE_IRQ_EXIT_ON_IRQ_STACK Add independent irq stacks for percpu to prevent kernel stack overflows. It is also compatible with VMAP_STACK by arch_alloc_vmap_stack. Tested-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Cc: Clément Léger <cleger@rivosinc.com> Link: https://lore.kernel.org/r/20230614013018.2168426-2-guoren@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-06-13 18:30:16 -07:00
static void noinstr handle_riscv_irq(struct pt_regs *regs)
riscv: entry: Convert to generic entry This patch converts riscv to use the generic entry infrastructure from kernel/entry/*. The generic entry makes maintainers' work easier and codes more elegant. Here are the changes: - More clear entry.S with handle_exception and ret_from_exception - Get rid of complex custom signal implementation - Move syscall procedure from assembly to C, which is much more readable. - Connect ret_from_fork & ret_from_kernel_thread to generic entry. - Wrap with irqentry_enter/exit and syscall_enter/exit_from_user_mode - Use the standard preemption code instead of custom Suggested-by: Huacai Chen <chenhuacai@kernel.org> Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Tested-by: Yipeng Zou <zouyipeng@huawei.com> Tested-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Cc: Ben Hutchings <ben@decadent.org.uk> Link: https://lore.kernel.org/r/20230222033021.983168-5-guoren@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-21 20:30:18 -07:00
{
struct pt_regs *old_regs;
irq_enter_rcu();
old_regs = set_irq_regs(regs);
handle_arch_irq(regs);
set_irq_regs(old_regs);
irq_exit_rcu();
riscv: stack: Support HAVE_IRQ_EXIT_ON_IRQ_STACK Add independent irq stacks for percpu to prevent kernel stack overflows. It is also compatible with VMAP_STACK by arch_alloc_vmap_stack. Tested-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Cc: Clément Léger <cleger@rivosinc.com> Link: https://lore.kernel.org/r/20230614013018.2168426-2-guoren@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-06-13 18:30:16 -07:00
}
asmlinkage void noinstr do_irq(struct pt_regs *regs)
{
irqentry_state_t state = irqentry_enter(regs);
riscv: Deduplicate IRQ stack switching With CONFIG_IRQ_STACKS, we switch to a separate per-CPU IRQ stack before calling handle_riscv_irq or __do_softirq. We currently have duplicate inline assembly snippets for stack switching in both code paths. Now that we can access per-CPU variables in assembly, implement call_on_irq_stack in assembly, and use that instead of redundant inline assembly. Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Tested-by: Nathan Chancellor <nathan@kernel.org> Reviewed-by: Guo Ren <guoren@kernel.org> Link: https://lore.kernel.org/r/20230927224757.1154247-10-samitolvanen@google.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-09-27 15:48:00 -07:00
if (IS_ENABLED(CONFIG_IRQ_STACKS) && on_thread_stack())
call_on_irq_stack(regs, handle_riscv_irq);
else
riscv: stack: Support HAVE_IRQ_EXIT_ON_IRQ_STACK Add independent irq stacks for percpu to prevent kernel stack overflows. It is also compatible with VMAP_STACK by arch_alloc_vmap_stack. Tested-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Cc: Clément Léger <cleger@rivosinc.com> Link: https://lore.kernel.org/r/20230614013018.2168426-2-guoren@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-06-13 18:30:16 -07:00
handle_riscv_irq(regs);
riscv: entry: Convert to generic entry This patch converts riscv to use the generic entry infrastructure from kernel/entry/*. The generic entry makes maintainers' work easier and codes more elegant. Here are the changes: - More clear entry.S with handle_exception and ret_from_exception - Get rid of complex custom signal implementation - Move syscall procedure from assembly to C, which is much more readable. - Connect ret_from_fork & ret_from_kernel_thread to generic entry. - Wrap with irqentry_enter/exit and syscall_enter/exit_from_user_mode - Use the standard preemption code instead of custom Suggested-by: Huacai Chen <chenhuacai@kernel.org> Reviewed-by: Björn Töpel <bjorn@rivosinc.com> Tested-by: Yipeng Zou <zouyipeng@huawei.com> Tested-by: Jisheng Zhang <jszhang@kernel.org> Signed-off-by: Guo Ren <guoren@linux.alibaba.com> Signed-off-by: Guo Ren <guoren@kernel.org> Cc: Ben Hutchings <ben@decadent.org.uk> Link: https://lore.kernel.org/r/20230222033021.983168-5-guoren@kernel.org Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-02-21 20:30:18 -07:00
irqentry_exit(regs, state);
}
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
#ifdef CONFIG_GENERIC_BUG
int is_valid_bugaddr(unsigned long pc)
{
bug_insn_t insn;
riscv: Support BUG() in kernel module The kernel module is loaded into vmalloc region which is located below to the PAGE_OFFSET. Hence the condition, pc < PAGE_OFFSET, in the is_valid_bugaddr() will filter out all trap exceptions triggered by kernel module. To support BUG() in kernel module, the condition is changed to pc < VMALLOC_START. Signed-off-by: Vincent Chen <vincentc@andestech.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2019-03-04 20:23:35 -07:00
if (pc < VMALLOC_START)
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
return 0;
maccess: rename probe_kernel_address to get_kernel_nofault Better describe what this helper does, and match the naming of copy_from_kernel_nofault. Also switch the argument order around, so that it acts and looks like get_user(). Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2020-06-17 00:37:55 -07:00
if (get_kernel_nofault(insn, (bug_insn_t *)pc))
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
return 0;
riscv: Add the support for c.ebreak check in is_valid_bugaddr() The macro __BUG_INSN currently is defined as the "ebreak" opcode. The is_valid_bugaddr() function compares the instruction pointed to by $sepc with macro __BUG_INSN to check whether the current trap exception is caused by an "ebreak" instruction. However, this check flow is possibly erroneous because if C extension is supported, the expected trap instruction "ebreak" is possibly translated to "c.ebreak" by the assembler. Therefore, it requires a mechanism to distinguish the length of the instruction in $spec and compare it to the correct trap instruction. Signed-off-by: Vincent Chen <vincentc@andestech.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Palmer Dabbelt <palmer@sifive.com>
2019-03-04 20:23:34 -07:00
if ((insn & __INSN_LENGTH_MASK) == __INSN_LENGTH_32)
return (insn == __BUG_INSN_32);
else
return ((insn & __COMPRESSED_INSN_MASK) == __BUG_INSN_16);
RISC-V: Init and Halt Code This contains the various __init C functions, the initial assembly kernel entry point, and the code to reset the system. When a file was init-related this patch contains the entire file. Signed-off-by: Palmer Dabbelt <palmer@dabbelt.com>
2017-07-10 18:00:26 -07:00
}
#endif /* CONFIG_GENERIC_BUG */
riscv: add VMAP_STACK overflow detection This patch adds stack overflow detection to riscv, usable when CONFIG_VMAP_STACK=y. Overflow is detected in kernel exception entry(kernel/entry.S), if the kernel stack is overflow and been detected, the overflow handler is invoked on a per-cpu overflow stack. This approach preserves GPRs and the original exception information. The overflow detect is performed before any attempt is made to access the stack and the principle of stack overflow detection: kernel stacks are aligned to double their size, enabling overflow to be detected with a single bit test. For example, a 16K stack is aligned to 32K, ensuring that bit 14 of the SP must be zero. On an overflow (or underflow), this bit is flipped. Thus, overflow (of less than the size of the stack) can be detected by testing whether this bit is set. This gives us a useful error message on stack overflow, as can be trigger with the LKDTM overflow test: [ 388.053267] lkdtm: Performing direct entry EXHAUST_STACK [ 388.053663] lkdtm: Calling function with 1024 frame size to depth 32 ... [ 388.054016] lkdtm: loop 32/32 ... [ 388.054186] lkdtm: loop 31/32 ... [ 388.054491] lkdtm: loop 30/32 ... [ 388.054672] lkdtm: loop 29/32 ... [ 388.054859] lkdtm: loop 28/32 ... [ 388.055010] lkdtm: loop 27/32 ... [ 388.055163] lkdtm: loop 26/32 ... [ 388.055309] lkdtm: loop 25/32 ... [ 388.055481] lkdtm: loop 24/32 ... [ 388.055653] lkdtm: loop 23/32 ... [ 388.055837] lkdtm: loop 22/32 ... [ 388.056015] lkdtm: loop 21/32 ... [ 388.056188] lkdtm: loop 20/32 ... [ 388.058145] Insufficient stack space to handle exception! [ 388.058153] Task stack: [0xffffffd014260000..0xffffffd014264000] [ 388.058160] Overflow stack: [0xffffffe1f8d2c220..0xffffffe1f8d2d220] [ 388.058168] CPU: 0 PID: 89 Comm: bash Not tainted 5.12.0-rc8-dirty #90 [ 388.058175] Hardware name: riscv-virtio,qemu (DT) [ 388.058187] epc : number+0x32/0x2c0 [ 388.058247] ra : vsnprintf+0x2ae/0x3f0 [ 388.058255] epc : ffffffe0002d38f6 ra : ffffffe0002d814e sp : ffffffd01425ffc0 [ 388.058263] gp : ffffffe0012e4010 tp : ffffffe08014da00 t0 : ffffffd0142606e8 [ 388.058271] t1 : 0000000000000000 t2 : 0000000000000000 s0 : ffffffd014260070 [ 388.058303] s1 : ffffffd014260158 a0 : ffffffd01426015e a1 : ffffffd014260158 [ 388.058311] a2 : 0000000000000013 a3 : ffff0a01ffffff10 a4 : ffffffe000c398e0 [ 388.058319] a5 : 511b02ec65f3e300 a6 : 0000000000a1749a a7 : 0000000000000000 [ 388.058327] s2 : ffffffff000000ff s3 : 00000000ffff0a01 s4 : ffffffe0012e50a8 [ 388.058335] s5 : 0000000000ffff0a s6 : ffffffe0012e50a8 s7 : ffffffe000da1cc0 [ 388.058343] s8 : ffffffffffffffff s9 : ffffffd0142602b0 s10: ffffffd0142602a8 [ 388.058351] s11: ffffffd01426015e t3 : 00000000000f0000 t4 : ffffffffffffffff [ 388.058359] t5 : 000000000000002f t6 : ffffffd014260158 [ 388.058366] status: 0000000000000100 badaddr: ffffffd01425fff8 cause: 000000000000000f [ 388.058374] Kernel panic - not syncing: Kernel stack overflow [ 388.058381] CPU: 0 PID: 89 Comm: bash Not tainted 5.12.0-rc8-dirty #90 [ 388.058387] Hardware name: riscv-virtio,qemu (DT) [ 388.058393] Call Trace: [ 388.058400] [<ffffffe000004944>] walk_stackframe+0x0/0xce [ 388.058406] [<ffffffe0006f0b28>] dump_backtrace+0x38/0x46 [ 388.058412] [<ffffffe0006f0b46>] show_stack+0x10/0x18 [ 388.058418] [<ffffffe0006f3690>] dump_stack+0x74/0x8e [ 388.058424] [<ffffffe0006f0d52>] panic+0xfc/0x2b2 [ 388.058430] [<ffffffe0006f0acc>] print_trace_address+0x0/0x24 [ 388.058436] [<ffffffe0002d814e>] vsnprintf+0x2ae/0x3f0 [ 388.058956] SMP: stopping secondary CPUs Signed-off-by: Tong Tiangen <tongtiangen@huawei.com> Reviewed-by: Kefeng Wang <wangkefeng.wang@huawei.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-06-20 20:28:55 -07:00
#ifdef CONFIG_VMAP_STACK
riscv: VMAP_STACK overflow detection thread-safe commit 31da94c25aea ("riscv: add VMAP_STACK overflow detection") added support for CONFIG_VMAP_STACK. If overflow is detected, CPU switches to `shadow_stack` temporarily before switching finally to per-cpu `overflow_stack`. If two CPUs/harts are racing and end up in over flowing kernel stack, one or both will end up corrupting each other state because `shadow_stack` is not per-cpu. This patch optimizes per-cpu overflow stack switch by directly picking per-cpu `overflow_stack` and gets rid of `shadow_stack`. Following are the changes in this patch - Defines an asm macro to obtain per-cpu symbols in destination register. - In entry.S, when overflow is detected, per-cpu overflow stack is located using per-cpu asm macro. Computing per-cpu symbol requires a temporary register. x31 is saved away into CSR_SCRATCH (CSR_SCRATCH is anyways zero since we're in kernel). Please see Links for additional relevant disccussion and alternative solution. Tested by `echo EXHAUST_STACK > /sys/kernel/debug/provoke-crash/DIRECT` Kernel crash log below Insufficient stack space to handle exception!/debug/provoke-crash/DIRECT Task stack: [0xff20000010a98000..0xff20000010a9c000] Overflow stack: [0xff600001f7d98370..0xff600001f7d99370] CPU: 1 PID: 205 Comm: bash Not tainted 6.1.0-rc2-00001-g328a1f96f7b9 #34 Hardware name: riscv-virtio,qemu (DT) epc : __memset+0x60/0xfc ra : recursive_loop+0x48/0xc6 [lkdtm] epc : ffffffff808de0e4 ra : ffffffff0163a752 sp : ff20000010a97e80 gp : ffffffff815c0330 tp : ff600000820ea280 t0 : ff20000010a97e88 t1 : 000000000000002e t2 : 3233206874706564 s0 : ff20000010a982b0 s1 : 0000000000000012 a0 : ff20000010a97e88 a1 : 0000000000000000 a2 : 0000000000000400 a3 : ff20000010a98288 a4 : 0000000000000000 a5 : 0000000000000000 a6 : fffffffffffe43f0 a7 : 00007fffffffffff s2 : ff20000010a97e88 s3 : ffffffff01644680 s4 : ff20000010a9be90 s5 : ff600000842ba6c0 s6 : 00aaaaaac29e42b0 s7 : 00fffffff0aa3684 s8 : 00aaaaaac2978040 s9 : 0000000000000065 s10: 00ffffff8a7cad10 s11: 00ffffff8a76a4e0 t3 : ffffffff815dbaf4 t4 : ffffffff815dbaf4 t5 : ffffffff815dbab8 t6 : ff20000010a9bb48 status: 0000000200000120 badaddr: ff20000010a97e88 cause: 000000000000000f Kernel panic - not syncing: Kernel stack overflow CPU: 1 PID: 205 Comm: bash Not tainted 6.1.0-rc2-00001-g328a1f96f7b9 #34 Hardware name: riscv-virtio,qemu (DT) Call Trace: [<ffffffff80006754>] dump_backtrace+0x30/0x38 [<ffffffff808de798>] show_stack+0x40/0x4c [<ffffffff808ea2a8>] dump_stack_lvl+0x44/0x5c [<ffffffff808ea2d8>] dump_stack+0x18/0x20 [<ffffffff808dec06>] panic+0x126/0x2fe [<ffffffff800065ea>] walk_stackframe+0x0/0xf0 [<ffffffff0163a752>] recursive_loop+0x48/0xc6 [lkdtm] SMP: stopping secondary CPUs ---[ end Kernel panic - not syncing: Kernel stack overflow ]--- Cc: Guo Ren <guoren@kernel.org> Cc: Jisheng Zhang <jszhang@kernel.org> Link: https://lore.kernel.org/linux-riscv/Y347B0x4VUNOd6V7@xhacker/T/#t Link: https://lore.kernel.org/lkml/20221124094845.1907443-1-debug@rivosinc.com/ Signed-off-by: Deepak Gupta <debug@rivosinc.com> Co-developed-by: Sami Tolvanen <samitolvanen@google.com> Signed-off-by: Sami Tolvanen <samitolvanen@google.com> Acked-by: Guo Ren <guoren@kernel.org> Tested-by: Nathan Chancellor <nathan@kernel.org> Link: https://lore.kernel.org/r/20230927224757.1154247-9-samitolvanen@google.com Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
2023-09-27 15:47:59 -07:00
DEFINE_PER_CPU(unsigned long [OVERFLOW_STACK_SIZE/sizeof(long)],
riscv: add VMAP_STACK overflow detection This patch adds stack overflow detection to riscv, usable when CONFIG_VMAP_STACK=y. Overflow is detected in kernel exception entry(kernel/entry.S), if the kernel stack is overflow and been detected, the overflow handler is invoked on a per-cpu overflow stack. This approach preserves GPRs and the original exception information. The overflow detect is performed before any attempt is made to access the stack and the principle of stack overflow detection: kernel stacks are aligned to double their size, enabling overflow to be detected with a single bit test. For example, a 16K stack is aligned to 32K, ensuring that bit 14 of the SP must be zero. On an overflow (or underflow), this bit is flipped. Thus, overflow (of less than the size of the stack) can be detected by testing whether this bit is set. This gives us a useful error message on stack overflow, as can be trigger with the LKDTM overflow test: [ 388.053267] lkdtm: Performing direct entry EXHAUST_STACK [ 388.053663] lkdtm: Calling function with 1024 frame size to depth 32 ... [ 388.054016] lkdtm: loop 32/32 ... [ 388.054186] lkdtm: loop 31/32 ... [ 388.054491] lkdtm: loop 30/32 ... [ 388.054672] lkdtm: loop 29/32 ... [ 388.054859] lkdtm: loop 28/32 ... [ 388.055010] lkdtm: loop 27/32 ... [ 388.055163] lkdtm: loop 26/32 ... [ 388.055309] lkdtm: loop 25/32 ... [ 388.055481] lkdtm: loop 24/32 ... [ 388.055653] lkdtm: loop 23/32 ... [ 388.055837] lkdtm: loop 22/32 ... [ 388.056015] lkdtm: loop 21/32 ... [ 388.056188] lkdtm: loop 20/32 ... [ 388.058145] Insufficient stack space to handle exception! [ 388.058153] Task stack: [0xffffffd014260000..0xffffffd014264000] [ 388.058160] Overflow stack: [0xffffffe1f8d2c220..0xffffffe1f8d2d220] [ 388.058168] CPU: 0 PID: 89 Comm: bash Not tainted 5.12.0-rc8-dirty #90 [ 388.058175] Hardware name: riscv-virtio,qemu (DT) [ 388.058187] epc : number+0x32/0x2c0 [ 388.058247] ra : vsnprintf+0x2ae/0x3f0 [ 388.058255] epc : ffffffe0002d38f6 ra : ffffffe0002d814e sp : ffffffd01425ffc0 [ 388.058263] gp : ffffffe0012e4010 tp : ffffffe08014da00 t0 : ffffffd0142606e8 [ 388.058271] t1 : 0000000000000000 t2 : 0000000000000000 s0 : ffffffd014260070 [ 388.058303] s1 : ffffffd014260158 a0 : ffffffd01426015e a1 : ffffffd014260158 [ 388.058311] a2 : 0000000000000013 a3 : ffff0a01ffffff10 a4 : ffffffe000c398e0 [ 388.058319] a5 : 511b02ec65f3e300 a6 : 0000000000a1749a a7 : 0000000000000000 [ 388.058327] s2 : ffffffff000000ff s3 : 00000000ffff0a01 s4 : ffffffe0012e50a8 [ 388.058335] s5 : 0000000000ffff0a s6 : ffffffe0012e50a8 s7 : ffffffe000da1cc0 [ 388.058343] s8 : ffffffffffffffff s9 : ffffffd0142602b0 s10: ffffffd0142602a8 [ 388.058351] s11: ffffffd01426015e t3 : 00000000000f0000 t4 : ffffffffffffffff [ 388.058359] t5 : 000000000000002f t6 : ffffffd014260158 [ 388.058366] status: 0000000000000100 badaddr: ffffffd01425fff8 cause: 000000000000000f [ 388.058374] Kernel panic - not syncing: Kernel stack overflow [ 388.058381] CPU: 0 PID: 89 Comm: bash Not tainted 5.12.0-rc8-dirty #90 [ 388.058387] Hardware name: riscv-virtio,qemu (DT) [ 388.058393] Call Trace: [ 388.058400] [<ffffffe000004944>] walk_stackframe+0x0/0xce [ 388.058406] [<ffffffe0006f0b28>] dump_backtrace+0x38/0x46 [ 388.058412] [<ffffffe0006f0b46>] show_stack+0x10/0x18 [ 388.058418] [<ffffffe0006f3690>] dump_stack+0x74/0x8e [ 388.058424] [<ffffffe0006f0d52>] panic+0xfc/0x2b2 [ 388.058430] [<ffffffe0006f0acc>] print_trace_address+0x0/0x24 [ 388.058436] [<ffffffe0002d814e>] vsnprintf+0x2ae/0x3f0 [ 388.058956] SMP: stopping secondary CPUs Signed-off-by: Tong Tiangen <tongtiangen@huawei.com> Reviewed-by: Kefeng Wang <wangkefeng.wang@huawei.com> Signed-off-by: Palmer Dabbelt <palmerdabbelt@google.com>
2021-06-20 20:28:55 -07:00
overflow_stack)__aligned(16);
asmlinkage void handle_bad_stack(struct pt_regs *regs)
{
unsigned long tsk_stk = (unsigned long)current->stack;
unsigned long ovf_stk = (unsigned long)this_cpu_ptr(overflow_stack);
console_verbose();
pr_emerg("Insufficient stack space to handle exception!\n");
pr_emerg("Task stack: [0x%016lx..0x%016lx]\n",
tsk_stk, tsk_stk + THREAD_SIZE);
pr_emerg("Overflow stack: [0x%016lx..0x%016lx]\n",
ovf_stk, ovf_stk + OVERFLOW_STACK_SIZE);
__show_regs(regs);
panic("Kernel stack overflow");
for (;;)
wait_for_interrupt();
}
#endif
Copy Permalink