Received: by 2002:a05:6a10:5bc5:0:0:0:0 with SMTP id os5csp886298pxb; Wed, 27 Oct 2021 14:28:48 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw+6O9p6mKuYZ+5FMQCT+FQ9WQ04nV+UDaBovKxwsEEua8zo6A5Irk3jwH8AjjJUkFggpt6 X-Received: by 2002:a17:906:dfd6:: with SMTP id jt22mr58833ejc.381.1635370128022; Wed, 27 Oct 2021 14:28:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1635370128; cv=none; d=google.com; s=arc-20160816; b=HDeRaqGprqoxqWMnTyB7CH79buDvxGrDe97PGZdE+AVs0nBSAAFG86m84hZby7VZob STW6q4RnCjhDkf9uwBLFmlIrba978cQqOBn9ACNexpfNja9PAOVdokyZHo5fQVwSyK2Y J0JWm3I/L/Q05Il+k6zLfuOkz+6R3wG9gmMIiEQytX5xFbI+F6Fhn1w3FUKHgJ98+C/T /fTgcw6i0bSgiOqbltY7gaNOd98LRWg7VEW+zekiH7JbyXRd1bnMEdUJKJGmvdEn8AiZ NRFByzQwbv4N1XXM36S7cP/9WxgWtwI3/tIauNG111vIgEH62uMy8hRG7ABYkMJ7/9xx HI4w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:subject:message-id:date:from:in-reply-to :references:mime-version:dkim-signature; bh=kT1BTsS+aS9ez9cTATxAqBX2TlYnWNIAtTj3TQgKO1k=; b=yIF5pIyYrYJ5tszpaHPaqeVFD+A8xK9TSq51zckDtfCfi0SbCicm0s11AFDMzTenTE Q4iLRpJg5vTMYD1OdwSHtRPxNEVL3sqxnukUV6TUPAKEyIpzFeVfWFUktReNAF9J0yax C2+TNKlg3SLuurj1qW17FSGwlS7Av8EJ9FH2FyYfZBoPRWdTZM87LgfCGPdsWiLdcLUM UDCjME0coZMgekQaxau0NtdQhf1HNjfJeiZwrHrqm5d3h9tYj/PCC+2NtZRdpj1qByrG I4u99kMZtj4IqqdtGV6AGkbR99Ja3NIPCGYlQuCcwyEMInhJXNbbJGWGBGvghqlW7qYX nMIA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=kPx7lW7H; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id dp11si1492689ejc.3.2021.10.27.14.28.23; Wed, 27 Oct 2021 14:28:48 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=kPx7lW7H; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242457AbhJ0OVE (ORCPT + 97 others); Wed, 27 Oct 2021 10:21:04 -0400 Received: from mail.kernel.org ([198.145.29.99]:43632 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242437AbhJ0OUz (ORCPT ); Wed, 27 Oct 2021 10:20:55 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 58F2D60F9D; Wed, 27 Oct 2021 14:18:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1635344310; bh=Ci0lo29N8P5kyUfw16N1daeiwbEtGaYvfxpqWpHCONk=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=kPx7lW7H5ZT2/17cstsUGQuXbbGHVXvQ/bpdLYt0a0sMpaHMLEIYT4NCxgNe0Y2eP 6nhfUt5zAKC54gh326+pA4nrg0XG/rYnlVSwnlOd2OFHWKIqPs7gNoABolbkuYEJUI 0GN/XoUoVdJD+8Rz52gAGH02DEHXg2MFoIeL+DHQAmStebLsFhZXry1x/XRlZ/FyTm 3Z5UbeB/yQkLu4Db6GKCLdCRiMmz5WByWXoeLfYfMqxaObuxmTvuk9rb04eBbl7dfZ UBTyEGJAdP76AoH5A55xEKLLvAT8SMES+8oLbv/ub7E/WFAmV8BITiegOgJjceYI6E djrSvewbY8hPQ== Received: by mail-ot1-f45.google.com with SMTP id 107-20020a9d0a74000000b00553bfb53348so3865356otg.0; Wed, 27 Oct 2021 07:18:30 -0700 (PDT) X-Gm-Message-State: AOAM532DAPoIjgZrBUTLCiMksOK3b1RtQ9+UbiuPteybVJO8A639YheK j+SWQuOfSRTT3Y5z3ySOnMJOK8GvvvQr4qmQjxM= X-Received: by 2002:a9d:5911:: with SMTP id t17mr25145832oth.30.1635344309561; Wed, 27 Oct 2021 07:18:29 -0700 (PDT) MIME-Version: 1.0 References: <20211013181658.1020262-1-samitolvanen@google.com> <20211026201622.GG174703@worktop.programming.kicks-ass.net> <20211027120515.GC54628@C02TD0UTHF1T.local> <20211027124852.GK174703@worktop.programming.kicks-ass.net> In-Reply-To: From: Ard Biesheuvel Date: Wed, 27 Oct 2021 16:18:17 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH v5 00/15] x86: Add support for Clang CFI To: Peter Zijlstra Cc: Mark Rutland , Sami Tolvanen , X86 ML , Kees Cook , Josh Poimboeuf , Nathan Chancellor , Nick Desaulniers , Sedat Dilek , Steven Rostedt , linux-hardening@vger.kernel.org, Linux Kernel Mailing List , llvm@lists.linux.dev Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 27 Oct 2021 at 16:03, Peter Zijlstra wrote: > > On Wed, Oct 27, 2021 at 03:30:11PM +0200, Ard Biesheuvel wrote: > > > As far as I can tell from playing around with Clang, the stubs can > > actually be executed directly, > > I had just finished reading the clang docs which suggest as much and was > about to try what the compiler actually generates. > > > they just jumps to the actual function. > > The compiler simply generates a jump table for each prototype that > > appears in the code as the target of an indirect jump, and checks > > whether the target appears in the list. > > > > E.g., the code below > > > > void foo(void) {} > > void bar(int) {} > > void baz(int) {} > > void (* volatile fn1)(void) = foo; > > void (* volatile fn2)(int) = bar; > > > > int main(int argc, char *argv[]) > > { > > fn1(); > > fn2 = baz; > > fn2(-1); > > } > > > > produces > > > > 0000000000400594 : > > 400594: d65f03c0 ret > > > > 0000000000400598 : > > 400598: d65f03c0 ret > > > > 000000000040059c : > > 40059c: d65f03c0 ret > > Right, so these are the actual functions ^. > > > 00000000004005a0
: > > 4005a0: a9bf7bfd stp x29, x30, [sp, #-16]! > > > > // First indirect call > > 4005a4: b0000088 adrp x8, 411000 <__libc_start_main@GLIBC_2.17> > > 4005a8: f9401508 ldr x8, [x8, #40] > > 4005ac: 90000009 adrp x9, 400000 <__abi_tag-0x278> > > 4005b0: 91182129 add x9, x9, #0x608 > > 4005b4: 910003fd mov x29, sp > > 4005b8: eb09011f cmp x8, x9 > > 4005bc: 54000241 b.ne 400604 // b.any > > 4005c0: d63f0100 blr x8 > > That's impenetrable to me, sorry. > This loads the value of fn1 in x8, and takes the address of the jump table in x9. Since it is only one entry long, it does a simple compare to check whether x8 appears in the jump table, and branches to the BRK at the end if they are different. > > // Assignment of fn2 > > 4005c4: 90000009 adrp x9, 400000 <__abi_tag-0x278> > > 4005c8: b0000088 adrp x8, 411000 <__libc_start_main@GLIBC_2.17> > > 4005cc: 91184129 add x9, x9, #0x610 > > 4005d0: f9001909 str x9, [x8, #48] > > I'm struggling here, x9 points to the branch at 400610, but then what? > > x8 is in .data somewhere? > This takes the address of the jump table entry of 'baz' in x9, and stores it in fn2 whose address is taken in x8. > > // Second indirect call > > 4005d4: f9401908 ldr x8, [x8, #48] > > 4005d8: 90000009 adrp x9, 400000 <__abi_tag-0x278> > > 4005dc: 91183129 add x9, x9, #0x60c > > 4005e0: cb090109 sub x9, x8, x9 > > 4005e4: 93c90929 ror x9, x9, #2 > > 4005e8: f100053f cmp x9, #0x1 > > 4005ec: 540000c8 b.hi 400604 // b.pmore > > 4005f0: 12800000 mov w0, #0xffffffff // #-1 > > 4005f4: d63f0100 blr x8 > > > > > > 4005f8: 2a1f03e0 mov w0, wzr > > 4005fc: a8c17bfd ldp x29, x30, [sp], #16 > > 400600: d65f03c0 ret > > 400604: d4200020 brk #0x1 > > > > 0000000000400608 <__typeid__ZTSFvvE_global_addr>: > > 400608: 17ffffe3 b 400594 > > > > 000000000040060c <__typeid__ZTSFviE_global_addr>: > > 40060c: 17ffffe3 b 400598 > > 400610: 17ffffe3 b 40059c > > And these are the stubs per type. > > > So it looks like taking the address is fine, although not optimal due > > to the additional jump. > > Right. > ... although it does seem that function_nocfi() doesn't actually work as expected, given that we want the address of .cfi and not the address of the stub. > > We could fudge around that by checking the > > opcode at the target of the call, or token paste ".cfi" after the > > symbol name in the static_call_update() macro, but it doesn't like > > like anything is terminally broken tbh. > > Agreed, since the jump table entries are actually executable it 'works'. > > I really don't like that extra jump though, so I think I really do want > that nocfi_ptr() thing. And going by: > > https://clang.llvm.org/docs/ControlFlowIntegrityDesign.html#forward-edge-cfi-for-indirect-function-calls > > and the above, that might be possible (on x86) with something like: > > /* > * Turns a Clang CFI jump-table entry into an actual function pointer. > * These jump-table entries are simply jmp.d32 instruction with their > * relative offset pointing to the actual function, therefore decode the > * instruction to find the real function. > */ > static __always_inline void *nocfi_ptr(void *func) > { > union text_poke_insn insn = *(union text_poke_insn *)func; > > return func + sizeof(insn) + insn.disp; > } > > But really, that wants to be a compiler intrinsic. Agreed. We could easily do something similar on arm64, but I'd prefer to avoid that too.