Received: by 2002:a05:6a10:5bc5:0:0:0:0 with SMTP id os5csp886715pxb; Wed, 27 Oct 2021 14:29:17 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy8FHDd5EPwEKULs01LGNd8st4nL716Awey6aVkCcgDJSQLIkPuRms2f8bxhw7kLUOnAr3f X-Received: by 2002:a63:8c12:: with SMTP id m18mr178154pgd.187.1635370157100; Wed, 27 Oct 2021 14:29:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1635370157; cv=none; d=google.com; s=arc-20160816; b=t9yOgBhrmiv0ZwednqO43xXa8I1gD9OD5lNIXt5rC1scrOhF39A8Ob/85cryYnveTS kqs4jg9ZA+tWsKNv0++TULcjSkPm39dnEhKA4JsaVJduEkcHJkPKHBqeTRPBs0iYZCrO xwv2t3vZ0lMqUhdXFjO6C1XI9/amUUr6ImlUT1i7YXmgrfYvhFKrqDpl08edNodDYpBE uXfizR9VvNcq2qmJsipLP3dvUjUOXPZk17n3xdfsW8yv6BcH6AC3trCGI8gtDQH2nrSp Tf1/g92Yu7A3e91TiuQAnk1rnB6mJyGt/6WeYK4ucRL9dRibs/SahGowxZjtU8bEi70v t0Hg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:subject:message-id:date:from:in-reply-to :references:mime-version:dkim-signature; bh=xT3+nmu87OUZUlUqDOD62awQnojwMPGi3jG6yVIYB20=; b=GWgcTDdkMrqE/rk6P80+tL4gjYBW7mKLDNzrwslPsjf48Ik2d2oo9qhlWjeUG8M/EQ 6TE+Ae9mFe7y90oINwOhW6OJ8llkqWRZikFbMzKVqVvVG1qEOBDYGwX4VLU1kIt6+A16 +jniJRhk60alyoxwZKO2iJujsI3fPJqmoeHsvO6s1K3Z5vzWB2zlrwupGnu+4oTeQDks ylh2X0E6HPbYiXC1xRI08DOF5kOsboG/Y2++o9fcq/abXOLk0I0nfzueMDIxbmq0pRgS c+v6jzFXmPEWbxt2tgL0UXEDevjrMgdamym7r8cXTaix+XPclVqumCvrKq507hNRkwVk ey8w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=Q5LslJIP; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id b186si408416pgc.552.2021.10.27.14.29.04; Wed, 27 Oct 2021 14:29:17 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=Q5LslJIP; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242840AbhJ0Pwz (ORCPT + 97 others); Wed, 27 Oct 2021 11:52:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34806 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242768AbhJ0Pwy (ORCPT ); Wed, 27 Oct 2021 11:52:54 -0400 Received: from mail-yb1-xb33.google.com (mail-yb1-xb33.google.com [IPv6:2607:f8b0:4864:20::b33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4EDB4C061745 for ; Wed, 27 Oct 2021 08:50:29 -0700 (PDT) Received: by mail-yb1-xb33.google.com with SMTP id u84so3828455yba.3 for ; Wed, 27 Oct 2021 08:50:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=xT3+nmu87OUZUlUqDOD62awQnojwMPGi3jG6yVIYB20=; b=Q5LslJIP1WUECn+19rRJj5nPB6Bw+vkyE5RDu6YMSkOWj1yqa/r0xmS3A1hkwuQW4o N0rir4ytbQGNZrYwQ8olQ86SR9M3AjkZavvc7zfuvQjyj46lk4YV/pmrZNHcSVzsh6kp 15QYxWKGONAT6kEVnPluF5lEzyZiXcI8dlV1CXqciS+NG+iTJxOnD3gPrOULCFVzQvDk 11Ew+fnTS1X6vbrkw9x1Dt4iwf5ZG5xXv7fUfn8iR7udBLczkl3G6KvjRAv0/9ajYEDG zfYTawSOgY+RRpWN6rkRcFHmeqb59QVbejF6wxvI5dVBAQ3XJf+8uwqnYVSZR4aIzf7q yiJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=xT3+nmu87OUZUlUqDOD62awQnojwMPGi3jG6yVIYB20=; b=IZPW57Z6mjGR+E1bmvRk87eSxvJaJxhb/vxIHOvzea7xFoOXQJpdXf9LhYXrCl9qnn 2gBFVgqFdKqzmReMnUjnAaTgFPuOiohg5/tez7xe9ZanMHqKkAFR8NoJ8hSop2HA1weh tirO+GF5V8MfVDLmx/RRsOvSDRLPWpbMd4CupApRVwM5ASDn+Gv/XMjcCsy/7VOdEDBF IJfzw9EcJeAlEPEyjhL4h3ztk7auMeWfVfGUG39Rc1d1MsdYbeOdzk1rXTs2Hdb/svzh Jg8QmXvp9lXUL5IqaK7h0RFs8XqN5zAAGcnq5kK++vj2qKxEBx6S/c+jfCC/5mItu4Le K2dw== X-Gm-Message-State: AOAM531Wg/3uiM4Wxz/Aoc7q8cPQJ0jKIRlaevH+rHtPnUD9Nzf022i/ VEjATbF/RgCkiIOKe3wZlsAtBtwBLUPQU3pFGlwPLA== X-Received: by 2002:a25:bf81:: with SMTP id l1mr984264ybk.289.1635349828212; Wed, 27 Oct 2021 08:50:28 -0700 (PDT) MIME-Version: 1.0 References: <20211013181658.1020262-1-samitolvanen@google.com> <20211026201622.GG174703@worktop.programming.kicks-ass.net> <20211027120515.GC54628@C02TD0UTHF1T.local> <20211027124852.GK174703@worktop.programming.kicks-ass.net> In-Reply-To: From: Sami Tolvanen Date: Wed, 27 Oct 2021 08:50:17 -0700 Message-ID: Subject: Re: [PATCH v5 00/15] x86: Add support for Clang CFI To: Ard Biesheuvel Cc: Peter Zijlstra , Mark Rutland , X86 ML , Kees Cook , Josh Poimboeuf , Nathan Chancellor , Nick Desaulniers , Sedat Dilek , Steven Rostedt , linux-hardening@vger.kernel.org, Linux Kernel Mailing List , llvm@lists.linux.dev Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Oct 27, 2021 at 7:18 AM Ard Biesheuvel wrote: > > On Wed, 27 Oct 2021 at 16:03, Peter Zijlstra wrote: > > > > On Wed, Oct 27, 2021 at 03:30:11PM +0200, Ard Biesheuvel wrote: > > > > > As far as I can tell from playing around with Clang, the stubs can > > > actually be executed directly, > > > > I had just finished reading the clang docs which suggest as much and was > > about to try what the compiler actually generates. > > > > > they just jumps to the actual function. > > > The compiler simply generates a jump table for each prototype that > > > appears in the code as the target of an indirect jump, and checks > > > whether the target appears in the list. > > > > > > E.g., the code below > > > > > > void foo(void) {} > > > void bar(int) {} > > > void baz(int) {} > > > void (* volatile fn1)(void) = foo; > > > void (* volatile fn2)(int) = bar; > > > > > > int main(int argc, char *argv[]) > > > { > > > fn1(); > > > fn2 = baz; > > > fn2(-1); > > > } > > > > > > produces > > > > > > 0000000000400594 : > > > 400594: d65f03c0 ret > > > > > > 0000000000400598 : > > > 400598: d65f03c0 ret > > > > > > 000000000040059c : > > > 40059c: d65f03c0 ret > > > > Right, so these are the actual functions ^. > > > > > 00000000004005a0
: > > > 4005a0: a9bf7bfd stp x29, x30, [sp, #-16]! > > > > > > // First indirect call > > > 4005a4: b0000088 adrp x8, 411000 <__libc_start_main@GLIBC_2.17> > > > 4005a8: f9401508 ldr x8, [x8, #40] > > > 4005ac: 90000009 adrp x9, 400000 <__abi_tag-0x278> > > > 4005b0: 91182129 add x9, x9, #0x608 > > > 4005b4: 910003fd mov x29, sp > > > 4005b8: eb09011f cmp x8, x9 > > > 4005bc: 54000241 b.ne 400604 // b.any > > > 4005c0: d63f0100 blr x8 > > > > That's impenetrable to me, sorry. > > > > This loads the value of fn1 in x8, and takes the address of the jump > table in x9. Since it is only one entry long, it does a simple compare > to check whether x8 appears in the jump table, and branches to the BRK > at the end if they are different. > > > > // Assignment of fn2 > > > 4005c4: 90000009 adrp x9, 400000 <__abi_tag-0x278> > > > 4005c8: b0000088 adrp x8, 411000 <__libc_start_main@GLIBC_2.17> > > > 4005cc: 91184129 add x9, x9, #0x610 > > > 4005d0: f9001909 str x9, [x8, #48] > > > > I'm struggling here, x9 points to the branch at 400610, but then what? > > > > x8 is in .data somewhere? > > > > This takes the address of the jump table entry of 'baz' in x9, and > stores it in fn2 whose address is taken in x8. > > > > > // Second indirect call > > > 4005d4: f9401908 ldr x8, [x8, #48] > > > 4005d8: 90000009 adrp x9, 400000 <__abi_tag-0x278> > > > 4005dc: 91183129 add x9, x9, #0x60c > > > 4005e0: cb090109 sub x9, x8, x9 > > > 4005e4: 93c90929 ror x9, x9, #2 > > > 4005e8: f100053f cmp x9, #0x1 > > > 4005ec: 540000c8 b.hi 400604 // b.pmore > > > 4005f0: 12800000 mov w0, #0xffffffff // #-1 > > > 4005f4: d63f0100 blr x8 > > > > > > > > > 4005f8: 2a1f03e0 mov w0, wzr > > > 4005fc: a8c17bfd ldp x29, x30, [sp], #16 > > > 400600: d65f03c0 ret > > > 400604: d4200020 brk #0x1 > > > > > > > 0000000000400608 <__typeid__ZTSFvvE_global_addr>: > > > 400608: 17ffffe3 b 400594 > > > > > > 000000000040060c <__typeid__ZTSFviE_global_addr>: > > > 40060c: 17ffffe3 b 400598 > > > 400610: 17ffffe3 b 40059c > > > > And these are the stubs per type. > > > > > So it looks like taking the address is fine, although not optimal due > > > to the additional jump. > > > > Right. > > > > ... although it does seem that function_nocfi() doesn't actually work > as expected, given that we want the address of .cfi and not the > address of the stub. This is because the example wasn't compiled with -fno-sanitize-cfi-canonical-jump-tables, which we use in the kernel. With non-canonical jump tables, continues to point to the function and .cfi_jt points to the jump table, and therefore, function_nocfi() returns the raw function address. > > > We could fudge around that by checking the > > > opcode at the target of the call, or token paste ".cfi" after the > > > symbol name in the static_call_update() macro, but it doesn't like > > > like anything is terminally broken tbh. > > > > Agreed, since the jump table entries are actually executable it 'works'. > > > > I really don't like that extra jump though, so I think I really do want > > that nocfi_ptr() thing. And going by: > > > > https://clang.llvm.org/docs/ControlFlowIntegrityDesign.html#forward-edge-cfi-for-indirect-function-calls > > > > and the above, that might be possible (on x86) with something like: > > > > /* > > * Turns a Clang CFI jump-table entry into an actual function pointer. > > * These jump-table entries are simply jmp.d32 instruction with their > > * relative offset pointing to the actual function, therefore decode the > > * instruction to find the real function. > > */ > > static __always_inline void *nocfi_ptr(void *func) > > { > > union text_poke_insn insn = *(union text_poke_insn *)func; > > > > return func + sizeof(insn) + insn.disp; > > } > > > > But really, that wants to be a compiler intrinsic. > > Agreed. We could easily do something similar on arm64, but I'd prefer > to avoid that too. I'll see what we can do. Note that the compiler built-in we previously discussed would have semantics similar to function_nocfi(). It would return the raw function address from a symbol name, but it wouldn't decode the address from an arbitrary pointer, so this would require something different. Sami