Received: by 2002:a05:6a10:5bc5:0:0:0:0 with SMTP id os5csp4529295pxb; Tue, 2 Nov 2021 11:11:27 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx7d9PtD1Uq3dHXlGN3DQtGs2Vxg/b7j+JXyIp6UJ2kqRNf0HuKsYeEpbrg1sUjGyzuyXvO X-Received: by 2002:a5e:8803:: with SMTP id l3mr27799093ioj.217.1635876687377; Tue, 02 Nov 2021 11:11:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1635876687; cv=none; d=google.com; s=arc-20160816; b=aB2XCmRFaT1gOydRae/RGTz6qi9bfnxSHP9LPOpo7wB0wZo6eAPryuG2wLDzEdaL6u YV4Q+xaKPmKQ4P4FbQZuxhtaPAkSHihWQdDOYmkeEWh2TJC6zPKKEwQt7c/hHV53zf6Q FkaOiS/i54Xc22cgsIx97+SUDkJZ4/ethIlB1US64MFachHlgf6/K6tdes8IQGb5VLYP beSrxwFUPoO558xB3d87IT4nVlCtlL9So7YxZ3aKaxtGL5dut8iGCx3yPQ/+IikSfY1Q R5Y7iJZQAw6B5z34hye0h6834ziRSNBmYxMgDFMb5bPa1wq6i6Z9Mmis0LAanSynwSES 1vhg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:in-reply-to:content-disposition:mime-version :references:message-id:subject:cc:to:from:date:dkim-signature; bh=kgD6YYbeofGJLBI9l1G71OKrQRNv5pZP6nuKYTSL9kc=; b=zpPubzN4pxr/yln4/70/qkSQLN4xPMZQZFUwAjtJolozMmUHaX5jtSaRwPF87QHwZi KDp7DwwkvZAV09IlzMJEMMbctZobXUbFfxvdZ5Z4wcxgOKzZFn9diTu8NJGjvAR0bF1G q8pnkaXLE2Q1whtPW3urICQlFDbfUgwG11RFTJjxI5nxzTFjzcm5UNzV4290j3fA9d6E Ws55xGPt8YD/JxucU+WODqwMJ4AGeDUOkK7L/yCPqeJ3AEDvpKEowoKmDhdZfHJ/Lpwg GZngXOJ3NSka4nQwnuEoXH1oi9eGRI/P8XBJ2gM44RHqX2EAjDEZLjrwz8guBXe8b24M 8O8w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=Vt24ScdT; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id u2si3910425jad.53.2021.11.02.11.11.15; Tue, 02 Nov 2021 11:11:27 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@chromium.org header.s=google header.b=Vt24ScdT; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=chromium.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230392AbhKBSMs (ORCPT + 99 others); Tue, 2 Nov 2021 14:12:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36234 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232217AbhKBSMr (ORCPT ); Tue, 2 Nov 2021 14:12:47 -0400 Received: from mail-pl1-x62c.google.com (mail-pl1-x62c.google.com [IPv6:2607:f8b0:4864:20::62c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 78B18C061714 for ; Tue, 2 Nov 2021 11:10:12 -0700 (PDT) Received: by mail-pl1-x62c.google.com with SMTP id p18so14260662plf.13 for ; Tue, 02 Nov 2021 11:10:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=kgD6YYbeofGJLBI9l1G71OKrQRNv5pZP6nuKYTSL9kc=; b=Vt24ScdTjW5lCKq9Nq2BKvZX+xTvcMf8gNvw0urACzABmE/S8n5BEgBg+DviNBjxgI LhFHz/tItrvep184fWXpioxVa/8PEJ55IE6j+Ar+B2P9mOp36YN5lm7CJDu2isGY9lRN 9Zde9MfFGleSGuzEjM50MLDId7fVhGv1ROPjk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=kgD6YYbeofGJLBI9l1G71OKrQRNv5pZP6nuKYTSL9kc=; b=22XvOZQUkRWQVyhZ67K+KUxpJlzuOPiVkbfJMfkRu/13OJXPaXCxvTsMfP5qSy5tCC DMnwakAmHjDu5Jf7Tis1CaicVfWJ+A1ty/wipSarShbyMDmeAo7wX/O6QmMOPPsw/JK1 M0FoGxFDQ0pEBhG9ShGsHFRdBldfBYPnC70t8EGSZXgelF10J6s+zbL1rR6uu9pVTxAP uig4prYxPCXWZ5ALSHyA3Pg2k6DDusNNhWt5E7ZedcC4GFo7xw3l+F+V60eKFtk7f3mc MZqoqcQp+I6c5jmddCJZgkcfPWy67Gc6FWR1b8UKZt6yAkHHFdTlQcG4alLU4ClT/aSZ QysA== X-Gm-Message-State: AOAM5323WEtjPEJ9Bb8xo5z4YRTPCBcnefd3zALElmJwUJDR7EU7JEFW SzdYMGVQWP+v1j9PCYVytBpERQ== X-Received: by 2002:a17:903:1c5:b0:141:fbe2:56c1 with SMTP id e5-20020a17090301c500b00141fbe256c1mr9943924plh.52.1635876611992; Tue, 02 Nov 2021 11:10:11 -0700 (PDT) Received: from www.outflux.net (smtp.outflux.net. [198.145.64.163]) by smtp.gmail.com with ESMTPSA id x9sm16184594pga.28.2021.11.02.11.10.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Nov 2021 11:10:11 -0700 (PDT) Date: Tue, 2 Nov 2021 11:10:10 -0700 From: Kees Cook To: Peter Zijlstra Cc: Ard Biesheuvel , Sami Tolvanen , Mark Rutland , X86 ML , Josh Poimboeuf , Nathan Chancellor , Nick Desaulniers , Sedat Dilek , Steven Rostedt , linux-hardening@vger.kernel.org, Linux Kernel Mailing List , llvm@lists.linux.dev Subject: Re: [PATCH] static_call,x86: Robustify trampoline patching Message-ID: <202111021040.6570189A5@keescook> References: <20211031163920.GV174703@worktop.programming.kicks-ass.net> <20211101090155.GW174703@worktop.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Nov 02, 2021 at 01:57:44PM +0100, Peter Zijlstra wrote: > On Mon, Nov 01, 2021 at 03:14:41PM +0100, Ard Biesheuvel wrote: > > On Mon, 1 Nov 2021 at 10:05, Peter Zijlstra wrote: > > > > How is that not true for the jump table approach? Like I showed earlier, > > > it is *trivial* to reconstruct the actual function pointer from a > > > jump-table entry pointer. > > > > > > > That is not the point. The point is that Clang instruments every > > indirect call that it emits, to check whether the type of the jump > > table entry it is about to call matches the type of the caller. IOW, > > the indirect calls can only branch into jump tables, and all jump > > table entries in a table each branch to the start of some function of > > the same type. > > > > So the only thing you could achieve by adding or subtracting a > > constant value from the indirect call address is either calling > > another function of the same type (if you are hitting another entry in > > the same table), or failing the CFI type check. > > Ah, I see, so the call-site needs to have a branch around the indirect > call instruction. > > > Instrumenting the callee only needs something like BTI, and a > > consistent use of the landing pads to ensure that you cannot trivially > > omit the check by landing right after it. > > That does bring up another point tho; how are we going to do a kernel > that's optimal for both software CFI and hardware aided CFI? > > All questions that need answering I think. I'm totally fine with designing a new CFI for a future option, but blocking the existing (working) one does not best serve our end users. There are already people waiting on x86 CFI because having the extra layer of defense is valuable for them. No, it's not perfect, but it works right now, and evidence from Android shows that it has significant real-world defensive value. Some of the more adventurous are actually patching their kernels with the CFI support already, and happily running their workloads, etc. Supporting Clang CFI means we actually have something to evolve from, where as starting completely over means (likely significant) delays leaving folks without the option available at all. I think the various compiler and kernel tweaks needed to improve kernel support are reasonable, but building a totally new CFI implementation is not: it _does_ work today on x86. Yes, it's weird, but not outrageously so. (And just to state the obvious, CFI is an _optional_ CONFIG: not everyone wants CFI, so it's okay if there are some sharp edges under some CONFIG combinations.) Regardless, speaking to a new CFI design below: > So how insane is something like this, have each function: > > foo.cfi: > endbr64 > xorl $0xdeadbeef, %r10d > jz foo > ud2 > nop # make it 16 bytes > foo: > # actual function text goes here > > > And for each hash have two thunks: > > > # arg: r11 > # clobbers: r10, r11 > __x86_indirect_cfi_deadbeef: > movl -9(%r11), %r10 # immediate in foo.cfi This requires the text be readable. I have been hoping to avoid this for a CFI implementation so we could gain the benefit of execute-only memory (available soon on arm64, and possible today on x86 under a hypervisor). But, yes, FWIW, this is very similar to what PaX RAP CFI does: the caller reads "$dest - offset" for a hash, and compares it against the hard-coded hash at the call site, before "call $dest". > xorl $0xdeadbeef, %r10 # our immediate > jz 1f > ud2 > 1: ALTERNATIVE_2 "jmp *%r11", > "jmp __x86_indirect_thunk_r11", X86_FEATURE_RETPOLINE > "lfence; jmp *%r11", X86_FEATURE_RETPOLINE_AMD > > > > # arg: r11 > # clobbers: r10, r11 > __x86_indirect_ibt_deadbeef: > movl $0xdeadbeef, %r10 > subq $0x10, %r11 > ALTERNATIVE "", "lfence", X86_FEATURE_RETPOLINE > jmp *%r11 > > > > And have the actual indirect callsite look like: > > # r11 - &foo > ALTERNATIVE_2 "cs call __x86_indirect_thunk_r11", > "cs call __x86_indirect_cfi_deadbeef", X86_FEATURE_CFI > "cs call __x86_indirect_ibt_deadbeef", X86_FEATURE_IBT > > Although if the compiler were to emit: > > cs call __x86_indirect_cfi_deadbeef > > we could probaly fix it up from there. It seems like this could work for any CFI implementation, though, if the CFI implementation always performed a call, or if the bounds of the inline checking were known? i.e. objtool could also find the inline checks just as well as the call, though? > Then we can at runtime decide between: > > {!cfi, cfi, ibt} x {!retpoline, retpoline, retpoline-amd} This does look pretty powerful, but I still don't think it precludes using the existing Clang CFI. I don't want perfect to be the enemy of good. :) -- Kees Cook