Received: by 2002:a6b:500f:0:0:0:0:0 with SMTP id e15csp1883901iob; Thu, 5 May 2022 10:03:34 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxfm09awvL8/MnclXZYhdODxaawwQAlOOzFjOtBdwCE44JxZukHi/OVWQnMLnVL4IvJkoI9 X-Received: by 2002:a17:907:da7:b0:6f4:ed59:c4e2 with SMTP id go39-20020a1709070da700b006f4ed59c4e2mr5609842ejc.83.1651770214525; Thu, 05 May 2022 10:03:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1651770214; cv=none; d=google.com; s=arc-20160816; b=c7duLoNTH1CpfamPV7fWqA3jhWC+Vks8rQKuAHeMWTPP3UL8/A/4b2NAvIONVTqiu0 caMGlxSOqBDZ5aMhPi7vFhezDfQvRqhPBaB5fcjRODiJvlIyUvqeENIVuWAevaOnRW5f as6EJWVlPjZV6IPPwCeuzGXcVq02RR/98QOJbV6GNdCDVpbVtJDpHKTVsf3Pq/AXkWha 9BFfNyklYuDhznxkJGD/7vNi0JF0og0CkUBld3P8aa4BFNeiWPZsJ59MBoSNoWAJ8bO7 bnaHRtf8lZk4u6OtY6zDUhBfQvop/Gkeftg91yqrA+XJv5P1lRd7N7uKldfXF5UtSdMw RC5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:in-reply-to:content-disposition:mime-version :references:message-id:subject:cc:to:from:date:dkim-signature; bh=iQi/ZFhqTl+9HgfZab97oEqJSaMHvbvR0mYRqBC6N6U=; b=QpeDFWht6EbSJz2VzeTKVvxDKvs/BTb0XUCIYQHbRdVpetcOZklEsT2WtYO0iE1n75 vs2tXBgz+nTIac6cOqDy1Xt1xl1JMwfHM1y5giaccREYhj/RkBrNDbN1nd9kRcYFTggZ r3bVAoBHpCt5T6dkQWWQKSVInYhphVfLsGxb3o8nJwZugCTlBw2N5gz24YdGwcA8nf8y fl+cXsaCHyFgvcedogDgRtiacPbB5oJkS5HFrytPAF0otYMYhN08yhcUsqL2ZbrsZfgD 9SABvbCdo0nHvn06cF4ZJJECmcPYtz3ety83fJSc0ogjVD3CWkVTZ5oN2xyspm77EfkA u7Qw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b="cbmCz1/2"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id fp31-20020a1709069e1f00b006e86856a2c0si2497593ejc.704.2022.05.05.10.03.09; Thu, 05 May 2022 10:03:34 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b="cbmCz1/2"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243460AbiECWG7 (ORCPT + 99 others); Tue, 3 May 2022 18:06:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48856 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238803AbiECWG5 (ORCPT ); Tue, 3 May 2022 18:06:57 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 62C70424AA for ; Tue, 3 May 2022 15:03:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1651615402; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=iQi/ZFhqTl+9HgfZab97oEqJSaMHvbvR0mYRqBC6N6U=; b=cbmCz1/23gXu5CJhxe9LU6KLoBDYlTRZycdZULh6hxDuDOUPJeiyvxHQjkI5Xyc1vKmuUn o4d9F9jApcD8haJBLeu9z2MzzmtW6K1cR/o1Eb9rkPLTOkJVvS4UxDMSj6Vo2uCAjkC526 ssiW07bCXuEZh47d9pd4tNkkrH0NBfI= Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-658-lIaRym7YPzKF4a_BwWfWKA-1; Tue, 03 May 2022 18:02:54 -0400 X-MC-Unique: lIaRym7YPzKF4a_BwWfWKA-1 Received: by mail-qv1-f72.google.com with SMTP id 11-20020a0562140d0b00b0045aac32023fso1712546qvh.19 for ; Tue, 03 May 2022 15:02:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=iQi/ZFhqTl+9HgfZab97oEqJSaMHvbvR0mYRqBC6N6U=; b=EbnbTMQgf1CV5KKarIhODuDbHT703KgVnmBb/vf9KOVEl212NI4zR1yEtoLrAp2P7f 075hRPkHB73up8po3o6iNlft64Y92zTF+6UACz+Rlk59yImZyfSDQrNaF6DFo6vcOXzn zVVQWtRNljl42tAJRZchlcrQpZ4iuqC8bZg6oZ50MFUjJ3IAWPx7AR6uUT33ljzpFSHg eWYIfZurrNPduSi99pQB51KfD7GT+HFO55Vi3SQpXTv6cI1b3+QKObnzkzyQcz4gVwOH jdADLCviKYLeXHIYoF3oxrwrHjKwBs9tbsn91BL/NMAaC2Oqw+JBR3k56JcEVSY5Tap9 Urhw== X-Gm-Message-State: AOAM530JSSW7zK2tOX7NshByE0nCOyPDMklv/Khxcy3pxJ81w/rAAHq8 HareVZx96UiIdYqApCZrdAY95LLfheKk54z6UHdb90sukDYAUyirTpo4Z9YXyrIgBJbJZ18beM/ xE3IC6+CReuLeJ3Gek9TZlWqK X-Received: by 2002:a05:622a:14c6:b0:2e2:3f:f938 with SMTP id u6-20020a05622a14c600b002e2003ff938mr17023692qtx.358.1651615368204; Tue, 03 May 2022 15:02:48 -0700 (PDT) X-Received: by 2002:a05:622a:14c6:b0:2e2:3f:f938 with SMTP id u6-20020a05622a14c600b002e2003ff938mr17023656qtx.358.1651615367925; Tue, 03 May 2022 15:02:47 -0700 (PDT) Received: from treble ([2600:1700:6e32:6c00::a]) by smtp.gmail.com with ESMTPSA id 141-20020a370693000000b0069ff862c84esm2671595qkg.3.2022.05.03.15.02.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 03 May 2022 15:02:47 -0700 (PDT) Date: Tue, 3 May 2022 15:02:44 -0700 From: Josh Poimboeuf To: Joao Moreira Cc: linux-kernel@vger.kernel.org, linux-hardening@vger.kernel.org, peterz@infradead.org, andrew.cooper3@citrix.com, keescook@chromium.org, samitolvanen@google.com, mark.rutland@arm.com, hjl.tools@gmail.com, alyssa.milburn@linux.intel.com, ndesaulniers@google.com, gabriel.gomes@linux.intel.com, rick.p.edgecombe@intel.com Subject: Re: [RFC PATCH 01/11] x86: kernel FineIBT Message-ID: <20220503220244.vyz5flk3gg3y6rbw@treble> References: <20220420004241.2093-1-joao@overdrivepizza.com> <20220420004241.2093-2-joao@overdrivepizza.com> <20220429013704.4n4lmadpstdioe7a@treble> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-3.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, May 02, 2022 at 10:17:42AM -0700, Joao Moreira wrote: > > > + asm("\t movq 0x90(%%rsp),%0" : "=r"(ret)); > > > + asm("\t movq 0x98(%%rsp),%0" : "=r"(caller)); > > > > This is making some questionable assumptions about the stack layout. > > > > I assume this function is still in the prototype stage ;-) > > Yeah, this is just a messy instrumentation to get reports about mismatching > prototypes (as the ones reported further down the series). > > The issue with having the call is that it bloats the binary, so the ud2 is > 3-bytes-per-function better. Yet, we may consider a FINEIBT_DEBUG config, > which can then enable a handler. This would be useful together with a fuzzer > or a stress tool to cover possible control-flow paths within the kernel and > map mismatching prototypes more properly I guess. It should be possible to have a non-fatal #UD2 handler. See for example how WARN() is implemented with __WARN_FLAGS in arch/x86/include/asm/bug.h . So hopefully we can just get rid of the need for the "call handler" thing altogether. > > Not sure what would happen for "ibt=off"? Maybe apply_ibt_endbr() could > > NOP out all the FineIBT stuff. > > Either that, or... > > I'm thinking about a way to have FineIBT interchangeable with KCFI. > Currently KCFI adds a 4 byte hash + 2 byte nops before function entry, to > allow for proper prototype checking. After that, there should be an ENDBR of > 4 bytes. This gives us 10 bytes in total. Then, my yet to be properly > thought idea would be patch these 10 bytes with: > > endbr > call fineibt_handler_<$HASH> > nop > > and then, on the caller side, patch the "cmp <$HASH>, -0x6(%r11); je; ud2; > call" sequence with a "sub 0x6, r11; mov $HASH, %r10; call %r11, add 0x6 > %r11". This would then allow the kernel to verify if the CPU is IBT capable > on boot time and only then setting the proper scheme. > > The downsides of having something like this would be that this sub r11/add > r11 sequence is kinda meh. We can avoid that by having two padding nops > after the original ENDBR, which will be skipped when the function is reached > directly by the linker optimization I'm working on, and that we can convert > into a JMP -offset that makes control flow reach the padding area before the > prologue and from where we can call the fineibt_handler function. The > resulting instrumentation would be something like: > > 1: > call fineibt_handler_<$HASH> > jmp 2f > > endbr > jmp 1b > 2: > > Also, it would prevent a paranoid user to have both schemes simultaneously > (there are reasons why people could want that). > > Any thoughts? I'm not really qualified to comment on this too directly since I haven't looked very much at the variations on FineIBT/CFI/KCFI, and what the protections and drawbacks are for each approach, and when it might even make sense to combine them for a "paranoid user". Since we have multiple similar and possibly competing technologies being discussed, one thing I do want to warn against is that we as kernel developers tend to err on the side of giving people too many choices and combinations which *never* get used. All those unused options can confuse the user and significantly add to complexity and maintenance overhead for us. Especially for invasive features like these. (Just to be clear, I'm not saying that's happening here, but it's something we need to be careful about.) Here, documentation is going to be crucial, for both reviewers and users. Something that describes when/why I should use X or Y or X+Y. If we truly want to add more options/combos for different use cases then we'll also need clear and concise documentation about which options/combos would be used under what circumstances. -- Josh