Received: by 2002:ad5:4acb:0:0:0:0:0 with SMTP id n11csp2997290imw; Sun, 17 Jul 2022 23:08:59 -0700 (PDT) X-Google-Smtp-Source: AGRyM1t9/rNI8ihFnCasAsa0sPxsbkzDSzNayrH6HvQ2lGukKykXcES9n0jsy5st+kFqea9US3dj X-Received: by 2002:a17:907:1c87:b0:6f0:29ea:cc01 with SMTP id nb7-20020a1709071c8700b006f029eacc01mr24893718ejc.671.1658124539674; Sun, 17 Jul 2022 23:08:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1658124539; cv=none; d=google.com; s=arc-20160816; b=Z8ba3oLbRfvfvWFBTM38j+OtG8YCN5a1FXX+bEm/JHa6OJvjlk8SM07Bo6hK6w6kex ejGDDWpjNjO+6tITNt9h6/IfTinwvBrSatP3vTsT0dNU99yfD9tmFqj3tCSrUN30dn64 CQymY8iJgIT6hhEQu1nBBEXLC3obhr9ZJgG3CczJcErr2snU+IpZIeVXPw3VJ5oVPmdU UW99s8rpWXIozT7rDDZKb6yyjXyFyUGMiRCPZ+lxSBBeRN0o1feK6MPt50+Rpuukmt1J FOxi/YS6kS90+/0q4Khjof93O00nVke8/zYsTNzn2b/o7HwZYUyc9UpZ70q4jAEvSMlz Vlag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=ZF9V6gFHNy+DlDhVCq9uRK7pJZ2p7JHGRKT0MLeDEgw=; b=wqZ46AzKPPVkfyPR0slCm8HYx/1mVTS7tKShcI/EN0GSGaLG611l5rG8CcjNZXrLaL Ozr8vDoV/i27PZwMz+NvwHguuCw7RYjFuPA3FjVdtneAG/omJOtCH0fU8Etpd3wk6K8U qJvaIcTgDHe74SnAY7wMTJrTTZ9/fRgOM2b8x36WdJCrhQsBd8CCw3u8fm1+wKozGLKX x5TP+SPqEbZOyLpvNkAtZOPril1MKDAaoSsQwhFthSOfIUJr/FdivoIPAsKXHUhNu6MC kdxMqBZ0h6LUafonPlK2M5fMJPIj9OkN3b5vbilE2WA8ehxYSTfg/hrgTFK2Snp4STb5 wSWQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id dt12-20020a170907728c00b0072b5895f57bsi15257221ejc.12.2022.07.17.23.08.33; Sun, 17 Jul 2022 23:08:59 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233446AbiGRFzQ convert rfc822-to-8bit (ORCPT + 99 others); Mon, 18 Jul 2022 01:55:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57640 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233317AbiGRFzM (ORCPT ); Mon, 18 Jul 2022 01:55:12 -0400 Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AECC110FEE for ; Sun, 17 Jul 2022 22:55:11 -0700 (PDT) Received: from pps.filterd (m0044010.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 26I0E1DD026860 for ; Sun, 17 Jul 2022 22:55:11 -0700 Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com (PPS) with ESMTPS id 3hbrnqf36w-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Sun, 17 Jul 2022 22:55:11 -0700 Received: from twshared30313.14.frc2.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.28; Sun, 17 Jul 2022 22:55:08 -0700 Received: by devbig932.frc1.facebook.com (Postfix, from userid 4523) id 87035A4C01FF; Sun, 17 Jul 2022 22:54:57 -0700 (PDT) From: Song Liu To: , , CC: , , , , Song Liu Subject: [PATCH v4 bpf-next 3/4] bpf, x64: Allow to use caller address from stack Date: Sun, 17 Jul 2022 22:54:48 -0700 Message-ID: <20220718055449.3960512-4-song@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220718055449.3960512-1-song@kernel.org> References: <20220718055449.3960512-1-song@kernel.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8BIT X-FB-Internal: Safe Content-Type: text/plain X-Proofpoint-GUID: xQv5o42ASPKG6fLfeWijuSW0yvAjToGr X-Proofpoint-ORIG-GUID: xQv5o42ASPKG6fLfeWijuSW0yvAjToGr X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-07-18_04,2022-07-15_01,2022-06-22_01 X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Jiri Olsa Currently we call the original function by using the absolute address given at the JIT generation. That's not usable when having trampoline attached to multiple functions, or the target address changes dynamically (in case of live patch). In such cases we need to take the return address from the stack. Adding support to retrieve the original function address from the stack by adding new BPF_TRAMP_F_ORIG_STACK flag for arch_prepare_bpf_trampoline function. Basically we take the return address of the 'fentry' call: function + 0: call fentry # stores 'function + 5' address on stack function + 5: ... The 'function + 5' address will be used as the address for the original function to call. Signed-off-by: Jiri Olsa Signed-off-by: Song Liu --- arch/x86/net/bpf_jit_comp.c | 13 +++++++++---- include/linux/bpf.h | 5 +++++ 2 files changed, 14 insertions(+), 4 deletions(-) diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c index 54c7f46c453f..e1b0c5ed0b7c 100644 --- a/arch/x86/net/bpf_jit_comp.c +++ b/arch/x86/net/bpf_jit_comp.c @@ -2119,10 +2119,15 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i if (flags & BPF_TRAMP_F_CALL_ORIG) { restore_regs(m, &prog, nr_args, regs_off); - /* call original function */ - if (emit_call(&prog, orig_call, prog)) { - ret = -EINVAL; - goto cleanup; + if (flags & BPF_TRAMP_F_ORIG_STACK) { + emit_ldx(&prog, BPF_DW, BPF_REG_0, BPF_REG_FP, 8); + EMIT2(0xff, 0xd0); /* call *rax */ + } else { + /* call original function */ + if (emit_call(&prog, orig_call, prog)) { + ret = -EINVAL; + goto cleanup; + } } /* remember return value in a stack for bpf prog to access */ emit_stx(&prog, BPF_DW, BPF_REG_FP, BPF_REG_0, -8); diff --git a/include/linux/bpf.h b/include/linux/bpf.h index a5bf00649995..7496842a4671 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -751,6 +751,11 @@ struct btf_func_model { /* Return the return value of fentry prog. Only used by bpf_struct_ops. */ #define BPF_TRAMP_F_RET_FENTRY_RET BIT(4) +/* Get original function from stack instead of from provided direct address. + * Makes sense for trampolines with fexit or fmod_ret programs. + */ +#define BPF_TRAMP_F_ORIG_STACK BIT(5) + /* Each call __bpf_prog_enter + call bpf_func + call __bpf_prog_exit is ~50 * bytes on x86. */ -- 2.30.2