Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <d6555f27-95bf-5474-3006-6f8d399ab556@intel.com>
Date:   Fri, 25 Nov 2022 21:44:29 +0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Firefox/102.0 Thunderbird/102.5.0
Subject: Re: [PATCH bpf v2] selftests/bpf: Fix "missing ENDBR" BUG for
 destructor kfunc
Content-Language: en-US
To:     Peter Zijlstra <peterz@infradead.org>,
        Jiri Olsa <olsajiri@gmail.com>
CC:     <jpoimboe@kernel.org>, <memxor@gmail.com>, <bpf@vger.kernel.org>,
        "Pengfei Xu" <pengfei.xu@intel.com>,
        Alexei Starovoitov <ast@kernel.org>,
        "Daniel Borkmann" <daniel@iogearbox.net>,
        Andrii Nakryiko <andrii@kernel.org>,
        "Martin KaFai Lau" <martin.lau@linux.dev>,
        Song Liu <song@kernel.org>, Yonghong Song <yhs@fb.com>,
        John Fastabend <john.fastabend@gmail.com>,
        KP Singh <kpsingh@kernel.org>,
        Stanislav Fomichev <sdf@google.com>,
        Hao Luo <haoluo@google.com>,
        "David S. Miller" <davem@davemloft.net>,
        Eric Dumazet <edumazet@google.com>,
        Jakub Kicinski <kuba@kernel.org>,
        Paolo Abeni <pabeni@redhat.com>,
        <linux-kernel@vger.kernel.org>, <netdev@vger.kernel.org>,
        "Chen, Hu1" <hu1.chen@intel.com>
References: <20221122073244.21279-1-hu1.chen@intel.com>
 <Y3zTF0CjQFt/dR2M@krava> <Y3zZQpHNQ8cRjKQY@hirez.programming.kicks-ass.net>
From:   "Chen, Hu1" <hu1.chen@intel.com>
In-Reply-To: <Y3zZQpHNQ8cRjKQY@hirez.programming.kicks-ass.net>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?eHRMREVHMklVeVB1RStLM0txazRDbGV0cS9HWnM5UE05Q055ZzB6REFMRm9v?=
 =?utf-8?B?SHE1cEFGZW5DKzBNby8vZisxZXVhMmc3NHRiK2hEUjFLelJ3bXZCdXR4Snp6?=
 =?utf-8?B?YWVpWjdqMzFZZW4zQWV6Zm9obFNCbEFTTTF4QWgvQ1NTWEFqSlBoaHp4djkw?=
 =?utf-8?B?WThxNkRhZTJwN3pvYnU4QmhkWmZ6KzJFeVFlTkV4RFI2Nm1XejBrT2VBaTE3?=
 =?utf-8?B?UGd6eitRWUpCNEhBTThhdFRpR0xoNXJtZmJQL0RnY09kS3IwZlJwVkRBalFM?=
 =?utf-8?B?Zng5Zkc5UVE1b3cyV2hBQUhGMVQyeXRXNEpabFNTOUVYcHNSRFpvSjhsdTN2?=
 =?utf-8?B?SUVOY3BRM01Yc1Q0RlFWenY1NEdXL1NxUFhLWk5MeVhDUXVwU0NrdlI5Rldh?=
 =?utf-8?B?dXhtR1A5anNRZ0ZGYnV4NlZTWFpTWXJkNElKZENab2VYd2VUM04rTk5GaHN6?=
 =?utf-8?B?cGN0Z0xKZ0UxQVRITHRXRVJlSXlvM1FNU0pGTHJ3azFnLzh6NWljYW0vd1p2?=
 =?utf-8?B?ZVlUb0xqbWluRDRaUjBWdmZXMnpXK2JJOG1vdTBtZGNyOXlnbk1Ha2x6VFhs?=
 =?utf-8?B?M2JodkF0S0FJdTBzdEJMUEVveitKWFFyVjBhNkNGT2hjZVQ4N1VvSlM3RUVn?=
 =?utf-8?B?TmtHeVoyNmg1N0s2S3dLbTF4ZnpjNkpJSTBTKzY2RHJzcW5rSEQ0Sll6NEkw?=
 =?utf-8?B?TEdORVp1U2tudzVkYTRpTEhWV0RIWFVFckNseXFwRGZ0WXNGd09Db2M2UkJ4?=
 =?utf-8?B?NlQyNHBuTkZiREZyaXBDUUNFMEt2SU1BZlFRZjNZT0RoRU12czNpR0padTd1?=
 =?utf-8?B?RVNCeFB4SFdOSGdmSWxVcWlNOWRWTXhJUzR4MXltcHdoYmZDU2tjRzVOcStH?=
 =?utf-8?B?YnZhdFdGejBjblhxN0RtZU8vRVpOdExLR1Fiejd3TzZkQW4rSU82bS9OVkZq?=
 =?utf-8?B?QTVkSDZ5TngzTGx2WHdLSzNWMVpKZGJBOStua2oxa3hPMTAzU20rRGRzSnAz?=
 =?utf-8?B?RUc3RkNXZkxxalFJOEFoTGk2TjIxU2tQSG1ENFNhWlBtWWk5N2lUc2VtK05r?=
 =?utf-8?B?TlB3SFhxR0JzTFJTUUhZd0NjMW9nZmJxcmZmUGhKeGt0SllUSzlkdFR0TzU3?=
 =?utf-8?B?OHY0Rk1lYUdJbzFkZGlGeDNvVlBHZUZMZ0RZRmkxOFJENUVSS1B1VU9oVlRM?=
 =?utf-8?B?aGpzTEMrYnd0SEt2VUVGMnlJdURwcE1BMHlOQ1FldXUvbDBnZGxLTmllZ0s3?=
 =?utf-8?B?NE5XemNDYmZFTU1GTTV6K1MxcVZZZFgrQW9Ma1dzRTFIRktCK3FMSlVaUmVm?=
 =?utf-8?B?ZndqeXVvSzIwSGEvTi9EZUhReDRsS25aWFR2VkNGK0YvbktSbzdmSllXQUdK?=
 =?utf-8?B?VVdRejRMOTFvcDJWMnFZRDlOUUl1R1FJcUNDcWJQbnZCdzVRTHZlR2VQdkhv?=
 =?utf-8?B?bVFhVzBLVGVPaGJNa1gwVDBISHFWSWJZRUdsVzY1WkhZRlQ0VlljNUMyZDVE?=
 =?utf-8?B?Wko3Q0dvbTY5bU9DUW0zVkhKNERlcFp2Q2k5azZ1SXhEc1JVV042NWNmUCtr?=
 =?utf-8?B?aFdGcDA2TzE5eFoyL2o0cDkwQXU2UzhvWjRVZDZpbllKSGFlTDZST0N4WC9z?=
 =?utf-8?B?c0xDdWVEZEc4Rk04eEhIV0JDTVRURDBBQnFrNm5IdWNmdGxvaC9NWitLbER2?=
 =?utf-8?B?OEdjWXBENTdpYXFDY0NsNmNCOTI3V2x0R0dUVG5TQWxsR1RtekVCVVE4MnBj?=
 =?utf-8?B?dlJ6aUlHTWVCTUR1VzNTNXJiQ1JzenpiOXc5WGJFbkpyVU96Nm5NOHArc3Jz?=
 =?utf-8?B?WElCRUVQL3hybXNpYVhELy8xZ1RLcGxtdnc3REo4Nk9HNDZwRUVLc3hYUVo0?=
 =?utf-8?B?UlQ4ckRyVXFLaS9sbnlQNzZoNWdJaFJSbTA5OU5QTW1mNHNlUzEySDZ2Ym5l?=
 =?utf-8?B?cXRTYVZudzFja3MxUHpZcGlmVk96RVJoWkVqdGtNY1BmbTd1MkpsVE83R1hu?=
 =?utf-8?B?Y3I4RTNyREJuSHpPZVZkR01jUUZxNmVPR2JGSlBlVlVMVUc5bDYyanZ1TFdy?=
 =?utf-8?B?cTdkdE95SlIwVXlmbzB1czYvRjFqeXpaTU9vMWozbFlZU1BTSlphbkRDSkRn?=
 =?utf-8?Q?9e9LBEZQiQ+UkMeoaLxzfexMq?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 88694d8d-54fc-4505-1dc9-08daceeb3418
X-MS-Exchange-CrossTenant-AuthSource: BY5PR11MB4449.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Nov 2022 13:44:41.5478
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: B4ehlrhFr5SJ9PvV/YLe0taMkyDrHUfeC+1LBzlCO5aBcJ75UNodqP9KpLRlbanVeRk4r6WFr9eK1iHMX7DH4w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR11MB7175
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,
        RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_NONE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 11/22/2022 10:14 PM, Peter Zijlstra wrote:
> On Tue, Nov 22, 2022 at 02:48:07PM +0100, Jiri Olsa wrote:
>> On Mon, Nov 21, 2022 at 11:32:43PM -0800, Chen Hu wrote:
>>> With CONFIG_X86_KERNEL_IBT enabled, the test_verifier triggers the
>>> following BUG:
>>>
>>>   traps: Missing ENDBR: bpf_kfunc_call_test_release+0x0/0x30
>>>   ------------[ cut here ]------------
>>>   kernel BUG at arch/x86/kernel/traps.c:254!
>>>   invalid opcode: 0000 [#1] PREEMPT SMP
>>>   <TASK>
>>>    asm_exc_control_protection+0x26/0x50
>>>   RIP: 0010:bpf_kfunc_call_test_release+0x0/0x30
>>>   Code: 00 48 c7 c7 18 f2 e1 b4 e8 0d ca 8c ff 48 c7 c0 00 f2 e1 b4 c3
>>> 	0f 1f 44 00 00 66 0f 1f 00 0f 1f 44 00 00 0f 0b 31 c0 c3 66 90
>>>        <66> 0f 1f 00 0f 1f 44 00 00 48 85 ff 74 13 4c 8d 47 18 b8 ff ff ff
>>>    bpf_map_free_kptrs+0x2e/0x70
>>>    array_map_free+0x57/0x140
>>>    process_one_work+0x194/0x3a0
>>>    worker_thread+0x54/0x3a0
>>>    ? rescuer_thread+0x390/0x390
>>>    kthread+0xe9/0x110
>>>    ? kthread_complete_and_exit+0x20/0x20
>>>
>>> This is because there are no compile-time references to the destructor
>>> kfuncs, bpf_kfunc_call_test_release() for example. So objtool marked
>>> them sealable and ENDBR in the functions were sealed (converted to NOP)
>>> by apply_ibt_endbr().
> 
> If there is no compile time reference to it, what stops an LTO linker
> from throwing it out in the first place?
>

Ah, my stupid.

The only references to this function from kernel space are:
    $ grep -r bpf_kfunc_call_test_release
    net/bpf/test_run.c:noinline void bpf_kfunc_call_test_release(struct prog_test_ref_kfunc *p)
    net/bpf/test_run.c:BTF_ID_FLAGS(func, bpf_kfunc_call_test_release, KF_RELEASE)
    net/bpf/test_run.c:BTF_ID(func, bpf_kfunc_call_test_release)

Macro BTF_ID_... puts the function names to .BTF_ids section. It looks
like:
__BTF_ID__func__bpf_kfunc_call_test_release__692

When running, it uses kallsyms_lookup_name() to find the function
address via names in .BTF_ids section.


Hi jirka,
Please kindly correct me if my understanding of BTF_ids is wrong.