Received: by 2002:a05:6a10:af89:0:0:0:0 with SMTP id iu9csp4933357pxb; Wed, 26 Jan 2022 00:38:20 -0800 (PST) X-Google-Smtp-Source: ABdhPJx+XCpIekQzjEUigj0dhitmyfhlIypdMBx6Ci35t8Dal9VPKDUvAQiwzvLkC0vTvikr7d3U X-Received: by 2002:a17:907:72cb:: with SMTP id du11mr10876102ejc.161.1643186300652; Wed, 26 Jan 2022 00:38:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1643186300; cv=none; d=google.com; s=arc-20160816; b=CIBnRY5A74U4Oyo7R8beK8ttzNZlYrBY8dbUGowtdkkfzqENq9OPK7OtaV2FHTAsmm szj4mI8MVI+R2esvsTvRpvl6qpKhG0DEhB4uYrT3tDfp+hdhvWVt+N0ejIbhzj8RPlUz IN2oelmF9245VBI2Lz8bQzO29iSQPzzzWSsPIZm11WUV1w3EzuKm3ZCkOK8OaenYllja aCx5IWhCHpZP6aujrPImnGEsB6vGJW6uHXla/B6BeV44Wn4823N55/v/2XMge9dN7xh/ xwZoJ7ufG7R4ForAz/+gzvtG7hlzuMWc6MlcoxXucHOZQ1+rDNc8GGMlDjOIPwQMz5fK N2NA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:subject:message-id:date:from:in-reply-to :references:mime-version:dkim-signature; bh=9X2ZcgdBzzi0ZX9fvwZzZ7gaHC1Kk1vB/k5NLqJ0IPE=; b=J/7nPVNLm2tk13WLru8+lwmUw0LAh9+XYnGVvZB0szNOaNTej7j8JK5tfeCk3ZMExb N/srBo7EHr+hQ1+jcUKhl1NagLEIqMuWIx+0vrnxaNbRkVIE8oAcupPJ7oq5PVUmytJf kQ5uqNCWtaTqDckaNAEJLwk+Y8cwxOR+V5b0V/fk7M9WkP9qHNQYV4ElMEG2k7vkf36b zM9Oz1DpgjlWGziMYZoaS6N79L6tmwhvp+if88s2lKohZoPra+5TR9Rhp1nTT8BTvWQm wnYm7nQqu5jCfA1SDmoUcYuxU4p92I7u4oOr2Yw+h0uSoqP8WZYs7AtdsQ7Zg4glCqvh n+oQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=DUKdgoAM; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id r14si7284271edw.155.2022.01.26.00.37.55; Wed, 26 Jan 2022 00:38:20 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=DUKdgoAM; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231243AbiAYUAI (ORCPT + 99 others); Tue, 25 Jan 2022 15:00:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57790 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230488AbiAYUAD (ORCPT ); Tue, 25 Jan 2022 15:00:03 -0500 Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0FDA2C06173B; Tue, 25 Jan 2022 12:00:03 -0800 (PST) Received: by mail-pl1-x636.google.com with SMTP id x11so14618548plg.6; Tue, 25 Jan 2022 12:00:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=9X2ZcgdBzzi0ZX9fvwZzZ7gaHC1Kk1vB/k5NLqJ0IPE=; b=DUKdgoAMoX+gtYjEAzMZvB6j2w8MF8dH1MImCtcnwgmo4ok+D3IhwYUWWo41gCtacw SqKchHEFlU7KIEM++Oxkss8hAnEsIHWbsdXvCSfeRk3hoWOk9bcN41OtdXSghcUWW1EH x0CjZap4fUmmdqu9rxgRMDC58TkhpfoCzgNIN2jejSmnzf1JOXxJoVu5sjYlLE5eFUvT 1FcCxdpWIe+P3iD8c/yngW0paL2qIPLAX/E2PYFwtlgL1e0gB1NPN8w6Qfnf3K3D62Xk /Ls6Rj0O8QQa4pHCHZ8HZqBfLOYx7CEn3w35ZvQZ7LGLviQ5sQ/L/nCnA2QNmIzGOEo/ bZkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=9X2ZcgdBzzi0ZX9fvwZzZ7gaHC1Kk1vB/k5NLqJ0IPE=; b=PIOycowuUSqBI3fM8Ei09+YDTFRcFmkQv9kaHoOfrMJT137Wz0+2veWTuBsj2ZwW7/ evD0bNaiHnS5yGDTULKg39uVwSthMc8xxC6x5WUWpEgNnEcsXsCn7C6zIoCAR/Mlzfq1 6jTWRp25JH7K9asHFMf2e0mMWrR9wNGsXw6cabQktBl8YSKSwBjXFzdvUCKiC8PoiK3G rmq9HdRdd7TX4iQUL+l52tKrSwF7Uo3CTF5bnRZpLr+ARCPO2yFoC2XnXURXtu8/qaZS F4nqPd2TsIGNfT8qedur3EQBzubNb7hzz+1aWCMRgDO81HI3GKHknmvq106XHRqgQhoX omPw== X-Gm-Message-State: AOAM532I25ggLT5yBYk4fZeYr33HIyJBRykMMOBLvAWwcilOMZA2tq34 X8xmyJPHNWzH7pVQOhi+xx1EUgylX7SqlHe7VFM= X-Received: by 2002:a17:902:cec6:b0:14b:81e8:ad1b with SMTP id d6-20020a170902cec600b0014b81e8ad1bmr2198226plg.116.1643140802319; Tue, 25 Jan 2022 12:00:02 -0800 (PST) MIME-Version: 1.0 References: <20220121194926.1970172-1-song@kernel.org> <20220121194926.1970172-7-song@kernel.org> <7393B983-3295-4B14-9528-B7BD04A82709@fb.com> <5407DA0E-C0F8-4DA9-B407-3DE657301BB2@fb.com> <5F4DEFB2-5F5A-4703-B5E5-BBCE05CD3651@fb.com> <5E70BF53-E3FB-4F7A-B55D-199C54A8FDCA@fb.com> <2AAC8B8C-96F1-400F-AFA6-D4AF41EC82F4@fb.com> In-Reply-To: From: Alexei Starovoitov Date: Tue, 25 Jan 2022 11:59:50 -0800 Message-ID: Subject: Re: [PATCH v6 bpf-next 6/7] bpf: introduce bpf_prog_pack allocator To: Song Liu Cc: Song Liu , Ilya Leoshkevich , bpf , Network Development , LKML , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Kernel Team , Peter Zijlstra , X86 ML Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Jan 24, 2022 at 11:21 PM Song Liu wrote: > > On Mon, Jan 24, 2022 at 9:21 PM Alexei Starovoitov > wrote: > > > > On Mon, Jan 24, 2022 at 10:27 AM Song Liu wrote: > > > > > > > > Are arches expected to allocate rw buffers in different ways? If not, > > > > I would consider putting this into the common code as well. Then > > > > arch-specific code would do something like > > > > > > > > header = bpf_jit_binary_alloc_pack(size, &prg_buf, &prg_addr, ...); > > > > ... > > > > /* > > > > * Generate code into prg_buf, the code should assume that its first > > > > * byte is located at prg_addr. > > > > */ > > > > ... > > > > bpf_jit_binary_finalize_pack(header, prg_buf); > > > > > > > > where bpf_jit_binary_finalize_pack() would copy prg_buf to header and > > > > free it. > > > > It feels right, but bpf_jit_binary_finalize_pack() sounds 100% arch > > dependent. The only thing it will do is perform a copy via text_poke. > > What else? > > > > > I think this should work. > > > > > > We will need an API like: bpf_arch_text_copy, which uses text_poke_copy() > > > for x86_64 and s390_kernel_write() for x390. We will use bpf_arch_text_copy > > > to > > > 1) write header->size; > > > 2) do finally copy in bpf_jit_binary_finalize_pack(). > > > > we can combine all text_poke operations into one. > > > > Can we add an 'image' pointer into struct bpf_binary_header ? > > There is a 4-byte hole in bpf_binary_header. How about we put > image_offset there? Actually we only need 2 bytes for offset. > > > Then do: > > int bpf_jit_binary_alloc_pack(size, &ro_hdr, &rw_hdr); > > > > ro_hdr->image would be the address used to compute offsets by JIT. > > If we only do one text_poke(), we cannot write ro_hdr->image yet. We > can use ro_hdr + rw_hdr->image_offset instead. Good points. Maybe let's go back to Ilya's suggestion and return 4 pointers from bpf_jit_binary_alloc_pack ? > > rw_hdr->image would point to kvmalloc-ed area for emitting insns. > > rw_hdr->size would already be populated. > > > > The JITs would write insns into rw_hdr->image including 'int 3' insns. > > At the end the JIT will do text_poke_copy(ro_hdr, rw_hdr, rw_hdr->size); > > That would be the only copy that will transfer everything into final > > location. > > Then kvfree(rw_hdr) > > The only problem is the asymmetry of allocating rw_hdr from bpf/core.c, > and freeing it from arch/bpf_jit_comp.c. But it doesn't bother me too much. Indeed. Asymmetry needs to be fixed. Let's then pass 4 pointers back into bpf_jit_binary_finalize_pack() which will call arch dependent weak function to do text_poke_copy or use default __weak function that returns eopnotsupp and then kvfree the rw_hdr ? I'd like to avoid callbacks. imo __weak is easier to follow.