Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <20220121194926.1970172-1-song@kernel.org> <20220121194926.1970172-7-song@kernel.org>
 <CAADnVQK6+gWTUDo2z1H6AE5_DtuBBetW+VTwwKz03tpVdfuoHA@mail.gmail.com>
 <7393B983-3295-4B14-9528-B7BD04A82709@fb.com> <CAADnVQJLHXaU7tUJN=EM-Nt28xtu4vw9+Ox_uQsjh-E-4VNKoA@mail.gmail.com>
 <5407DA0E-C0F8-4DA9-B407-3DE657301BB2@fb.com> <CAADnVQLOpgGG9qfR4EAgzrdMrfSg9ftCY=9psR46GeBWP7aDvQ@mail.gmail.com>
 <5F4DEFB2-5F5A-4703-B5E5-BBCE05CD3651@fb.com> <CAADnVQLXGu_eF8VT6NmxKVxOHmfx7C=mWmmWF8KmsjFXg6P5OA@mail.gmail.com>
 <5E70BF53-E3FB-4F7A-B55D-199C54A8FDCA@fb.com> <adec88f9-b3e6-bfe4-c09e-54825a60f45d@linux.ibm.com>
 <2AAC8B8C-96F1-400F-AFA6-D4AF41EC82F4@fb.com>
In-Reply-To: <2AAC8B8C-96F1-400F-AFA6-D4AF41EC82F4@fb.com>
From:   Alexei Starovoitov <alexei.starovoitov@gmail.com>
Date:   Mon, 24 Jan 2022 21:21:28 -0800
Message-ID: <CAADnVQKgdMMeONmjUhbq_3X39t9HNQWteDuyWVfcxmTerTnaMw@mail.gmail.com>
Subject: Re: [PATCH v6 bpf-next 6/7] bpf: introduce bpf_prog_pack allocator
To:     Song Liu <songliubraving@fb.com>
Cc:     Ilya Leoshkevich <iii@linux.ibm.com>, Song Liu <song@kernel.org>,
        bpf <bpf@vger.kernel.org>,
        Network Development <netdev@vger.kernel.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Alexei Starovoitov <ast@kernel.org>,
        Daniel Borkmann <daniel@iogearbox.net>,
        Andrii Nakryiko <andrii@kernel.org>,
        Kernel Team <Kernel-team@fb.com>,
        Peter Zijlstra <peterz@infradead.org>, X86 ML <x86@kernel.org>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Mon, Jan 24, 2022 at 10:27 AM Song Liu <songliubraving@fb.com> wrote:
> >
> > Are arches expected to allocate rw buffers in different ways? If not,
> > I would consider putting this into the common code as well. Then
> > arch-specific code would do something like
> >
> >  header = bpf_jit_binary_alloc_pack(size, &prg_buf, &prg_addr, ...);
> >  ...
> >  /*
> >   * Generate code into prg_buf, the code should assume that its first
> >   * byte is located at prg_addr.
> >   */
> >  ...
> >  bpf_jit_binary_finalize_pack(header, prg_buf);
> >
> > where bpf_jit_binary_finalize_pack() would copy prg_buf to header and
> > free it.

It feels right, but bpf_jit_binary_finalize_pack() sounds 100% arch
dependent. The only thing it will do is perform a copy via text_poke.
What else?

> I think this should work.
>
> We will need an API like: bpf_arch_text_copy, which uses text_poke_copy()
> for x86_64 and s390_kernel_write() for x390. We will use bpf_arch_text_copy
> to
>   1) write header->size;
>   2) do finally copy in bpf_jit_binary_finalize_pack().

we can combine all text_poke operations into one.

Can we add an 'image' pointer into struct bpf_binary_header ?
Then do:
int bpf_jit_binary_alloc_pack(size, &ro_hdr, &rw_hdr);

ro_hdr->image would be the address used to compute offsets by JIT.
rw_hdr->image would point to kvmalloc-ed area for emitting insns.
rw_hdr->size would already be populated.

The JITs would write insns into rw_hdr->image including 'int 3' insns.
At the end the JIT will do text_poke_copy(ro_hdr, rw_hdr, rw_hdr->size);
That would be the only copy that will transfer everything into final
location.
Then kvfree(rw_hdr)

wdyt?