Received: by 2002:a05:6a10:af89:0:0:0:0 with SMTP id iu9csp5199267pxb; Wed, 26 Jan 2022 06:58:58 -0800 (PST) X-Google-Smtp-Source: ABdhPJyjopa/JpTvqhXI0+aDN+zlMOLHFRaVGMjaKgLqKNln+cUBRMUwfmDkyYdBcbW71mMtp4k4 X-Received: by 2002:a17:90b:38c9:: with SMTP id nn9mr9018832pjb.219.1643209137943; Wed, 26 Jan 2022 06:58:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1643209137; cv=none; d=google.com; s=arc-20160816; b=nSVBbYKiGBniua97+IfA+gg6y41N5Kylwkux6AM6vLcnZ08Tie4EThl+43127ajV+l +BF1V9E9qznnPBmdBh9owj41DPx5YNYP7ALo9hLaWzojUPrpIANXPpYKbm5aYdYw8sfL scidU9yDH5vweBoHcuaU+yMLcdL8mIFKZESq8LG3+aS0QFvvcEUx3Wi/DQbT4pWsLkj5 pqSjtPsPg9jjJl4PoW+GggB2lX3XPSC1pQBrlw9yLDda0pBRDDU430hyj9dXPhINe6SX HtgK6N5uRRXrWPviFM0LMkTnpCKRTxus7cCeVHAp7CVGjYJiULYPfNRJmF6SK1OM2KuH d3Cg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:subject:message-id:date:from:in-reply-to :references:mime-version:dkim-signature; bh=WE+J2QanX13xauZhQxfLJNljbHtY1NDRSbKC9uOnr2o=; b=KhpzwmDapQ6N6UJE9+CV47sizhBMJtCbrrBxmBV7HW5DjPmfkJCWu9huhnvqPFhX/E 8aTaeTCZwNEEbIIO/xUCkVU4FEFDdgqK5csp36VbYfYFoVbi5zbnN2+Vv22MDRuOqNgd 12YyASLe4SkFvcG7Ygl/kE8J71UytiiAP76c6vOjNUiZnWOvmYeqg0jydAMqeJFXAQR4 VugdXAq7b7aipd/y5YqjJP67gKrTsQMznfW0Q3ri+edWZGRjGcRtdnqrNDFDO8UjrgIr rPaALLWZNeFQkRJ7TTIGU2U4oSoxulnabRzpAsx2M+UYG1txOMjt6Uklf5BioSSYwDMo 5FbA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=gSkvFnU4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id v7si19185059plo.89.2022.01.26.06.58.45; Wed, 26 Jan 2022 06:58:57 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b=gSkvFnU4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235676AbiAZBUe (ORCPT + 99 others); Tue, 25 Jan 2022 20:20:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46506 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235637AbiAZBUe (ORCPT ); Tue, 25 Jan 2022 20:20:34 -0500 Received: from mail-pf1-x436.google.com (mail-pf1-x436.google.com [IPv6:2607:f8b0:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E5352C06161C; Tue, 25 Jan 2022 17:20:33 -0800 (PST) Received: by mail-pf1-x436.google.com with SMTP id i65so21310943pfc.9; Tue, 25 Jan 2022 17:20:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=WE+J2QanX13xauZhQxfLJNljbHtY1NDRSbKC9uOnr2o=; b=gSkvFnU4pgpgk4OsxmF61xsJl8bNGgbJUvL0FPtC0ZyRHBYyUCz/8NhllYwdnlaOxp c7IZ9JrEco5VSg8M9ZLd6O/dcDonX9EfaWBB5O11HhC8VxvrcNeLHXXTstA9onTn9oQF wbhggwvilhc5x5sPaAE8zbZPIlW5npLzVyENZqQXpNR6kNdDl2x0lrMWQdPbTUNb9CN8 Qx7BO5x55eW9twzQbRyNo/SfbvjbzQcTSsYi9LDfgYOuSSc3NHiBT6vlO/tgOJBfP9j4 hn7rrmh/xmubjESQ6cVSAEmRjuvlJbA21jBhNQ7YBdGTqKKHZJITOkwRVxQC0ctVwkOf dLhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=WE+J2QanX13xauZhQxfLJNljbHtY1NDRSbKC9uOnr2o=; b=cIot47b26lhJDrktmtyOAZZlXerhtqs/LWf78MZhVxCnxQkvOy+/xCeciJvBOsTi5Y QdxZTe3fLNzk3RKRBLn8NB05QAFEd0qd5dzwDL528DkWVAUSJBxWDURprV5eW4YY/SAZ mXegIoQ4oEUFiZYhiAO11JICJEX1X5V1KCW553q6EJSsMz4jDvuBH5N9VYNnrKmt0X9w 7w0Oy/XPGrvPStmtKJp/rz8bYt9ulXxq9zJyYa4nLsis0ScDXG4l/TbvjWtZ9+HWEt+k 9bFqoMazWguYEklStnr5xY2HYuAfmOzXdyecPAMMmcTigAFCxuSA1h+5G5THikucpZjG PxvQ== X-Gm-Message-State: AOAM531z3s8b7mDPRyVkbm3w0fo9Mh2mai7/uwj35Bokrz9ICMAc+ObU 1LNyys3pfSD3DoLUAu+Ix2bpTt2+1tTFXd3BI5TBwaVN998= X-Received: by 2002:aa7:888d:0:b0:4c2:7965:950d with SMTP id z13-20020aa7888d000000b004c27965950dmr20945167pfe.46.1643160032469; Tue, 25 Jan 2022 17:20:32 -0800 (PST) MIME-Version: 1.0 References: <20220121194926.1970172-1-song@kernel.org> <20220121194926.1970172-7-song@kernel.org> <7393B983-3295-4B14-9528-B7BD04A82709@fb.com> <5407DA0E-C0F8-4DA9-B407-3DE657301BB2@fb.com> <5F4DEFB2-5F5A-4703-B5E5-BBCE05CD3651@fb.com> <5E70BF53-E3FB-4F7A-B55D-199C54A8FDCA@fb.com> <2AAC8B8C-96F1-400F-AFA6-D4AF41EC82F4@fb.com> In-Reply-To: From: Alexei Starovoitov Date: Tue, 25 Jan 2022 17:20:21 -0800 Message-ID: Subject: Re: [PATCH v6 bpf-next 6/7] bpf: introduce bpf_prog_pack allocator To: Song Liu Cc: Song Liu , Ilya Leoshkevich , bpf , Network Development , LKML , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Kernel Team , Peter Zijlstra , X86 ML Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Jan 25, 2022 at 4:50 PM Song Liu wrote: > > On Tue, Jan 25, 2022 at 4:38 PM Alexei Starovoitov > wrote: > > > > On Tue, Jan 25, 2022 at 3:09 PM Song Liu wrote: > > > > > > On Tue, Jan 25, 2022 at 2:48 PM Alexei Starovoitov > > > wrote: > > > > > > > > On Tue, Jan 25, 2022 at 2:25 PM Song Liu wrote: > > > > > > > > > > On Tue, Jan 25, 2022 at 12:00 PM Alexei Starovoitov > > > > > wrote: > > > > > > > > > > > > On Mon, Jan 24, 2022 at 11:21 PM Song Liu wrote: > > > > > > > > > > > > > > On Mon, Jan 24, 2022 at 9:21 PM Alexei Starovoitov > > > > > > > wrote: > > > > > > > > > > > > > > > > On Mon, Jan 24, 2022 at 10:27 AM Song Liu wrote: > > > > > > > > > > > > > > > > > > > > Are arches expected to allocate rw buffers in different ways? If not, > > > > > > > > > > I would consider putting this into the common code as well. Then > > > > > > > > > > arch-specific code would do something like > > > > > > > > > > > > > > > > > > > > header = bpf_jit_binary_alloc_pack(size, &prg_buf, &prg_addr, ...); > > > > > > > > > > ... > > > > > > > > > > /* > > > > > > > > > > * Generate code into prg_buf, the code should assume that its first > > > > > > > > > > * byte is located at prg_addr. > > > > > > > > > > */ > > > > > > > > > > ... > > > > > > > > > > bpf_jit_binary_finalize_pack(header, prg_buf); > > > > > > > > > > > > > > > > > > > > where bpf_jit_binary_finalize_pack() would copy prg_buf to header and > > > > > > > > > > free it. > > > > > > > > > > > > > > > > It feels right, but bpf_jit_binary_finalize_pack() sounds 100% arch > > > > > > > > dependent. The only thing it will do is perform a copy via text_poke. > > > > > > > > What else? > > > > > > > > > > > > > > > > > I think this should work. > > > > > > > > > > > > > > > > > > We will need an API like: bpf_arch_text_copy, which uses text_poke_copy() > > > > > > > > > for x86_64 and s390_kernel_write() for x390. We will use bpf_arch_text_copy > > > > > > > > > to > > > > > > > > > 1) write header->size; > > > > > > > > > 2) do finally copy in bpf_jit_binary_finalize_pack(). > > > > > > > > > > > > > > > > we can combine all text_poke operations into one. > > > > > > > > > > > > > > > > Can we add an 'image' pointer into struct bpf_binary_header ? > > > > > > > > > > > > > > There is a 4-byte hole in bpf_binary_header. How about we put > > > > > > > image_offset there? Actually we only need 2 bytes for offset. > > > > > > > > > > > > > > > Then do: > > > > > > > > int bpf_jit_binary_alloc_pack(size, &ro_hdr, &rw_hdr); > > > > > > > > > > > > > > > > ro_hdr->image would be the address used to compute offsets by JIT. > > > > > > > > > > > > > > If we only do one text_poke(), we cannot write ro_hdr->image yet. We > > > > > > > can use ro_hdr + rw_hdr->image_offset instead. > > > > > > > > > > > > Good points. > > > > > > Maybe let's go back to Ilya's suggestion and return 4 pointers > > > > > > from bpf_jit_binary_alloc_pack ? > > > > > > > > > > How about we use image_offset, like: > > > > > > > > > > struct bpf_binary_header { > > > > > u32 size; > > > > > u32 image_offset; > > > > > u8 image[] __aligned(BPF_IMAGE_ALIGNMENT); > > > > > }; > > > > > > > > > > Then we can use > > > > > > > > > > image = (void *)header + header->image_offset; > > > > > > > > I'm not excited about it, since it leaks header details into JITs. > > > > Looks like we don't need JIT to be aware of it. > > > > How about we do random() % roundup(sizeof(struct bpf_binary_header), 64) > > > > to pick the image start and populate > > > > image-sizeof(struct bpf_binary_header) range > > > > with 'int 3'. > > > > This way we can completely hide binary_header inside generic code. > > > > The bpf_jit_binary_alloc_pack() would return ro_image and rw_image only. > > > > And JIT would pass them back into bpf_jit_binary_finalize_pack(). > > > > From the image pointer it would be trivial to get to binary_header with &63. > > > > The 128 byte offset that we use today was chosen arbitrarily. > > > > We were burning the whole page for a single program, so 128 bytes zone > > > > at the front was ok. > > > > Now we will be packing progs rounded up to 64 bytes, so it's better > > > > to avoid wasting those 128 bytes regardless. > > > > > > In bpf_jit_binary_hdr(), we calculate header as image & PAGE_MASK. > > > If we want s/PAGE_MASK/63 for x86_64, we will have different versions > > > of bpf_jit_binary_hdr(). It is not on any hot path, so we can use __weak for > > > it. Other than this, I think the solution works fine. > > > > I think it can stay generic. > > > > The existing bpf_jit_binary_hdr() will do & PAGE_MASK > > while bpf_jit_binary_hdr_pack() will do & 63. > > The problem with this approach is that we need bpf_prog_ksym_set_addr > to be smart to pick bpf_jit_binary_hdr() or bpf_jit_binary_hdr_pack(). We can probably add a true JIT image size to bpf_prog_aux. bpf_prog_ksym_set_addr() is approximating the end: prog->aux->ksym.end = addr + hdr->pages * PAGE_SIZE which doesn't have to include all the 'int 3' padding after the end. Or add a flag to bpf_prog_aux. Ideally bpf_jit_free() would stay generic too.