Received: by 2002:ad5:474a:0:0:0:0:0 with SMTP id i10csp1308654imu; Wed, 23 Jan 2019 14:38:21 -0800 (PST) X-Google-Smtp-Source: ALg8bN5P1FhjQBm+srkQ89m733Nx8rzwB+u3Aq5W1Zt4c6nVU1rbn78FS6x30imUiWqpgMoCoVL1 X-Received: by 2002:a62:b80a:: with SMTP id p10mr3892603pfe.32.1548283101803; Wed, 23 Jan 2019 14:38:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548283101; cv=none; d=google.com; s=arc-20160816; b=TWC2CP03Wfd2dnof+fNJ1OKhhqIwmbew/QULoRnjoaob6EUpeI+fVY0Ofh7ZF5Gmtu XjecfgbVZzwDl446rlCD+GLGe1i5ast620Fr8HlvxZKc8Z/i7QSlzifDl3qcJ1okqEPc jLubFie8zz5lu3RApiCNeysRMNv9eeOCX1t77av55lk06kMQ7iTBg9vtmP9UNynG5Uc2 pQK7OkIULpULzC8nPm5ib3o/21i+dVvMs0j9G6148TESwJKXKJDnWZbn3KyclXInP89F j9nL52SMkeqwb7G112EUfTdJ0Pf05QuSIk9hiKIXGTsuMY/fO7i8YKSq1mCKVOWYB619 MIWQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:dkim-signature; bh=W4B8lCha9ZTaw69yBTneXvxWuhyv5HaHtF9PM+NeshY=; b=HiGF30GPtzQ2vh6aQy+23KvVHBaIW/Os/UoNMk1o84oIGCyW6XW5DvC1MVuqO4dx0C IvsUr6rJKSTZjZplf3bBhn/NoLY89pdUy9IeVNHqZpFGTT2ihmlvSz7oGPj0HoxRpuP+ bAhsMEJtPCYpULIB+JH+82ZuAGkkFqioU1H6iS1v+A/1vGsXEhucJbCZg8f9Qm/wJxBR XkEYomVEH7vDjtMnAoYTFnCLoFCsEI4nRXud7crVvl8eYVUHi3LAKt60xlm/L66vmQe2 PCZyWyYHMaQ70ks4Vyy/5Xxq1Zhviu2A8suThJs14j8ZOQsn35wonmnM088t7DlBxE/r 3H1Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=XoLHPkMG; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 43si20619852plb.176.2019.01.23.14.38.06; Wed, 23 Jan 2019 14:38:21 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=XoLHPkMG; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726986AbfAWWiB (ORCPT + 99 others); Wed, 23 Jan 2019 17:38:01 -0500 Received: from mail-ua1-f49.google.com ([209.85.222.49]:44189 "EHLO mail-ua1-f49.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726313AbfAWWiB (ORCPT ); Wed, 23 Jan 2019 17:38:01 -0500 Received: by mail-ua1-f49.google.com with SMTP id d19so1307749uaq.11 for ; Wed, 23 Jan 2019 14:38:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=W4B8lCha9ZTaw69yBTneXvxWuhyv5HaHtF9PM+NeshY=; b=XoLHPkMG0sjoqXANSoKCIxZjaGp5ZrPI3pNVduYbjj/qH0LRZRRtWb3fXkJjwaqcFO 65sIYsoQBmWv+AIWJWMR5UMTb3vwnHFwgi8qhA7m+QiWeWxnBp4OzemSJv1t7pLKz3QJ efHx+GNbBhX1157wGglCRIFIg6YmQNmbp379ICdiqcwxrooQw083uu4rsYT1vpFaCNx+ QqPJaM6EpnEdXsBchxF6cykk4rKRQIK9w+qSDEazsGVyp0DDV3QZWi4iuX2vn+n2T8cd JnspAPMM8encGI7z7lHMbx9okZema1B3oOgPebuLq8ppvmogV70mccNEB5p8hjvSuw1s lkyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=W4B8lCha9ZTaw69yBTneXvxWuhyv5HaHtF9PM+NeshY=; b=nIM18KZ6/CD/YWIYlhVqErYlfvY8cRkUMoq3Xo2llD5QoHjXC43TaONYF3iiuEiEbp Wi1jllk4nfm2aV7m+JrtxRLRPJavZfBW2TCpWF67J9q182JXTFZL5us6ZgYv9EB2RQ0K KyHwBLMW3LdzQ+4nTXYMr5/LTjEWIn+XGJM+IUhP8AhoicoJjDR+dWn5TSbN8eUi8+Lj 9YLikOTt7w88gYE1U/WeW1CoRCdfXLujJM4FzLokTINiy6bIqMGv7sR344ixvqWJcHnl VRk1FN0KI8gIBS42Bu4/IjQqe5K38OTLZV2aGVCEjx4msCjYcVfnQjkhMCGWOGSKw9En uVLw== X-Gm-Message-State: AJcUukfqc72Br3yb0O5c3+coN1S8sUNR7pvH/R5SEeLmeu6aOx0PQPbc alsdNf9Uxp0DF9G4JakjfwODq/Vd2Aow1KvkEEz4Bg== X-Received: by 2002:ab0:63d7:: with SMTP id i23mr1704478uap.11.1548283079017; Wed, 23 Jan 2019 14:37:59 -0800 (PST) MIME-Version: 1.0 References: <20190118225543.86996-1-joel@joelfernandes.org> <20190119102800.GB17723@infradead.org> <20190119103606.GA17400@kroah.com> <8BD4CB7A-944C-4EC5-A198-1D85C9EE28D6@zytor.com> <20190120161003.GB23827@google.com> <20190121014553.GD23827@google.com> <20190122133901.GA189736@google.com> <117d2f96-b0e9-2376-69b7-836fa0c52539@opersys.com> In-Reply-To: <117d2f96-b0e9-2376-69b7-836fa0c52539@opersys.com> From: Daniel Colascione Date: Wed, 23 Jan 2019 14:37:47 -0800 Message-ID: Subject: Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel To: Karim Yaghmour Cc: Joel Fernandes , "H. Peter Anvin" , Greg KH , Christoph Hellwig , linux-kernel , Andrew Morton , ast@kernel.org, atish patra , Borislav Petkov , Ingo Molnar , Jan Kara , Jonathan Corbet , Kees Cook , kernel-team@android.com, "open list:DOCUMENTATION" , Manoj Rao , Masahiro Yamada , Paul McKenney , "Peter Zijlstra (Intel)" , Randy Dunlap , rostedt@goodmis.org, Thomas Gleixner , "maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT)" , yhs@fb.com Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Jan 23, 2019 at 1:29 PM Karim Yaghmour wrote: > > By the way, we can easily write a script to just extract the .ko directly - > > if the whole "load it as a module" thing bothers you. The kheaders.ko can > > just be thought of as a tarball. There's already a script to extract > > /proc/config.gz in the same/similar way: scripts/extract-ikconfig > > If I may add a few more thoughts here ... in no specific order: > > From that point of view, if something comes from or is rooted in > mainline, instead of being mandated, it's usually easier to find it > across the board. A perfect example of this is ftrace. The fact that > it's in mainline has enabled google to: a) instrument their entire stack > to log events to it (see > http://www.opersys.com/downloads/cc-slides/android-debug/slides-main-181012.html#/82 > and > http://www.opersys.com/downloads/cc-slides/android-debug/slides-main-181012.html#/83), > and b) provide app-developer-facing tools (see > https://developer.android.com/studio/command-line/systrace). Since this > tracing functionality is now integrated into Android Studio (look for > "System Trace" here: > https://developer.android.com/studio/profile/cpu-profiler), it's very > much "standard android" and additional proof, if any was needed, that > tracing is useful to everyone. > > A few years back I was asked by a customer to put together some class > material for internal Android debugging and performance analysis > (commercial disclaimer applies, but slides/exercises are under > "courseware": > http://www.opersys.com/training/android-debug-and-performance). ftrace > was very much in those early versions and it was great to show people > that they could use it "out of the box". Recently I wanted to update > this class material to cover eBPF and its applicability in Android. Holy > cow. That turned out to be less obvious than necessary and somewhat > peculiar to pull off. In this specific case, Joel tried a few things > (see > http://www.opersys.com/downloads/cc-slides/android-debug/slides-main-181012.html#/111) > before eventually settling on loading a Debian chroot jail into a live > Android (https://github.com/joelagnel/adeb) ... all of which require a > proper set of kernel headers to properly function. Don't get me wrong, > Joel's Androdeb makes this easy, but it's still outside the standard > Android MO. > > In short, let's just say that, contrary to ftrace, I don't see the path > for eBPF to being part of the standard toolset used by app developers > any time soon. The recently-released bpftrace might help in that regard, > but the kernel headers aren't too far in that regard either. While I think there's definitely a place for eBPF as part of the Android performance toolkit, I think most users will end up using it through rich front-end performance collection and analysis tools (of the sort I'm working on) rather than directly as a first-line window into the operation of the system. Below this level is probably something like bpftrace, and below that, raw eBPF and ftrace manipulation. It's also worth noting that much of the time, system analysis is retrospection, not inspection (e.g., investigating the causes of rare and hard-to-reproduce bad behavior), and so iteration via interactive specification of eBPF programs isn't a practical path forward. It's still useful, even in this scenario, to be able (as part of higher-level tools) attach "canned" eBPF programs to the kernel to extract certain generally-useful bits of information, and in this capacity, Joel's header module would be useful. > Personally I advocated a more aggressive approach with Joel in private: > just put the darn headers straight into the kernel image, it's the > *only* artifact we're sure will follow the Android device whatever > happens to it (like built-in ftrace). I was thinking along similar lines. Ordinarily, we make loadable kernel modules. What we kind of want here is a non-loadable kernel module --- or a non-loadable section in the kernel image proper. I'm not familiar with early-stage kernel loader operation: I know it's possible to crease discardable sections in the kernel image, but can we create sections that are never slurped into memory in the first place? If not, maybe loading and immediately discarding the header section is good enough. > To that end, I even had some crazy > ideas on how to compress the headers even further than with std > compression algorithms -- here's a snippet from an email I sent Joel > some time back detailing such a hack: > > Since C headers have fairly constrained semantics and since the types of semantics generally used to name structs, etc. in the Linux kernel are well established, we can likely devise a very customized compression algorithm for the purpose. Would such a thing really do better than LZMA? LZMA already has very clever techniques for eliminating long-range redundancies in compressible text, including redundancies at the sub-byte level. I can certainly understand the benefit of stripping comments, since removing comments really does decrease the total amount of information the compressor has to preserve, but I'm not sure how much the encoding scheme you propose below would help, since it reminds me of the encoding scheme that LZMA would discover automatically. > Whether such craziness makes sense or is adopted or not isn't mine to > chart, but I certainly can't see eBPF reaching the same mass deployment > ftrace has within the Android ecosystem until there's a way to use it > without having to chase kernel headers independently of kernel images. > There are "too many clicks" involved and someone somewhere will drop the > ball if it's not glued to the kernel in some way shape or form. Any > solution that solves this is one I'd love to hear about. I agree. There definitely needs to be a "just collect a damn trace" button that works on any device, and for this button to work and incorporate eBPF, the system needs to be able to describe itself.