Received: by 2002:ad5:474a:0:0:0:0:0 with SMTP id i10csp5931371imu; Sun, 20 Jan 2019 23:42:58 -0800 (PST) X-Google-Smtp-Source: ALg8bN7teOmIpNV3GbOIhOWKqmj73rBIxaReeZicrSwGN6Cx+x9j+mWpzdVDve/oK8Nj8PU9OFPl X-Received: by 2002:a62:5dd1:: with SMTP id n78mr28412476pfj.58.1548056578467; Sun, 20 Jan 2019 23:42:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548056578; cv=none; d=google.com; s=arc-20160816; b=LqT7A8QM1rtOmjwr4Vffb1Ni8fAWJ7omV30NnZAs7VXzCHIguEwSWWq7peyMODgBxs qb/wx8Ixm17oShjcCFKhzbwBF84ifctZi2nykXim/hvpOQyXy7bVWcZVnfreVrll+7et ct6Tq5uDLwTpDOFo+MVh9zzYaD3Td89NggLY3EuRSmk/Q5Ur+7DOzfJh3IcbWq0U2faK fjae8b7MjpO0HdMch0GFXWrPLzfs0oi5wf7s8aeWm85lBM9OYGUZ7pwZ2D0AE1X8p3+m nzXCwWElM0BDJoyYVziC5L2mpE+u80LtHq+vJuj2DgrrSZZEC4Plt8nhGPASS/Wkezrc Q7yQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:user-agent:in-reply-to :content-transfer-encoding:content-disposition:mime-version :references:message-id:subject:cc:to:from:date:dkim-signature; bh=lUq5dmEXLPCHwnDf9zu/uZyKFZ+oNDbDsnI3kWjYAeY=; b=P+11Bl3sdUyaW1PBwz6JLdRHiKuhJH+lIKdJJzL/qssbC32Kkokjr1fw0kdkjakYgB LVSVWFjP+p+O0pwQyX8dAwyk2/eLHNS9HUCpJeN3uo1ZcOjTdTliVrW7+0/j/ZBI1RXS QUWOH+iS65ifK0NMMiPU9ecZtMOffkuYq/52zVvcdrIDSFL+e5JdJF/RHsG014/VVh3V fFkIw9hLsFkO7cpDsPdDiA21BYbXJtaBYE4jnfnohMfUS16I11F+zeLJzvratXy9LNG0 yZYppYZg6FHHSqw6aQLrA4fvy1kcwDWX3/j+glG7KmtYLwj1wT8j0FcOMttUsxtuvl08 HrGw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@android.com header.s=20161025 header.b=mXqn10Z6; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=android.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id s13si12862137pgc.509.2019.01.20.23.42.43; Sun, 20 Jan 2019 23:42:58 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@android.com header.s=20161025 header.b=mXqn10Z6; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=android.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728515AbfAUHlb (ORCPT + 99 others); Mon, 21 Jan 2019 02:41:31 -0500 Received: from mail-pl1-f193.google.com ([209.85.214.193]:47078 "EHLO mail-pl1-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728037AbfAUHla (ORCPT ); Mon, 21 Jan 2019 02:41:30 -0500 Received: by mail-pl1-f193.google.com with SMTP id t13so9383402ply.13 for ; Sun, 20 Jan 2019 23:41:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=android.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to :user-agent; bh=lUq5dmEXLPCHwnDf9zu/uZyKFZ+oNDbDsnI3kWjYAeY=; b=mXqn10Z6B0mQYeGBqKKsc/sf+D3V9H1GMGz+y+R4uM6YvdWO5Fti33zx7H6ZNlrvnL 0zTACm1YOa1xa5D6yMJu7YTnUu1RAAwry2hr/eRUSxywCtdJr5zmHzTTSxoKMONSfBLY T50XaFhoJjZn6O63a0k3fZI/trTXhF3wDTmYldBA+AMEY58EJyNLxVdxu0VBpEqVQFgI zDHOrYo7eW5YpFspE1gPc141fHMfqvU+NqqlwVfu+xrenSNI2br2E2yhfsk3zQuaGQme 1mfum3nZ18k4dVAwen/rlEMa6VAdwZALn+GAzh+WaLPHretA4ebKrUbAATd4cViJEXhY 3m+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:content-transfer-encoding :in-reply-to:user-agent; bh=lUq5dmEXLPCHwnDf9zu/uZyKFZ+oNDbDsnI3kWjYAeY=; b=HOPYrzq2Knpbg138oKby6ehqU+MLVhrRvpASCkFw/itNshwEo75brE43eProTHGJCR GAg/rpisVVyaRHH41cOvz+MNhm31jFPrhfLXw3uKtEBNIyOWDDo60a2J4BQ573iUxbWD ZISFFf6acGq+fWLfNY9dhmdNtLhQyGVpxEjfMlbDX5Oe51VMuPEYY+BW0Xb5+/O7NbE3 vKvHEnNkkP8bywJCjnw70k8Vyht54i5PTbwqatpUvizr2YsKmInKIxb1LpvYBvxl4Sa9 j0HYRA2RjlKIob31oN1xh8quV5nIboMIJbxMK0uuMNg5xK2KbDF+QWMw9Q3YzUyYRxUt vi9w== X-Gm-Message-State: AJcUukcLdRU6cOA72p2pY84weFoq3xeU1h24+tqvfSup3HAQ77hzFbFE LcEMo5CsV7fZcrRwRe+d5Oovcw== X-Received: by 2002:a17:902:9a9:: with SMTP id 38mr28235912pln.204.1548045514833; Sun, 20 Jan 2019 20:38:34 -0800 (PST) Received: from localhost (c-73-170-36-70.hsd1.ca.comcast.net. [73.170.36.70]) by smtp.gmail.com with ESMTPSA id p2sm14593359pfb.28.2019.01.20.20.38.33 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sun, 20 Jan 2019 20:38:34 -0800 (PST) Date: Sun, 20 Jan 2019 20:38:32 -0800 From: Sandeep Patil To: hpa@zytor.com Cc: Joel Fernandes , Greg KH , Christoph Hellwig , linux-kernel@vger.kernel.org, Andrew Morton , ast@kernel.org, atishp04@gmail.com, Borislav Petkov , dancol@google.com, Ingo Molnar , Jan Kara , Jonathan Corbet , karim.yaghmour@opersys.com, Kees Cook , kernel-team@android.com, linux-doc@vger.kernel.org, Manoj Rao , Masahiro Yamada , paulmck@linux.vnet.ibm.com, "Peter Zijlstra (Intel)" , rdunlap@infradead.org, rostedt@goodmis.org, Thomas Gleixner , "maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT)" , yhs@fb.com Subject: Re: [RFC] Provide in-kernel headers for making it easy to extend the kernel Message-ID: <20190121043832.GA70794@google.com> References: <20190118225543.86996-1-joel@joelfernandes.org> <20190119102800.GB17723@infradead.org> <20190119103606.GA17400@kroah.com> <8BD4CB7A-944C-4EC5-A198-1D85C9EE28D6@zytor.com> <20190120161003.GB23827@google.com> <20190121014553.GD23827@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Sun, Jan 20, 2019 at 06:49:56PM -0800, hpa@zytor.com wrote: > On January 20, 2019 5:45:53 PM PST, Joel Fernandes wrote: > >On Sun, Jan 20, 2019 at 01:58:15PM -0800, hpa@zytor.com wrote: > >> On January 20, 2019 8:10:03 AM PST, Joel Fernandes > > wrote: > >> >On Sat, Jan 19, 2019 at 11:01:13PM -0800, hpa@zytor.com wrote: > >> >> On January 19, 2019 2:36:06 AM PST, Greg KH > >> > wrote: > >> >> >On Sat, Jan 19, 2019 at 02:28:00AM -0800, Christoph Hellwig > >wrote: > >> >> >> This seems like a pretty horrible idea and waste of kernel > >memory. > >> >> > > >> >> >It's only a waste if you want it to be a waste, i.e. if you load > >the > >> >> >kernel module. > >> >> > > >> >> >This really isn't any different from how /proc/config.gz works. > >> >> > > >> >> >> Just add support to kbuild to store a compressed archive in > >> >initramfs > >> >> >> and unpack it in the right place. > >> >> > > >> >> >I think the issue is that some devices do not use initramfs, or > >> >switch > >> >> >away from it after init happens or something like that. Joel has > >> >all > >> >> >of > >> >> >the looney details that he can provide. > >> >> > > >> >> >thanks, > >> >> > > >> >> >greg k-h > >> >> > >> >> Yeah, well... but it is kind of a losing game... the more > >in-kernel > >> >stuff there is the less smiley are things to actually be supported. > >> > > >> >It is better than nothing, and if this makes things a bit easier and > >> >solves > >> >real-world issues people have been having, and is optional, then I > >> >don't see > >> >why not. > >> > > >> >> Modularizing is it in some ways even crazier in the sense that at > >> >that point you are relying on the filesystem containing the module, > >> >which has to be loaded into the kernel by a root user. One could > >even > >> >wonder if a better way to do this would be to have "make > >> >modules_install" park an archive file – or even a directory as > >opposed > >> >to a symlink – with this stuff in /lib/modules. We could even > >provide a > >> >tmpfs shim which autoloads such an archive via the firmware loader; > >> >this might even be generically useful, who knows. > >> > > >> >All this seems to assume where the modules are located. In Android, > >we > >> >don't > >> >have /lib/modules. This patch generically fits into the grand scheme > >> >things > >> >and I think is just better made a part of the kernel since it is not > >> >that > >> >huge once compressed, as Dan also pointed. The more complex, and the > >> >more > >> >assumptions we make, the less likely people writing tools will get > >it > >> >right > >> >and be able to easily use it. > >> > > >> >> > >> >> Note also that initramfs contents can be built into the kernel. > >> >Extracting such content into a single-instance tmpfs would again be > >a > >> >possibility > >> > > >> >Such an approach would bloat the kernel image size though, which may > >> >not work > >> >for everyone. The module based approach, on the other hand, gives an > >> >option > >> >to the user to enable the feature, but not have it loaded into > >memory > >> >or used > >> >until it is really needed. > >> > > >> >thanks, > >> > > >> > - Joel > >> > >> Well, where are the modules? They must exist in the filesystem. > > > >The scheme of loading a module doesn't depend on _where_ the module is > >on the > >filesystem. As long as a distro knows how to load a module in its own > >way (by > >looking into whichever paths it cares about), that's all that matters. > >And > >the module contains compressed headers which saves space, vs storing it > >uncompressed on the file system. > > > >To remove complete reliance on the filesystem, there is an option of > >not > >building it as a module, and making it as a built-in. > > > >I think I see your point now - you're saying if its built-in, then it > >becomes kernel memory that is lost and unswappable. Did I get that > >right? > >I am saying that if that's a major concern, then: > >1. Don't make it a built-in, make it a module. > >2. Don't enable it at for your distro, and use a linux-headers package > >or > >whatever else you have been using so far that works for you. > > > >thanks, > > > > - Joel > > My point is that if we're going to actually solve a problem, we need to make it so that the distro won't just disable it anyway, and it ought to be something scalable; otherwise nothing is gained. > > I am *not* disagreeing with the problem statement! > > Now, /proc isn't something that will autoload modules. A filesystem *will*, although you need to be able to mount it; furthermore, it makes it trivially to extend it (and the firmware interface provides an . easy way to feed the data to such a filesystem without having to muck with anything magic.) > > Heck, we could even make it a squashfs image that can just be mounted. > > So, first of all, where does Android keep its modules, and what is actually included? Is /sbin/modprobe used to load the modules, as is normal? We might even be able to address this with some fairly trivial enhancements to modprobe; specifically to search in the module paths for something that isn't a module per se. > FWIW, 'modprobe' does exist on Android. Although most of the times, Init's builtin insmod function[1] gets used. The module locations are /{system, vendor,odm}/lib/modules and yes, modprobe can be made to load modules from there too. [2] - ssp > The best scenario would be if we could simply have the tools find the location equivalent of /lib/modules/$version/source... [1] https://android.googlesource.com/platform/system/core/+/master/init/builtins.cpp#230 [2] https://source.android.com/devices/architecture/kernel/modular-kernels > -- > Sent from my Android device with K-9 Mail. Please excuse my brevity. > >