Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Thu, 25 May 2023 09:54:33 -0700
From:   Lucas De Marchi <lucas.demarchi@intel.com>
To:     Petr Pavlu <petr.pavlu@suse.com>
CC:     Luis Chamberlain <mcgrof@kernel.org>, <gregkh@linuxfoundation.org>,
        <rafael@kernel.org>, <song@kernel.org>,
        <lucas.de.marchi@gmail.com>, <christophe.leroy@csgroup.eu>,
        <peterz@infradead.org>, <rppt@kernel.org>, <dave@stgolabs.net>,
        <willy@infradead.org>, <vbabka@suse.cz>, <mhocko@suse.com>,
        <dave.hansen@linux.intel.com>, <colin.i.king@gmail.com>,
        <jim.cromie@gmail.com>, <catalin.marinas@arm.com>,
        <jbaron@akamai.com>, <rick.p.edgecombe@intel.com>,
        <yujie.liu@intel.com>, <david@redhat.com>, <tglx@linutronix.de>,
        <hch@lst.de>, <patches@lists.linux.dev>,
        <linux-modules@vger.kernel.org>, <linux-mm@kvack.org>,
        <linux-kernel@vger.kernel.org>, <pmladek@suse.com>,
        <prarit@redhat.com>, <torvalds@linux-foundation.org>,
        <lennart@poettering.net>
Subject: Re: [PATCH 2/2] module: add support to avoid duplicates early on load
Message-ID: <fd2jkkgi5j2wakpaip3jbmj2b66s2euv2wwosade5h73nqi77r@4t4pdpyeemwv>
References: <20230524213620.3509138-1-mcgrof@kernel.org>
 <20230524213620.3509138-3-mcgrof@kernel.org>
 <8fc5b26b-d2f6-0c8f-34a1-af085dbef155@suse.com>
Content-Type: text/plain; charset="us-ascii"; format=flowed
Content-Disposition: inline
In-Reply-To: <8fc5b26b-d2f6-0c8f-34a1-af085dbef155@suse.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?QgjI4l7dOQ+ZMeaodi966RX0xH+dV7H9t1Ph8EMi2C3gNE1SRqlINx17Epxh?=
 =?us-ascii?Q?RTpn8Gf0sr7RZi429eCqGuaxsIFWIwdQx8CZcdXmReponnHjnaRAnsrbYNPH?=
 =?us-ascii?Q?vKs3k5aJjGeH5g3AZYGfdEFeFIK1un1wUuG9EMcXs7OV761r07xUnxnV3uAz?=
 =?us-ascii?Q?7h4A5qV/2pAXTZ2df+Qitoe3HL99HrAkpLMbW7AWV2D5JJWJDUq2pWpBPA3t?=
 =?us-ascii?Q?AFlR9vLZgUWmj3KARD0wx0TEgpNJub61NNhGNNEgAe/YUGOsT7Vynz3eEXOo?=
 =?us-ascii?Q?+BErCI8xomT78FnFaPe5GrAJCta2MVBALvZwl6wOBjwp+gGfwgLMjDuxMKb/?=
 =?us-ascii?Q?aWckSgObcBc82kkf/UiDdIw8UCorG55wcYaa2nM1pPQPMjNZgJOnKC8kGR0D?=
 =?us-ascii?Q?/kMFUGtN2NQF7HsIf3xGTGepYkpFTiTSRdFXFODXzNm1gMI+hvpnawpquwb+?=
 =?us-ascii?Q?8dsZe+OLf5NueAhYXcDgnVqnLLfbNX92GDPm9uihm3302XQGfjnmWnn1WySK?=
 =?us-ascii?Q?0XVPzSJlFxEeQETxdUb/7JSgVjN6nK7xVSDsjsxoV6qKW+XcjbfwAKdayvgU?=
 =?us-ascii?Q?JFuTlGLxYN1BHkO1MOIGWlvzPVMe7MTBkGRMyEINexXjvRb+1SItmCGeRXxV?=
 =?us-ascii?Q?1Jqs+6QhN6anFJ3+huQ7Z+9l438omMWkXZOYr0QlF3LpH4kv555kyrcMGVqf?=
 =?us-ascii?Q?bA5CFl6WCqp4xazEh1AmHN17LGK4G/ia9/SQVmAcO6p2neu1IT6WYHwH4tvY?=
 =?us-ascii?Q?sAt+Y2snFi0bLM1yH+8w8XQPz/wS1Fc9Ipf/3ODWGEW9QqVerS6YsW0J3oAJ?=
 =?us-ascii?Q?IfvyTXVo435iDBH8Uo1DbQfgt/FS2d1GILDLyptY82Fs+81foCIbMdjd6p9A?=
 =?us-ascii?Q?zNXzrq3tQvbDgfkZkaBUVZTUgYrAQLHDEUs8gNrs3lfq2uW/I1t1+Ih+2cda?=
 =?us-ascii?Q?dFkx+xgiScveSoLfhX8WEDSkw35YqLxoDl5CkiCqN1ZqmLcG0iTKzEZwFYCt?=
 =?us-ascii?Q?fWSU57FPizjoVLfttv2RJP0f/0jrL0XhIbx/HANpfQG0S0Z2w0LvOfr1CVER?=
 =?us-ascii?Q?81bXhqzoJgF1UfU6FAGtrpkWAiBF9CyBdeVhnRdzsYzmqz9LLecyKRC58qgA?=
 =?us-ascii?Q?CI6tGuqy7CiuXRFbrXS/DCgOeotsfj3K7CUJAgtTau6fi/O5grp8ED+r4pxn?=
 =?us-ascii?Q?TvOtqPBNL68kYSmdQalEzxqXCnqgkVy+ZCNQvWlvdiE/x23FIJ1rZtBOVXZR?=
 =?us-ascii?Q?/pSXCMYiZXTZt/qHWXcJ5YMXD+CxFsxIV6C6HJTWeDFVdVMEOkkM7je7Uml1?=
 =?us-ascii?Q?BQWNwGuZqDiIim4bbEH6sVauFeevNtMK57Q1HmlbxgIqSq9H3MsP/cOESK9X?=
 =?us-ascii?Q?Vzd8NUgtL0m9rFFff6Eyao2pr+pejTC4H4KUAWY2oVEnoT94bVIVbO3a0xfA?=
 =?us-ascii?Q?Xh5fRG47yqKVuUbb51w1sKfzsOXJpTFdKX3cosxz1aCk4s9zGmA1ycMe3mZc?=
 =?us-ascii?Q?15fGdKqPzuvCG6B4WoqZPkefO+KKnJD1zyA/2JLkIEooSITZnp4my3ydYyDr?=
 =?us-ascii?Q?P2hyBmU11lDLfb+Xq4SysEp/9kPGIhPnD+HwJ4p6F6eKkKNNdBjxAlJUUl9p?=
 =?us-ascii?Q?hQ=3D=3D?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 574af3bf-0145-4609-90c1-08db5d40b93b
X-MS-Exchange-CrossTenant-AuthSource: CY5PR11MB6139.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 May 2023 16:54:37.3332
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: KabrB1jtHIoArAUJPNO+bwn+9ZSssbfjaa9IvfgDXjjbey9ioHBJEqeias57qWSscS5+rQx8yIjusE9wC8iiI2ksLWHOKsgEl7NQjqPX5A0=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR11MB5627
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thu, May 25, 2023 at 01:40:32PM +0200, Petr Pavlu wrote:
>On 5/24/23 23:36, Luis Chamberlain wrote:
>> Add support to use the new kread_uniq_fd() to avoid duplicate kernel
>> reads on modules. At the cost of about ~945 bytes to your kernel size,
>> enabling this on a 255 CPU x86_64 qemu guest this saves about ~1.8 GiB
>> of memory during boot which would otherwise be free'd, and reduces boot
>> time by about ~11 seconds.
>>
>> Userspace loads modules through finit_module(), this in turn will
>> use vmalloc space up to 3 times:
>>
>>   a) The kernel_read_file() call
>>   b) Optional module decompression
>>   c) Our final copy of the module
>>
>> Commit 064f4536d139 ("module: avoid allocation if module is already
>> present and ready") shows a graph of the amount of vmalloc space
>> observed allocated but freed for duplicate module request which end
>> up in the trash bin. Since there is a linear relationship with the
>> number of CPUs eventually this will bite us and you end up not being
>> able to boot. That commit put a stop gap for c) but to avoid the
>> vmalloc() space wasted on a) and b) we need to detect duplicates
>> earlier.
>>
>> We could just have userspace fix this, but as reviewed at LSFMM 2023
>> this year in Vancouver, fixing this in userspace can be complex and we
>> also can't know when userpace is fixed. Fixing this in kernel turned
>> out to be easy with the inode and with a simple kconfig option we can
>> let users / distros decide if this full stop gap is worthy to enable.
>
>kmod normally uses finit_module() only if a module is not compressed,
>otherwise it decompresses it first and then invokes init_module().

that is for historical reasons, because the kernel didn't support to
uncompress the module by itself.

>
>Looking at Fedora and openSUSE Tumbleweed, they compress kernel modules
>with xz and zstd, respectively. They also have their kernels built
>without any CONFIG_MODULE_COMPRESS_{GZIP,XZ,ZSTD} options.
>
>It means that these and similarly organized distributions end up using
>init_module(), and adding complexity to optimize finit_module() wouldn't
>actually help in their case.

true, but the change in kmod should be trivial now that the kernel has
the proper support in place and the algorithms support match the ones
kmod has.  I will take a look at switching the logic around to just pass
the fd to the kernel so it can also deduplicate the requests.

thanks for the reminder,

Lucas De Marchi

>
>-- Petr