Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Song Liu <songliubraving@fb.com>
To:     Peter Zijlstra <peterz@infradead.org>
CC:     Song Liu <song@kernel.org>, bpf <bpf@vger.kernel.org>,
        lkml <linux-kernel@vger.kernel.org>,
        Linux-MM <linux-mm@kvack.org>,
        "linux-modules@vger.kernel.org" <linux-modules@vger.kernel.org>,
        Luis Chamberlain <mcgrof@kernel.org>,
        Steven Rostedt <rostedt@goodmis.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Masami Hiramatsu <mhiramat@kernel.org>,
        "naveen.n.rao@linux.ibm.com" <naveen.n.rao@linux.ibm.com>,
        "davem@davemloft.net" <davem@davemloft.net>,
        "anil.s.keshavamurthy@intel.com" <anil.s.keshavamurthy@intel.com>,
        "keescook@chromium.org" <keescook@chromium.org>,
        "hch@infradead.org" <hch@infradead.org>,
        "dave@stgolabs.net" <dave@stgolabs.net>,
        "daniel@iogearbox.net" <daniel@iogearbox.net>,
        Kernel Team <Kernel-team@fb.com>,
        "x86@kernel.org" <x86@kernel.org>,
        "dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
        "rick.p.edgecombe@intel.com" <rick.p.edgecombe@intel.com>,
        "akpm@linux-foundation.org" <akpm@linux-foundation.org>
Subject: Re: [PATCH bpf-next 1/3] mm/vmalloc: introduce vmalloc_exec which
 allocates RO+X memory
Thread-Topic: [PATCH bpf-next 1/3] mm/vmalloc: introduce vmalloc_exec which
 allocates RO+X memory
Thread-Index: AQHYlpK25UY5kW1ND0S/krM8qG//OK18FxyAgABbw4A=
Date:   Wed, 13 Jul 2022 15:48:35 +0000
Message-ID: <7C927986-3665-4BD6-A339-D3FE4A71E3D4@fb.com>
References: <20220713071846.3286727-1-song@kernel.org>
 <20220713071846.3286727-2-song@kernel.org>
 <Ys6cWUMHO8XwyYgr@hirez.programming.kicks-ass.net>
In-Reply-To: <Ys6cWUMHO8XwyYgr@hirez.programming.kicks-ass.net>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?2CMULXxOgKEYNi698WQ6S2TeeDXfBlWvNbUp0Snx57YBXL/eDZJF31zJMfOT?=
 =?us-ascii?Q?D8qy77cpZIjmK2yVDNCbFLTJGoxHqtiJKYtl93cI3lva3EznCpFtwGRvM4Ry?=
 =?us-ascii?Q?CDgTJSlr0byrXOtZCyJyHSo6tTNZZm+aiKSOQ5l877lvz3lSeftnXEjWG7SU?=
 =?us-ascii?Q?pUyq0s5wPBQrUkKMy9W5zdxQuNlNAFON7q6cXbVSiK9U9C5kaMpIkP0YPUyw?=
 =?us-ascii?Q?2zUyWH9rClUaPL/6O6oRwU6bqo9mT5tnYQN9eUeDlrg98X9E9dWDkt6PvQ4g?=
 =?us-ascii?Q?bWnyxHZ9ola2fYhUSMLX7KWkGsGhxJlFthbZW51cy8pcDC7oMnPyk6E3yH2K?=
 =?us-ascii?Q?4gYBYakTmApI1JL/n2zfYZDgKFXBx5pc0rgTABYqlJhMkvc+xn6CPFBzosB9?=
 =?us-ascii?Q?Tm6KcesOU02TFhipqFk9fuUApzR/hYkB9ZlPzwj/abAvcoKKoUUjF5nkhev7?=
 =?us-ascii?Q?8KCjjMJtdqieSHrDcf4srrmZRf0IRupQjsrzeLRMqn31DlEOmjkuIE99xlQX?=
 =?us-ascii?Q?7tT5BlqZF/lSntuFovHp7d5NmlftqHRPYiW73P1l2QJ4Q1ve3rgaaPfvWYe7?=
 =?us-ascii?Q?4z+cGf6ziu97fxms3VSX0yyGTuWqJecJtJzXLZgDQo2uSXL9OO8ZrKtZ1h8r?=
 =?us-ascii?Q?zzDhIo70DAbc4VLT6hXXI7o2UYfnK49sKzSv3EjmC1smZyiBAPO3fNfzxLLF?=
 =?us-ascii?Q?VicOd/GZNrTJxs/2oBOu4JBWbaGslbyHKD+zbbVt5lZZXCMgZ+mXqLLgLanp?=
 =?us-ascii?Q?hexRgTIkd4Wefc5InwBMIY/L4RO/GLTy0f9HvxQNdCBgWZv5IoaVRYaJg5KZ?=
 =?us-ascii?Q?/NOTKPfzEZwNMzaSRBjqxR7hFPMsl7E5nKLy/JEArWnKv7mw4pAr9ClFBtAd?=
 =?us-ascii?Q?EcyFK7bAIGj6GxPllalbPwmfnAdyhRVxkryAbL7D18QG+LoOuwGnM2NCAM96?=
 =?us-ascii?Q?yXg+VDq5hEd2WOe5adC5FHMo1mTiMQjPei1I9EypPSli8HG1zntKwd1+BGSL?=
 =?us-ascii?Q?W2dncr27SPrfaNNKbr9ms3OUGqgSPeAIAKWlYlWf01cqC71UNUgensRTXndf?=
 =?us-ascii?Q?Y58nk0j5twIDfF4q+rjavYWyGKQrsiZk7Bnn9M6ZC8cOfMeBsB3C9AejQWOf?=
 =?us-ascii?Q?sLZ9DTjr+m5CiO7oiwT+YEqkpKbHbkZINRJgEvp0sfOMHlPAExNG8TwAgWyJ?=
 =?us-ascii?Q?IyEDnl2a8zfO4ehSKebJVQGE2qO60O9XspnEDvZ++8EsoSj97xEI826ArNYL?=
 =?us-ascii?Q?MZOQVKM0c9vGtF34o5Kz7er4v/YCzeJqxFZR+MTFabnqoch1az2ETZhep9q2?=
 =?us-ascii?Q?6M330K3zfMHpoabZYs/UkNQbPrWFBJkmLHG/e1/wDvggFIBE2JydcqfXJPfr?=
 =?us-ascii?Q?aew+pJRUf9TkW/sQywCBTdsHku2u0PSBdAuyWkgzxz5oSSwOIPWniZERGlb2?=
 =?us-ascii?Q?H3HCIR4yUL5oGu4zo9Gdv+zpnvB3QZIAgxhQKadO0g28pmM5gIu4JOKnjcGU?=
 =?us-ascii?Q?AOjREoYKZVr6CTewcsO+PthkCtBmxCad2ALfEx/F0F3C9ZW4H8SNwXMPa/He?=
 =?us-ascii?Q?ISF7A9Szw0OZ6EzEihST/BvJBdqqwUQlWN24PvckvgwnoxeCL3+TTgs/Op68?=
 =?us-ascii?Q?96SWZcZiwSN5K+OlFp0zrFU=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-ID: <F385C12D76F4DE45A924897DEE4C9110@namprd15.prod.outlook.com>
MIME-Version: 1.0
X-OriginatorOrg: fb.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SA1PR15MB5109.namprd15.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4be26d81-0d70-4119-dd52-08da64e725b2
X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Jul 2022 15:48:35.8892
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: sWMNyqKuWHgHtpkb6vPPq4fVEhpf3gLo7OEIggAJLkWUO8nGXlEzMH0GY06fVBQ00z65jAkFHqKs4BGn52wWKw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR15MB4297
X-Proofpoint-ORIG-GUID: n2YKRCvW-1Jk1Bei-v-9dcVEzGlaHmyi
X-Proofpoint-GUID: n2YKRCvW-1Jk1Bei-v-9dcVEzGlaHmyi
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1
 definitions=2022-07-13_05,2022-07-13_03,2022-06-22_01
X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


> On Jul 13, 2022, at 3:20 AM, Peter Zijlstra <peterz@infradead.org> wrote:
> 
> On Wed, Jul 13, 2022 at 12:18:44AM -0700, Song Liu wrote:
>> Dynamically allocated kernel texts, such as module texts, bpf programs,
>> and ftrace trampolines, are used in more and more scenarios. Currently,
>> these users allocate meory with module_alloc, fill the memory with text,
>> and then use set_memory_[ro|x] to protect the memory.
>> 
>> This approach has two issues:
>> 1) each of these user occupies one or more RO+X page, and thus one or
>>    more entry in the page table and the iTLB;
>> 2) frequent allocate/free of RO+X pages causes fragmentation of kernel
>>    direct map [1].
>> 
>> BPF prog pack [2] addresses this from the BPF side. Now, make the same
>> logic available to other users of dynamic kernel text.
>> 
>> The new API is like:
>> 
>>  void *vmalloc_exec(size_t size);
>>  void vfree_exec(void *addr, size_t size);
>> 
>> vmalloc_exec has different handling for small and big allocations
>> (> PMD_SIZE * num_possible_nodes). bigger allocations have dedicated
>> vmalloc allocation; while small allocations share a vmalloc_exec_pack
>> with other allocations.
>> 
>> Once allocated, the vmalloc_exec_pack is filled with invalid instructions
> 
> *sigh*, again, INT3 is a *VALID* instruction.

I am fully aware "invalid" or "illegal" is not accurate, but I am not 
sure what to use. Shall we call them "safe" instructions?

> 
>> and protected with RO+X. Some text_poke feature is required to make
>> changes to the vmalloc_exec_pack. Therefore, vmalloc_exec requires changes
>> from the arch (to provide text_poke family APIs), and the user (to use
>> text poke APIs to make any changes to the memory).
> 
> I hate the naming; this isn't just vmalloc, this is a whole different
> allocator build on top of things.
> 
> I'm also not convinced this is the right way to go about doing this;
> much of the design here is because of how the module range is mixing
> text and data and working around that.

Hmm.. I am not sure mixed data/text is the only problem here. 

> 
> So how about instead we separate them? Then much of the problem goes
> away, you don't need to track these 2M chunks at all.

If we manage the memory in < 2MiB granularity, either 4kB or smaller, 
we still need some way to track which parts are being used, no? I mean
the bitmap.  

> 
> Start by adding VM_TOPDOWN_VMAP, which instead of returning the lowest
> (leftmost) vmap_area that fits, picks the higests (rightmost).
> 
> Then add module_alloc_data() that uses VM_TOPDOWN_VMAP and make
> ARCH_WANTS_MODULE_DATA_IN_VMALLOC use that instead of vmalloc (with a
> weak function doing the vmalloc).
> 
> This gets you bottom of module range is RO+X only, top is shattered
> between different !X types.
> 
> Then track the boundary between X and !X and ensure module_alloc_data()
> and module_alloc() never cross over and stay strictly separated.
> 
> Then change all module_alloc() users to expect RO+X memory, instead of
> RW.
> 
> Then make sure any extention of the X range is 2M aligned.
> 
> And presto, *everybody* always uses 2M TLB for text, modules, bpf,
> ftrace, the lot and nobody is tracking chunks.
> 
> Maybe migration can be eased by instead providing module_alloc_text()
> and ARCH_WANTS_MODULE_ALLOC_TEXT.

If we have the text/data separation, can we just put text after _etext? 

Right now, we allocate huge pages for _stext to round_down(_etext, 2MB),
and 4kB pages for round_down(_etext, 2MB) to round_up(_etext, 4kB). To 
make this more efficient, we can allocate huge pages for _stext to 
round_up(_etext, 2MB), and use _etext to round_up(_etext, 2MB) as the
first pool of memory for module_alloc_text(). Once we used all the 
memory there, we allocate more huge pages after round_up(_etext, 2MB).

I am not sure how to make this work, but I guess this is similar to 
the idea you are describing here? However, we will need some bitmap 
to track the usage of these memory pools, right?

Thanks,
Song