Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Song Liu <songliubraving@fb.com>
To:     Christoph Hellwig <hch@infradead.org>,
        Luis Chamberlain <mcgrof@kernel.org>,
        Linus Torvalds <torvalds@linux-foundation.org>
CC:     Song Liu <song@kernel.org>, bpf <bpf@vger.kernel.org>,
        Linux Memory Management List <linux-mm@kvack.org>,
        open list <linux-kernel@vger.kernel.org>,
        Alexei Starovoitov <ast@kernel.org>,
        Daniel Borkmann <daniel@iogearbox.net>,
        Kernel Team <Kernel-team@fb.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        "Edgecombe, Rick P" <rick.p.edgecombe@intel.com>,
        Claudio Imbrenda <imbrenda@linux.ibm.com>
Subject: Re: [PATCH v4 bpf 0/4] vmalloc: bpf: introduce VM_ALLOW_HUGE_VMAP
Thread-Topic: [PATCH v4 bpf 0/4] vmalloc: bpf: introduce VM_ALLOW_HUGE_VMAP
Thread-Index: AQHYUOjRr/eYt5HVI0ucQZg4JV/u/6zxVckAgACoe4CAAPedAA==
Date:   Sat, 16 Apr 2022 19:55:00 +0000
Message-ID: <4AD023F9-FBCE-4C7C-A049-9292491408AA@fb.com>
References: <20220415164413.2727220-1-song@kernel.org>
 <YlnCBqNWxSm3M3xB@bombadil.infradead.org> <YlpPW9SdCbZnLVog@infradead.org>
In-Reply-To: <YlpPW9SdCbZnLVog@infradead.org>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?LB2lqU+e9hw9MosFMILJASzzJAF1/OVe+FriN+OfuuIIDC0DLxc/+A/W2+tK?=
 =?us-ascii?Q?ca9Kn33QWRhtyg01E5GJCL9wT7gf2zUOcFTvP/pJi6j6FN2JkRuOIRl6jud0?=
 =?us-ascii?Q?r52KRA06JlQwBkwLGm0FwG3q2QEqBd+6IxTVFe0FB6oXFIx4FSX0HOzNIInt?=
 =?us-ascii?Q?wg1kopzBMpvXRPJj6FG3Z/iWCh2LmUmtGcL61+1/TK5wpH43bridh/5v902A?=
 =?us-ascii?Q?S4i7J4gEjoICSpXXGvAF2J45hQe6NBnd/jqUDLS8SjhrTvRRnT8BpETQzDFh?=
 =?us-ascii?Q?tsKxSMh/LADolbek3j23XkDuVVxCF/sGolvapGQZPGqTVesBlPNmn9FLZ9IR?=
 =?us-ascii?Q?R84i3/YXFEW7nrs3IenwhY3ArKeMngFdYwDDBa3iiu2EPtAJmbr5CRpNkA06?=
 =?us-ascii?Q?45UmRredXjJwkOZy3G83hVLCfqtzWFxzw3hnniPzzACM+1W7x5OhB0gRC5T+?=
 =?us-ascii?Q?YBXHa9tfrvTZMyzNOSo76q9eE91/MJrnaVQ98rWf4pcZ8BSDgeQJL4pnkRRZ?=
 =?us-ascii?Q?+9HpdsBDDkBTPaUPfbvmlGJJOeSxNCh0eoNvgW/zsNr38Le5NniSuzvdKBGN?=
 =?us-ascii?Q?SjO2pHIdvRm1X8oVI2ydC93sNh8XL2vgubCnKVbOfs0M3qLt9GFeDS8g+wsd?=
 =?us-ascii?Q?wJTrvQlycMX0oipvZMcx7WEpOQHS6jN7woKHOZJ6hjp8c5bcRji7Lqj+BlB4?=
 =?us-ascii?Q?O4A5d+CygcPQfcRE6gHD5XmL8vo2I1zcYw35BVD4n8daLXqy1YvXZO+ybYSU?=
 =?us-ascii?Q?xSkvseFI/V9ZQ9pJ0fbmZTAxEnpauERbz/JfOagFUScPS+1et3gToMu+NPjt?=
 =?us-ascii?Q?A9J3uKJvaaBVWcuU7zoPCZVQ9TmwmDvXSDElSat17qbHi1rMmnGmT+g93UAL?=
 =?us-ascii?Q?g8vmH5qW5ElKaJu69cqYWf6HULQ03Q3ciVYzd0Gpzepi2AZDx/bJzHbK6fnZ?=
 =?us-ascii?Q?SYN34Q5u3gfAjIyBvG+rzbt0BatNFjjOSX0VSMFzLjFwX5rz0Ir7Z5BvOM8s?=
 =?us-ascii?Q?rt3XhMOFh2VUHYY6B/EnfT1DRlMP5MNRwgKwpsgOC0Q2CQwPG6WrAH+92pz9?=
 =?us-ascii?Q?vaPsvfhIM7ikSCnRDjtuVrZ8yMOGa3/1AaKpE+nSpKtdbzRDve5oxW/+sErc?=
 =?us-ascii?Q?5GP63+1QpJSe0uosI27AcMWQvDzl/H1hwm54fCGm0zqJ+En5DgYBFkwBRJ3G?=
 =?us-ascii?Q?9QZoQuR3yFf5t7Bus8uwUI7te7JNyyNL8ekB4ApoczbwGQCTpQnvowslbMcR?=
 =?us-ascii?Q?bXVDD8Ul43wp2GaJN2IfRMjIL74zOQRWsHR9YoipZDiLR4IjIT6VhMJVkuLo?=
 =?us-ascii?Q?sdxKl8eKIQosBxGcwJRt8zZGPB8lUMofdld0mAkcXV63VfdKawLfXbWlQH9O?=
 =?us-ascii?Q?EZ1oJsc0McMtFYKKmr4solw/bjyeU6bgrvYyHee+/V+tQEREeOftsVqn4+o/?=
 =?us-ascii?Q?LL33G1RpehY1/cCnMq6ejpsk8RAehjml9A0TwZj+6olIO5x/Jjn2tcGfhanr?=
 =?us-ascii?Q?mfMjEy9hG3fvg4+NxVk5WeS8UWVVCYlr+uHu9wM5ytywZPnJj5p52BkveSqG?=
 =?us-ascii?Q?UxxqSJWVy0rxQItQErBYr9s+Vy29KTp4VDIqm1NPWzNvE2bfgYDCgRbbnUNy?=
 =?us-ascii?Q?sF32Mu4LXmRGx6gKUwg5lGvtuNFqvHai+15VkwrUeuls5YHl5hhWJyu9KOT1?=
 =?us-ascii?Q?4QzqSJ+lqnjoqNwv+fNeHOXHUuriMer/nrR5UmxrwRptggM7/pmD5kp8PqiX?=
 =?us-ascii?Q?uiliuJHIHPEPZJm6sPideqJv2MO1uy4=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-ID: <BAD8ABAD9E1DFB4F8BF7E00F388E316C@namprd15.prod.outlook.com>
X-OriginatorOrg: fb.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SA1PR15MB5109.namprd15.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 63155dae-8eda-49b7-477e-08da1fe2fd99
X-MS-Exchange-CrossTenant-originalarrivaltime: 16 Apr 2022 19:55:00.3471
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 27RHXNNGuJwC25N0VC251Jf1HxFwLcXd6K+jfsyyFfxRVi5tVrFVPvVA+pbHH4r5TEjmpRB0Cu3t3B37LQxydA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR15MB2882
X-Proofpoint-GUID: bJxexOqTTyc8jVwJJsSQTGpDj6NRE0P1
X-Proofpoint-ORIG-GUID: bJxexOqTTyc8jVwJJsSQTGpDj6NRE0P1
X-Proofpoint-UnRewURL: 0 URL was un-rewritten
MIME-Version: 1.0
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.205,Aquarius:18.0.858,Hydra:6.0.486,FMLib:17.11.64.514
 definitions=2022-04-16_08,2022-04-15_01,2022-02-23_01
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


> On Apr 15, 2022, at 10:08 PM, Christoph Hellwig <hch@infradead.org> wrote:
> 
> On Fri, Apr 15, 2022 at 12:05:42PM -0700, Luis Chamberlain wrote:
>> Looks good except for that I think this should just wait for v5.19. The
>> fixes are so large I can't see why this needs to be rushed in other than
>> the first assumptions of the optimizations had some flaws addressed here.
> 
> Patches 1 and 2 are bug fixes for regressions caused by using huge page
> backed vmalloc by default.  So I think we do need it for 5.18.  The
> other two do look like candidates for 5.19, though.

Thanks Luis and Christoph for your kind inputs on the set. 

Here are my analysis after thinking about it overnight. 

We can discuss the users of vmalloc in 4 categories: module_alloc, BPF 
programs, alloc_large_system_hash, and others; and there are two archs
involved here: x86_64 and powerpc. 

With whole set, the behavior is like:

              |           x86_64            |       powerpc
--------------------------------------------+----------------------
module_alloc  |                        use small pages 
--------------------------------------------+----------------------
BPF programs  |      use 2MB pages          |  use small changes
--------------------------------------------+----------------------
large hash    |           use huge pages when size > PMD_SIZE
--------------------------------------------+----------------------
other-vmalloc |                      use small pages 


Patch 1/4 fixes the behavior of module_alloc and other-vmalloc. 
Without 1/4, both these users may get huge pages for size > PMD_SIZE 
allocations, which may be troublesome([3] for example). 

Patch 3/4 and 4/4, together with 1/1, allows BPF programs use 2MB 
pages. This is the same behavior as before 5.18-rc1, which has been 
tested in bpf-next and linux-next. Therefore, I don't think we need
to hold them until 5.19. 

Patch 2/4 enables huge pages for large hash. Large hash has been 
using huge pages on powerpc since 5.15. But this is new for x86_64. 
If we ship 2/4, this is a performance improvement for x86_64, but
it is less tested on x86_64 (didn't go through linux-next). If we 
ship 1/4 but not 2/4 with 5.18, we will see a small performance 
regression for powerpc. 

Based on this analysis, I think we should either 
  1) ship the whole set with 5.18; or
  2) ship 1/4, 3/4, and 4/4 with 5.18, and 2/4 with 5.19. 

With option 1), we enables huge pages for large hash on x86_64 
without going through linux-next. With option 2), we take a small
performance regression with 5.18 on powerpc. 

Of course, we can ship a hybrid solution by gating 2/4 for powerpc 
only in 5.18, and enabling it for x86_64 in 5.19. 

Does this make sense? Please let me know you comments and 
suggestions on this. 

Thanks,
Song

[3] https://lore.kernel.org/lkml/14444103-d51b-0fb3-ee63-c3f182f0b546@molgen.mpg.de/