Received-SPF: pass (google.com: domain of linux-kernel+bounces-145853-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Date: Mon, 15 Apr 2024 15:57:59 -0400
From: "Liam R. Howlett" <Liam.Howlett@oracle.com>
To: "Pierre-Loup A. Griffais" <pgriffais@valvesoftware.com>
Cc: David Hildenbrand <david@redhat.com>,
        Oleksandr Natalenko <oleksandr@natalenko.name>,
        linux-kernel@vger.kernel.org, vincentdelor@free.fr,
        Andrew Morton <akpm@linux-foundation.org>, linux-mm@kvack.org
Subject: Re: Increase Default vm_max_map_count to Improve Compatibility with
 Modern Games
Message-ID: <ht35j3ig4reppggvpcjpjc7gotg6ndrjwvwfiqvxrwrtr4iauh@kw46hnsliy5l>
Mail-Followup-To: "Liam R. Howlett" <Liam.Howlett@oracle.com>, 
	"Pierre-Loup A. Griffais" <pgriffais@valvesoftware.com>, David Hildenbrand <david@redhat.com>, 
	Oleksandr Natalenko <oleksandr@natalenko.name>, linux-kernel@vger.kernel.org, vincentdelor@free.fr, 
	Andrew Morton <akpm@linux-foundation.org>, linux-mm@kvack.org
References: <566168554.272637693.1710968734203.JavaMail.root@zimbra54-e10.priv.proxad.net>
 <13499186.uLZWGnKmhe@natalenko.name>
 <1a91e772-4150-4d28-9c67-cb6d0478af79@redhat.com>
 <ugjnhfiwuoslupkbmrrbbxxdosotzpqjfljebzj4dkuibmf6sr@kuew2qemsen4>
 <8f6e2d69-b4df-45f3-aed4-5190966e2dea@valvesoftware.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <8f6e2d69-b4df-45f3-aed4-5190966e2dea@valvesoftware.com>
User-Agent: NeoMutt/20231103
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?us-ascii?Q?sp9BmWKeVJ4AAXF+0+mIytBbMLIGAH7fDAc/yVPKINOyBmr5dwv2f0aS+7lh?=
 =?us-ascii?Q?5BhjZ7JLrcyGyCCeu1roB6yx4XvufPw7UuiAkFLNul9cGBzRVgKHUmr5rmgp?=
 =?us-ascii?Q?Gyas8CsT3HEmXYBHMrhUcTsrEuF+JxzxMzvS1uRxiuWBWAao8gQqIK4B85tL?=
 =?us-ascii?Q?cXLPtNKUyjgx31DklhZKFHJh5jlSIRCZBczHHsTfxEz/F7hWM3BL2VP+M2eZ?=
 =?us-ascii?Q?ZqaaLtkagjq0H4HXbkmXcprOjOaATp13yHambUue5YZyWQghiJZ72CIIfvKQ?=
 =?us-ascii?Q?G93MQuXUrphefkHNr660oGGFAlZNlX/1Z56mlU7TDvsO4Cbx6uPd2Wtfq5vV?=
 =?us-ascii?Q?W476YKHD0UdoWyt0zFJB2JTJFU7NKyLpqaP47fW5WfUeJwYrBI7IYOQlxNPe?=
 =?us-ascii?Q?u+y7Y9Dr5Hu5x505taTFJCVVMunMOQK/8i9ZCknNNhPXFam37FSRtTbPGtSx?=
 =?us-ascii?Q?WxHzO6DmbyQ0EvL8Lx+fizQci3gtNVPmnm7pD2MVCMTniO88iDT3TZLrSiR7?=
 =?us-ascii?Q?TBjWYgBlIb5WmxvZWRYArsapLxY54aG3V6jKoQv4id8x5TfzbrPtKikrZa5W?=
 =?us-ascii?Q?7AjELwZcodhSL75g+ZC2Be42qX1QgAt87s2zKxPRhQou5MqfpiHKrcGnvWK8?=
 =?us-ascii?Q?2yCXo/buGl6Afl8kUCwVWuIz98BvT3xmUbPLju8njPoLKWz90eQpvUINGSao?=
 =?us-ascii?Q?JN+is2tKUPkxN5xzQyRSVdTYqeBWwZddxmj4GZjxpYH42MKL7d1ybMTrl7b3?=
 =?us-ascii?Q?gIQk603t6JDcstBJkGoVH/9jSmmjb4R6ckh6MAJ8bLWfB/aZoNkMCBh8Wr7b?=
 =?us-ascii?Q?1u8MVY37L8EtxsyUNSm9tmH8esT4ui2/CpPgqxOH2UTEUMWFWFN5Dxrg7/Tt?=
 =?us-ascii?Q?XiVGLnTyMDm5k134w1YmHew/Vm3m9v99KkRI+Ofuv8hNqSV4YFGSTByjsyYd?=
 =?us-ascii?Q?l44HvMsB0i8xLysiHqZqsHFJ7qC5OPotQJjIv4nEhg4HTrGSLDC1q+unPn15?=
 =?us-ascii?Q?xcCts2jbRv4z7t2WlVrpLq1BkZN9+yMf9xHljMFnLX636eqBTAJX0icSZZAj?=
 =?us-ascii?Q?lmJT1SFouHdT9/B3dzX5Xgnl35RGP0/CPQ3fS2ZZiKLdKt7faDOEEjYR56Gz?=
 =?us-ascii?Q?8kXIl57SYa6n1LERh2JTlZUvLXeYif5vaI+IfoNuaSHuvHjuQRTqmzAV8MT+?=
 =?us-ascii?Q?psPfulL0pPOwL4OPcNh6WIDBkJ+bCWDR4MyiDuzrv838xf+/nu1XLAiIu9aT?=
 =?us-ascii?Q?b+ZFqj6WIpKaWG/93sVcrV7z/8BPyZj99ANF/jamqjtjjfSiRnsaqDGVdivl?=
 =?us-ascii?Q?cqtpxfYvstCS5JU8+0z1nJiK1JeO3vmrhoo6o7BOzAjb8SUaa1jn8wbIFcA4?=
 =?us-ascii?Q?mWDZBhnqRUmuQiIPH99lG1YZBzrLU0/Pks5prH2UzpcyZWzpdaEcI6ZTeMO3?=
 =?us-ascii?Q?M+iF/fQYQUnBarN8noClWljQQb3OrAPFigAoFbzWFL9Lg48lmjiBnlYj2Qqs?=
 =?us-ascii?Q?RmzdTjjzz8YeDlTG/+KRyZWHkemj33w+uEKt/AGZUrZIY04TvYp+Esgkpnc3?=
 =?us-ascii?Q?NjnV6e6Pt18a/o+32Q4echPSCRIc5ZIq3fq6asxy?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	cCvND5/WZA76mzGCRrOI6K7N42HHe0XPlBySKkc5wxlycCMqDCo2iT27aKqKvVTm2J3D7YFY2Yf/qudUTYcz2o0bcwHB3UjevrO1HslTq1srUGeJ86u402+AxCbh2frD73/maarAaZlHOW25LT+ajI6N2i9a+fwX2x/m0PKZRxUoI5db6W/UgHjx/BroEEfBa8u8A7HbmeszgUFXOHLiMhXu8Ay5lCFu+mZo2aVdu55HBADnlnHAR2OStb1OyspozniYEKhl7WUmgLNR0FWQ51igCK5duNiaTjijsm+M6WtVmM1I/2eSr0WzR4nkJ91arXjzH7of5U/GlD8+GHw3q74mKbLMgVXHEN714v1RdtaBB30ZPzFrct//QHaUnNNUwK4Q9nKDV5qjN6EnmSl1bAQv/8WKAE8CZogueOGp4Y+SdzwZayhFCbk6+IpT0oWp/Mrk/aEcLE2B/BWUaRdTIXxvpvBgiPlqMGuXOSVjdckIF8YzOzBXX2S5Bn5BXvJkXpD3LVSEsQI6Nsz8zciQpfEFq2hYFqdjhzIQgKCswHDOPBOP/rkeFVKELT33Tz8J0tun8I4viqoJ7A+1p0L0o4X7lbPiZ5+bl4xZC8Hb2yg=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f09a9b2d-3ab7-4245-6358-08dc5d865ac4
X-MS-Exchange-CrossTenant-AuthSource: DS0PR10MB7933.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Apr 2024 19:58:01.0997
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 0zH883UnTXvgQV8KSlLL8v1bKGQgcmGsx5gY1xDfFjlsU2WsF8AYcCCvIwCtWP6toV4n5QOXZjPnR+myamtDBw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA2PR10MB4715
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2024-04-15_17,2024-04-15_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 adultscore=0
 mlxlogscore=999 suspectscore=0 bulkscore=0 spamscore=0 mlxscore=0
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2404010000 definitions=main-2404150132
X-Proofpoint-GUID: l-hqDgRu-KiNgNV1OB5v2em11MiQQn2X
X-Proofpoint-ORIG-GUID: l-hqDgRu-KiNgNV1OB5v2em11MiQQn2X

* Pierre-Loup A. Griffais <pgriffais@valvesoftware.com> [240414 20:22]:
> 
> 
..

> > 
> > To be clear, what you are doing here is akin to adding more memory to
> > your system when there is a memory leak.  This is not the solution you
> > should be pushing.  Ironically, this is using more memory and performing
> > worse than it should.  At best, the limit increase is a workaround for
> > buggy programs.
> > 
> > At worst, you are enabling bad things to keep happening and normalising
> > poor programming choices.  Please put pressure on the applications that
> > clearly have issues.
> 
> We don't get to prescribe what those applications do. The fact of the matter
> is that there are several high-performance memory allocators in wide use by
> game applications that make heavy internal use of mmap(), and that using
> hundreds of thousands of different memory mappings is well supported on the
> platform those applications were written for. (or mapping regions with
> different permissions, which results in different regions after platform
> translation to Linux happens within Wine)

Thank you for the information on the situation that causes the kernel to
use such a large number of vmas.

The mmap operations will run faster if there are significantly less
vmas.  Having such a large number of objects will cause the faulting of
information into the memory to be slower, and that would hold true for
all platforms.

If this is for high-performance, then it would be unlikely that it was
designed to run with 65,530 objects to search.  It is also odd that
there are several allocators running into the same issue. If I were to
guess, the allocators are trying to bypass the operating systems use of
memory and implement another way of tracking it specific to your usecase
for speed.  It sounds like it is being translated incorrectly and
causing a monster data structure to track it on the kernel side.

If it's a translation layer in wine making a decision on how to
translate a particular set of calls then it could be fixed, or at least
examined for inefficiencies.

Either way, the performance will be sub-optimal on the page fault path
(probably the most common) and any other path that uses such a large
number of vmas.

> 
> Pointing out that there exists one game that doesn't happen to do that is
> not terribly useful for the purpose of this discussion.

I provided the data I could collect reasonably quickly, but the scale of
the difference was the important part of my statement.

> 
> The problem statement seems pretty simple - distributions that want to
> support those usecases out of the box can make that change, like we've done
> for years on SteamOS. On those that don't, users of those applications will
> have to discover and learn to apply the change by hand after having a likely
> sub-par experience trying to get their game up and running.

This number of vmas is indicating an issue with the utilisation of the
virtual memeory areas.  Increasing the limit is allowing the game to
run, but it will not be performant. It is unfortunate that the solution
was to increase the value.

> 
> I've yet to hear a specific downside of making the change other than a real
> concern about DoS of kernel memory in another discussion - it seems to me
> like there is much lower hanging fruit for DoSing a Linux system you have
> shell access to, at the moment.

Poor performance is the downside. The specific downside is the overly
large data structure that the kernel has to navigate on every page fault
or any other vma operation.  This isn't specific to changing the number,
but to the fact that it needed to be changed in the first place.

Is there an upper limit of vmas that you have seen?  Can you provide a
copy of the mappings when you see this for testing?  This works out to a
5 level maple tree.

Thanks,
Liam