Received-SPF: pass (google.com: domain of linux-kernel+bounces-214895-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Date: Fri, 14 Jun 2024 08:30:20 -0400
From: "Liam R. Howlett" <Liam.Howlett@oracle.com>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: Bert Karwatzki <spasswolf@web.de>, linux-mm@kvack.org,
        linux-kernel@vger.kernel.org, linux-next@vger.kernel.org
Subject: Re: commit 1c29a32ce65f4cd0f1c causes Bad rss-counter state and
 firefox-esr crash in linux-next-20240613
Message-ID: <w5uejhiciolye2ikgsdjim25z7lau7km3tu6t2vby3kuxeshos@osowmu4ecng5>
Mail-Followup-To: "Liam R. Howlett" <Liam.Howlett@oracle.com>, 
	Andrew Morton <akpm@linux-foundation.org>, Bert Karwatzki <spasswolf@web.de>, linux-mm@kvack.org, 
	linux-kernel@vger.kernel.org, linux-next@vger.kernel.org
References: <f4ea65e28ac47e6acb13f401f46a414a2f50f7bf.camel@web.de>
 <20240613170351.cc3e7ccfa1bb198b57f31352@linux-foundation.org>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240613170351.cc3e7ccfa1bb198b57f31352@linux-foundation.org>
User-Agent: NeoMutt/20231103
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?us-ascii?Q?T2Ulpn2hBbn2WcOuaUCXrzf/KfUV9sJcEg4PCVR+WSqca+izXLjCxz5lOOaF?=
 =?us-ascii?Q?VwKM4ZQFCHVEGycPqU2PwFcrm9VYbtnCF0CX0eIyGvJmCsMD+ZpOuHCWp/ww?=
 =?us-ascii?Q?HL28PMAhdxRyE8aHutc+XnskNxL44eyLFs31TeCzl23DxkiVkpJX82DzFhQB?=
 =?us-ascii?Q?Iyn0SF1krUF4bL7lNl0CDMW9wmKNRfFeW2baw5HtJ5ESCKiiq3sllUPGOmb1?=
 =?us-ascii?Q?+weXyw4NtqR1KewSXPO40OzcIoTzDT7QWiF2ZLRiqJdpGo5q1S2coftIVfJ2?=
 =?us-ascii?Q?Hb3JnEH1Cmo1GoTxgvtJNiwvnEidt3HBbSnz7FiQ705+zKmPsK/ve2vkI+pO?=
 =?us-ascii?Q?xtRqlsgkZxaS1lU8fP1X4UQmhC0SSpG/C2maGzLEPKMVJueMRqoIF+1Vyncd?=
 =?us-ascii?Q?BJMr8RccAXSmXXK68rUVDH4KD9sUoehTMOjD2YG1XLKhfkQ53VzZJ7tPy2wT?=
 =?us-ascii?Q?afnuEaVYXKVft5/BGPCOP+uz465KSnJDkIgI9c6Qof18TO3Tw+KXm2x/7qWj?=
 =?us-ascii?Q?pT21cP464fMnmMO2hS0D8E9VBy0ZtCkglUuseyq/40q+oruWRN15fxiMb4ih?=
 =?us-ascii?Q?aDgJyOC/dsTQwUdBTguhCNtssmVEJaGW61VI/NgGjlT/7ZtPkkPaZvjCd0Eo?=
 =?us-ascii?Q?hhWGsLkM5Z/TSz9w+NnMPDAm16DX5+f2OXF/Ez+TXnN1+lF3D8k224zrGVIF?=
 =?us-ascii?Q?sHeNkr4xsHtbIeKdr8FdNrpugJx2wqacnQgdQfx/bWOtseR+aZSEDvZ067j6?=
 =?us-ascii?Q?dNJCjX18ar5sIZ/cY3VY2vQNCql9clEDueJSK5O413skGug/o5E+GiAz9eOo?=
 =?us-ascii?Q?pevNOUUwjwLmZ7tnVVyHkAHs0i5ZzZo3LYhl6BRJRfQly+UmlPXRcnXL8emK?=
 =?us-ascii?Q?ZlEYgAnmWrmtRQ84yyutVrskH2RWonks4ycwyZwkKmpU01aup+bbsMuXJnA+?=
 =?us-ascii?Q?RVJqBq3mskHLTSLjIHo99VCnGFuthg9YBtnqiplFOwDpDF3FSv1OxK4h/vYJ?=
 =?us-ascii?Q?btcjYV4QFJy/1Z0NMFinr5yEe6+3LTjt/9Zz/cW7q/naZn3o4NzfUoHpd0DZ?=
 =?us-ascii?Q?Ij996zesuf0BfsXVEmZXTBzQID3uVEqC3kWvarztisjjmpIm4/SlZu4xzBm+?=
 =?us-ascii?Q?g2ZveoTVQ76xed9ewfNHeBmVpp0ZRXufizp9zzV/KdRv+gNEyDmMCV67UpYT?=
 =?us-ascii?Q?NUs3Oxt6VfJWA0hDQAlvtutlXAXNLDPmH/K3j8MGen+UcXOkAdSXI3t5DKkW?=
 =?us-ascii?Q?msI7n6Ra9nk4ugVdttazbIMd9/mWNh2DO7760SayETQTB0LGNuPP5GiuG9uV?=
 =?us-ascii?Q?NJA3faTj1TATLpsWwJNI7z1CCC5VETBnfclM9STujNp5y/IUT1EMZJoISbfN?=
 =?us-ascii?Q?V4ubreEJcAtD5Pc/Z6JwhLpwflOvr1cSjB9XDWMTyLSOidDS/TSBT48e/tUL?=
 =?us-ascii?Q?Y4oQmjxWJH4tOGDm3N8mq2eS2B+JDKsJ3KKJNTI3J2aO4yZ9QUoxcFKDK1AA?=
 =?us-ascii?Q?XcXlQv14cuVguIQXNnITl6dYv6NWONRx4kZCK++z9vBs777SI9qALKo3Pg4J?=
 =?us-ascii?Q?FvBR1GCrPOKmIPq0qK0LWoFYuW3YIL96PApQZ3iw?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	m7hLMG4lTMukq5GI9PTxE/h5IYuK1S37huy1U8xpaUOA76ynMGSJWsf+dvu5zvBKWLv6b+/L99QZTCPAEWBFtL6IWxIWx5uKSQkDjRGtWj6eXzLqqx/XOR9Ec+UcUgMAAnVXRX99K582OAL4GQ3JoGNFN+7ghemZiSbR5WrynXsOidawEaVqTa3wCMWIfEE6x0RlsZ3aytio3q0GCGu/wlxSxkKWBy8vEW61DthTan0AkIwgZUoirGMvDceovp5fvAZCHydH5iYYzwFaKiy2R9JVaSOAm570cY7eGGYnLifg3kzjTG1U0RyAbhvinO3hyqD3928/r5fpeJ/SXEkHoG59RDiJHbzA/vON3sVveIxYmqVpwxaDNjZa3BUQx20fw7LOiNRz5d9o23YkSGYjuDXLmzsx5V9MB47JE7Z4ZWwqokSq1fdC7/kd7j/5yRffr46Sj8P/VSk+TLFWMyjrPsIGF1F3SiM9i++vPyqmr6CSkUK+b/dYEQKi16e1TUQd5QUnf7dl8wvrcQ0oLDFT94dOIdpN+kSf4azsRZ/SLBVztYiVC5Nppy51eKeilu1U6aWuL6B927FCmty8apbM0UI2ZeZBUCGUY2ia+kbrAMI=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9f4b1a5b-ae28-46d1-2d15-08dc8c6dc310
X-MS-Exchange-CrossTenant-AuthSource: DS0PR10MB7933.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Jun 2024 12:30:23.2694
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: /vw2dgCuR6XF5poYzrY7Per7TblXr4JE6MzRWTypzUT7Dz1G5PktHl9zCTUGL2AcZH48S3v/9WUiIF0YpgdqqA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR10MB5044
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.28.16
 definitions=2024-06-14_09,2024-06-14_03,2024-05-17_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=999
 bulkscore=0 malwarescore=0 spamscore=0 phishscore=0 mlxscore=0
 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2405010000 definitions=main-2406140085
X-Proofpoint-GUID: uncMMp66nCiDCu2x1Dyrsv64mSEOkGlS
X-Proofpoint-ORIG-GUID: uncMMp66nCiDCu2x1Dyrsv64mSEOkGlS

* Andrew Morton <akpm@linux-foundation.org> [240613 20:03]:
> On Fri, 14 Jun 2024 01:40:54 +0200 Bert Karwatzki <spasswolf@web.de> wrote:
> 
> > Since linux-next-20240613 firefox-esr crashes after several minutes of browsing
> > giving the following error messages in dmesg:
> > [ T2343] BUG: Bad rss-counter state mm:00000000babe0c39 type:MM_ANONPAGES val:86
> > [ T4063] show_signal_msg: 16 callbacks suppressed
> > [ T4063] Isolated Web Co[4063]: segfault at 396d1686c000 ip 0000396d1686c000 sp
> > 00007ffd767b30a8 error 14 likely on CPU 7 (core 3, socket 0)
> > [ T4063] Code: Unable to access opcode bytes at 0x396d1686bfd6.
> > [ T4211] BUG: Bad rss-counter state mm:00000000cd9fc541 type:MM_ANONPAGES
> > val:817
> > [ T3798] BUG: Bad rss-counter state mm:00000000432d87c2 type:MM_ANONPAGES
> > val:181
> > [ T5548] BUG: Bad rss-counter state mm:00000000034aa27a type:MM_ANONPAGES
> > val:242
> > [ T3823] BUG: Bad rss-counter state mm:0000000099734197 type:MM_ANONPAGES
> > val:137
> > [    T1] BUG: Bad rss-counter state mm:000000005e5e2f2f type:MM_ANONPAGES val:28
> 
> Let's hope Linus doesn't read this.  Why are we nuking the entire
> planet just because some counter went wonky?

I think I know what's going on, and it's more than just the counters
being off here.  The counters are the symptom of what is happening.

> 
> > (these are the error messages of several crashes and the error seems to affect
> > other processes, too (T1))
> > 
> > The crash can be provoked to appear in ~1min by opening large numbers of tabs in
> > firefox-esr (by holding pressing ctrl+t for some time). With this I bisected the
> > error to commit "1c29a32ce65f mm/mmap: use split munmap calls for MAP_FIXED" and
> > reverting this commit in linux-next-20240613 fixes the issue for me.
> 
> Thanks, that must have taken a lot of time.

Yes, thank you for all that work and apologies in creating this
frustrating situation.

Andrew, please drop the set from your branch.  I need to write some more
tests, but I suspect I will need to do some work around the vma_merge()
function, which is never a fun endeavor.

Regards,
Liam