Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 20 Jan 2023 12:08:15 -0500
From:   "Liam R. Howlett" <Liam.Howlett@Oracle.com>
To:     Matthew Wilcox <willy@infradead.org>
Cc:     Suren Baghdasaryan <surenb@google.com>,
        Michal Hocko <mhocko@suse.com>, akpm@linux-foundation.org,
        michel@lespinasse.org, jglisse@google.com, vbabka@suse.cz,
        hannes@cmpxchg.org, mgorman@techsingularity.net, dave@stgolabs.net,
        peterz@infradead.org, ldufour@linux.ibm.com,
        laurent.dufour@fr.ibm.com, paulmck@kernel.org, luto@kernel.org,
        songliubraving@fb.com, peterx@redhat.com, david@redhat.com,
        dhowells@redhat.com, hughd@google.com, bigeasy@linutronix.de,
        kent.overstreet@linux.dev, punit.agrawal@bytedance.com,
        lstoakes@gmail.com, peterjung1337@gmail.com, rientjes@google.com,
        axelrasmussen@google.com, joelaf@google.com, minchan@google.com,
        jannh@google.com, shakeelb@google.com, tatashin@google.com,
        edumazet@google.com, gthelen@google.com, gurua@google.com,
        arjunroy@google.com, soheil@google.com, hughlynch@google.com,
        leewalsh@google.com, posk@google.com, linux-mm@kvack.org,
        linux-arm-kernel@lists.infradead.org,
        linuxppc-dev@lists.ozlabs.org, x86@kernel.org,
        linux-kernel@vger.kernel.org, kernel-team@android.com
Subject: Re: [PATCH 39/41] kernel/fork: throttle call_rcu() calls in
 vm_area_free
Message-ID: <20230120170815.yuylbs27r6xcjpq5@revolver>
Mail-Followup-To: "Liam R. Howlett" <Liam.Howlett@Oracle.com>,
        Matthew Wilcox <willy@infradead.org>,
        Suren Baghdasaryan <surenb@google.com>,
        Michal Hocko <mhocko@suse.com>, akpm@linux-foundation.org,
        michel@lespinasse.org, jglisse@google.com, vbabka@suse.cz,
        hannes@cmpxchg.org, mgorman@techsingularity.net, dave@stgolabs.net,
        peterz@infradead.org, ldufour@linux.ibm.com,
        laurent.dufour@fr.ibm.com, paulmck@kernel.org, luto@kernel.org,
        songliubraving@fb.com, peterx@redhat.com, david@redhat.com,
        dhowells@redhat.com, hughd@google.com, bigeasy@linutronix.de,
        kent.overstreet@linux.dev, punit.agrawal@bytedance.com,
        lstoakes@gmail.com, peterjung1337@gmail.com, rientjes@google.com,
        axelrasmussen@google.com, joelaf@google.com, minchan@google.com,
        jannh@google.com, shakeelb@google.com, tatashin@google.com,
        edumazet@google.com, gthelen@google.com, gurua@google.com,
        arjunroy@google.com, soheil@google.com, hughlynch@google.com,
        leewalsh@google.com, posk@google.com, linux-mm@kvack.org,
        linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org,
        x86@kernel.org, linux-kernel@vger.kernel.org,
        kernel-team@android.com
References: <20230109205336.3665937-1-surenb@google.com>
 <20230109205336.3665937-40-surenb@google.com>
 <Y8k+syJu7elWAjRj@dhcp22.suse.cz>
 <CAJuCfpEAL9y70KJ_a=Z_kJpJnNC-ge1aN2ofTupeQ5-FaKh84g@mail.gmail.com>
 <Y8pWW9Am3mDP53qJ@dhcp22.suse.cz>
 <CAJuCfpHeuckG8YuNTgdDcNHNzJ3sQExD_f1hwXG_xmS7Z-925g@mail.gmail.com>
 <CAJuCfpF20nuP6Meib9h7NVrJv+wybYS==vZFQXxUW6n-ir9bvQ@mail.gmail.com>
 <Y8rGJq8LvX2C+Cr7@casper.infradead.org>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Y8rGJq8LvX2C+Cr7@casper.infradead.org>
User-Agent: NeoMutt/20220429
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?kaXRpA8ciJSLTcnT8FlPEbxU5t3FKre0MruHvV+S+v/t8xe/WDKfXrSg3YtY?=
 =?us-ascii?Q?D1tRgsrR6oY25x8gLSlaNRYXErnkyyk0RkqoXcZpsxwrStHgw9+qsrap3QSf?=
 =?us-ascii?Q?CJBSERjy1XStkcvnhAs3qU/DLAeFudNFWqzpG6DoICQu73XFCFijVArZOwnE?=
 =?us-ascii?Q?eSTjnBdPG6jUOzeN8EJiJO4Dq1aVD2M+lMyr9Qc8YGfaddXoZRStUrWRZKlZ?=
 =?us-ascii?Q?RmmVzmmqOde7hnsdMynlLfLpfugeE+UNSmxseqFFvZ9qRyts5Oq1rj1DBNA9?=
 =?us-ascii?Q?itWMCupnH1p8c/Z6Qf6JCDgsXpN8Pl9AINI+yAhVcm+ewKOZv6iN4Kih1B/m?=
 =?us-ascii?Q?A3TpQr2h762mmHtfc/y8yGe+IvnTus2AJ5Ovq1UPJg33qKUYh70R4YGezZc0?=
 =?us-ascii?Q?jy7LeBInG3NV4pwnvHPaHPEYYcDK1kjkaKqfmkaOYQwJUrqAZlI7JFYL7tui?=
 =?us-ascii?Q?E6i69QcrnvyOsaOB7+CHkcWVWlaxO9Zs5ohi1dKdf9oi1bZP+3x8HuTVWh4j?=
 =?us-ascii?Q?AfYt2+qlzIORJFb4t/atPDG30nWiB8pZnRvPuMyKlRMb74uAkvCK6xRDtZW/?=
 =?us-ascii?Q?D5SjWXjeNCFIKHseKdCJYfovzzqOyvMHAolTiczCrxhEIYYeOUjZtvtBKfHB?=
 =?us-ascii?Q?MH0X+NcunhiTB0i3x/4RUWjLpewv9sI62oFfw5xxrMzNux0HLeLNJ1xPtb/6?=
 =?us-ascii?Q?qarNUR4vjdMFssPJegz3gpht92/xAsNSom49VVkbEcAziMeE/Zn10qKnIkxW?=
 =?us-ascii?Q?wGSCA+95zaTA/MdUtgpngwtrx6s/AJRMKX2b5zyeog9EUz1sMzBbf6Y40lmL?=
 =?us-ascii?Q?AAsqfqso+Cf7zulD8DJgmIQYu0lo+DgEPhMdoAGZ1xG8fCYt8BujAoqIc3Mg?=
 =?us-ascii?Q?19wmbWSm3ZkDsqADc4fP31Kup6xylCEeR9HkFIG52QvYf58l6u4Ah6SAbYlA?=
 =?us-ascii?Q?Knnda56FKgXKxFiB9+a3qgFNWVjMUD6DAoCoa/6/3o2HkF88RI6Lu+2BRV0i?=
 =?us-ascii?Q?u4NVkStMoXLblTSGWimyY+z32eILQMnhQXxWP6O0LAh7iNs5JhCPsjkqFbS+?=
 =?us-ascii?Q?ulC1LHfUpMwDIX3Kf2RW+taBQ+h+lQyam2nN+gsfa3qag1T98v8+Q1IgeeHI?=
 =?us-ascii?Q?p7XElYAou3UOAi/+2Qz0oPBsGNP3wkrpOv1p2tnwZfcoaxUVFPta6d/keB2Q?=
 =?us-ascii?Q?OSDlfiKiMGw7EoWl839tzPfJB0xgnXtXJ1GPai2o1LKLUF3Ovpwa6dbcEnk1?=
 =?us-ascii?Q?jeU3CKhRfH5lCsZdCTsZQlOw30TzLe9vKtqfxfSS95ckCgoMDr1I3Alu72j5?=
 =?us-ascii?Q?4C7FBJqznEQSjkr3m8JOy02RrKIGME6iCWjib+SmdMTcX/UIIKNxYW8q2CIN?=
 =?us-ascii?Q?aucaAJZIoisqCtJIVnPxxXW2PoeMCJPiiC25Y2GmcKX6/6zRxINuwOfSH6yK?=
 =?us-ascii?Q?KPfJ/gsFEPB4DwZ/sA9QdPs7YOu8EQDmU2OwsJp5vEQUD0SJ2UiLcuF+SslF?=
 =?us-ascii?Q?jEGmtP0hLMGtyrUTIB3QkAyOG/jyvoT2m8l9Fm51BlBhl09T6luO3Ph65KqE?=
 =?us-ascii?Q?57JpWZ3xxp9Bcj5txbG/l6CAC8WgEgzeu9B0KNSqTHqP/fbSOReqdlR/ZPUW?=
 =?us-ascii?Q?vA=3D=3D?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: =?us-ascii?Q?ynL1LT2h30mNXy3xHndWdX/TvQkQsT2SpZzg5u7nUUggVeWeZxTkCRS1uGlR?=
 =?us-ascii?Q?nx2r5HaFdJb4dcajrKjXSlSqeXRFqaPkkGJ2MX9L5aw+UsgquGGVyf6ctb51?=
 =?us-ascii?Q?HrDiNp7SaomN9I2VCPmZ6dT3L0TgHdyuqwxEf7SviuxXzU4pwBSIiMc4Z9tE?=
 =?us-ascii?Q?s8SxgcQhrHrtnrcGbhvx6eMJWW+r1dc3C+7pr8yX2x9Xin1fnPybWxms9dx8?=
 =?us-ascii?Q?Wt18KAUxPM5Gi50alB6/qtJRt/X+znRJffoqFYIva5WiBaRs9u9sAJcoOATz?=
 =?us-ascii?Q?IZztzrBtM8koNbYq2LqEinALJQPQIMFh3Fd84Q224SFJTQ9XnQgrIbqoJ/77?=
 =?us-ascii?Q?7XmYk558kiM3SsUhZcjfrYR+iFxqq+EHaHyUDxxfKAnZBZG4pCXnS+k67q6E?=
 =?us-ascii?Q?SBUohaoW/Dwhc/Myl+L58eUFHAKx/JN0YRpURnDdQlplvzwjSkhsm0G/3cnB?=
 =?us-ascii?Q?4vw7huwZ5CQ5LjF0eSvuOb/EprH+NpuBOu1HxtjoTrbWAgx3EIyOIX+dFA0o?=
 =?us-ascii?Q?q0NhP9uHLvZBImDUZGKT/iTIC1R5177yVtVvG0vyMDPRnPJzFnsY2NHsX1gU?=
 =?us-ascii?Q?r6m78cRxbyp2BuF9BPWN0ubg2hzoCzEHEM97ZNXU/jDj8gJH9CBvoDz/pNLY?=
 =?us-ascii?Q?KpX3PhdJLEEdLk1aEBnhyoZ2W9I571LzfSNf1mKpkEq0yVS/YmoIDxj/Dogs?=
 =?us-ascii?Q?wTemqlgjgl5SYXXbw3pYyxAvQYmWK64I8wt72D59Aip29Ut3zH59j4zaZwZz?=
 =?us-ascii?Q?0PAe3WAK2n7LSJdpELRr1TBRTDMST4mmYa0rdkrWaxfjGRfmpHyGudavfaJr?=
 =?us-ascii?Q?1y7Nf8nnPVriy2qG6WUZSBLbIAepZ+tu4MjhS7z7wXCf4v8+F/tLDNBRyr3H?=
 =?us-ascii?Q?BL9AtBj5zzqdOxPG4l1bA/mk+P5/e3QeuW19BbrUmNJb1bvhnUqorbDVJg5Q?=
 =?us-ascii?Q?SuZ9p6svaotTWpR8UH8ozVbbq2T9FDnkQM4xw9sV9JbEPkkHX6kvagr+ZWdY?=
 =?us-ascii?Q?Xa7/RTFlygNxS3D7FsF5ivaA4SHB8Q8WukV4pGxuYUdH1+4QEbCJR3AQ62Q7?=
 =?us-ascii?Q?ZOw0egh4Mpn7EOGrWfXcmVm/BGj1ktxWVq0dmU+VjMX0I1Xqh5ltPOf2V9ZO?=
 =?us-ascii?Q?JzAsIKqTkFw0N0q6bzMNcGj9UA+xVdZrNdlTPi4AgrqgpNT1ESB8UJUxgCTa?=
 =?us-ascii?Q?mMY38RSb4x0Le0zLzbcCcU0juGjET0HFmA5qYEXO+6GHLINnSCpwFgtatqq4?=
 =?us-ascii?Q?sNpLGLLOm4eclGwj6d3im8zey9QxD4/D9qPwClybjjhnmEA8DJZTqhI+YTXp?=
 =?us-ascii?Q?AmZWBj6+3QB4onACsfSgiYzQb6IPtSkumS2/E5crCG2L1HJdS1JoHtSJnvR8?=
 =?us-ascii?Q?a0V+srVQwi3ykhgKTTZU0HUgi1lnOlGgvTg5GyD+nf0mzumK29ILXZ9v7B3I?=
 =?us-ascii?Q?dDuFThhKYlKjJgU3d+BC4coYOcfERM4TTxoXDpJv86sykJjiIm2W6U+gpDXn?=
 =?us-ascii?Q?x2mMiFfswaD6iZ0+nmmfeKU0j+6xONQVmib7?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8c5c1bdf-8826-4e51-a397-08dafb08f68d
X-MS-Exchange-CrossTenant-AuthSource: SN6PR10MB3022.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Jan 2023 17:08:34.3946
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: dQZXM//pXjgDcHWrgZYGRzqLZ/+UCybXd/TW/Jcwh8lehOWQCuuLX3uiIuPyI/Ib/9AjaFDq/puVeMX3W734iA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR10MB6005
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.219,Aquarius:18.0.930,Hydra:6.0.562,FMLib:17.11.122.1
 definitions=2023-01-20_09,2023-01-20_01,2022-06-22_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 bulkscore=0 suspectscore=0
 spamscore=0 malwarescore=0 mlxscore=0 mlxlogscore=999 adultscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000
 definitions=main-2301200164
X-Proofpoint-GUID: UG1YrPNHsYVTSLOF9E4iOvTDWHilwB-8
X-Proofpoint-ORIG-GUID: UG1YrPNHsYVTSLOF9E4iOvTDWHilwB-8
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

* Matthew Wilcox <willy@infradead.org> [230120 11:50]:
> On Fri, Jan 20, 2023 at 08:45:21AM -0800, Suren Baghdasaryan wrote:
> > On Fri, Jan 20, 2023 at 8:20 AM Suren Baghdasaryan <surenb@google.com> wrote:
> > >
> > > On Fri, Jan 20, 2023 at 12:52 AM Michal Hocko <mhocko@suse.com> wrote:
> > > >
> > > > On Thu 19-01-23 10:52:03, Suren Baghdasaryan wrote:
> > > > > On Thu, Jan 19, 2023 at 4:59 AM Michal Hocko <mhocko@suse.com> wrote:
> > > > > >
> > > > > > On Mon 09-01-23 12:53:34, Suren Baghdasaryan wrote:
> > > > > > > call_rcu() can take a long time when callback offloading is enabled.
> > > > > > > Its use in the vm_area_free can cause regressions in the exit path when
> > > > > > > multiple VMAs are being freed. To minimize that impact, place VMAs into
> > > > > > > a list and free them in groups using one call_rcu() call per group.
> > > > > >
> > > > > > After some more clarification I can understand how call_rcu might not be
> > > > > > super happy about thousands of callbacks to be invoked and I do agree
> > > > > > that this is not really optimal.
> > > > > >
> > > > > > On the other hand I do not like this solution much either.
> > > > > > VM_AREA_FREE_LIST_MAX is arbitrary and it won't really help all that
> > > > > > much with processes with a huge number of vmas either. It would still be
> > > > > > in housands of callbacks to be scheduled without a good reason.
> > > > > >
> > > > > > Instead, are there any other cases than remove_vma that need this
> > > > > > batching? We could easily just link all the vmas into linked list and
> > > > > > use a single call_rcu instead, no? This would both simplify the
> > > > > > implementation, remove the scaling issue as well and we do not have to
> > > > > > argue whether VM_AREA_FREE_LIST_MAX should be epsilon or epsilon + 1.
> > > > >
> > > > > Yes, I agree the solution is not stellar. I wanted something simple
> > > > > but this is probably too simple. OTOH keeping all dead vm_area_structs
> > > > > on the list without hooking up a shrinker (additional complexity) does
> > > > > not sound too appealing either.
> > > >
> > > > I suspect you have missed my idea. I do not really want to keep the list
> > > > around or any shrinker. It is dead simple. Collect all vmas in
> > > > remove_vma and then call_rcu the whole list at once after the whole list
> > > > (be it from exit_mmap or remove_mt). See?
> > >
> > > Yes, I understood your idea but keeping dead objects until the process
> > > exits even when the system is low on memory (no shrinkers attached)
> > > seems too wasteful. If we do this I would advocate for attaching a
> > > shrinker.
> > 
> > Maybe even simpler, since we are hit with this VMA freeing flood
> > during exit_mmap (when all VMAs are destroyed), we pass a hint to
> > vm_area_free to batch the destruction and all other cases call
> > call_rcu()? I don't think there will be other cases of VMA destruction
> > floods.
> 
> ... or have two different call_rcu functions; one for munmap() and
> one for exit.  It'd be nice to use kmem_cache_free_bulk().

Do we even need a call_rcu on exit?  At the point of freeing the VMAs we
have set the MMF_OOM_SKIP bit and unmapped the vmas under the read lock.
Once we have obtained the write lock again, I think it's safe to say we
can just go ahead and free the VMAs directly.