Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Tue, 7 Sep 2021 12:42:01 -0700
From:   Roman Gushchin <guro@fb.com>
To:     Shakeel Butt <shakeelb@google.com>
CC:     Tejun Heo <tj@kernel.org>, Jens Axboe <axboe@kernel.dk>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        kernel test robot <oliver.sang@intel.com>,
        Vasily Averin <vvs@virtuozzo.com>,
        Alexander Viro <viro@zeniv.linux.org.uk>,
        Alexey Dobriyan <adobriyan@gmail.com>,
        Andrei Vagin <avagin@gmail.com>,
        Borislav Petkov <bp@alien8.de>, Borislav Petkov <bp@suse.de>,
        Christian Brauner <christian.brauner@ubuntu.com>,
        Dmitry Safonov <0x7f454c46@gmail.com>,
        "Eric W. Biederman" <ebiederm@xmission.com>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        "H. Peter Anvin" <hpa@zytor.com>, Ingo Molnar <mingo@redhat.com>,
        "J. Bruce Fields" <bfields@fieldses.org>,
        Jeff Layton <jlayton@kernel.org>,
        Jiri Slaby <jirislaby@kernel.org>,
        Johannes Weiner <hannes@cmpxchg.org>,
        Kirill Tkhai <ktkhai@virtuozzo.com>,
        Michal Hocko <mhocko@kernel.org>,
        Oleg Nesterov <oleg@redhat.com>,
        Serge Hallyn <serge@hallyn.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Vladimir Davydov <vdavydov.dev@gmail.com>,
        Yutian Yang <nglaive@gmail.com>,
        Zefan Li <lizefan.x@bytedance.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        LKML <linux-kernel@vger.kernel.org>, <lkp@lists.01.org>,
        kernel test robot <lkp@intel.com>,
        "Huang, Ying" <ying.huang@intel.com>,
        Feng Tang <feng.tang@intel.com>,
        Zhengjun Xing <zhengjun.xing@linux.intel.com>
Subject: Re: [memcg] 0f12156dff: will-it-scale.per_process_ops -33.6%
 regression
Message-ID: <YTfAiaIbxM0CqEnj@carbon.dhcp.thefacebook.com>
References: <20210907150757.GE17617@xsang-OptiPlex-9020>
 <dbc9955d-6c28-1dd5-b842-ef39a762aa3b@kernel.dk>
 <CAHk-=wgrF65BpP6P6_a6+k+C8V+JOBiEx0X-c9idiM87uk32FQ@mail.gmail.com>
 <774dee70-69bd-9f95-d197-4cff83e4e633@kernel.dk>
 <YTedOVCV3s6Z210f@carbon.dhcp.thefacebook.com>
 <YTeeBf64yUwj01Sf@slm.duckdns.org>
 <YTeh1WkseQtyboM9@carbon.dhcp.thefacebook.com>
 <CALvZod4=uY=5Zp_GCzjjMzCbs16856dR85SBWv-dpNc4kBrO+Q@mail.gmail.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <CALvZod4=uY=5Zp_GCzjjMzCbs16856dR85SBWv-dpNc4kBrO+Q@mail.gmail.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?yyH9lM33OZpwNBAGoxzmyX1m0+B2qtDiGJK5Mynxjpjv0zq5kjIqlqLj7oqZ?=
 =?us-ascii?Q?iUFGE0mI3te0hC7ZKW0MBM3uY+m5AgKCLDq/+74tfoWQ5JEIK6ZVRMQJVRkY?=
 =?us-ascii?Q?+eib1WRfTtwolTx86i4A86upD//eCGdDQ1d5h86aOVJHN0SOEmaX3GIfsMUO?=
 =?us-ascii?Q?5hUMsD72H6bAOoOj7QkXmbjgxRxZrln19PGdYbOGR2lojZc9wCfzE+JJ9P/Q?=
 =?us-ascii?Q?9qX4IPVXmH2J5s6RhSjlbx+HcrW/MpbaXqkcc6lbI817PqqDqPjAVSGHGtaE?=
 =?us-ascii?Q?ed69SMf86Q1NZP+uPYl3u6FCTHcKSX679dJagQWnud98jNQeK9q4NYX5nrRG?=
 =?us-ascii?Q?kWk1FLkDVMRd1Eyo5LbKbzxIJsxynxgI8hdzdAGvqFZarUHeV+3wEUMg9KKY?=
 =?us-ascii?Q?rtd1O4xbnLDaXVW/pVapzUsdfkmgA5hVoIUSt9UPtY+Hm+P8UpifHuoiLKfT?=
 =?us-ascii?Q?iQsjp47HXw72k0pDJwd1mZbb++OW8putwmJ1DeLeSflJC2LOY9y5vkbTpj3X?=
 =?us-ascii?Q?PZjcUQlPIYYiEpcmLV+7c6q2E7vXcHSXdfw6w8VFxPdGW/iWeF008/K6F981?=
 =?us-ascii?Q?76EXdJj/w1fNPP3rqC5IoPl5+qLgzmellgOf4EPNLpXuPp2T0gsTlvI10VXC?=
 =?us-ascii?Q?fun47yr7yhQEbteRDEeSZeDvBXwmykvq5Yx1XW51l3/7h0PlNYi8IoWxg1YC?=
 =?us-ascii?Q?FxQYrWNe6GCdzxiWrYyk8ezNd+CdfNMNUSvrxeMtUBydisWSv6UneN7ZXa72?=
 =?us-ascii?Q?em6vpLpBItodyTLD7AcjR9vzFuFihaAP1bE+9GxL+GKQ83HOOHn4efZ634pK?=
 =?us-ascii?Q?0uTeeV+W2d6ZgAA2eLDnXYC9FXnftyP7Ho16O4FUWIxG60+JqngS/727PRff?=
 =?us-ascii?Q?C4Bix42p/Q3Ltv5o2r4NCUZ4EJpW8E3S35VVKin2llIKfKSkpIqgsI4swX7f?=
 =?us-ascii?Q?+Rz7JyQe0TcSSUnj94InQRX+FWtkIuyCrn5crNnxwULUhfGUNI1Rom985Pf9?=
 =?us-ascii?Q?9ttnKSDmajg7ykGRLlIoowLXNI6pcgLy4YDHNAj8YMRzRUzXGJNDtuhyYep6?=
 =?us-ascii?Q?Nza+OpYikYtILyhtVqXK4KVcchoeRlA1+5m5T0JXUIFUmeSwNRgkdV8Jxsnn?=
 =?us-ascii?Q?uICtL9EuB+lLzrsvc0stKHGeTNg1WU1XuCrFvTgv9fWJNVpoJ6hrMeynP38a?=
 =?us-ascii?Q?Kc4eu6qyUVcOE3eKG3n0lyeT0MMFM3xqYlNTPPytify1UJQYW/ID/wMfDntc?=
 =?us-ascii?Q?fzsZWhonNoyPvsdJK1hkFBwEFB8H9NhHCOBXVTDxDDQKZjeqkv+2R+oNTnyC?=
 =?us-ascii?Q?BnEnl/1ICNYqmotJpIPO34UlzEdDHDUo2fErNnvU63bmTQ=3D=3D?=
X-MS-Exchange-CrossTenant-Network-Message-Id: da73e15d-fc4b-42ea-8805-08d97237932a
X-MS-Exchange-CrossTenant-AuthSource: BYAPR15MB4136.namprd15.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Sep 2021 19:42:06.9590
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: wtVfs4Fyt8A4ycadrZBzkskTsznWBmwlV/52OrH3zuGh016rO/hhovqW51eXx0oc
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR15MB4273
X-OriginatorOrg: fb.com
X-Proofpoint-GUID: CsRE8ZSoxbzpiG_LNHbgzC9rMO-ArMPA
X-Proofpoint-ORIG-GUID: CsRE8ZSoxbzpiG_LNHbgzC9rMO-ArMPA
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391,18.0.790
 definitions=2021-09-07_07:2021-09-07,2021-09-07 signatures=0
X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 phishscore=0 adultscore=0
 spamscore=0 lowpriorityscore=0 bulkscore=0 suspectscore=0 mlxscore=0
 impostorscore=0 clxscore=1015 mlxlogscore=999 priorityscore=1501
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109070126
X-FB-Internal: deliver
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Tue, Sep 07, 2021 at 10:48:06AM -0700, Shakeel Butt wrote:
> On Tue, Sep 7, 2021 at 10:31 AM Roman Gushchin <guro@fb.com> wrote:
> >
> > On Tue, Sep 07, 2021 at 07:14:45AM -1000, Tejun Heo wrote:
> > > Hello,
> > >
> > > On Tue, Sep 07, 2021 at 10:11:21AM -0700, Roman Gushchin wrote:
> > > > There are two polar cases:
> > > > 1) a big number of relatively short-living allocations, which lifetime is well
> > > >    bounded (e.g. by a lifetime of a task),
> > > > 2) a relatively small number of long-living allocations, which lifetime
> > > >    is potentially indefinite (e.g. struct mount).
> > > >
> > > > We can't use the same approach for both cases, otherwise we'll run into either
> > > > performance or garbage collection problems (which also lead to performance
> > > > problems, but delayed).
> > >
> > > Wouldn't a front cache which expires after some seconds catch both cases?
> >
> > I'm not sure. For the second case we need to pack allocations from different
> > tasks/cgroups into a small number of shared pages. It means the front cache
> > should be really small/non-existing. For the first case we likely need a
> > substantial cache. Maybe we can do something really smart with scattering
> > the cache over multiple pages, but I really doubt.
> 
> I think we need to prototype this to sensibly evaluate. Let me know if
> you want to take a stab at this otherwise I can try.

If you have time and are ready to jump in, please, go on. Otherwise I can start
working on it in few weeks. In any case, I'm happy to help with discussions, code
reviews & whatever else I can do.

Thanks!