Date:   Thu, 23 Dec 2021 12:27:37 -0800
From:   Roman Gushchin <guro@fb.com>
To:     Shakeel Butt <shakeelb@google.com>
CC:     Johannes Weiner <hannes@cmpxchg.org>,
        Michal Hocko <mhocko@kernel.org>,
        Muchun Song <songmuchun@bytedance.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        Linux MM <linux-mm@kvack.org>,
        LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v2] memcg: add per-memcg vmalloc stat
Message-ID: <YcTbuaNqd+ILLTe+@carbon.dhcp.thefacebook.com>
References: <20211222052457.1960701-1-shakeelb@google.com>
 <YcPYz4N0sXEVFaJA@carbon.dhcp.thefacebook.com>
 <CALvZod4HWG2+8WF6=OVwst1S205Z_3gB=AtbvnQ7jDpmFt0iZg@mail.gmail.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <CALvZod4HWG2+8WF6=OVwst1S205Z_3gB=AtbvnQ7jDpmFt0iZg@mail.gmail.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?uqTROVqDBD/Gc1RgDDJvVWQl8u6scMOdtKqaAPDi0PN+m52mf4GvsYMRgjdH?=
 =?us-ascii?Q?t7v+Jk1CxlXYPgLO2u4cAGfXTuKCIqp39kAEBZ8JFUVknfIcwLuj0RGFY8od?=
 =?us-ascii?Q?5nOPeOrlHYe2fDU4hUDUAPSzonyRpKIx1fe8zr7M+RqJMcsM5GNA5SAT8ehy?=
 =?us-ascii?Q?AjB0QmWardfNupVkvgTOm00JqIyHua/Qj+xf2quCkpbVxZIVlDRGHPBlBXht?=
 =?us-ascii?Q?Kfp5BW3w/3wU5gT6ad+ZsGwt0egT1RTZVGfQeWeg3xznnVI5YGpj22rViC0N?=
 =?us-ascii?Q?oxDggnRA483CubxHThTKEowA8Axgg9TdO0sPMpqmGHQXjhuU7T6/IXBlniYE?=
 =?us-ascii?Q?SFtMcq7Yk+ECDrTKL1iWHpTqTqeuLYPOQU6nqE01TrSqePTEro7bXB2lo4f6?=
 =?us-ascii?Q?SAXLEI/bTJ+oasWBDDqXLwwDUty7J95RV0cqow+lIoE1sj/KL185tdoRK5YV?=
 =?us-ascii?Q?WC76tPDZ2oDaee3dPWJded/ajwebU6Ze6lOpDBM59Tipr+ucH2zScSdFES/N?=
 =?us-ascii?Q?5+Lgm0YhedwMMU14r8krDgCaS203VzHCPVRl3BcYcKqsowOrij3OaC+BCLlo?=
 =?us-ascii?Q?IlmQOnkfPVvjWkqzSpYLiA0kYaAQAQA5Zdwn5wKXcYuq13SlJFlM3Wj9tLFQ?=
 =?us-ascii?Q?9iLwQlJGftIixSEX8QAOY/k3iSUb0qVJ5IKHcAhPNPFsrjT+97xMMVAc7ocr?=
 =?us-ascii?Q?8FfpDla+SoyUGYbIDdbfFjDBD/v67k8SCWSuAgw8pjHxiUvoW630me4qaXLx?=
 =?us-ascii?Q?hiaEIZAlRptleHuXEnHFC5mBEazj0P+VhjUvNp72O6IdMwL5998VcmYk3uAy?=
 =?us-ascii?Q?9o6jgqlD7RGEWDr5uJpN7U4zeNqqZ9Dnwd4S1v5pBh23dVLstx5LSRAQ1dva?=
 =?us-ascii?Q?4bdjlgkdC60SbY4G1qBlRa64G3NDIkUi8mFwlby8Ayfr/RjBKuItU7wzEflF?=
 =?us-ascii?Q?314wQOxjzIQFG6Y98apc+/NUh5yZjNkYOh1z4al7tSoeIDhf/59gVM/y6I81?=
 =?us-ascii?Q?mR1FWbtjLNvMV24KR0xXk9b33M1dyeLFmY8WdLJMhj9T0MoriKicfqfihr9L?=
 =?us-ascii?Q?KDVtuTzCY9+td5Pac7BQm325eHMpHeF5ZbLEvYERGkRbQtLJ36Ju/O9I+p7e?=
 =?us-ascii?Q?Ci9lR9AjtkvuMn3pQyf2gcsSfbSAafcUZk9VWH8RNBSVRxBeBBaTljkPFc0c?=
 =?us-ascii?Q?/v+5yjvDf94dmZGLmwL4sZad44q7IosVNwpSlIaYQKdaZty0ok0Pmq3+cZDt?=
 =?us-ascii?Q?QxaTxoVNeIiqySjPx0gQr8Ge0ogaz1r0wspqT1EXlCVIwF83Or2cC0VU1YxQ?=
 =?us-ascii?Q?Fja83RgAW8raxz4NNnsW5QSgInFRsBgEu5L//ZaVtChQw0M7LMZPsEx6Yczg?=
 =?us-ascii?Q?MW3dzW+mKp48zhYI40ybpaOf9XIKqPSVbZ3d5pF0au3ksu9FNOjHN0nki+xu?=
 =?us-ascii?Q?32Kdg5RciHzQtB1GG3Gnnu4gQqqCN+0mq1Rzr6IqHAW1yxpSp2V8DZF9ZbkI?=
 =?us-ascii?Q?SspcFstThHWEWsVKhVNKR1m+8wT3KY/DY7uz+ZW3vq1IPO8Io5E9vuY/0aTE?=
 =?us-ascii?Q?nLyQD+y1UjB48ZEvbRn69gynlm8k6VOT1EKNdA6L?=
X-MS-Exchange-CrossTenant-Network-Message-Id: e735fcd4-567b-4904-7af0-08d9c652ab6a
X-MS-Exchange-CrossTenant-AuthSource: BYAPR15MB4136.namprd15.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Dec 2021 20:27:41.7323
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: CF5OmAb5Xvv0v0M92jPFsV3UB8Vvd2+ZW7zcj7XGrq+pwlgiV591DBA8EEZvHlNh
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR15MB3221
X-OriginatorOrg: fb.com
X-Proofpoint-GUID: TJdLYB16QxRe0cDNXsrwrO-7bXScvdD6
X-Proofpoint-ORIG-GUID: TJdLYB16QxRe0cDNXsrwrO-7bXScvdD6
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.205,Aquarius:18.0.790,Hydra:6.0.425,FMLib:17.11.62.513
 definitions=2021-12-23_04,2021-12-22_01,2021-12-02_01
X-Proofpoint-Spam-Details: rule=fb_outbound_notspam policy=fb_outbound score=0 impostorscore=0
 suspectscore=0 malwarescore=0 mlxlogscore=908 phishscore=0
 lowpriorityscore=0 mlxscore=0 clxscore=1015 adultscore=0
 priorityscore=1501 spamscore=0 bulkscore=0 classifier=spam adjust=0
 reason=mlx scancount=1 engine=8.12.0-2110150000
 definitions=main-2112230103
X-FB-Internal: deliver
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thu, Dec 23, 2021 at 09:44:26AM -0800, Shakeel Butt wrote:
> On Wed, Dec 22, 2021 at 6:03 PM Roman Gushchin <guro@fb.com> wrote:
> >
> > On Tue, Dec 21, 2021 at 09:24:57PM -0800, Shakeel Butt wrote:
> > > The kvmalloc* allocation functions can fallback to vmalloc allocations
> > > and more often on long running machines. In addition the kernel does
> > > have __GFP_ACCOUNT kvmalloc* calls. So, often on long running machines,
> > > the memory.stat does not tell the complete picture which type of memory
> > > is charged to the memcg. So add a per-memcg vmalloc stat.
> > >
> > > Signed-off-by: Shakeel Butt <shakeelb@google.com>
> > >
> > > ---
> > > Changelog since v1:
> > > - page_memcg() within rcu lock as suggested by Muchun.
> > >
> > >  Documentation/admin-guide/cgroup-v2.rst |  3 +++
> > >  include/linux/memcontrol.h              | 21 +++++++++++++++++++++
> > >  mm/memcontrol.c                         |  1 +
> > >  mm/vmalloc.c                            |  5 +++++
> > >  4 files changed, 30 insertions(+)
> > >
> > > diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
> > > index 82c8dc91b2be..5aa368d165da 100644
> > > --- a/Documentation/admin-guide/cgroup-v2.rst
> > > +++ b/Documentation/admin-guide/cgroup-v2.rst
> > > @@ -1314,6 +1314,9 @@ PAGE_SIZE multiple when read back.
> > >         sock (npn)
> > >               Amount of memory used in network transmission buffers
> > >
> > > +       vmalloc (npn)
> > > +             Amount of memory used for vmap backed memory.
> > > +
> >
> > It's a bid sad that this counter will partially intersect with others
> > (e.g. percpu and stack), but I don't see how it can be easily fixed.
> 
> I checked those again. For vmap based stack we do vmalloc() without
> __GFP_ACCOUNT and charge the stack page afterwards with
> memcg_kmem_charge_page() interface.
> 
> I think we do the same for percpu as well i.e. not use GFP_ACCOUNT for
> underlying memory but later use objcg infrastructure to charge at
> finer grain.
> 
> So, I think at least percpu and stack should not intersect with
> vmalloc per-memcg stat.

Ah, ok then, thanks for checking!

In general, it seems that if an allocation is backed by multiple layers
of mm code (e.g. percpu on top of vmalloc), we choose one layer to do
both memcg statistics and accounting. This makes total sense.

Thanks!