Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Tue, 27 Apr 2021 16:17:33 +0200
From:   Michal Hocko <mhocko@suse.com>
To:     Alexander Sosna <alexander@sosna.de>
Cc:     Chris Down <chris@chrisdown.name>, linux-mm@kvack.org,
        linux-kernel@vger.kernel.org
Subject: Re: [PATCH] Prevent OOM casualties by enforcing memcg limits
Message-ID: <YIgc1G496grYBbSH@dhcp22.suse.cz>
References: <ea6db5cc-f862-7c4b-d872-acb29c2d8193@sosna.de>
 <YIdWMC/iAdanDjLh@chrisdown.name>
 <410a58ba-d746-4ed6-a660-98b5f99258c3@sosna.de>
 <YIfGbd4wupW4mdHy@dhcp22.suse.cz>
 <c7e0a2f9-0b83-2d9b-8ec1-8141d5dca554@sosna.de>
 <YIf/bOhWIKPuwIzg@dhcp22.suse.cz>
 <93fcbc37-8c8c-a752-a191-ff8e3dc02eb1@sosna.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <93fcbc37-8c8c-a752-a191-ff8e3dc02eb1@sosna.de>
Precedence: bulk

On Tue 27-04-21 15:43:25, Alexander Sosna wrote:
> 
> On 27.04.21 14:11, Michal Hocko wrote:
[...]
> > Well, I am afraid that a reliable and easy solutions would be extremely
> > hard to find. A memcg aware overcommit policy is certainly possible but
> > as I've said it would require an additional accounting, it would be
> > quite unreliable - especially with small limits where the mapped (and
> > accounted) address space is not predominant. A lack of background
> > reclaim (kswapd in the global case) would result in ENOMEM reported even
> > though there is reclaimable memory to satisfy the reserved address space
> > etc.
> 
> Thank you very much for this information.  Would you share the opinion
> that it would be too hacky to define an arbitrary memory threshold here?
>  One could say that below a used memory of X the memory cgroup limit is
> not enforced by denying a malloc().  So that the status quo behavior is
> only altered when the memory usage is above X.  This would mitigate the
> problem with small limits and does not introduce new risks or surprises,
> because in this edge case it will behaves identical to the current kernel.

It will not. Please read again about the memory reclaim concern. There
is no background reclaim so (and I believe Chris has mentioned that in
other email) the only way to balance memory consumption (e.g. caches)
would be memory allocations which are excluded from the virtual memory
accounting. That can lead to a hard to predict behavior.

> >> Could
> >> you elaborate on where you see "a lot of fallouts"?  overcommit_memory 2
> >> is only set when needed for the desired workload.
> > 
> > My above comment was more general to the approach Linux is embracing
> > overcommit and relies on oom killer to handle fallouts. This to change
> > would lead to lot of fallouts. E.g. many syscalls returning unexpected
> > and unhandled ENOMEM etc.
> 
> We are talking about a special use case here.  Do you see a problem in
> the domain where and how overcommit_memory=2 is used today?

yes I do. I believe I have already provided some real challenges. All
that being said, a virtual memory overcommit control could be
implemented but I am not sure this is worth the additional complexity
and overhead introduced by the additional accounting.
-- 
Michal Hocko
SUSE Labs