Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   ebiederm@xmission.com (Eric W. Biederman)
To:     Pratik Sampat <psampat@linux.ibm.com>
Cc:     Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        containers@lists.linux.dev, containers@lists.linux-foundation.org,
        legion@kernel.org, akpm@linux-foundation.org,
        christian.brauner@ubuntu.com, hannes@cmpxchg.org,
        mhocko@kernel.org, Alexey Makhalov <amakhalov@vmware.com>,
        llong@redhat.com, pratik.r.sampat@gmail.com
References: <ac76aada-f94d-d596-9b3c-1dca5a9914d0@linux.ibm.com>
Date:   Thu, 22 Jul 2021 10:22:55 -0500
In-Reply-To: <ac76aada-f94d-d596-9b3c-1dca5a9914d0@linux.ibm.com> (Pratik
        Sampat's message of "Thu, 22 Jul 2021 13:23:33 +0530")
Message-ID: <874kcmmkz4.fsf@disp2133>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
Subject: Re: [RFD] Provide virtualized CPU system information for containers
Precedence: bulk


As stated I think this idea is a non-starter.

There is a real problem that there are applications that have a
legitimate need to know what cpu resources are available for them to use
and we don't have a good interfaces for them to request that
information.

I think MESOS solved this by passing a MAX_CPUS environment variable,
and at least the JVM was modified to use that variable.

That said as situations can be a bit more dynamic and fluid having
something where an application can look and see what resources are
available from it's view of the world seems reasonable.

AKA we need something so applications can stop conflating physical
cpu resources that are available with cpu resources that are allowed
to be used in an application.

This might be as simple as implementing a /proc/self/cpus_available
file.

Without the will to go through find existing open source applications
that care and update them so that they will use the new interface I
don't think anything will really happen.

The problem I see with changing existing interfaces that describe the
hardware is that the definition becomes unclear and so different
applications can legitimately expect different things, and it would
become impossible to implement what is needed correctly.

The problem I see with using cgroup interfaces is that they are not
targeted at end user applications and but rather are targeted at the
problem of controlling access to a resource.  Using them report what is
available again gets you into the multiple master problem.  Especially
as cgroups may not be the only thing in the system controlling access to
your resource.

So I really think the only good solution that people won't mind is to go
through the applications figure out what information is legitimately
needed from an application perspective, and build an interface tailored
for applications to get that information.

Then applications can be updated to use the new interface, and as the
implementation of the system changes the implementation in the kernel
can be updated to keep the applications working.

Eric