Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <CALPaoCj-zav4x6H3ffXo_O+RFan8Qb-uLy-DdtkaQTfuxY4a0w@mail.gmail.com>
 <b2e020b1-f6b2-e236-a042-4eb2fd27d8b0@intel.com> <IA1PR11MB6097236CFF891041DBA42ECB9B5F9@IA1PR11MB6097.namprd11.prod.outlook.com>
 <Y0BhzKkksSjSeE3W@agluck-desk3.sc.intel.com> <81a7b4f6-fbb5-380e-532d-f2c1fc49b515@intel.com>
 <CALPaoCjdeRjyX5L6BBX688ZM21eMwetuL9QLF1+GEDUskGcU2w@mail.gmail.com> <7b09fb62-e61a-65b9-a71e-ab725f527ded@intel.com>
In-Reply-To: <7b09fb62-e61a-65b9-a71e-ab725f527ded@intel.com>
From:   Peter Newman <peternewman@google.com>
Date:   Wed, 19 Oct 2022 11:08:51 +0200
Message-ID: <CALPaoCg4zrODVoXF2y2b+LRYq_+jnV8yv5qB+T_3Z264cV82GQ@mail.gmail.com>
Subject: Re: [RFD] resctrl: reassigning a running container's CTRL_MON group
To:     Reinette Chatre <reinette.chatre@intel.com>
Cc:     Tony Luck <tony.luck@intel.com>,
        "Yu, Fenghua" <fenghua.yu@intel.com>,
        "Eranian, Stephane" <eranian@google.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        James Morse <james.morse@arm.com>,
        Babu Moger <Babu.Moger@amd.com>,
        Gaurang Upasani <gupasani@google.com>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

Hi Reinette,

On Wed, Oct 12, 2022 at 7:23 PM Reinette Chatre
<reinette.chatre@intel.com> wrote:
> What if resctrl adds support to rdtgroup_kf_syscall_ops for
> the .rename callback?
>
> It seems like doing so could enable users to do something like:
> mv /sys/fs/resctrl/groupA/mon_groups/containerA /sys/fs/resctrl/groupB/mon_groups/
>
> Such a user request would trigger the "containerA" monitor group
> to be moved to another control group. All tasks within it could be moved to
> the new control group (their CLOSIDs are changed) while their RMIDs
> remain intact.

I think this will be the best approach for us, since we need separate
counters for every job. Unless you were planning to implement this very
soon, I will prototype it for the container manager team to try out and
submit patches for review if it works for them.

> I just read James's response and I do not know how this could be made to
> work with the Arm monitoring when it arrives. Potentially there
> could be an architecture specific "move monitor group" call.

AFAICT all we could do in that situation is hope there are plenty of
CLOSIDs, since we wouldn't be able to create any additional monitoring
groups.

What's still unclear to me is exactly how an application would interpret
the reported CLOSID and RMID counts to decide whether it should create
lots of MON groups vs CTRL_MON groups, given that the RMID count would
mean something semantically different on MPAM. I would not want to see
the container manager asking itself "am I on an ARM system?" when
calculating how many containers' bandwidth usage it can count. (Maybe
James has an answer to this question.)

Thanks!
-Peter