Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <CALPaoCj-zav4x6H3ffXo_O+RFan8Qb-uLy-DdtkaQTfuxY4a0w@mail.gmail.com>
 <b2e020b1-f6b2-e236-a042-4eb2fd27d8b0@intel.com> <IA1PR11MB6097236CFF891041DBA42ECB9B5F9@IA1PR11MB6097.namprd11.prod.outlook.com>
 <Y0BhzKkksSjSeE3W@agluck-desk3.sc.intel.com> <81a7b4f6-fbb5-380e-532d-f2c1fc49b515@intel.com>
 <CALPaoCjdeRjyX5L6BBX688ZM21eMwetuL9QLF1+GEDUskGcU2w@mail.gmail.com>
 <7b09fb62-e61a-65b9-a71e-ab725f527ded@intel.com> <CALPaoCg4zrODVoXF2y2b+LRYq_+jnV8yv5qB+T_3Z264cV82GQ@mail.gmail.com>
 <b931062a-1a70-2331-03cc-6bfa69a51bee@intel.com>
In-Reply-To: <b931062a-1a70-2331-03cc-6bfa69a51bee@intel.com>
From:   Peter Newman <peternewman@google.com>
Date:   Thu, 20 Oct 2022 10:48:20 +0200
Message-ID: <CALPaoCj8ps2YRH5T1eCRY-8MLwGi25cfbd0KNtjYXTq5baAXOQ@mail.gmail.com>
Subject: Re: [RFD] resctrl: reassigning a running container's CTRL_MON group
To:     Reinette Chatre <reinette.chatre@intel.com>
Cc:     Tony Luck <tony.luck@intel.com>,
        "Yu, Fenghua" <fenghua.yu@intel.com>,
        "Eranian, Stephane" <eranian@google.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        James Morse <james.morse@arm.com>,
        Babu Moger <Babu.Moger@amd.com>,
        Gaurang Upasani <gupasani@google.com>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Thu, Oct 20, 2022 at 1:54 AM Reinette Chatre
<reinette.chatre@intel.com> wrote:
> It is still not clear to me how palatable this will be on Arm systems.
> This solution also involves changing the CLOSID/PARTID like your original
> proposal and James highlighted that it would "mess up the bandwidth counters"
> because of the way PARTID.PMG is used for monitoring. Perhaps even a new
> PMG would need to be assigned during such a monitor group move. One requirement
> for this RFD was to keep usage counts intact and from what I understand
> this will not be possible on Arm systems.  There could be software mechanisms
> to help reduce the noise during the transition. For example, some new limbo
> mechanism that avoids re-assigning the old PARTID.PMG, while perhaps still
> using the old PARTID.PMG to read usage counts for a while? Or would the
> guidance just be that the counters will have some noise after the move?

I'm going to have to follow up on the details of this in James's thread.
It sounded like we probably won't be able to create enough mon_groups
under a single control group for the rename feature to even be useful.
Rather, we expect the PARTID counts to be so much larger than the PMG
counts that creating more mon_groups to reduce the number of control
groups wouldn't make sense.

At least in our use case, we're literally creating "classes of service"
to prioritize memory traffic, so we want a small number of control
groups to represent the small number of priority levels, but enough
RMIDs to count every job's traffic independently. For MPAM to support
this MBM/MBA use case in exactly this fashion, we'd have to develop the
monitors-not-matching-on-PARTID use case better in the MPAM
architecture. But before putting much effort into that, I'd want to know
if there's any payoff beyond being able to use resctrl the same way on
both implementations.

-Peter