Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37;
MIME-Version: 1.0
References: <20231201005720.235639-1-babu.moger@amd.com> <d97cbeba-af6d-4b64-b5c8-32dc437a67b6@intel.com>
In-Reply-To: <d97cbeba-af6d-4b64-b5c8-32dc437a67b6@intel.com>
From:   Peter Newman <peternewman@google.com>
Date:   Fri, 8 Dec 2023 11:45:54 -0800
Message-ID: <CALPaoCjNgv4rFpbovyayNynuAqYGP0rVLv=djVnDO1LR+zU55g@mail.gmail.com>
Subject: Re: [PATCH 00/15] x86/resctrl : Support AMD QoS RMID Pinning feature
To:     Reinette Chatre <reinette.chatre@intel.com>
Cc:     Babu Moger <babu.moger@amd.com>, corbet@lwn.net,
        fenghua.yu@intel.com, tglx@linutronix.de, mingo@redhat.com,
        bp@alien8.de, dave.hansen@linux.intel.com,
        James Morse <james.morse@arm.com>, x86@kernel.org,
        hpa@zytor.com, paulmck@kernel.org, rdunlap@infradead.org,
        tj@kernel.org, peterz@infradead.org, seanjc@google.com,
        kim.phillips@amd.com, jmattson@google.com,
        ilpo.jarvinen@linux.intel.com, jithu.joseph@intel.com,
        kan.liang@linux.intel.com, nikunj@amd.com,
        daniel.sneddon@linux.intel.com, pbonzini@redhat.com,
        rick.p.edgecombe@intel.com, rppt@kernel.org,
        maciej.wieczor-retman@intel.com, linux-doc@vger.kernel.org,
        linux-kernel@vger.kernel.org, eranian@google.com, dhagiani@amd.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Tue, Dec 5, 2023 at 3:17=E2=80=AFPM Reinette Chatre
<reinette.chatre@intel.com> wrote:
> On 11/30/2023 4:57 PM, Babu Moger wrote:
> > c. Read the monitor states. There will be new file "monitor_state"
> >    for each monitor group when ABMC feature is enabled. By default,
> >    both total and local MBM events are in "unassign" state.
> >
> >       #cat /sys/fs/resctrl/monitor_state
> >       total=3Dunassign;local=3Dunassign
> >
> > d. Read the event mbm_total_bytes and mbm_local_bytes. Note that MBA
> >    events are not available until the user assigns the events explicitl=
y.
> >    Users need to assign the counters to monitor the events in this mode=
.
> >
> >       #cat /sys/fs/resctrl/mon_data/mon_L3_00/mbm_total_bytes
> >       Unavailable
>
> How is the llc_occupancy event impacted when ABMC is enabled? Can all RMI=
Ds
> still be used to track cache occupancy?
>
> >
> >       #cat /sys/fs/resctrl/mon_data/mon_L3_00/mbm_local_bytes
> >       Unavailable
>
> I believe that "Unavailable" already has an accepted meaning within curre=
nt
> interface and is associated with temporary failure. Even the AMD spec sta=
tes "This
> is generally a temporary condition and subsequent reads may succeed". In =
the
> scenario above there is no chance that this counter would produce a value=
 later.
> I do not think it is ideal to overload existing interface with different =
meanings
> associated with a new hardware specific feature ... something like "Disab=
led" seems
> more appropriate.

Could we hide event counter files if they're not enabled? Is there
value in displaying the value of a non-running counter that will be
reset the next time it's enabled?


>
> Considering this we may even consider using these files themselves as a
> way to enable the counters if they are disabled. For example, just
> "echo 1 > /sys/fs/resctrl/mon_data/mon_L3_00/mbm_total_bytes" can be used
> to enable this counter. No need for a new "monitor_state". Please note th=
at this
> is not an official proposal since there are two other use cases that stil=
l need to
> be considered as we await James's feedback on how this may work for MPAM =
and
> also how this may be useful on AMD hardware that does not support ABMC bu=
t
> users may want to get similar benefits ([1])

We plan to use the ABMC counters as a window to sample the MB/s rate
of a very large number of groups, so there's a serious concern about
the number of write syscalls this will take, as they will add up
quickly for a large RMID*domain count.

To that end, the ideal would be the ability to re-assign all ABMC
counters on all domains in a single system call.

-Peter