Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20221207112924.3602960-1-peternewman@google.com>
 <f58e6af2-aa16-9461-d40d-1e4e52ee6943@intel.com> <CALPaoCiB-vOuXJYkaLLsxSKHcjT55q1RSNBjhHUWmPL9rFdF8A@mail.gmail.com>
 <74cfd689-3c03-5f41-d01c-efab04ce4197@intel.com> <CALPaoChbJNYBXvOwftSxApo_ca6BLC7Ej21cDAaKdj9LOJumxw@mail.gmail.com>
 <6f0f4c45-039b-ffaf-b30a-d9e2aae0cc74@intel.com>
In-Reply-To: <6f0f4c45-039b-ffaf-b30a-d9e2aae0cc74@intel.com>
From:   Peter Newman <peternewman@google.com>
Date:   Fri, 16 Dec 2022 14:54:46 +0100
Message-ID: <CALPaoCjAUOO=2wECEQF+weKsto5LNi1-8nVi_QTLs7B+fvRb5A@mail.gmail.com>
Subject: Re: [PATCH] x86/resctrl: Fix event counts regression in reused RMIDs
To:     Reinette Chatre <reinette.chatre@intel.com>
Cc:     Fenghua Yu <fenghua.yu@intel.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
        "H. Peter Anvin" <hpa@zytor.com>,
        James Morse <james.morse@arm.com>,
        Shaopeng Tan <tan.shaopeng@fujitsu.com>,
        Jamie Iles <quic_jiles@quicinc.com>,
        linux-kernel@vger.kernel.org, eranian@google.com,
        Babu Moger <Babu.Moger@amd.com>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

Hi Reinette,

On Wed, Dec 14, 2022 at 8:17 PM Reinette Chatre
<reinette.chatre@intel.com> wrote:
> On 12/14/2022 6:21 AM, Peter Newman wrote:
> > mbm_state is arch-independent, so I think putting it here would require
> > the MPAM version to copy this and for get_mbm_state() to be exported.
>
> You are correct, it is arch independent ... so every arch is expected to
> have it.
> I peeked at your series and that looks good also - having cleanup done in
> a central place helps to avoid future mistakes.
>
> >> am = get_arch_mbm_state(hw_dom, rmid, eventid);
> >> if (am) {
> >> memset(am, 0, sizeof(*am));
> >> /* Record any initial, non-zero count value. */
> >> ret = __rmid_read(rmid, eventid, &val);
> >> if (!ret)
> >> am->prev_msr = val;
> >> }
> >>
> >> }
> >>
> >> Having this would be helpful as reference to Babu's usage.
> >
> > His usage looks a little different.
> >
> > According to the comment in Babu's patch:
> >
> > https://lore.kernel.org/lkml/166990903030.17806.5106229901730558377.stgit@bmoger-ubuntu/
> >
> > + /*
> > + * When an Event Configuration is changed, the bandwidth counters
> > + * for all RMIDs and Events will be cleared by the hardware. The
> > + * hardware also sets MSR_IA32_QM_CTR.Unavailable (bit 62) for
> > + * every RMID on the next read to any event for every RMID.
> > + * Subsequent reads will have MSR_IA32_QM_CTR.Unavailable (bit 62)
> > + * cleared while it is tracked by the hardware. Clear the
> > + * mbm_local and mbm_total counts for all the RMIDs.
> > + */
> > + resctrl_arch_reset_rmid_all(r, d);
> >
> > If all the hardware counters are zeroed as the comment suggests, then
> > leaving am->prev_msr zero seems correct. __rmid_read() would likely
> > return an error anyways. The bug I was addressing was one of reusing
> > an RMID which had not been reset.
>
> You are correct, but there are two things to keep in mind though:
> * the change from which you copied the above snippet introduces a new
>   _generic_ utility far away from this call site. It is thus reasonable to
>   assume that this utility should work for all use cases, not just the one
>   for which it is created. Since there are no other use cases at this time,
>   this may be ok, but I think at minimum the utility will benefit from
>   a snippet indicating the caveats of its use as a heads up to any future users.
> * the utility does not clear struct mbm_state contents. Again, this is ok
>   for this usage since AMD does not support the software controller but
>   as far as a generic utility goes the usage should be clear to avoid
>   traps for future changes.

To this end, would it help if I pulled the rr->first case into a
separate function like this:

-               resctrl_arch_reset_rmid(rr->r, rr->d, rmid, rr->evtid);
-               m = get_mbm_state(rr->d, rmid, rr->evtid);
-               if (m)
-                       memset(m, 0, sizeof(struct mbm_state));
+               resctrl_reset_rmid(rr->r, rr->d, rmid, rr->evtid);

I'm open to suggestions on the name.

-Peter