Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   "Yu, Fenghua" <fenghua.yu@intel.com>
To:     James Morse <james.morse@arm.com>,
        "x86@kernel.org" <x86@kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
CC:     "Chatre, Reinette" <reinette.chatre@intel.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        H Peter Anvin <hpa@zytor.com>,
        Babu Moger <Babu.Moger@amd.com>,
        "shameerali.kolothum.thodi@huawei.com" 
        <shameerali.kolothum.thodi@huawei.com>,
        D Scott Phillips OS <scott@os.amperecomputing.com>,
        "carl@os.amperecomputing.com" <carl@os.amperecomputing.com>,
        "lcherian@marvell.com" <lcherian@marvell.com>,
        "bobo.shaobowang@huawei.com" <bobo.shaobowang@huawei.com>,
        "tan.shaopeng@fujitsu.com" <tan.shaopeng@fujitsu.com>,
        "xingxin.hx@openanolis.org" <xingxin.hx@openanolis.org>,
        "baolin.wang@linux.alibaba.com" <baolin.wang@linux.alibaba.com>,
        Jamie Iles <quic_jiles@quicinc.com>,
        Xin Hao <xhao@linux.alibaba.com>,
        "peternewman@google.com" <peternewman@google.com>
Subject: RE: [PATCH v2 02/18] x86/resctrl: Access per-rmid structures by index
Thread-Topic: [PATCH v2 02/18] x86/resctrl: Access per-rmid structures by
 index
Thread-Index: AQHZJ3hM+VcWIzEppUSZYtjY9I+xpa6hP2/A
Date:   Tue, 17 Jan 2023 18:29:08 +0000
Message-ID: <IA1PR11MB6097E206D4A9404BC3D78CD89BC69@IA1PR11MB6097.namprd11.prod.outlook.com>
References: <20230113175459.14825-1-james.morse@arm.com>
 <20230113175459.14825-3-james.morse@arm.com>
In-Reply-To: <20230113175459.14825-3-james.morse@arm.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?vCEzSFieatqh3eyn9z2bKjOJlYOED6oQMIUKQN31joKO5uTposnWdDnbi1ZD?=
 =?us-ascii?Q?TO7Wx7PF+hcTx7FvPqT5vGjt5qIN3HGT5hHalg388l5++lskddMBWSXPJ0OR?=
 =?us-ascii?Q?vfsEO8jpl1SfTF1JKgf/rrERu3YFvjjq5BiWfhLQdBqpAP0+kcYkL9vtx1pb?=
 =?us-ascii?Q?d+C8VNNhwY8V8YJEh2uhHfWtj3ZgMSOD2rHTBj3yO4IcNRMjS3kLwlGpqMgo?=
 =?us-ascii?Q?acqsemNfqASLHQNUWP3ioNCFa+FTmiSWtJPktQJpIzcwmclOt2T4YLCXZnDA?=
 =?us-ascii?Q?xnTGsT/OA6u1q78fmn51l7cbVL4gO+9o0euOZuaHmiOl53PXCLfCiYF5/oEt?=
 =?us-ascii?Q?/in7Y47KLnSwimVNs05Ot6AEGYWTpGb29rh0idVzkMTCkr/yWZsLmFbsypO4?=
 =?us-ascii?Q?deFpD1sA9MdsTX695Cksy5STc+fsPEG698U4sqT6uCZnEWF1E4Nr3tGRvZjL?=
 =?us-ascii?Q?B2Cw9A1DoDf9KASrzI4CKCGXiNyzUXthLBZboWdMB8y1ptV63DTJIjPaF/eq?=
 =?us-ascii?Q?S5s+fBEkUaqTv73xORIvQTaXDIcdu2X2lpcY8OJAn5f1QvhRPKovdIIBj2fI?=
 =?us-ascii?Q?/kugcYyMgo7cE4tVyX9GsCo2QzdcFSmF7DnGXza47+3E+UyIojpLe1Stw6/c?=
 =?us-ascii?Q?JqBtOL8rGMWnUghe4qLd+VIhnJfKMyWJKK0c8lW007lhkUgCZ+57mGjtpSOT?=
 =?us-ascii?Q?X7jh+NbcPZw/u+VbjMp4DEcmCIHRc03ssYM96UaEXNhplIEXaH+YYEeLwYdp?=
 =?us-ascii?Q?NTi6i4c/KdiS1hJo8v2uZW1SYiphhx8ThN4Db1Eba8cRwZD9aCsAW2CEGVnd?=
 =?us-ascii?Q?b7OLQIS3q27JDdwuI/a77Lr66dxwrcaXyk6UtleolxrYmUpkAxCp3C7ACblj?=
 =?us-ascii?Q?/tSFr/k99pw7ky6oP0ZJWE4nQdMG2Nc9oLJhmtyn794566fgJx7B6syOyiYY?=
 =?us-ascii?Q?UM5g/r95B6aPqbpvl3CDwuC4XFO/hhGGKG1njV5kdi71xqfapIDDqRwqXiEz?=
 =?us-ascii?Q?yC2I8AcSJiBMTvkKpZpQVVCmQcvqeTofi5c8Hthq4crrmU1z0XUA37umORtp?=
 =?us-ascii?Q?jXN51IuazZVsRlmnN+nHs7Y5dYa4HyHd1PLPLc1sjBiuQk0d634lOvUde7ko?=
 =?us-ascii?Q?2647lMti9/Viw+NDNuhETZoJ/hjkpM6Byh9kMD68EJxQb7NCa8i2E/INRztg?=
 =?us-ascii?Q?T2nj1y0iVjNrXRiCvQTCHY+YuZ1QKkReZeBrLNtGXJkU7duuAUbWOOZ5jwR8?=
 =?us-ascii?Q?TiG8WVXwtHY6z3AVgvPBD0dkMHVkRidQq22ghwJXGvJSC93h8lMGhgXdA3N6?=
 =?us-ascii?Q?v/kHVFbaWFC/iN3+aFGdU7BmmaZVITpyKbuQzlI64TbdDJZvAltCMlWF7INB?=
 =?us-ascii?Q?RNJ/S4Qx9ZaaM7slHInZjAAsQaj/Hlpxi0oYMbaxM80yfMcn94j3qq9mLTc5?=
 =?us-ascii?Q?/b/y8m3aPoVe2bSR7hIZQBjE8rRxrGjIBenXx/GRgzcGhHE1RUEZDaR7RtxX?=
 =?us-ascii?Q?a0+wtb5AeaCMXtY2hegcM2c8Cz80SRbm/RLzHSvDkhRTSbSfC4fNCHE6uSDt?=
 =?us-ascii?Q?MfRw55Zt0D7ovKSHim+wRStGCCjKi2mww+kelLN5?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: IA1PR11MB6097.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9c03e392-d014-4d5f-aa51-08daf8b8b896
X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Jan 2023 18:29:08.0430
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: FpNuv6/05I9KwK00cxdZePlAcHlxq9xSHzML2ZgPOoHtGejm/iSx+Atp/kc6AtTh7dMS91rx18Frmx4fV0xZXQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR11MB7541
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        SPF_HELO_PASS,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi, James,

> Because of the differences between Intel RDT/AMD QoS and Arm's MPAM
> monitors, RMID values on arm64 are not unique unless the CLOSID is also
> included. Bitmaps like rmid_busy_llc need to be sized by the number of un=
ique
> entries for this resource.
>=20
> Add helpers to encode/decode the CLOSID and RMID to an index. The domain'=
s
> busy_rmid_llc and the rmid_ptrs[] array are then sized by index. On x86, =
this is
> always just the RMID. This gives resctrl a unique value it can use to sto=
re
> monitor values, and allows MPAM to decode the closid when reading the
> hardware counters.
>=20
> Tested-by: Shaopeng Tan <tan.shaopeng@fujitsu.com>
> Signed-off-by: James Morse <james.morse@arm.com>
> ---
> Changes since v1:
>  * Added X86_BAD_CLOSID macro to make it clear what this value means
>  * Added second WARN_ON() for closid checking, and made both _ONCE()
> ---
>  arch/x86/include/asm/resctrl.h         | 24 ++++++++
>  arch/x86/kernel/cpu/resctrl/internal.h |  2 +
> arch/x86/kernel/cpu/resctrl/monitor.c  | 79 +++++++++++++++++---------
> arch/x86/kernel/cpu/resctrl/rdtgroup.c |  7 ++-
>  4 files changed, 82 insertions(+), 30 deletions(-)
>=20
> diff --git a/arch/x86/include/asm/resctrl.h b/arch/x86/include/asm/resctr=
l.h
> index 52788f79786f..44d568f3577c 100644
> --- a/arch/x86/include/asm/resctrl.h
> +++ b/arch/x86/include/asm/resctrl.h
> @@ -7,6 +7,13 @@
>  #include <linux/sched.h>
>  #include <linux/jump_label.h>
>=20
> +/*
> + * This value can never be a valid CLOSID, and is used when mapping a
> + * (closid, rmid) pair to an index and back. On x86 only the RMID is
> + * needed.
> + */
> +#define X86_RESCTRL_BAD_CLOSID		~0
> +
>  /**
>   * struct resctrl_pqr_state - State cache for the PQR MSR
>   * @cur_rmid:		The cached Resource Monitoring ID
> @@ -94,6 +101,23 @@ static inline void resctrl_sched_in(void)
>  		__resctrl_sched_in();
>  }
>=20
> +static inline u32 resctrl_arch_system_num_rmid_idx(void)
> +{
> +	/* RMID are independent numbers for x86. num_rmid_idx=3D=3Dnum_rmid
> */
> +	return boot_cpu_data.x86_cache_max_rmid + 1; }
> +
> +static inline void resctrl_arch_rmid_idx_decode(u32 idx, u32 *closid,
> +u32 *rmid) {
> +	*rmid =3D idx;
> +	*closid =3D X86_RESCTRL_BAD_CLOSID;
> +}
> +
> +static inline u32 resctrl_arch_rmid_idx_encode(u32 closid, u32 rmid) {

s/u32 closid/u32 ignored/, please.

> +	return rmid;
> +}
> +

Thanks.

-Fenghua