Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
IronPort-SDR: lB8NWLRZB9tgM/jn4ZvFHq+2vpR18jT/62KCs6eRv42I5LNbLEm40ksLT0gG013rt69Z2Vseaf
 gZiMtsogwnqqNSia5vNP6hXTJ+PWs7I3yLe6twvLQwBs4GCAbd8IVw/eAY8XKEzZyt/SCR3LQs
 DpC3G/zrNu+TrboGu7InOj0wFE/rVOXAt2j4qxZJfEUr5ujVhLu+mxlXSHhHpnK6rEmoMSqKRL
 VdeiUFghLyXiO+3QEdpKK8rWW46IF2IOHqLyMrIl7gyRHVy0Nquq7Pe39FpW49/OS5B9Z0ESXy
 /yA=
From:   "tan.shaopeng@fujitsu.com" <tan.shaopeng@fujitsu.com>
To:     "'fenghua.yu@intel.com'" <fenghua.yu@intel.com>,
        "'reinette.chatre@intel.com'" <reinette.chatre@intel.com>
CC:     "'linux-kernel@vger.kernel.org'" <linux-kernel@vger.kernel.org>,
        "'linux-arm-kernel@lists.infradead.org'" 
        <linux-arm-kernel@lists.infradead.org>,
        'James Morse' <james.morse@arm.com>,
        "misono.tomohiro@fujitsu.com" <misono.tomohiro@fujitsu.com>
Subject: RE: About add an A64FX cache control function into resctrl
Thread-Topic: About add an A64FX cache control function into resctrl
Thread-Index: AdctARUohIvM7pb1S52qTlM+pjp6RAJht/AA
Date:   Wed, 21 Apr 2021 08:37:10 +0000
Message-ID: <OSAPR01MB214657641D532FB8D112DD528B479@OSAPR01MB2146.jpnprd01.prod.outlook.com>
References: <OSAPR01MB214600C7923AEF7C35B02E648B739@OSAPR01MB2146.jpnprd01.prod.outlook.com>
In-Reply-To: <OSAPR01MB214600C7923AEF7C35B02E648B739@OSAPR01MB2146.jpnprd01.prod.outlook.com>
Accept-Language: ja-JP, en-US
Content-Language: ja-JP
Content-Type: text/plain; charset="iso-2022-jp"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: 7a4170e9-ab58-4c6d-d268-08d904a0a77d
X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Apr 2021 08:37:10.0605
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: a19f121d-81e1-4858-a9d8-736e267fd4c7
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: nYqkTb1SIhqCaMf/6bP9uYGVA1SwleuRYQNJZfIQ8e8UN7+k1FbxV8zYJYrA1UwUBQijgn1+isjpMtNbiNAC4XkXhoPs67Wn1IjkS32rXVA=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: OSAPR01MB2948
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi,

Ping... any comments&advice about add an A64FX cache control function into =
resctrl?

Best regards
Tan Shaopeng

> Hello
>=20
>=20
> I'm Tan Shaopeng from Fujitsu Limited.
>=20
> I=1B$B!G=1B(Bm trying to implement Fujitsu A64FX=1B$B!G=1B(Bs cache relat=
ed features.
> It is a cache partitioning function we called sector cache function that =
using
> the value of the tag that is upper 8 bits of the 64bit address and the va=
lue of the
> sector cache register to control virtual cache capacity of the L1D&L2 cac=
he.
>=20
> A few days ago, when I sent a driver that realizes this function to
> ARM64 kernel community, Will Deacon and Arnd Bergmann suggested an idea
> to add the sector cache function of A64FX into resctrl.
> https://lore.kernel.org/linux-arm-kernel/CAK8P3a2pFcNTw9NpRtQfYr7A5Oc
> Z=3DAs2kM0D_sbfFcGQ_J2Q+Q@mail.gmail.com/
>=20
> Based on my study, I think the sector cache function of A64FX can be adde=
d
> into the allocation features of resctrl after James' resctrl rework has f=
inished.
> But, in order to implement this function, more interfaces for resctrl are=
 need.
> The details are as follow, and could you give me some advice?
>=20
> [Sector cache function]
> The sector cache function split cache into multiple sectors and control t=
hem
> separately. It is implemented on the L1D cache and
> L2 cache in the A64FX processor and can be controlled individually for L1=
D
> cache and L2 cache. A64FX has no L3 cache. Each L1D cache and L2 cache
> has 4 sectors. Which L1D sector is used is specified by the value of [57:=
56] bits
> of address, how many ways of sector are specified by the value of registe=
r
> (IMP_SCCR_L1_EL0).
> Which L2 sector is used is specified by the value of [56] bits of address=
, and
> how many ways of sector are specified by value of register
> (IMP_SCCR_ASSIGN_EL1, IMP_SCCR_SET0_L2_EL1,
> IMP_SCCR_SET1_L2_EL1).
>=20
> For more details of sector cache function, see A64FX HPC extension
> specification (1.2. Sector cache) in https://github.com/fujitsu/A64FX
>=20
> [Difference between resctrl(CAT) and this sector cache function]
> L2/L3 CAT (Cache Allocation Technology) enables the user to specify some
> physical partition of cache space that an application can fill.
> A64FX's L1D/L2 cache has 4 sectors and 16ways. This sector function enabl=
es
> a user to specify number of ways each sector uses.
> Therefore, for CAT it is enough to specify a cache portion for each cache=
_id
> (socket). On the other hand, sector cache needs to specify cache portion =
of
> each sector for each cache_id, and following extension to resctrl interfa=
ce is
> needed to support sector cache.
>=20
> [Idear for A64FX sector cache function control interface (schemata file
> details)]
> L1:<cache_id0>=3D<cwbm>,<cwbm>,<cwbm>,<cwbm>;<cache_id1>=3D<cw
> bm>,<cwbm>,<cwbm>,<cwbm>;=1B$B!D=1B(B
> L2:<cache_id0>=3D>=3D<cwbm>,<cwbm>,<cwbm>,<cwbm>;<cache_id1>=3D
> <cwbm>,<cwbm>,<cwbm>,<cwbm>;=1B$B!D=1B(B
>=20
> =1B$B!&=1B(BL1: Add a new interface to control the L1D cache.
> =1B$B!&=1B(B<cwbm>,<cwbm>,<cwbm>,<cwbm>=1B$B!'=1B(BSpecify the number of =
ways for each
> sector.
> =1B$B!&=1B(Bcwbm=1B$B!'=1B(BSpecify the number of ways in each sector as =
a bitmap (percentage),
>   but the bitmap does not indicate the location of the cache.
> * In the sector cache function, L2 sector cache way setting register is
>   shared among PEs (Processor Element) in shared domain. If two PEs
>   which share L2 cache belongs to different resource groups, one resource
>   group's L2 setting will affect to other resource group's L2 setting.
> * Since A64FX does not support MPAM, it is not necessary to consider
>   how to switch between MPAM and sector cache function now.
>=20
> Some questions:
> 1.I'm still studying about RDT, could you tell me whether RDT has
>   the similar mechanism with sector cache function?
> 2.In RDT, L3 cache is shared among cores in socket. If two cores which
>   share L3 cache belongs to different resource groups, one resource
>   group's L3 setting will affect to other resource group's L3 setting?
> 3.Is this approach acceptable? could you give me some advice?
>=20
>=20
> Best regards
> Tan Shaopeng