Date:   Mon, 23 Jan 2023 15:11:12 -0800 (PST)
From:   Ilkka Koskinen <ilkka@os.amperecomputing.com>
To:     Robin Murphy <robin.murphy@arm.com>
cc:     will@kernel.org, mark.rutland@arm.com,
        linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org,
        ilkka@os.amperecomputing.com
Subject: Re: [PATCH] Partially revert "perf/arm-cmn: Optimise DTC counter
 accesses"
In-Reply-To: <b41bb4ed7283c3d8400ce5cf5e6ec94915e6750f.1674498637.git.robin.murphy@arm.com>
Message-ID: <6d4a6e3-eba9-a5a2-fc41-393d2b5d755@os.amperecomputing.com>
References: <b41bb4ed7283c3d8400ce5cf5e6ec94915e6750f.1674498637.git.robin.murphy@arm.com>
Content-Type: text/plain; format=flowed; charset=US-ASCII
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?JVz7eowFOs9V0mxapX0YLJU1rVSV7STYRMBDNbD4RsTighAPigpeKS/HItuw?=
 =?us-ascii?Q?njhW5apDSLYm1g+bdhiPsx2aBDh69FlIPuipPVPryCZISNvoAd6oRSbavsuI?=
 =?us-ascii?Q?FQct/3jq61LC9Uu66vI7ZlebKt3DSM9q07go/Pl9OELK4BwdyUU88JfGKFCn?=
 =?us-ascii?Q?AuQJMKcLdBpCJC0oWjOBz3ozaC3dkFvi1xXvkCj2HIeV/l0FMU0wIltGrEV/?=
 =?us-ascii?Q?KXYIJKWCUW4A7arv6tJWXvGWT3YKCnDepA5aDwfWibKAHLMXb853PobRqINL?=
 =?us-ascii?Q?YjZCWID4x+EyszhuvL2X56e+ZbKi9fe6siYhKr2z/LHWkIYtbBlAtWY3nnge?=
 =?us-ascii?Q?rgRDuCeOVm5lUBt50FP1R3NcHrXTNhcezJfq/IjEQkCEThyYwx1u1hsBV+fn?=
 =?us-ascii?Q?srQqP7phnKfxQkvO1i+X2jyzjnyFaZLY8jiFtcF2Vjt6hHqrpCtRRs/RoI3m?=
 =?us-ascii?Q?UNOSvgf1qSLrJEPJCdFCfb8gYZw4jFxsnqhZLSW6ZIvgxZ3zGQf0xd0iErDm?=
 =?us-ascii?Q?HgXdcZg0A7qDiEFg0T9oFBxVarEUW9t90GCdVPnuX7vQG1OI4PVU4chKGLaW?=
 =?us-ascii?Q?GTxjvi9QaNXi8TCZ9yDGNuiwuwWJrb2sSEq5WTHU86pJgYSf8Zt7awL/fxyj?=
 =?us-ascii?Q?LmfydfsVqSWlQoW9HegeNhr7yMTCZat4E+IVfXedj9dEkiJ2RpnPWqJsiKdz?=
 =?us-ascii?Q?CexnKM+uQ3i4mUjsGCh6UkUZNapxFvKscJbcApju3KaJjOdjNR7RfqJUZJEP?=
 =?us-ascii?Q?k3WOVE2l0hTmsyHIoJuBZHhNd4kmU9S9I+xWw4eX/K8bSqthIdkFDxcphqTr?=
 =?us-ascii?Q?QXR7H87MHKdL8rBGRiIZOC5UEqkL3cG88s9pXFgNzksjGawBFsradVsNMlDQ?=
 =?us-ascii?Q?E+7gwbH5eDdTqXQnctIOb9gJrHDf8mZ4QEmeVH+glOi5KCnSgOSnbSfUNhKa?=
 =?us-ascii?Q?7+Jfp+GHO1y8TajoMh0StPJJh3Fo7VZGMysP3llfdHwKV4RiYHNDxFK3xbv7?=
 =?us-ascii?Q?ElILZQxJYkXBddqH2kqBXmkdhg9e+gOAy+KMuiqruStpkgkEdde1Uni6bdI0?=
 =?us-ascii?Q?nUHf15YrBiieBlQsWFJTRUN0gHELxRuUywHqDwXWfKYZQHgtfiDZMVI/E+sT?=
 =?us-ascii?Q?Ki3dR/+ZzwYnLKRDskqgPoLhR89lE2DRJy0/IuapYAexOKRfac0Vt0HWZckw?=
 =?us-ascii?Q?Jih2qpz6+MFXcAZFYPwZk2als9Bc9w77qdmmaiN4KM5uecZmmE7ptURoigZB?=
 =?us-ascii?Q?aT6ndij26NTnPscIlZ1dCaJZcsKxpJddRsN2cpEpnzwWYwaj7VlRvXE1gRuh?=
 =?us-ascii?Q?KlP2t50+jMAAfEKm7KySHyBiSVyaKqDTY7QDk0iPBOoy6g6IqfN48b1f+0HM?=
 =?us-ascii?Q?7cZNj0wWY69WcHdfphnemsczQRrnSbkTd6qzKI4thnQFAJQxj/13zPNAxbcV?=
 =?us-ascii?Q?N6u7Kd90zBIBr0wC3Tq3FBW27DjfcZ4y/7/q8vL2lAup0f1ZlybUypwHdJBF?=
 =?us-ascii?Q?VSo8j3pY/QTfuO5pucjI/24K5shjgPccZdWPNcTIpAFhE6SRWWLNqOSEQR38?=
 =?us-ascii?Q?V+QiLXVnVsiE1PmQ8DWl919G19n7JeNx+Bc4tpriFEFoL4CyLIvT2E5xPtT5?=
 =?us-ascii?Q?hvbHtIQiN/UvEkI+54Dw7vg=3D?=
X-OriginatorOrg: os.amperecomputing.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6f1dd8da-ea9f-4d2f-c37a-08dafd97275b
X-MS-Exchange-CrossTenant-AuthSource: DM5PR0102MB3590.prod.exchangelabs.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Jan 2023 23:11:26.9966
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: NW6NqPkf7yGo+X/QCvtAWATOpC2I5jLjbxqHV4btl2fUAuccIfptyNav4f7uc4hUEttNY0LE937gbwgOvq931JNQM3qVpqbU3RVpp+hEvltGwwzpuXL2/neevMOZIjLb
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR01MB6812
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Robin,

On Mon, 23 Jan 2023, Robin Murphy wrote:
> It turns out the optimisation implemented by commit 4f2c3872dde5 is
> totally broken, since all the places that consume hw->dtcs_used for
> events other than cycle count are still not expecting it to be sparsely
> populated, and fail to read all the relevant DTC counters correctly if
> so.
>
> If implemented correctly, the optimisation potentially saves up to 3
> register reads per event update, which is reasonably significant for
> events targeting a single node, but still not worth a massive amount of
> additional code complexity overall. Getting it right within the current
> design looks a fair bit more involved than it was ever intended to be,
> so let's just make a functional revert which restores the old behaviour
> while still backporting easily.
>
> Fixes: 4f2c3872dde5 ("perf/arm-cmn: Optimise DTC counter accesses")
> Reported-by: Ilkka Koskinen <ilkka@os.amperecomputing.com>
> Signed-off-by: Robin Murphy <robin.murphy@arm.com>

Thanks for the patch. It looks good to me and seems to work fine.

Cheers, Ilkka

> ---
> drivers/perf/arm-cmn.c | 7 ++++++-
> 1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/perf/arm-cmn.c b/drivers/perf/arm-cmn.c
> index b80a9b74662b..1deb61b22bc7 100644
> --- a/drivers/perf/arm-cmn.c
> +++ b/drivers/perf/arm-cmn.c
> @@ -1576,7 +1576,6 @@ static int arm_cmn_event_init(struct perf_event *event)
> 			hw->dn++;
> 			continue;
> 		}
> -		hw->dtcs_used |= arm_cmn_node_to_xp(cmn, dn)->dtc;
> 		hw->num_dns++;
> 		if (bynodeid)
> 			break;
> @@ -1589,6 +1588,12 @@ static int arm_cmn_event_init(struct perf_event *event)
> 			nodeid, nid.x, nid.y, nid.port, nid.dev, type);
> 		return -EINVAL;
> 	}
> +	/*
> +	 * Keep assuming non-cycles events count in all DTC domains; turns out
> +	 * it's hard to make a worthwhile optimisation around this, short of
> +	 * going all-in with domain-local counter allocation as well.
> +	 */
> +	hw->dtcs_used = (1U << cmn->num_dtcs) - 1;
>
> 	return arm_cmn_validate_group(cmn, event);
> }
> -- 
> 2.36.1.dirty
>
>