Date:   Mon, 15 Nov 2021 10:51:36 +0000
Message-ID: <87czn18zev.wl-maz@kernel.org>
From:   Marc Zyngier <maz@kernel.org>
To:     Dougall <dougallj@gmail.com>
Cc:     Alyssa Rosenzweig <alyssa@rosenzweig.io>,
        linux-arm-kernel@lists.infradead.org, devicetree@vger.kernel.org,
        linux-kernel@vger.kernel.org, Mark Rutland <mark.rutland@arm.com>,
        Will Deacon <will@kernel.org>,
        Hector Martin <marcan@marcan.st>,
        Sven Peter <sven@svenpeter.dev>,
        Rob Herring <robh+dt@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>
Subject: Re: [PATCH 8/8] drivers/perf: Add Apple icestorm/firestorm CPU PMU driver
In-Reply-To: <CAHT7dO9G6tRfQCC_62OwgkoHQeUbz4AQwseX=t2S3YVYjoHCXA@mail.gmail.com>
References: <20211113115429.4027571-1-maz@kernel.org>
        <20211113115429.4027571-9-maz@kernel.org>
        <YY+32A1gdLzCP0TA@sunset>
        <CAHT7dO9G6tRfQCC_62OwgkoHQeUbz4AQwseX=t2S3YVYjoHCXA@mail.gmail.com>
User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue)
 FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1
 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO)
MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue")
Content-Type: text/plain; charset=US-ASCII
Precedence: bulk

On Sun, 14 Nov 2021 02:43:14 +0000,
Dougall <dougallj@gmail.com> wrote:
> 
> Apple distributes names (and descriptions and affinity masks) for 55
> of the events with macOS in the file /usr/share/kpep/a14.plist
> (exposed to users in Instruments.app). Many of those 55 events were
> added in macOS 12, so it's good to check the latest version. I use
> the command "plutil -convert json -o - /usr/share/kpep/a14.plist" to
> get these as JSON.

As it appears, the perf tool can ingest an event description from a
json file, and none of it has to be in the kernel itself.

So if someone was to provide a tool to convert the macOS file into
something that perf can understand, it would be great, and wouldn't
require any distribution of otherwise tainted material (distribute the
tool, and not the data).

> 
> There are many more events that I have discovered experimentally,
> but this work is unusually hard to verify, so I'd be inclined to
> stick with what's documented.
>
> However, I have observed a few oddities that might be of interest.
> 
> The counter 0x9B (INST_LDST) works on PMCs 5, 6 and 7, but gives
> different results for paired AMX instructions on PMC 7 (7 counts
> instructions, while 5 and 6 count pairs as one). Apple addresses
> this by restricting the affinity mask to PMC 7. This is also seen
> on undocumented counter 0x96, which counts integer stores. (For
> context, microarchitecturally non-load-store AMX operations appear
> as stores, as they just need to be posted to the AMX coprocessor on
> commit. Consecutive non-load-store AMX operations can be paired
> (fused), such that they issue as one uop, which is where this
> anomaly can be seen.)

Interesting. I guess we're unlikely to see any AMX support anytime
soon on Linux, unless we can make it fit the architected SME model
(and even that would be pretty controversial).

> Undocumented counters 0xF1 through 0xFF appear to be operation
> counters, meaning their result depends on events selected on other
> counters. There are three threshold registers (PMTRHLD2, PMTRHLD4,
> PMTRHLD6) which can specify a threshold (in number of cycles) for
> the operation counter on the PMC with the same number. There is also
> a mapping register (PMMAP), which contains a 3-bit field for each
> counter from PMC2 to PMC7, each specifying a PMC index which can be
> used as an input to the operation. Binary operations only use
> PMC2/4/6 and use PMC(n+1) as their other input. These operation
> counters may also behave differently depending on the value
> currently in the corresponding PMC (specifically counters F9/FA
> which implement shortest/longest run of non-zero counts).

Weeee... I'm sure there are super interesting uses for this, but I'd
rather have something simple for a start. Thanks for the heads up
though, this is extremely interesting!

> This is complicated, and it's not exposed to the user by macOS, so I
> wouldn't worry about supporting it for now.

We're in strong agreement here.

> Despite all this, the
> events and features on the P and E cores seem to be the same, so I
> don't expect a need to distinguish between them in the future.

That'd be the first big-little implementation to have consistent
events across the board. Amazing! :D

> (I've been meaning to write all this up properly, but haven't got
> around to it, sorry!)

No worries, and thanks for taking the time to write this email!

	M.

-- 
Without deviation from the norm, progress is not possible.