Message-ID: <3136ff3c-9b5a-383a-efc5-43b3d174765b@amd.com>
Date:   Wed, 17 Nov 2021 10:00:55 +0530
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.3.1
Subject: Re: [RFC] perf/amd/ibs: Move ibs pmus under perf_sw_context
Content-Language: en-US
To:     Peter Zijlstra <peterz@infradead.org>
Cc:     kim.phillips@amd.com, mingo@redhat.com, acme@kernel.org,
        mark.rutland@arm.com, alexander.shishkin@linux.intel.com,
        jolsa@redhat.com, namhyung@kernel.org, tglx@linutronix.de,
        bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org,
        hpa@zytor.com, linux-perf-users@vger.kernel.org,
        linux-kernel@vger.kernel.org, santosh.shukla@amd.com,
        sandipan.das@amd.com, Ravi Bangoria <ravi.bangoria@amd.com>
References: <20211115094838.3359-1-ravi.bangoria@amd.com>
 <YZJB5FVHpzhwFMtZ@hirez.programming.kicks-ass.net>
 <7ee6951e-eda1-7872-4158-e3bbb9eb31b7@amd.com>
 <YZJNZRuXjND9zqjL@hirez.programming.kicks-ass.net>
From:   Ravi Bangoria <ravi.bangoria@amd.com>
In-Reply-To: <YZJNZRuXjND9zqjL@hirez.programming.kicks-ass.net>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?MkNhNDN5ZWIxcmNpMmMzcnZHTjhJUEhoOWl2UHd5SHFkMC9wWW9TMi9FUGZ0?=
 =?utf-8?B?ZmdjYlQrTG5CRitCRFZ4UjFJcTNuTmlPSE85QkpWYmJ3MnEzcnRYQVphUXRi?=
 =?utf-8?B?cTFmUzREMDI3ZTFlNzFXZndxRkt0TmVvYjBzeXNrdjRQL2tMZ0RFUVNTYnFm?=
 =?utf-8?B?OFg1ejBsckRRYkkwT1RDK0VtMkp1WkhpU3JIWllMUjdvamVETENBRk9ZTWNp?=
 =?utf-8?B?MkRCMU9sbjV6YTV5RHgvT0dFV0YvOXdqcWpjWi8yZHpZbkUzQmRSYXNjTWRF?=
 =?utf-8?B?WWZWcEhpRDNQNFpKSnlOb21FNGErWmhXYkRnbHEyeGM0TURPTE16Y1FSTlpQ?=
 =?utf-8?B?N0liQ1V6TkxZampmdklCd1NGYUNLcUN0YWh2cnBGUzZ1OGxVT0x4bXF2ZHBy?=
 =?utf-8?B?VXd3dmJuM3pTaEhGN3ZpdVAvMTRVYjA3TjQ4Mk80L0V0TlFvdWlnZStRT1Jv?=
 =?utf-8?B?M2Rld3pnYTYzMis5dzNFQlRrTzZWY1V3Z3FDWENXUVo5SXlpL3hybmdQNEpt?=
 =?utf-8?B?M0pHdTNTLzRjUlFIcDBkWTAvWlJTWXF4TWg3bktaUjBnVmtkYW5LQmRMcUR5?=
 =?utf-8?B?S21ldlBFaGJCUTRveU1IYVlsZWhwOVFBbzM2WTQySDhXeWhxZ2FLNE14Rldq?=
 =?utf-8?B?TDcxYlJWdkRYZXNITnBZMm12SFdNdlNPZ1pNVFA4YkU5L1VXWmJPOGw5KzBh?=
 =?utf-8?B?ckRzaHNqUnVJZmZWeXRjK05CSmw0Y09JejBWcmgvalVmeWJRRjFGOEg2QlZv?=
 =?utf-8?B?cm9yQkUvQmlQTEV6cURKQTZkT1RuUlpzcm1jZWRHcjUvekZnOXN5VmRVSUky?=
 =?utf-8?B?UUgrd0o5UmlJQ2cvcTVpNnExQk9SbVRweklRMUcxeXUzS1lRcS9scWhNQVRZ?=
 =?utf-8?B?dTgyOFl6aWRTQzJhcnljVGViWXptRkgwMnBraHhqYUxpMG55d0VuMmR0WCth?=
 =?utf-8?B?UWdOdUhmQVpmYzZsYm1wQWNxWUF1RkNSOEhWakErQ05ZMFFQU1BaQ1ByZmxv?=
 =?utf-8?B?NzVyS29JNUJGUnJQa2xRMUZwS0FyRVhHalFtR1orazJnejBZbVF0N096M29h?=
 =?utf-8?B?SFJpWVV3ci8xMHY5UitTazlEMDR1c3ZRTnYyM3hqTFN1Q2xyWW1iVmxWOEhW?=
 =?utf-8?B?UlIzMXVLS0piOSszdVk0dWN2V05ncEZtWUUvN09EbHQrS1ZuOUpuSU9BdUgx?=
 =?utf-8?B?NGZLOWxtU3VhMVlsOTljeElQZjJyNVlRRm9nZVYzSHFqKzdCeHp6ZE0vNzdv?=
 =?utf-8?B?SnZxU3V3WDlwSDRlUUlQV28zdHZLaUhwZ1dBMVU5UkdURndpZVh2ODZDVFNW?=
 =?utf-8?B?R1JkU0RFT2xUdDc2eVNzb3ZkeXVucWNyck4ySGVVUTFyOE9DK2craUdqTXJI?=
 =?utf-8?B?Vyt6dkd6S0pkZDBLUWllUEl0UWh1OGVSV2JLSTN6QzRBQnVySXJSWnhXSWlF?=
 =?utf-8?B?bEVqNzNCUzZ1MjFFTWxlQmpVMjh1U2l3OExXOVJ5Z2VZNFRyRTJRblk3Wm54?=
 =?utf-8?B?ays3c1NRTHNyV1drWXo3NlBpa21PL2djSWJGN2FtNGNTdXBBQjZFZkJyN0pT?=
 =?utf-8?B?M3pPRmltcXdnaTEwT2o1aUV2TlNrR0NJeFVNN05RZ093NkVzUTIvVUhTMW9R?=
 =?utf-8?B?WVFFSk41SFJOVjNIK3JidFZwdlV6OW1XWWRmWEsrSUg5RlBoM2hMUDFRK2pv?=
 =?utf-8?B?Vm05and6aGhsL2NGUjdRdTV5M3FCVWpqb2hUQU10VnNlQTJkTGg3OFg2Rmlp?=
 =?utf-8?B?Y1BFWEFvZERKS2h4SzRIWWxwSWhCK1UreFZIbmw2dDBQWEV5OWQ0T21telQ0?=
 =?utf-8?B?MlVJSUhQUy9UdU5VK2hJdXhrL0FyQkI2aEREN2FlTW82QW5QbkRMVWhhSVN1?=
 =?utf-8?B?WERYRnhrYXpISGJIOWV2Uy9CRGNTVW5wTnRvY0pvUGpwSUoxcGVaR2IzUG9J?=
 =?utf-8?B?MXQ2SGlzV29uS0hTUm9rbDRCYWExcFVjUHB6Wm1UK1ZIQnlFTkVPV1crYmNv?=
 =?utf-8?B?RHlLanhaUjdEZlFIMGJYMW9yTFFpWTgxZmlneTBuem5wTUtqdUNPcWs1MVhw?=
 =?utf-8?B?WXl3ODZEWXJ3cjB3Mlh1UTMvelc2VE0ybHh0ejNUcXFMUDJ4ZFB2WURIYXVl?=
 =?utf-8?B?LzJJNTYyS0l2U01KRjJKcTNLQklBTkFPNTBqVDh4bjZDVFJMeWFRMi93ckds?=
 =?utf-8?Q?A00L95TQH36/y8p0kNcxPUs=3D?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 29400ac3-9c8b-4985-f2d6-08d9a98316d3
X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB2654.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Nov 2021 04:31:14.2932
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: DFuP+rY95fQOSLOZ2DqWBW7NzqhZzmrA+cGZa2gTnjzTZwbj5tNv4ef8BjLWYM7us78pEdvbC3txmVt5AWdykw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4461
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 15-Nov-21 5:37 PM, Peter Zijlstra wrote:
> On Mon, Nov 15, 2021 at 05:31:21PM +0530, Ravi Bangoria wrote:
>>
>>
>> On 15-Nov-21 4:47 PM, Peter Zijlstra wrote:
>>> On Mon, Nov 15, 2021 at 03:18:38PM +0530, Ravi Bangoria wrote:
>>>> Ideally, a pmu which is present in each hw thread belongs to
>>>> perf_hw_context, but perf_hw_context has limitation of allowing only
>>>> one pmu (a core pmu) and thus other hw pmus need to use either sw or
>>>> invalid context which limits pmu functionalities.
>>>>
>>>> This is not a new problem. It has been raised in the past, for example,
>>>> Arm big.LITTLE (same for Intel ADL) and s390 had this issue:
>>>>
>>>>   Arm:  https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Flkml%2F20160425175837.GB3141%40leverpostej&amp;data=04%7C01%7Cravi.bangoria%40amd.com%7C8b30ce7e52c0414dc65e08d9a8307149%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637725748302455206%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=59fepiNH9du3ZR0owjPcquQ4YLc%2B8hE74H%2BpdnQggQY%3D&amp;reserved=0
>>>>   s390: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Flkml%2F20160606082124.GA30154%40twins.programming.kicks-ass.net&amp;data=04%7C01%7Cravi.bangoria%40amd.com%7C8b30ce7e52c0414dc65e08d9a8307149%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637725748302455206%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=cSEGVgRbClDXeGJmEuGr1P8YXSIvhx%2FxbQjmdydQiuU%3D&amp;reserved=0
>>>>
>>>> Arm big.LITTLE (followed by Intel ADL) solved this by allowing multiple
>>>> (heterogeneous) pmus inside perf_hw_context. It makes sense as they are
>>>> still registering single pmu for each hw thread.
>>>>
>>>> s390 solved it by moving 2nd hw pmu to perf_sw_context, though that 2nd
>>>> hw pmu is count mode only, i.e. no sampling.
>>>>
>>>> AMD IBS also has similar problem. IBS pmu is present in each hw thread.
>>>> But because of perf_hw_context restriction, currently it belongs to
>>>> perf_invalid_context and thus important functionalities like per-task
>>>> profiling is not possible with IBS pmu. Moving it to perf_sw_context
>>>> will:
>>>>  - allow per-task monitoring
>>>>  - allow cgroup wise profiling
>>>>  - allow grouping of IBS with other pmu events
>>>>  - disallow multiplexing
>>>>
>>>> Please let me know if I missed any major benefit or drawback of
>>>> perf_sw_context. I'm also not sure how easy it would be to lift
>>>> perf_hw_context restriction and start allowing more pmus in it.
>>>>
>>>> Suggestions?
>>>
>>> Same as I do every time this comes up... this patch is still lingering
>>> and wanting TLC:
>>>
>>>   https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Flkml%2F20181010104559.GO5728%40hirez.programming.kicks-ass.net%2F&amp;data=04%7C01%7Cravi.bangoria%40amd.com%7C8b30ce7e52c0414dc65e08d9a8307149%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637725748302455206%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=sn7QOMMjrzsbBsvdEAPttouujDel0JuDptIFMHkm3Q0%3D&amp;reserved=0
>>
>> Thanks for the pointer Peter. I have looked at the patch and it is quite complex,
>> altering the very way perf event scheduling works.
>>
>> I don't dispute that is the right 'fix' for the issue, but do you think adding a
>> new perf context can help alleviate some of the issues in the interim?
> 
> And take away the motivation for people to do the right thing? How does
> that work out in my favour?
> 

:-) Understood Peter. I will take a stab at making your patch upstream ready.

Ravi