Subject: Re: [PATCH 0/2] arm64 SMMUv3 PMU driver with IORT support
To: Robin Murphy <robin.murphy@arm.com>,
        Linu Cherian <linuc.decode@gmail.com>,
        Neil Leeder <nleeder@codeaurora.org>
Cc: Will Deacon <will.deacon@arm.com>, Mark Rutland <mark.rutland@arm.com>,
        Mark Langsdorf <mlangsdo@redhat.com>, Jon Masters <jcm@redhat.com>,
        Timur Tabi <timur@codeaurora.org>, linux-kernel@vger.kernel.org,
        Mark Brown <broonie@kernel.org>, Mark Salter <msalter@redhat.com>,
        linux-arm-kernel@lists.infradead.org, linu.cherian@cavium.com,
        Sunil.Goutham@cavium.com, ynorov@caviumnetworks.com
References: <1501876754-1064-1-git-send-email-nleeder@codeaurora.org>
 <20171210023549.GA22492@virtx40>
 <999b7132-2de8-2872-1067-fc7e02cbc57a@arm.com>
From: Marc Zyngier <marc.zyngier@arm.com>
Organization: ARM Ltd
Message-ID: <7312293d-8897-95d8-9045-9e97a6c150eb@arm.com>
Date: Mon, 18 Dec 2017 15:39:22 +0000
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.5.0
MIME-Version: 1.0
In-Reply-To: <999b7132-2de8-2872-1067-fc7e02cbc57a@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-GB
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2700
Lines: 61

Thanks for putting me in the loop Robin.

On 18/12/17 14:48, Robin Murphy wrote:
> On 10/12/17 02:35, Linu Cherian wrote:
>> Hi,
>>
>>
>> On Fri Aug 04, 2017 at 03:59:12PM -0400, Neil Leeder wrote:
>>> This adds a driver for the SMMUv3 PMU into the perf framework.
>>> It includes an IORT update to support PM Counter Groups.
>>>
>>
>> In one of Cavium's upcoming SOC, SMMU PMCG implementation is such a way
>> that platform bus id (Device ID in ITS terminmology)is shared with that of SMMU.
>> This would be a matter of concern for software if the SMMU and SMMU PMCG blocks
>> are managed by two independent drivers.
>>
>> The problem arises when we want to alloc/free MSIs for these devices
>> using the APIs, platform_msi_domain_alloc/free_irqs.
>> Platform bus id being same for these two hardware blocks, they end up sharing the same
>> ITT(Interrupt Translation Table) in GIC ITS and hence alloc, free and management
>> of this shared ITT becomes a problem when they are managed by two independent
>> drivers.
> 
> What is the problem exactly? IIRC resizing a possibly-live ITT is a 
> right pain in the bum to do - is it just that?

Understatement of the day! ;-) Yes, it is a massive headache, and will
either result in a temporary loss of interrupts (at some point you have
to unmap the ITT), or with spurious interrupts (you generate interrupts
for all the MSIs you've blackholed when unmapping the ITT).

> 
>> We were looking into the option of keeping the SMMU PMCG nodes as sub nodes under
>> SMMUv3 node, so that SMMUv3 driver could probe and figure out the total vectors
>> required for SMMU PMCG devices and make a common platform_msi_domain_alloc/free_irqs
>> function call for all devices that share the platform bus id.
> 
> I'm not sure how scalable that approach would be, since it's not 
> entirely obvious how to handle PMCGs associated with named components or 
> root complexes (rather than directly with SMMU instances). We certainly 
> don't want to end up spraying similar PMCG DevID logic around all manner 
> of GPU/accelerator/etc. drivers in future (whilst PMCGs for device TLBs 
> will be expected to have distinct IDs from their host devices, they 
> could reasonably still overlap with other PMCGs/SMMUs).
> 
>> Would like to know your expert opinion on what would be the right approach
>> to handle this case ?
> 
> My gut feeling says the way to deal with this properly is in the ITS 
> code, but I appreciate that that's a lot easier said than done :/

I can revive the hack I once wrote for that (and that was hoping to
forever forget), but that's pretty disgusting, and subject to the above
caveat.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...